What type of interconnects and connectors link accelerator cards in AI data centers?

May 22, 2025 by Aharon Etengoff

Collected at: https://www.eeworldonline.com/what-type-of-interconnects-and-connectors-link-accelerator-cards-in-ai-data-centers/

Many data centers are packed with racks of high-performance graphics processing units (GPUs) and tensor processing units (TPUs). These accelerators process massive artificial intelligence (AI) and machine learning (ML) datasets, executing complex operations in parallel and exchanging data at high speed. This article explores the interconnects and connectors that link AI accelerator clusters together.

Scaling AI compute with accelerators and clustered architectures

AI accelerators such as GPUs, TPUs, and, in some cases, field-programmable gate arrays (FPGAs) run large language models (LLMs) using parallel processing to handle complex computations at scale. These devices divide complex workloads into smaller tasks and execute billions of operations simultaneously. Most AI models are built on neural networks, which benefit from this massively parallel architecture to accelerate both training and inference.

As shown in Figure 1, AI accelerators are typically deployed in tightly coupled clusters to efficiently share data, synchronize computations, and scale training across thousands of processing units.

This configuration helps meet the low-latency, high-performance demands of AI workloads. It also improves throughput, minimizes bottlenecks, and enables real-time inference for complex, compute-intensive tasks.

High-level interconnect architectures and protocols

Data centers use specialized interconnect technologies to link AI accelerator clusters to operate efficiently at scale, enabling high-speed communication within and across nodes. These interconnects support massive data exchange, synchronized processing, and the parallel execution of complex workloads. Common AI accelerator interconnects include:

NVLink — NVIDIA’s proprietary, high-bandwidth interconnect facilitates direct GPU-to-GPU communication with low latency and high energy efficiency. It supports rapid synchronization and data sharing across accelerators using dedicated connectors and NVSwitch technology. NVLink scales efficiently in multi-GPU environments by enabling memory pooling, allowing GPUs to share a unified address space and operate as a single, high-performance compute unit. As shown in Figure 2, NVLink 4.0 delivers up to 900 GB/s of bidirectional bandwidth on the H100 GPU.

UALink  — the Ultra Accelerator Link is an open interconnect standard designed to scale clusters of up to 1,024 AI accelerators within a single computing pod. The 1.0 specification supports 200G per lane and enables dense, memory-semantic connections with Ethernet-class bandwidth and PCIe-level latency. UALink supports read, write, and atomic transactions across nodes and defines a common protocol stack for scalable multi-node systems. UALink is positioned as a high-performance alternative for scaling within accelerator pods by targeting lower latency than typical Ethernet for inter-node communication.

Compute Express Link (CXL) enables coherent, low-latency communication between CPUs, GPUs, and other accelerators. It improves resource utilization across heterogeneous systems by supporting cache coherency, memory pooling, resource sharing, and memory disaggregation. CXL 1.1 and 2.0 operate over PCIe 5.0, while CXL 3.0 and later leverage PCIe 6.0 or beyond, enabling transfer speeds of up to 64 GT/s and bidirectional bandwidth of 128 GB/s.

High-speed Ethernet facilitates data movement between accelerator clusters distributed across servers and nodes. Technologies such as 400 GbE and 800 GbE enable high-throughput communication using NICs and optical or copper cabling. While Ethernet introduces higher latency than NVLink or UALink, it offers broad interoperability and flexible deployment at the rack and data center levels.

Optical interconnects and form factors; optical links transmit data at high speeds over extended distances, linking accelerator clusters across racks and nodes. Compared to copper-based connections, they consume less power and overcome signal integrity challenges such as attenuation and EMI. These interconnects often rely on standardized form factors, such as Quad Small Form-factor Pluggable (QSFP), Quad Small Form-factor Pluggable Double Density (QSFP-DD), and Octal Small Form-factor Pluggable (OSFP), which function as the physical interface for both electrical and optical Ethernet connections. These same form factors are also widely used for other high-speed optical interconnects in data centers, such as InfiniBand and proprietary optical links, further extending their role in scalable compute infrastructure.

Physical connectors and interfaces for AI accelerators

High-performance interconnects rely on various physical-layer components, including connectors, slots, and cabling interfaces. These components help maintain signal integrity, mechanical compatibility, and scalable system design. They transmit electrical and optical signals across boards, devices, and systems, facilitating the reliable operation of clustered AI infrastructure.

Although interconnects define the communication protocols and signaling standards, they rely on these physical interfaces to function effectively at scale. Common connector and interface technologies are described below.

PCIe interface connects accelerator cards to host systems and other components. Although newer generations, such as PCIe 5.0 and 6.0, offer scalable bandwidth, they may act as bottlenecks in tightly coupled multi-accelerator environments. Retimers are often used to maintain signal integrity over longer board traces.

Mezzanine connectors are used in the Open Compute Project’s Open Accelerator Infrastructure (OAI). They support high-density module-to-module connections, reduce signal loss, manage impedance, and simplify mechanical integration in modular accelerator designs.

Active electrical cables (AECs) integrate digital signal processors within copper cabling to boost signal strength over longer distances. This enables electrical links to maintain data integrity beyond the reach of passive cables.

High-speed board-to-board connectors enable direct module communication at data rates up to 224 Gbps using PAM4 modulation. They support dense, low-latency communication within AI platforms and tightly integrated accelerator clusters.

Optical connectors — QSFP, QSFP-DD, and OSFP form factors are the physical interface for both optical and short-range electrical Ethernet connections. These transceiver formats are widely deployed across NICs, switch ports, and optical modules and support PAM4 modulation to maintain signal performance across various deployment scenarios.

Liquid-cooled connectors

As shown in Figure 3, an increasing number of high-performance AI accelerator racks rely on liquid cooling. Many of the connectors used in these systems must meet stringent mechanical and thermal requirements to ensure safe, reliable operation.

These connectors typically withstand temperatures up to 50°C (122°F), support coolant flow rates up to 13 liters per minute (LPM), and maintain low pressure drops around 0.25 pounds per square inch (psi). They provide leak-free operation with water-based and dielectric fluids, prevent corrosion, and integrate easily with in-rack manifolds.

Most liquid-cooled connectors incorporate quick-disconnect functionality for dripless maintenance access. Large internal diameters — often around 5/8 inch — support high flow rates across AI racks. Some offer hybrid designs that combine high-speed data transmission with liquid cooling channels. Others support compatibility with three-inch square stainless-steel tubing or feature ruggedized construction to withstand temperature fluctuations, pressure changes, and vibration.

Summary

AI data centers depend on various interconnects and physical connectors to link accelerator cards, enable high-speed data exchange, and facilitate large-scale parallel processing. These components are critical in maintaining performance, signal integrity, and mechanical reliability across tightly coupled clusters.

References

A Deep Dive into the Copper and Optical Interconnects Weaving AI Clusters Together, Marvell
The Evolution of AI Interconnects, Marvell
Open Accelerator Infrastructure, Molex
NVIDIA NVLink-C2C, Nvidia
UCIe For 1.6T Interconnects In Next-Gen I/O Chiplets For AI Data Centers, Semiconductor Engineering
New Connectivity Solutions for the AI Data Center, ConnectorSupplier
New Mezzanine Connector that Supports the Open Computing Project, Arrow
PCIe 6.0 and CXL: The Perfect Alignment for AI and ML Workloads, Signal Integrity Journal