T-Head Panmai 920: Building Smarter 10,000-GPU Clusters
At the 2026 Apsara Conference in Hangzhou on September 22, T-Head presented its full-stack data center chip portfolio spanning compute, storage, and networking.
At the center of its networking lineup is the Panmai 920, T-Head’s first 400G SmartNIC. The chip combines several technologies aimed at the networking challenges of large-scale AI clusters, including an integrated PCIe switch, per-packet multi-path RDMA spraying, programmable congestion control, and a 512-core programmable processing engine.
These features target a fundamental problem in modern AI infrastructure: increasing the number of GPUs does not automatically translate into proportional increases in useful compute.
Once clusters scale into the thousands or tens of thousands of accelerators, network efficiency becomes a critical component of overall system performance. GPU resources can remain idle while waiting for synchronization, data transfers, congestion recovery, or communication between distributed workers.
Panmai 920 is designed around the premise that improving the network fabric can be as important as increasing GPU compute capacity.
🔗 Why Networking Becomes a Compute Problem #
An AI cluster can be viewed as a distributed computing system in which GPUs continuously exchange data during training and inference.
For large Transformer models, one of the most important communication operations is AllReduce. During training, GPUs calculate gradients independently and then synchronize those gradients across the cluster.
As the number of GPUs increases, the volume and concurrency of network traffic also increase.
A cluster containing thousands of GPUs can therefore encounter situations where accelerators spend significant amounts of time waiting for communication rather than performing computation.
The resulting bottleneck is not necessarily insufficient GPU performance. Instead, the limiting factor can be the network’s ability to move data efficiently between nodes.
This is one reason SmartNICs have become increasingly important in large-scale AI infrastructure. By moving networking, communication, and data-processing functions closer to the accelerator and away from the host CPU, SmartNICs can reduce the amount of system overhead associated with distributed workloads.
🚧 Three Networking Bottlenecks in 10,000-GPU Clusters #
Large-scale AI clusters face several networking problems that become increasingly difficult as the number of accelerators grows.
Hash Polarization #
Traditional Ethernet networks commonly use ECMP, or Equal-Cost Multi-Path routing, to distribute traffic across multiple paths.
The problem is that conventional ECMP generally performs hashing at the flow level rather than independently distributing every packet.
AI training traffic can contain a relatively small number of extremely large and long-lived flows. If several heavy flows are assigned to the same physical link, that link can become congested while other available paths remain underutilized.
This imbalance is commonly referred to as hash polarization.
For a 400G network, leaving substantial portions of available link capacity unused because of uneven flow placement can significantly reduce effective cluster bandwidth.
Incast Congestion #
AI training also produces highly synchronized traffic patterns.
During collective communication operations, many nodes may transmit data toward the same switch port or destination at approximately the same time. This creates an Incast pattern characterized by short, intense traffic bursts.
If the switch cannot absorb the burst in its buffers, packets can be dropped.
In RoCE networks, packet loss can be particularly expensive because reliable RDMA transport must recover from the missing data. Traditional retransmission behavior can cause additional traffic and further reduce effective bandwidth.
At large cluster scales, even a relatively small packet-loss rate can therefore have a disproportionate effect on application performance.
Long-Tail Latency #
Distributed training is also highly sensitive to latency variance.
A synchronization operation cannot complete until the required participants have received and processed the necessary data. If a small number of packets experience extreme delays, the entire operation can be held up.
This creates a long-tail latency problem: most network operations may complete quickly, while occasional operations take dramatically longer.
In large-scale training, those rare delays can accumulate into significant idle time across otherwise powerful GPU resources.
🧠 Panmai 920’s Approach to Multi-Path RDMA #
Different networking architectures address these problems in different ways.
InfiniBand provides an established high-performance interconnect designed around low-latency communication, congestion management, and advanced multipath capabilities. However, it is a proprietary ecosystem and can involve higher infrastructure costs and vendor-specific dependencies.
RoCEv2 instead implements RDMA over Ethernet. It benefits from the broader Ethernet ecosystem, but congestion control becomes a central engineering challenge.
T-Head’s Panmai 920 takes a different approach by combining:
- A self-developed networking ASIC
- Per-packet multi-path RDMA spraying
- Selective retransmission
- Hardware out-of-order packet handling
- Programmable congestion-control engines
The overall strategy is to move more intelligence into the SmartNIC so that congestion, packet distribution, and recovery can be handled closer to the endpoints.
📡 Per-Packet Spraying Targets Hash Polarization #
The most distinctive networking feature of Panmai 920 is per-packet spraying.
Instead of assigning an entire flow to a single path through conventional flow-based ECMP hashing, individual packets can be distributed across multiple available paths.
This allows traffic from a single connection to use multiple links concurrently.
In theory, this can significantly improve bandwidth utilization because a large flow is no longer constrained by the particular path selected by a flow-level hash.
For a 400G SmartNIC, the ability to spread traffic at packet granularity is particularly important in large AI clusters where individual communication flows can be extremely large.
Out-of-Order Packets Become the New Challenge #
Per-packet routing introduces another problem: packets traveling through different physical paths can arrive out of order.
Out-of-order delivery is relatively manageable in some networking protocols, but RDMA workloads are more sensitive to packet ordering and loss-recovery behavior.
If packet reordering is handled poorly, a technique designed to improve path utilization can instead introduce additional retransmissions and latency.
Panmai 920 addresses this through a combination of hardware-level out-of-order packet handling and selective retransmission.
Instead of requiring every packet to arrive in strict sequence, the receiving SmartNIC can buffer packets that arrive early and identify the specific packets that were actually lost.
Only the missing packets then need to be retransmitted.
T-Head’s reported benchmark results indicate that in a 1,000-GPU training cluster, this mechanism reduced job completion time by 14% and reduced switch queue depth by approximately 90%.
🎛️ Programmable Congestion Control for Different Workloads #
Congestion control is another major component of large-scale RoCE deployments.
Different algorithms make different trade-offs, and no single strategy is necessarily optimal for every AI workload.
DCQCN #
DCQCN uses ECN, or Explicit Congestion Notification, to provide congestion feedback in RoCEv2 networks.
It can respond quickly to congestion, but highly bursty AI traffic can produce oscillation and require careful parameter tuning.
HPCC #
HPCC uses precise telemetry information, including in-band network telemetry, to estimate network conditions and proactively control traffic.
This can provide more precise congestion management but places demanding requirements on hardware telemetry and clock synchronization.
RTT-Based Probing #
Round-trip-time-based approaches monitor latency changes to detect emerging congestion.
This approach can be useful for burst-oriented workloads, including large checkpoint transfers, where traffic behavior differs significantly from steady-state training communication.
Programmable Algorithms #
Panmai 920 integrates multiple congestion-control engines while exposing programmable interfaces that allow operators to customize network behavior.
This matters because a large model training workload, an online inference service, and a storage-intensive pipeline can produce very different traffic patterns.
Rather than permanently binding the SmartNIC to one congestion-control algorithm, programmability allows the networking behavior to evolve with the workload.
T-Head reports that in bursty traffic scenarios, Panmai 920 can reduce queue-depth fluctuation by approximately 90% and long-tail latency by approximately 95%.
🔌 Integrated PCIe Switch Enables Direct GPU Connectivity #
Another major feature of Panmai 920 is its integrated PCIe Switch.
Traditional SmartNIC architectures commonly connect the NIC to the host CPU through PCIe. If the NIC needs to exchange data with a GPU, data may need to traverse host memory or additional PCIe connections depending on the system design.
That adds latency and can consume CPU memory bandwidth.
One alternative is to deploy an external PCIe switch connecting the SmartNIC, GPUs, and storage devices directly. This can enable more direct data movement but adds another component, increasing board complexity, power consumption, cost, and potentially the number of failure points.
Panmai 920 integrates the PCIe switch directly into its SoC.
This allows the SmartNIC to establish direct PCIe connectivity with GPUs and storage devices without requiring a separate external PCIe switch.
GPUDirect RDMA #
The architecture enables GPUDirect RDMA, allowing network data to be transferred directly into GPU VRAM rather than passing through CPU memory first.
This reduces unnecessary data movement and can lower communication overhead.
The capability is particularly relevant to AI inference workloads where large amounts of data may need to be streamed continuously between the network and GPU.
Potential examples include:
- KV Cache transfers
- Intermediate Attention results
- Distributed inference
- GPU-to-network data pipelines
- Direct network-to-GPU data movement
T-Head says integrating the PCIe switch into Panmai 920 can reduce total system cost by approximately 30% compared with designs requiring a separate PCIe switch.
🧩 512 Programmable Cores Turn the NIC Into a Software-Defined Data Plane #
Panmai 920 also integrates 512 hyper-threaded processing cores.
The significance of these cores is not simply the raw core count. They enable the SmartNIC to function as a programmable data-plane processor rather than a fixed-function network controller.
Traditional NICs rely heavily on dedicated hardware logic for packet processing and forwarding. Once deployed, changing fundamental behavior can require new hardware.
Panmai 920 instead combines programmable processing with a P4-based networking model.
P4 is a domain-specific language designed to define packet-processing behavior in programmable network data planes.
This architecture can potentially allow operators to modify or extend networking functions through software.
Examples include:
- Updating congestion-control algorithms
- Customizing packet-priority policies
- Implementing AI-agent traffic scheduling
- Adding network telemetry logic
- Offloading Virtio networking functions
- Supporting container network isolation
This programmability can extend the useful operational life of the hardware because the SmartNIC does not have to remain tied to a single workload or networking strategy.
A deployment initially optimized for LLM training could potentially be reconfigured for inference, agent workloads, or other data-center applications without replacing the physical NIC.
⚡ Power Efficiency and Total Cost of Ownership #
T-Head reports that Panmai 920 consumes approximately 24W at full load.
According to the company, this represents approximately a 70% reduction compared with FPGA-based alternatives.
At large deployment scales, networking power consumption becomes a significant component of data-center operating costs.
For a cluster containing hundreds or thousands of accelerator nodes, reducing power consumption at the NIC level can translate into meaningful savings in addition to the benefits of improved network performance.
The integrated PCIe switch can also reduce component count by eliminating the need for a separate PCIe switch chip in compatible system designs.
Together, lower power consumption and reduced board complexity can affect the total cost of ownership beyond the SmartNIC’s purchase price alone.
🏗️ Panmai 920 Fits Into T-Head’s Broader Data Center Strategy #
Panmai 920 is part of a larger T-Head strategy covering multiple layers of data-center infrastructure.
The company’s portfolio includes:
- Zhenwu Series: AI compute chips
- Yitian Series: Arm server CPUs
- Panmai Series: SmartNICs
- Zhenyue Series: Storage controller chips
The strategic significance comes from integrating these components rather than evaluating each chip independently.
Compute, storage, and networking are tightly coupled in large AI systems. A bottleneck in any one layer can reduce the utilization of the others.
For example, faster accelerators are of limited value if the network cannot deliver data quickly enough. Likewise, a high-performance storage system can remain underutilized if the host or network becomes the limiting factor.
A vertically coordinated hardware platform can therefore provide opportunities for system-level optimization that are difficult to achieve when every component comes from a different vendor.
🔍 The Larger Challenge: Keeping Thousands of GPUs Busy #
The Panmai 920 illustrates a broader change in AI infrastructure design.
As AI clusters move toward thousands or tens of thousands of GPUs, networking is no longer simply an interconnect between compute nodes. It becomes part of the computational architecture itself.
The relevant performance metrics increasingly include:
- Effective bandwidth utilization
- Collective communication efficiency
- Packet-loss recovery
- Congestion convergence
- Tail latency
- GPU-to-GPU communication overhead
- Network-to-GPU data-transfer efficiency
- SmartNIC power consumption
Features such as per-packet spraying, selective retransmission, programmable congestion control, and direct GPU connectivity all target different portions of this system-level problem.
The objective is ultimately to reduce the amount of time GPUs spend waiting for the network.
📝 Final Takeaways #
T-Head’s Panmai 920 approaches large-scale AI networking by combining high-speed Ethernet connectivity with programmable packet processing and direct accelerator connectivity.
Its major features include:
- 400G SmartNIC networking
- Per-packet multi-path RDMA spraying to address hash polarization
- Selective retransmission for more efficient packet-loss recovery
- Hardware out-of-order packet handling
- Programmable congestion control
- Integrated PCIe Switch
- GPUDirect RDMA support
- 512 hyper-threaded programmable cores
- P4-based programmable data-plane capabilities
- Approximately 24W full-load power consumption, according to T-Head
T-Head’s reported testing indicates substantial reductions in queue depth, long-tail latency, and job completion time under selected AI workloads.
The broader significance of Panmai 920 lies in treating networking as an active component of AI compute infrastructure rather than a passive connection between GPUs.
At 10,000-GPU scale, the performance of the cluster depends not only on how much compute each accelerator provides, but also on how efficiently thousands of accelerators can exchange data and synchronize.
Panmai 920 represents T-Head’s attempt to address that problem through tighter integration of networking, PCIe connectivity, programmable processing, and AI-oriented congestion management. The effectiveness of this architecture at broader production scale will ultimately depend on workload characteristics, software integration, deployment topology, and real-world cluster behavior.