AMD Helion Packs 32,768 Zen 6 CPU Cores Into One Rack

AMD Helion Packs 32,768 Zen 6 CPU Cores Into One Rack

AMD has introduced Helion, an open rack-scale reference design built specifically for agentic AI infrastructure. Announced on October 12, 2026, the platform scales up to 32,768 CPU cores based on AMD’s sixth-generation EPYC processors, supports configurations with up to 16 Instinct MI350P GPUs, and provides up to 102 Tbps of aggregate networking bandwidth.

Unlike conventional AI racks designed primarily around dense GPU clusters, Helion emphasizes the CPU-intensive execution layer required by autonomous agents. While GPUs handle model inference, CPUs coordinate agents, execute generated code, invoke tools, manage workflows, and process the supporting workloads that surround AI applications.

AMD is offering two primary configurations: a CPU-only rack for hyperscalers that already operate separate GPU inference infrastructure, and an integrated enterprise rack combining EPYC processors, Instinct GPUs, and general-purpose compute resources.

The platform follows the Open Compute Project (OCP) hardware standards and supports both air-cooled and direct-liquid-cooled configurations. AMD plans to make Helion available through ecosystem partners in November 2026, with the specification being contributed to OCP for further development and adoption.

The announcement reflects a broader architectural change in AI infrastructure. As AI systems evolve from individual model requests into autonomous workflows, CPU execution capacity, agent density, networking, and infrastructure efficiency are becoming increasingly important alongside GPU performance.

🏗️ AMD Helion: Two Rack Configurations for Different AI Workloads

#

AMD’s central design principle is that agentic AI is not a single, uniform workload. Personal assistants, enterprise agents, code execution environments, and technical computing agents place different demands on CPU performance, memory, power, and GPU acceleration.

A fixed server configuration cannot optimize every workload simultaneously. Helion therefore provides an adaptable rack blueprint that allows system manufacturers and customers to choose the appropriate balance of compute resources.

Hyperscale CPU-only rack

#

The hyperscale configuration is designed for cloud operators, neocloud providers, and sovereign AI infrastructure operators that already have dedicated GPU clusters for model inference.

In this configuration, the rack concentrates on agent execution rather than duplicating GPU capacity that may already exist elsewhere in the data center.

Its maximum configuration supports 32,768 Zen 6 CPU cores per rack, using AMD’s sixth-generation EPYC 9006 series processors.

Typical workloads include:

• Agent orchestration: Scheduling tasks, managing execution state, and coordinating multiple agents.
• Code execution: Running generated code, automated tests, and isolated development environments.
• Tool integration: Interacting with databases, APIs, storage systems, and enterprise applications.
• Workflow management: Executing multi-step processes that invoke models and external services repeatedly.
• Distributed application services: Supporting the backend infrastructure required by large agent fleets.

The CPU-only model separates agent execution from model inference infrastructure. Agents can run across a dense CPU fleet while invoking remote GPU-hosted models when computationally intensive inference is required.

This decoupling allows hyperscalers to scale CPU capacity according to the number and complexity of active agents, rather than tying every additional execution environment to another GPU.

Enterprise integrated rack

#

The enterprise configuration combines EPYC CPU nodes, Instinct MI350P GPU nodes, and general-purpose servers in one rack.

It is intended for organizations that want to deploy agent execution, model inference, databases, storage, and other supporting services within a more integrated infrastructure footprint.

AMD specifies support for up to 16 Instinct MI350P GPUs per rack. Each 5U GPU server accommodates eight accelerators, allowing a fully configured rack to contain two GPU servers.

The GPU servers operate as independent nodes rather than forming a tightly coupled scale-up GPU domain across the two servers. This arrangement targets enterprise inference and general acceleration rather than frontier-model training architectures that require extensive accelerator-to-accelerator communication within one tightly integrated system.

The integrated design offers several potential advantages:

• Simplified deployment: CPU execution and GPU inference resources can be provisioned within one rack architecture.
• Flexible resource allocation: Operators can adjust CPU density and GPU capacity according to application requirements.
• Infrastructure consolidation: General-purpose workloads can share the rack with AI execution and inference services.
• Open hardware integration: Standardized rack and server components give OEMs flexibility when designing complete systems.

Actual configurations will depend on the number of CPU trays, GPU servers, general-purpose nodes, and networking components installed.

⚡ CPU Density, Power Profiles, and Node Configurations

#

Helion’s differentiator is its ability to scale CPU resources independently of GPU capacity. AMD provides several example server configurations to address distinct agentic AI workloads.

Three CPU node types

#

The reference design describes three CPU node categories, each optimized for a different performance target.

These configurations illustrate the flexibility of the rack design rather than a mandatory specification for every Helion deployment.

Dense nodes prioritize the number of execution environments that can be supported within a power budget. Balanced nodes allocate more power to sustain demanding enterprise workloads, while performance nodes emphasize per-agent responsiveness for workloads that benefit from higher CPU frequencies.

The appropriate configuration depends on the workload’s parallelism, memory requirements, scheduling overhead, and latency targets.

Why CPU density matters for agentic AI

#

Traditional AI infrastructure often measures capacity in GPUs, accelerator FLOPS, or model throughput. Agentic systems introduce another important dimension: the number of concurrent workflows that can execute efficiently.

An individual agent may need to run code, retrieve information, process files, invoke tools, communicate with other services, and submit multiple inference requests before completing a task.

The actual model inference may consume only part of the total end-to-end execution time. The remaining work can place substantial demands on CPU resources and general-purpose infrastructure.

As a result, maximizing GPU density does not necessarily maximize the number of complete agent workflows a system can support.

Helion addresses this by treating CPU execution as a first-class infrastructure requirement. Instead of adding GPUs whenever more agents are needed, operators can scale the CPU execution layer independently and use shared or dedicated GPU infrastructure for inference.

For hyperscalers, the goal is to maximize the number of agents served per watt. For enterprise deployments, the objective may be to balance concurrent execution, application responsiveness, and local inference capacity.

Air cooling and liquid cooling

#

Helion supports both air-cooled and direct-liquid-cooled rack configurations.

Air cooling can simplify deployment in facilities designed around conventional server infrastructure, although high-density racks still require sufficient airflow and carefully managed power budgets.

Direct liquid cooling provides another option for configurations with higher thermal density. It can support more demanding processor and accelerator arrangements where conventional airflow would impose greater cooling constraints.

Offering both approaches gives system manufacturers more flexibility when adapting Helion to different data center environments.

🌐 Up to 102 Tbps of Networking Bandwidth

#

AMD specifies up to 102 Tbps of aggregate networking bandwidth for Helion’s maximum configuration.

This figure represents aggregate rack networking capacity, not the speed of an individual network port. The actual available bandwidth depends on the installed NICs, switches, server configuration, and network topology.

Networking is particularly important for agentic AI because agent workflows frequently communicate with resources beyond the CPU node executing them.

An agent may retrieve data from a database, access shared storage, call a remote model endpoint, execute a tool in another environment, and pass intermediate results to another service. These operations create network traffic even when the agent itself performs relatively little intensive numerical computation.

Open networking and rack integration

#

Helion follows OCP hardware conventions intended to improve interoperability across server manufacturers and infrastructure suppliers.

The design includes support for the OCP ORv3 rack architecture and standardizes several important hardware layers:

• Rack and busbar: OCP ORv3.
• Host processor module: Data Center Modular Hardware System (DC-MHS).
• Management and security: Data Center Secure Control Module (DC-SCM).
• Power shelf: High Power Rack v2 (HPRv2).
• Networking: OCP NIC 3.0.
• Silicon root of trust: Caliptra.

These specifications address different parts of the infrastructure stack, including physical integration, power distribution, host hardware, management, and security.

Standardization does not mean that every component from every manufacturer will work together without qualification. Firmware, cooling, power delivery, mechanical compatibility, and system integration still require validation.

Nevertheless, a standardized rack blueprint gives OEMs a common foundation for developing compatible systems rather than forcing customers to adopt one fully proprietary hardware configuration.

🧠 AMD Token Router: Coordinating Frontier Models and Local Inference

#

Helion also introduces a software-level concept designed to connect CPU-based agent execution with model inference services.

AMD describes an integrated Token Router that can dynamically direct an agent’s model calls between cloud-based frontier models and on-rack open-source models.

This creates the possibility of choosing different inference resources according to the requirements of each request.

Routing model calls according to workload needs

#

Not every model invocation requires the same level of capability.

A simple classification task, routine summarization request, or lightweight transformation may be suitable for a locally hosted open-source model. A more demanding task involving complex reasoning may benefit from a larger frontier model hosted in a remote GPU cluster.

A routing layer can help determine which inference endpoint should process a request, taking account of factors such as:

• Latency: Whether a response must be returned within a particular time budget.
• Cost: Whether a smaller local model can complete the task at a lower serving cost.
• Model capability: Whether the task requires a more capable model.
• Resource availability: Whether local inference capacity is available or heavily loaded.
• Operational policies: Whether a request must remain within a designated infrastructure environment.

For enterprise deployments, this can help balance quality, cost, and responsiveness across different inference services.

The Token Router also fits naturally into the distinction between execution and inference. CPUs coordinate agent workflows, while a routing layer directs individual model requests to the inference resources best suited to them.

Why routing matters at scale

#

If a single agent executes several steps, it may invoke a model multiple times. A large fleet can therefore generate a substantial volume of inference traffic, even when individual requests are relatively small.

Routing appropriate tasks to local models could reduce dependence on more expensive frontier inference resources. Conversely, retaining access to larger models can preserve quality when tasks require more advanced reasoning.

The practical benefit depends on the quality of routing decisions, model compatibility, latency measurements, serving capacity, and the software integration between agents and inference endpoints.

The Token Router should therefore be understood as a key component of AMD’s proposed software architecture, rather than a guarantee that every request will automatically reach an optimal model.

🔓 OCP Compliance and the Open Hardware Strategy

#

AMD plans to contribute the Helion specification to the Open Compute Project community, extending the open-platform strategy used for its Helios rack-scale AI infrastructure.

This is an important distinction between an open reference design and a finished proprietary rack product.

AMD provides the core design direction and compute platform, while OEMs and ODMs can develop complete systems and adapt configurations to customer requirements.

Ecosystem participation

#

AMD identifies ecosystem partners including Dell, Foxconn, HPE, Lenovo, QCT, Supermicro, and Wiwynn.

Supermicro and other participating hardware providers can use the design to develop market-ready systems. Their products may differ in CPU node selection, GPU capacity, cooling, networking, and other implementation details.

This partner-driven model can expand the range of available configurations while reducing the need for each customer to independently design and qualify an entire rack from scratch.

OCP participation also creates a route for additional industry feedback and potential evolution of the specification.

Open standards versus proprietary rack architectures

#

The AI infrastructure market is increasingly competitive at the rack level, not just at the individual processor or accelerator level.

NVIDIA’s integrated systems combine CPUs, GPUs, networking, and software into tightly coordinated platforms. AMD’s strategy also includes a fully integrated rack-scale system, Helios, but Helion emphasizes modularity and CPU-focused agent execution.

The difference is not simply open versus closed hardware. Both companies develop integrated infrastructure, while their rack designs make different trade-offs in system integration, resource allocation, and hardware flexibility.

Helion’s use of OCP conventions gives customers and system builders a standardized foundation for configuration and interoperability. The extent of that flexibility will depend on how the specification is implemented by manufacturers and adopted across the ecosystem.

For additional context, see KAD’s analysis of AMD Helios, Zen 6 EPYC, Instinct MI455X, and rack-scale AI infrastructure.

📈 AMD Helion vs. Helios: Two Complementary AI Infrastructure Strategies

#

Helion is not a replacement for AMD’s Helios rack-scale AI system. The two platforms address different parts of the growing AI infrastructure market.

AMD introduced Helios in July 2026 as a rack-scale platform for frontier-model training and large-scale inference. Its architecture combines 72 Instinct MI455X GPUs with 18 sixth-generation EPYC processors, alongside high-speed networking and the ROCm software stack.

Helion extends the roadmap toward CPU-dominated agent execution.

How the two platforms differ

#

The platforms can complement one another within a wider infrastructure environment.

Helios provides the concentrated GPU capacity required by demanding model training and inference jobs. Helion provides CPU capacity for the execution work surrounding those models, with integrated GPU inference available for enterprises that prefer a combined deployment.

The distinction becomes increasingly relevant as AI applications evolve beyond individual prompts into longer, multi-stage processes involving tools, data access, and coordination among several agents.

For more context on this shift, KAD discusses how AI infrastructure is evolving around autonomous agents and specialized computing resources.

The broader rack-scale competition

#

The market is shifting toward complete infrastructure architectures in which CPUs, GPUs, networking, storage, and software must operate as a coordinated system.

NVIDIA is advancing its Vera and Rubin platform roadmap, while AMD is developing complementary rack-scale systems for both frontier AI and agentic execution workloads.

In this environment, the competitive question is not simply which company offers the fastest processor. It is which architecture can deliver the required application throughput at an acceptable cost, power level, and operating complexity.

Helion’s CPU-centric approach reflects the possibility that autonomous AI systems will need substantially more general-purpose execution capacity than conventional model-serving infrastructure was originally designed to provide.

✅ Conclusion: Productizing the CPU Execution Layer for Agentic AI

#

AMD Helion extends the company’s rack-scale strategy beyond GPU-dominated AI infrastructure by addressing the CPU execution requirements of autonomous agents.

The reference design scales to 32,768 Zen 6 CPU cores per rack, supports up to 16 Instinct MI350P GPUs, and provides up to 102 Tbps of aggregate networking capacity. Its CPU-only hyperscale configuration targets agent execution at high density, while its integrated enterprise configuration combines CPU processing, local inference, and general-purpose workloads.

The platform also incorporates a Token Router concept for directing model requests between frontier and open-source inference resources. Combined with its OCP-based design, this gives customers and hardware partners flexibility to balance compute capacity, inference performance, power consumption, and deployment requirements.

The central idea is that agentic AI infrastructure needs more than GPUs. It requires a scalable execution layer capable of running tools, managing workflows, executing code, and coordinating model calls across distributed systems.

Helios and Helion address complementary parts of that infrastructure: one emphasizes concentrated GPU capacity for advanced model computation, while the other productizes flexible CPU execution at rack scale.

Helion’s ultimate success will depend on real-world deployment density, energy efficiency, software integration, and the economics of operating large agent fleets. Its planned availability through ecosystem partners in November 2026 will provide an initial test of whether the open rack-scale approach can meet these requirements at production scale.