Google TPU 8th Gen: 8t, 8i, 8tx and 8i Max Explained

Google TPU 8th Gen: 8t, 8i, 8tx and 8i Max Explained

Google is taking a different approach to AI accelerator development: instead of building one increasingly powerful chip for every workload, it is creating specialized systems around a modular silicon platform.

On November 9, 2026, Google announced two additional eighth-generation Tensor Processing Unit (TPU) configurations: TPU 8tx and TPU 8i Max. Together with the previously introduced TPU 8t and TPU 8i, these additions bring the eighth-generation portfolio to four distinct systems within a single calendar year.

The expansion reflects two problems becoming increasingly difficult to solve with conventional AI infrastructure: protecting sensitive information during inference and supplying enough memory bandwidth for increasingly large context windows.

Google’s strategy is to adapt packaging, host processors, networking, and security features around existing silicon rather than develop an entirely new chip design for every workload.

The result is a four-system portfolio targeting different stages of the AI lifecycle:

This is more than a product lineup expansion. It illustrates how AI infrastructure is shifting from optimizing raw accelerator performance toward designing complete systems around specific production workloads.

🧩 Google’s Modular TPU Strategy: Building the System Around the Chip

#

Google’s announcement makes an important distinction between a new chip and a new accelerator system.

Developing a new silicon design involves transistor placement, circuit architecture, physical verification, manufacturing validation, and a lengthy production process. These activities can take years.

A system configuration can evolve much faster.

By treating compute silicon as a modular building block, Google can change how accelerators are packaged, connected, secured, and integrated with host processors without necessarily requiring a new chip tape-out.

This approach allows the company to respond to changing AI workloads on a shorter development cycle.

For example, a training cluster prioritizes distributed communication bandwidth and sustained compute utilization. An inference cluster may prioritize request latency, memory access, and the ability to handle multiple concurrent sessions.

A confidential inference environment introduces another requirement: sensitive model weights and user data must remain protected even while the accelerator is actively processing them.

These workloads do not necessarily need completely different compute architectures. They need different system-level trade-offs.

The four systems target different parts of the AI lifecycle

#

Google’s eighth-generation portfolio can be understood as four specialized configurations built around distinct operational requirements.

• TPU 8t focuses on large-scale foundation model training and maximizing useful compute output per watt.
• TPU 8i targets low-latency reasoning workloads, where inter-chip communication and memory access directly affect response times.
• TPU 8tx combines training-oriented compute with confidential computing capabilities for regulated workloads.
• TPU 8i Max increases compute and memory density through dual-chip packaging, targeting extremely large context windows.

Google previously reported that TPU 8t delivers up to 2.7 times better training performance than its preceding generation, while TPU 8i provides up to 80% better inference price-performance.

These are Google’s published comparisons, not universal performance guarantees across every model or deployment configuration.

The important architectural change is that the eighth-generation platform now addresses more than training and inference throughput. It also targets security boundaries, memory capacity, and the operational requirements of autonomous agents.

🔐 TPU 8tx: Hardware-Enforced Confidential Computing for AI

#

TPU 8tx is designed for organizations that need to process sensitive information without exposing proprietary data or model weights during computation.

Typical customers include financial institutions, healthcare organizations, government agencies, and enterprises operating under strict data protection requirements.

Traditional encryption protects information at rest and in transit. However, applications generally need to decrypt data into memory before performing computations.

This creates an exposure window.

If an attacker compromises the host environment or gains unauthorized access to memory, sensitive information may become accessible even though storage and network encryption are enabled.

TPU 8tx addresses this problem by extending hardware-level confidential computing to the accelerator itself.

Reusing TPU 8t silicon

#

One of the more interesting design decisions is that TPU 8tx does not introduce an entirely new compute die.

Google builds it around the same underlying silicon used by TPU 8t, the training-focused configuration.

The specialized capabilities come from system integration, including host processor selection, security architecture, and networking topology.

This reduces the need to develop and validate another independent accelerator design while allowing Google to target a different customer segment.

It also demonstrates why evaluating AI hardware solely by its chip specifications can be misleading. Host processors, memory architecture, interconnects, and isolation mechanisms can substantially change the behavior of an otherwise similar accelerator.

AMD EPYC Turin provides the host platform

#

TPU 8tx pairs its accelerator infrastructure with fifth-generation AMD EPYC processors, codenamed Turin.

The host platform provides CPU resources and PCIe connectivity for the TPU system and its supporting network hardware.

Google’s configuration uses dual-row Turin arrangements with 128 PCIe Gen5 lanes, providing connectivity for four TPUs and high-speed networking components.

This matters because accelerator performance is not determined by compute throughput alone.

If host-to-device transfers or network communication become bottlenecks, expensive accelerator resources can remain underutilized.

The PCIe Gen5 host configuration provides the required I/O connectivity for the current system design without introducing unnecessary platform complexity.

Confidential computing protects data during processing

#

The main security feature of TPU 8tx is its hardware-enforced Trusted Execution Environment (TEE).

A TEE creates an isolated execution boundary intended to protect sensitive data and code from unauthorized access, including access from parts of the surrounding infrastructure.

For AI workloads, this has several implications:

• Proprietary model weights can remain protected during execution.
• Sensitive enterprise datasets can be processed within an isolated hardware environment.
• Inference requests can be handled without exposing plaintext data to untrusted components of the infrastructure.
• Organizations can fine-tune models using private information while maintaining stronger security boundaries.

Google describes the performance overhead of this protection as near zero.

That claim is relevant because confidential computing has traditionally involved trade-offs involving memory encryption, isolation, and additional hardware operations.

The actual security guarantees still depend on the complete implementation, including attestation, key management, firmware security, host isolation, and the trust model of the surrounding cloud infrastructure.

A TEE should not be treated as a replacement for application-level authorization or secure software design. It provides another layer in the security architecture.

Boardfly networking reduces inference latency

#

TPU 8tx also adopts the Boardfly network topology developed for TPU 8i.

This is a departure from the three-dimensional torus topology used by TPU 8t for large-scale distributed training.

The two approaches target different communication patterns.

A 3D torus connects accelerator nodes through a structured network that supports large distributed workloads. It is well suited to communication patterns associated with scaling foundation model training.

Boardfly is optimized for lower-latency communication in inference and reasoning workloads.

According to Google, this topology reduces the maximum number of network hops from 16 on TPU 8t to seven on TPU 8tx.

Google also reports up to 2x lower latency compared with TPU 8t, with near-zero performance overhead from its confidential computing implementation.

For multi-step AI agents, this matters because a single user request may trigger several model calls, tool executions, and intermediate reasoning stages.

Reducing communication delays across those stages can improve end-to-end response time.

Training and inference can run within the same secure environment

#

Another advantage of TPU 8tx is that it combines training-oriented compute with an inference-focused network.

This gives organizations more flexibility when working with proprietary models and private datasets.

Potential workloads include:

• Fine-tuning foundation models using confidential enterprise data.
• Distilling large models into smaller deployment-oriented versions.
• Running secure inference for autonomous agents.
• Performing reinforcement learning using private operational information.
• Customizing models without transferring sensitive datasets between separate infrastructure environments.

This reduces the need to maintain completely separate clusters for every stage of model customization and deployment.

For regulated enterprises, keeping the entire workflow within a consistent security boundary can simplify infrastructure design and data governance.

🧠 TPU 8i Max: Addressing the Memory Wall in Long-Context AI

#

TPU 8i Max targets a different infrastructure limitation.

As AI agents become capable of working across entire software repositories, large document collections, and hours of video, memory bandwidth increasingly limits performance.

Raw compute throughput is not always the bottleneck. The accelerator also needs to receive enough data quickly enough to keep its processing units occupied.

This is commonly described as the memory wall.

Google addresses the problem by combining two low-latency TPU 8i chips into a single multi-chip package.

Dual-chip packaging doubles key resources

#

TPU 8i Max integrates two TPU 8i dies within one package.

Compared with a standard single-die TPU 8i, Google says the Max configuration provides:

• Twice the compute capacity.
• Twice the on-chip SRAM.
• Twice the HBM bandwidth.
• Higher compute density within the existing rack footprint.

This is a coordinated increase in processing and memory resources.

Simply adding more compute units would not solve the memory wall if the accelerator could not supply those units with data fast enough.

By scaling compute, SRAM, and HBM bandwidth together, TPU 8i Max aims to maintain higher utilization during memory-intensive workloads.

Why HBM bandwidth matters for large context windows

#

Large language models rely heavily on the Key-Value (KV) cache during autoregressive inference.

The KV cache stores intermediate attention information from previously processed tokens so the model does not need to recompute the same information for every subsequent token.

As context length and concurrent request counts increase, KV-cache memory requirements can grow substantially.

This becomes particularly relevant for autonomous agents.

An agent working across a large codebase may need to retain information about numerous source files, dependencies, previous tool calls, and intermediate reasoning steps.

A video-analysis system may need to process extended sequences containing large amounts of multimodal information.

Both workloads can create substantial memory traffic.

Increasing HBM bandwidth helps move data between high-bandwidth memory and the accelerator’s compute units more efficiently.

Increasing SRAM capacity provides additional fast on-chip storage for frequently accessed data.

These resources work together to reduce the likelihood that compute units remain idle while waiting for memory operations.

Multi-chip packaging improves rack-level compute density

#

TPU 8i Max also targets data center efficiency.

Google says the dual-chip configuration fits twice the silicon into the existing rack footprint, allowing operators to deploy more compute capacity per square foot and per watt.

This is relevant because data center expansion is constrained by more than accelerator availability.

Power delivery, cooling capacity, physical rack space, networking, and facility infrastructure all influence how much compute can be deployed.

Increasing accelerator density without proportionally expanding the facility footprint can improve infrastructure economics.

However, higher silicon density also increases thermal and power-delivery demands. The complete system design must account for those requirements rather than treating packaging density as a standalone performance metric.

For workloads involving million-token contexts and multiple cooperating agents, TPU 8i Max is designed to provide more compute and memory bandwidth within a tighter physical deployment.

📊 TPU 8t vs 8i vs 8tx vs 8i Max

#

The four configurations are easier to understand when compared by their intended workload and system-level design.

The distinction is not that one configuration is universally faster than another.

Each system is optimized around a different bottleneck.

TPU 8t prioritizes distributed training performance. TPU 8i focuses on inference latency. TPU 8tx adds hardware-level confidentiality, while TPU 8i Max increases memory and compute density for demanding context-heavy workloads.

This specialization allows Google to avoid forcing customers into a single accelerator architecture regardless of their actual application requirements.

⚙️ The Software Stack Is Becoming Just as Important as the Hardware

#

Google’s November announcement also covered software infrastructure intended to make TPUs easier to adopt and operate.

This is a significant part of the strategy.

Historically, specialized accelerators have required developers to work with hardware-specific frameworks, compilers, and optimized execution paths. That creates additional engineering overhead compared with deploying workloads on more familiar GPU environments.

Google is attempting to reduce this barrier through broader framework support and infrastructure-level improvements.

TorchTPU and PyTorch support

#

Google has open-sourced TorchTPU, allowing PyTorch applications and Hugging Face models to target TPU hardware.

The objective is to reduce the amount of code modification required when moving existing machine learning workloads onto Google’s accelerators.

Google is also providing TPU backends for open-source inference engines such as vLLM and SGLang.

For developers, this matters because framework compatibility can have a greater impact on adoption than theoretical hardware performance.

A powerful accelerator is difficult to justify if migrating an existing model requires rewriting large portions of the training or inference stack.

The practical evaluation should still include operator coverage, model compatibility, compiler maturity, debugging support, and performance stability across production workloads.

TPU agent tools automate hardware optimization

#

Google also introduced an open-source TPU agent suite, currently in preview.

It integrates with coding environments such as Claude Code, Cursor, and Google Antigravity.

The tools are intended to automate tasks such as:

• TPU environment onboarding.
• Performance bottleneck diagnosis.
• Custom kernel optimization.
• Hardware-specific configuration.
• Kernel autotuning through natural-language interaction.

This approach could reduce the amount of specialized knowledge required to optimize workloads for TPU infrastructure.

However, automated tuning does not eliminate the need to understand memory behavior, tensor layouts, communication overhead, and compiler-generated execution plans.

It changes how developers interact with those details rather than removing them.

XLA Serving Engine

#

For production inference, Google introduced the open-source XLA Serving Engine.

It allows models developed and trained in Python to be served through a low-latency C++ runtime across TPUs, GPUs, and CPUs.

Separating the development environment from the production serving runtime can help reduce runtime overhead and provide more predictable deployment behavior.

For infrastructure teams, the ability to target multiple hardware platforms through a common serving architecture also makes migration and workload scheduling easier to manage.

🔄 TPU Sync: Building a Unified Memory Hierarchy for AI Agents

#

One of the more interesting software announcements is TPU Sync, Google’s proposed model-state layer for TPU clusters.

As agentic workloads grow, keeping every piece of model state inside accelerator memory becomes increasingly expensive and restrictive.

The KV cache is a particularly important example.

During long-context inference, the KV cache can consume a substantial portion of available HBM. When multiple requests or agents run concurrently, memory pressure can increase rapidly.

TPU Sync aims to manage this problem by coordinating multiple levels of the infrastructure:

• Accelerator HBM.
• Host system DRAM.
• Shared storage.
• Data center networking.

Instead of treating these resources as isolated components, TPU Sync creates a coordinated memory hierarchy for managing model state.

Offloading KV caches

#

The system can move less immediately needed KV-cache data away from accelerator memory and retrieve it when required.

This allows an AI service to retain more context without requiring every token’s associated state to remain permanently in HBM.

For long-running agent sessions, this is particularly useful because agents frequently alternate between computation and waiting.

An agent might generate a plan, call an external database, wait for a tool response, and then resume reasoning.

Keeping all associated state in expensive accelerator memory during these idle periods is inefficient.

A tiered memory architecture offers a way to reduce that cost.

Disaggregated prefill and decode

#

TPU Sync also addresses the separation between prefill and decode workloads.

During prefill, the model processes the input prompt and builds the initial KV cache.

During decode, it generates output tokens incrementally using the accumulated context.

These phases have different performance characteristics and can benefit from different resource allocations.

Disaggregating prefill and decode allows infrastructure operators to assign resources according to the needs of each stage.

However, this introduces a data movement challenge: the KV cache must be transferred between the relevant execution nodes.

Google says TPU Sync is designed to streamline this process and synchronize model state across distributed resources.

The company also reports that its approach can reduce KV transfer latency and Time to First Token (TTFT) by more than 90% in its described configurations.

As with other vendor performance claims, actual results depend on workload characteristics, model architecture, context size, network topology, and baseline configuration.

☁️ Google Kubernetes Engine: Scaling TPU Infrastructure More Flexibly

#

Google also announced improvements to TPU cluster orchestration through Google Kubernetes Engine (GKE).

Two capabilities are particularly relevant: Cloud Pathways and dynamic slicing.

Cloud Pathways

#

Cloud Pathways brings the runtime architecture used by Google DeepMind for Gemini workloads to Google Cloud customers.

It provides a single-controller approach for orchestrating large TPU systems across multiple slices.

Instead of managing every accelerator group as an entirely separate deployment, developers can coordinate distributed workloads through a unified control plane.

The platform supports workloads such as:

• Foundation model pre-training.
• Reinforcement learning.
• Multi-host inference.
• Large-scale distributed AI execution.

Google also describes features for handling hardware disruptions, including Python sidecars, sub-minute slice hot-swapping, and elastic resharding.

These capabilities are relevant for long-running training jobs, where restarting an entire workload after a hardware failure can waste substantial compute time.

GKE dynamic slicing

#

Dynamic slicing addresses another operational limitation: rigid accelerator provisioning.

Traditional accelerator deployments often require physical resources to be provisioned into predefined node pools or cluster configurations.

Changing those allocations can involve time-consuming infrastructure operations.

GKE dynamic slicing uses optical circuit switching (OCS) to separate physical hardware provisioning from logical TPU slice formation.

This allows operators to dynamically assemble TPU resources into larger configurations or divide them into smaller isolated workloads.

Google describes two main operations:

• Super-slicing combines TPU cubes into larger logical topologies.
• Sub-slicing divides available resources into smaller isolated configurations.

The company reports that this approach can reduce job startup times by up to 5x and accelerate fault recovery by up to 4.5x.

For cloud operators, the main advantage is improved resource utilization. Hardware can be allocated according to workload requirements instead of remaining locked into rigid configurations.

🏗️ What Google’s Four-System TPU Strategy Means for AI Infrastructure

#

The introduction of TPU 8tx and TPU 8i Max highlights a broader shift in accelerator design.

For years, much of the AI hardware competition centered on peak compute throughput, memory capacity, and the number of accelerators that could be connected into a cluster.

Those metrics still matter, but production AI systems increasingly face more specialized requirements.

Enterprise inference needs stronger data protection. Autonomous agents need large persistent context windows. Long-running workloads need better state management. Cloud operators need more flexible resource allocation.

Google is addressing these problems by treating the accelerator as one component of a larger system.

The four configurations illustrate that approach:

+---------------------------------------------------------------------+
|                  Google 8th-Generation TPU Portfolio                |
+-------------------+-------------------------------------------------+
| TPU 8t            | Large-scale foundation model training           |
| TPU 8i            | Low-latency inference and real-time reasoning   |
| TPU 8tx           | Confidential computing and secure inference     |
| TPU 8i Max        | Massive context windows and high rack density   |
+----------------------+----------------------------------------------+

The strategy also reflects Google’s attempt to differentiate its infrastructure from conventional accelerator offerings.

Rather than competing exclusively on individual chip specifications, the company is combining custom silicon, AMD host processors, advanced networking, multi-chip packaging, confidential computing, and software orchestration.

For customers, the real question is whether these configurations deliver better performance, security, and operating costs for their specific workloads.

Google’s announcement does not yet provide every detail needed for a complete purchasing comparison. Pricing, regional availability, and several absolute system-level specifications remain unspecified.

What it does establish is the direction of the platform.

AI infrastructure is moving beyond the idea of a single accelerator optimized for everything. As training, inference, confidential processing, and autonomous reasoning develop different requirements, specialized systems are becoming a more practical way to scale.

Google’s eighth-generation TPU portfolio is a clear example of that transition: four configurations, a shared modular hardware foundation, and a software stack designed to make the entire system easier to deploy and operate.