REPS Explained: Lightweight Load Balancing for AI Clusters at Scale
Modern AI training clusters place unprecedented demands on data center networks. Large-scale distributed training relies on collective communication operations such as AllReduce, AllGather, and All-to-All, producing highly synchronized traffic bursts that can saturate network links within microseconds.
While today’s hyperscale networks provide enormous aggregate bandwidth, inefficient load balancing often prevents clusters from utilizing it effectively. A handful of congested links can stall thousands of GPUs, extending training times and significantly increasing infrastructure costs.
To address these challenges, researchers proposed REPS (Recycled Entropy Packet Spraying)—a lightweight load balancing mechanism specifically designed for next-generation AI fabrics. Instead of blindly distributing packets or statically assigning flows, REPS continuously learns which paths perform well, reuses them aggressively, and rapidly isolates failed links with minimal hardware state.
The result is a practical solution that combines high throughput, fast fault recovery, and extremely low deployment overhead.
🚀 Why Traditional Load Balancing Falls Short #
Conventional data center load balancing techniques were designed primarily for cloud services with relatively stable traffic patterns.
Large AI clusters behave very differently.
They generate:
- Massive synchronized traffic bursts
- Highly dynamic communication patterns
- Frequent all-to-all communication
- Extremely high sensitivity to network latency
- Significant performance penalties from even brief packet loss
Two challenges dominate these environments.
Bursty Collective Communication #
Distributed training repeatedly synchronizes model parameters across thousands of accelerators.
Operations such as:
- AllReduce
- AllGather
- All-to-All
produce bursts that can temporarily exceed available link capacity, even when average network utilization remains moderate.
Without intelligent traffic distribution, certain links become overloaded while others remain underutilized.
Failures Become Exceptionally Expensive #
In conventional cloud workloads, a temporary network failure may have only a modest impact.
In GPU clusters, however, every accelerator waits for the slowest participant.
A single failed link can delay synchronization across an entire training job, making the performance impact dramatically larger than in traditional applications.
Fast failure recovery is therefore essential.
⚠️ Limitations of Existing Approaches #
Several established techniques attempt to distribute traffic across multiple network paths.
Each has important trade-offs.
Equal-Cost Multi-Path (ECMP) #
ECMP remains the most widely deployed load balancing mechanism in Ethernet data centers.
Packets are assigned to paths by hashing flow identifiers such as:
- Source address
- Destination address
- Source port
- Destination port
- Protocol
While computationally efficient, ECMP has a major weakness.
Every packet belonging to the same flow follows exactly the same path.
If multiple large flows hash onto one link, congestion develops even when alternative paths remain idle.
Recovery from failures also depends on routing updates, which may require milliseconds—far too slow for AI workloads.
Oblivious Packet Spraying (OPS) #
OPS eliminates static flow assignment.
Instead, every packet is independently sprayed across available paths.
This improves path diversity but introduces another problem.
OPS has no awareness of:
- Link congestion
- Available bandwidth
- Failed links
Consequently, packets continue traversing congested or broken paths until higher-level mechanisms intervene.
Flow-Based Scheduling #
Mechanisms such as MPTCP and Flowlet Switching divide traffic into smaller units to improve load distribution.
However, they require substantially more per-flow state and respond relatively slowly to failures.
They also align poorly with emerging out-of-order transport protocols used in AI networking, including Ultra Ethernet.
🧠 Core Idea Behind REPS #
REPS stands for Recycled Entropy Packet Spraying.
Its central insight is remarkably simple:
Reuse network paths that have already demonstrated good performance instead of continuously selecting paths at random.
Rather than treating every packet independently, REPS gradually builds a lightweight cache of reliable routing choices.
Only proven, uncongested paths are reused.
Congested or failed paths are discarded.
This feedback loop allows the network to converge toward stable load balancing while remaining highly adaptive.
⚙️ Key Building Blocks #
REPS relies on three lightweight mechanisms already supported by modern Ethernet infrastructure.
Entropy Values (EV) #
Modern switches performing ECMP hashing use packet header fields to determine path selection.
REPS manipulates these fields by assigning different Entropy Values (EVs).
Changing an EV changes the resulting network path.
Instead of explicitly selecting routes, REPS indirectly controls routing through entropy manipulation.
A typical 16-bit field provides over 65,000 possible entropy values.
ECN-Based Congestion Feedback #
Rather than measuring latency directly, REPS relies on Explicit Congestion Notification (ECN).
When switch queues exceed configured thresholds:
- Switches mark packets with ECN.
- Receivers return ECN information within acknowledgements.
- Senders learn whether a path experienced congestion.
This avoids introducing new hardware features or proprietary telemetry.
Lightweight Circular Buffer #
Each connection maintains a tiny circular buffer containing only successful EVs.
The memory requirement is approximately:
- 25 bytes per connection
For comparison, traditional multipath protocols may require hundreds of bytes of state for each subflow.
This small footprint makes REPS practical for FPGA SmartNICs and hardware network accelerators.
🔄 Two-Phase Scheduling Strategy #
REPS operates in two distinct phases.
Exploration Phase #
When a connection begins, no routing history exists.
REPS initially behaves similarly to OPS.
Packets are assigned random entropy values to explore multiple available paths.
As acknowledgements return, the sender gradually learns which routes perform well.
Reuse Phase #
Once sufficient information has been collected:
- Successful EVs are cached.
- Congested EVs are discarded.
- Future packets preferentially reuse verified paths.
Only when the cache becomes empty does REPS resume exploration.
This simple strategy greatly reduces random congestion while maintaining adaptability.
⚡ Fast Failure Recovery #
One of REPS’s most innovative features is its Freezing Mode.
Traditional routing protocols require time to detect failures and propagate updated routing information.
During this interval, packets continue flowing toward failed links.
REPS reacts much faster.
Failure Detection #
If acknowledgements fail to arrive before timeout:
- The corresponding path is considered failed.
Entering Freezing Mode #
Rather than continuing random exploration, REPS immediately:
- Stops generating new entropy values.
- Uses only previously validated paths.
- Completely avoids suspected failures.
This dramatically reduces packet loss during transient failures.
Recovery #
After a predefined interval:
- Limited exploration resumes.
- The failed path is tested.
- If recovery succeeds, normal operation resumes.
- Otherwise, freezing continues.
Even incorrect failure detection has limited impact because previously verified paths remain available.
🏗️ Design Advantages #
REPS offers several practical advantages over existing packet spraying techniques.
Extremely Low Hardware Overhead #
Deployment requires:
- Existing ECMP support
- Standard ECN capability
- Minimal SmartNIC firmware updates
No switch redesign is necessary.
FPGA implementations reportedly consume less than 0.04% of available hardware resources.
Automatic Adaptation to Asymmetric Networks #
Real-world networks are rarely perfectly balanced.
Links may differ because of:
- Hardware generations
- Temporary congestion
- Background traffic
- Partial failures
Unlike random spraying, REPS naturally shifts traffic toward higher-performing paths using continuous feedback.
Excellent Failure Resilience #
Under transient link failures, REPS isolates problematic paths almost immediately.
Compared with OPS, evaluations report:
- Up to 100× higher performance
- More than 70× lower packet loss
Even multiple simultaneous failures are handled efficiently.
Broad Compatibility #
REPS integrates cleanly with modern networking technologies, including:
- DCTCP
- RDMA congestion control
- EQDS
- Ultra Ethernet
- RoCEv2
It also supports hybrid deployments alongside conventional ECMP traffic, enabling incremental rollout within existing production data centers.
📊 Why REPS Produces More Stable Networks #
The researchers explain REPS using a classic balls-and-bins model.
Imagine:
- Packets are balls.
- Network links are bins.
OPS #
Every packet is thrown randomly.
Eventually, some bins accumulate far more balls than others.
Queues become unpredictable and congestion spikes appear.
REPS #
Only successful throws are remembered.
Those successful choices are repeatedly reused.
As the system evolves, traffic naturally converges toward balanced utilization.
Theoretical analysis shows queue growth remains bounded, preventing explosive congestion even under sustained load.
📈 Evaluation Results #
Researchers evaluated REPS through both large-scale simulations and FPGA-based hardware deployments.
Workloads included:
- Incast traffic
- Permutation traffic
- Tornado traffic
- Production data center traces
- AI collective communication
Symmetric Networks #
Compared with ECMP:
- Up to 6× faster
Compared with OPS:
- Approximately 25% higher performance
Queue fluctuations were also significantly reduced.
Asymmetric Networks #
REPS demonstrated:
- Up to 5× improvement over ECMP
- Up to 2× improvement over OPS
Traffic naturally migrated away from degraded links.
Link Failures #
Transient failures produced particularly strong gains.
Compared with OPS:
- Up to 100× higher throughput
- Over 35% shorter AI training completion times
- Dramatically lower packet loss
Hardware Validation #
FPGA SmartNIC testing confirmed:
- Higher effective throughput
- Shorter flow completion times
- Near-elimination of packet loss during failures
These results indicate that REPS performs well not only in simulation but also in practical hardware environments.
🎯 Deployment Scenarios #
REPS is especially well suited for environments that combine massive parallel communication with strict latency requirements.
Representative use cases include:
- Large-scale distributed AI training clusters
- Ultra Ethernet deployments
- RoCEv2-based GPU fabrics
- FPGA SmartNIC platforms
- Hybrid cloud and AI infrastructure
- Existing Ethernet data centers seeking incremental upgrades
Because deployment requires only NIC firmware changes while preserving existing switch infrastructure, organizations can introduce REPS gradually without disruptive network redesigns.
🔍 Conclusion #
REPS rethinks packet spraying by replacing blind randomness with lightweight feedback-driven path reuse.
Instead of permanently binding flows to fixed routes or continuously selecting paths at random, REPS identifies high-quality paths, caches them with minimal state, and rapidly isolates failures through its freezing mechanism.
The design achieves three objectives that are increasingly important for hyperscale AI infrastructure:
- Efficient load balancing under highly bursty communication patterns
- Near-instant recovery from transient network failures
- Minimal hardware and deployment complexity
As AI clusters continue expanding toward hundreds of thousands of accelerators and Ultra Ethernet gains broader adoption, approaches such as REPS illustrate how relatively simple algorithmic changes can substantially improve network utilization, reliability, and overall training efficiency without requiring major changes to existing data center infrastructure.
REPS: Recycled Entropy Packet Spraying for Adaptive Load Balancing and Failure Mitigation