PSN-PATH Solves False Packet Loss Detection in Multipath RDMA

PSN-PATH Solves False Packet Loss Detection in Multipath RDMA

Remote Direct Memory Access (RDMA) is a key technology in high-performance data centers, enabling network adapters to transfer data directly between registered memory regions while reducing the CPU overhead associated with conventional networking. It is widely used in distributed AI training, cloud storage, and other latency-sensitive workloads.

However, running RDMA over lossy Ethernet networks becomes more complicated when multipath transmission is introduced. Multiple network paths can improve bandwidth utilization, but their different queueing delays can cause packets to arrive out of order. Traditional RDMA loss-detection mechanisms may mistake this reordering for genuine packet loss, triggering unnecessary retransmissions that waste bandwidth and increase latency.

The paper PSN-PATH: When Multipath RDMA Meets Lossy Networks, presented at ACM SIGCOMM 2026, proposes a solution. Developed by researchers from China University of Geosciences, Huazhong University of Science and Technology, Nanjing University, and Tianjin University, PSN-PATH uses packet sequence numbers and path information to distinguish genuine losses from normal cross-path reordering.

Its key idea is to check packet-sequence continuity within each path rather than across the entire multipath stream. Combined with dynamic path management, the design reduces unnecessary retransmissions while redirecting traffic away from degraded paths.

In the paper’s reported experiments, PSN-PATH achieved 88.2 Gbps of effective throughput on its FPGA prototype, reduced redundant retransmissions to 0.8%, and improved goodput by up to 5.11 times in large-scale simulations under high-load, lossy conditions.

🌐 1. Why Multipath RDMA Struggles on Lossy Networks

#

RDMA can reduce the overhead of data movement between servers, but its reliability mechanisms must still detect missing packets and recover from transmission errors.

RoCE (RDMA over Converged Ethernet) traditionally operates in networks designed to minimize packet loss, often using Priority-based Flow Control (PFC). PFC allows a congested downstream device to pause upstream traffic temporarily when its buffers approach capacity.

Although this can protect traffic from packet drops, it introduces other problems. A localized congestion event can affect unrelated traffic sharing the same links, resulting in head-of-line blocking and pause propagation. Under unfavorable conditions, PFC can also contribute to congestion spreading and deadlock scenarios.

An alternative is to allow the Ethernet network to drop packets and let the endpoints recover from losses. This can reduce dependence on a lossless network configuration, but it places greater demands on the RDMA transport’s loss-detection and retransmission logic.

Multipath transmission adds another complication.

1.1 Why multiple paths can hurt performance

#

Suppose a sender distributes packets across four network paths. One path has a short queue, while another experiences temporary congestion.

Even if the sender transmits packets in sequence, their arrival order may differ:

Packet 3 can arrive before packet 2 simply because it traveled through a faster path.

This is ordinary out-of-order delivery, not evidence that packet 2 has been lost.

Traditional RDMA reliability logic tracks Packet Sequence Numbers (PSNs) to detect missing data. When it observes a gap, however, it may not know whether the missing packet was dropped or is still queued along a slower path.

If it retransmits too aggressively, the network receives unnecessary duplicate traffic. Those retransmissions compete with original packets for bandwidth and can worsen congestion.

If it waits too long before deciding that a packet is missing, genuine losses take longer to recover from.

The resulting trade-off can undermine the throughput advantage that multipath transmission is supposed to provide.

⚠️ 2. The Core Problem: Confusing Reordering with Packet Loss

#

PSNs identify packet positions within an RDMA transmission sequence. In a conventional in-order delivery model, observing a later packet while an earlier packet remains missing can indicate a lost packet.

Multipath transmission weakens that inference because the network paths may have different delays.

2.1 Why selective retransmission is not enough

#

Some RDMA designs, including approaches such as IRN and other optimized retransmission schemes, improve recovery by retransmitting selected missing packets rather than repeatedly sending an entire trailing sequence of data.

Selective retransmission reduces the amount of duplicate data when packet loss has been identified correctly. It does not completely solve the problem of determining whether a packet is actually missing.

If ordinary path-induced reordering is interpreted as loss, the sender can still retransmit data that is already in transit. The duplicate traffic consumes resources without contributing new application data.

Conversely, waiting for additional packets, timeout events, or bitmap conditions before acting can delay recovery from real losses.

PSN-PATH addresses this earlier decision: before retransmitting a packet, determine whether the observed sequence gap violates the ordering expected on that specific path.

According to the paper, existing multipath RDMA schemes can experience a sharp increase in unnecessary retransmissions when paths have different delays and the network is lossy. PSN-PATH uses path-aware sequence checking to reduce that ambiguity.

🧩 3. PSN-PATH’s Core Principle: Check Ordering Within Each Path

#

PSN-PATH replaces the assumption that all packets must arrive in a single global sequence with a more targeted rule: packets assigned to the same stable path should preserve their expected order.

This works under the paper’s network model, in which packets following the same stable physical path are expected to experience FIFO ordering. If path assignment remains unchanged, a later packet arriving before an earlier packet assigned to that same path indicates a sequence disruption that can be treated as evidence of loss.

Packets belonging to different paths, by contrast, may arrive in either order without implying a failure.

3.1 Map PSNs to logical paths

#

Assume the sender uses \(K\) logical paths. PSN-PATH assigns packets to logical paths using a modulo operation:

For \(K=4\), the assignment becomes:

Each logical path receives every fourth packet in the sequence.

Now suppose packet 8 arrives while packet 7 is still missing. Because packets 7 and 8 belong to different logical paths, the receiver should not automatically interpret their arrival order as evidence of packet loss.

Instead, the receiver checks the earlier packet on packet 8’s own path: packet 4.

Under the stable-path FIFO assumption, packet 8 should not arrive ahead of packet 4 if both were transmitted through the same unchanged path and no reordering occurred within that path. If packet 4 is missing when packet 8 arrives, the gap provides evidence of an actual loss.

This transforms an ambiguous global sequence gap into a more informative path-local ordering check.

3.2 Logical paths versus physical network paths

#

A logical path index is not necessarily the same as a physical route through the data center.

PSN-PATH derives a UDP source port using the logical path index and an offset. The resulting packet flow interacts with the network’s existing Equal-Cost Multi-Path (ECMP) hashing mechanism to influence forwarding.

The switches do not need to understand PSN-PATH’s internal sequence-mapping algorithm. They continue forwarding packets using their existing mechanisms.

The mapping relies on network behavior that preserves the relevant ordering within a stable path. Changes in ECMP configuration, route selection, or the packet-to-path mapping can affect that assumption, which is why dynamic remapping requires additional handling.

🔀 4. Three Core Modules Inside PSN-PATH

#

PSN-PATH implements three main functional modules: the Path Mapper, Path Recognizer, and Path Manager.

Together, they answer three questions:

• Which logical path should carry a packet?
• Does a missing PSN represent a genuine loss or normal reordering?
• When should traffic move away from a degraded path?

The proposed implementation performs this logic at the RDMA NIC, without requiring modifications to Ethernet switches or application-level protocol changes.

4.1 Path Mapper: Associate packet sequences with paths

#

The Path Mapper calculates the logical path for each packet based on its PSN and the current path count.

For \(K\) paths, the basic mapping is:

The mapper then uses the configured path-to-port mapping, including the UDP source-port offset, to influence which physical route ECMP selects.

This separates the logical assignment of packets from the physical forwarding decisions made by the network.

The mapping is straightforward while the path set remains stable. The more difficult case occurs when a path becomes congested or starts losing packets and the system needs to move traffic elsewhere.

4.2 FPSN: Preserve correctness during path remapping

#

Dynamic path switching creates a subtle problem.

Suppose logical Path 3 initially carries packets 3, 7, 11, and 15. After packet 7 is sent, the system changes the physical mapping for subsequent packets.

Packet 7 may still be queued on the original route, while packet 11 travels along a newly selected, faster route. Packet 11 can therefore arrive before packet 7 even if neither packet was lost.

Without additional metadata, the receiver might incorrectly compare packets sent under different mappings and trigger an unnecessary retransmission.

PSN-PATH uses FPSN (First PSN of the new mapping) as a boundary marker. FPSN identifies where the new mapping begins, allowing the receiver to distinguish packets that belong to the old mapping from those governed by the new mapping.

This prevents the receiver from treating a gap across a mapping boundary as an ordinary same-path sequence violation.

The mechanism is important because path-aware ordering is valid only when the receiver knows which mapping governed each packet.

FPSN does not eliminate every need for timeout recovery. For example, if the final packet before a path switch is lost and no later packet on the old mapping arrives to expose the gap, the normal retransmission timeout can still be necessary. PSN-PATH accelerates recovery when a later same-path packet provides sufficient evidence, rather than eliminating timers for all possible loss scenarios.

4.3 Path Recognizer: Detect losses using the preceding packet

#

The Path Recognizer performs the receiver-side ordering check. It uses the RDMA receive bitmap to track which PSNs have arrived and obtains the path or mapping information needed to interpret each packet.

When a packet with PSN \(n\) arrives, the receiver checks the packet that precedes it on the same logical path:

For example, with four paths, the predecessor of packet 12 on its own path is packet 8, not packet 11.

The resulting decision is:

Case A: The same-path predecessor has arrived.

The sequence is continuous on that path. Even if the current packet is out of order relative to packets traveling along other paths, the receiver can accept it without treating the cross-path reordering as packet loss.

Case B: The same-path predecessor has not arrived.

Under the stable-path ordering assumption, the receiver has evidence of a sequence gap within the path. It can immediately generate a negative acknowledgment (NACK) for the missing predecessor, prompting the sender to retransmit it.

This is the central loss-detection mechanism in PSN-PATH.

The design reuses the existing receive bitmap instead of maintaining a completely independent set of packet-arrival states for every path. The path relationship is derived from the packet sequence and mapping metadata, reducing the storage overhead that per-path tracking could introduce.

Because loss detection no longer depends solely on waiting for the global sequence bitmap to fill or for a timeout to expire, eligible losses can be recovered sooner.

The sender can also redirect retransmissions through a healthy path rather than repeatedly using a degraded route.

🧭 5. Path Manager: Avoid Repeated Traffic on Degraded Paths

#

Correctly detecting packet loss is only part of the solution. If retransmissions and new packets continue using a path that repeatedly drops data, the system can suffer recurring recovery overhead.

The Path Manager evaluates each path’s observed delivery behavior and updates the set of available paths.

5.1 Score paths using ACKs and NACKs

#

PSN-PATH tracks acknowledgment success counts and negative acknowledgment loss counts for each path. The paper describes a path score of the form:

Here, \(\beta\) is a penalty coefficient that determines how strongly observed losses affect a path’s score.

A path with a favorable delivery history receives a higher score. Repeated losses reduce its score and can trigger exploratory probing or removal.

The implementation uses a dual-threshold mechanism called DT-Switch (Dual-Threshold Adaptive Switching) to distinguish early signs of degradation from more severe failures.

5.2 Exploration threshold: Probe alternatives

#

When a path’s score falls to the exploration threshold, the system treats it as potentially degraded but not necessarily unusable.

Rather than immediately removing it, the Path Manager probes alternative paths. If a candidate shows better behavior, traffic can migrate gradually.

This allows the system to test whether a better route is available without abruptly moving all traffic at once. A gradual migration can also reduce the large changes in arrival order that may occur during sudden path remapping.

5.3 Removal threshold: Evict a severely degraded path

#

When the score drops below the removal threshold, the path is considered sufficiently unhealthy to be removed from the active path set.

New packets are no longer assigned to it.

This two-threshold strategy avoids making every minor loss trigger immediate path replacement while still allowing the sender to stop using a path that repeatedly fails.

The paper describes path switching as an update to the mapper’s internal offset parameters and the UDP source-port mapping, rather than a change requiring complex coordination with the switches.

If a newly probed path also exhibits loss, the switch can be aborted and the original mapping restored, providing a mechanism for avoiding unstable repeated path changes.

🎛️ 6. PSN-PATH Complements Congestion Control

#

PSN-PATH is not a replacement for congestion control. The two systems address different questions.

Congestion-control algorithms such as DCQCN and HPCC regulate transmission behavior using network feedback or congestion information. PSN-PATH focuses on interpreting packet arrival order, accelerating loss recovery, and avoiding paths that repeatedly perform poorly.

The paper evaluates PSN-PATH in combination with these congestion-control approaches. The mechanisms can complement one another: congestion control reduces network pressure, while path-aware detection prevents ordinary cross-path reordering from unnecessarily triggering retransmissions.

Neither mechanism can solve every network problem independently. Accurate loss detection does not create additional physical bandwidth, and congestion control cannot by itself remove the ambiguity between genuine loss and out-of-order arrivals.

🧪 7. Experimental Results: Throughput, Retransmissions, and Hardware Cost

#

The authors implemented a hardware prototype using Vitis High-Level Synthesis (HLS) on Xilinx Alveo U50 FPGA NICs. They also conducted large-scale simulations using ns-3 to evaluate performance under different traffic and loss conditions.

The evaluation compared PSN-PATH with other multipath RDMA approaches, including MP-CX5, MP-IRN, ORNIC, and Falcon.

7.1 Reduced FPGA resource consumption

#

Because PSN-PATH reuses the global receive bitmap and checks path-local predecessors, it avoids maintaining independent full receive-state structures for every path.

Compared with the conventional multipath RDMA design used as the paper’s resource baseline, the reported reductions were:

BRAM usage fell particularly sharply because extensive per-path receive-state storage was not required.

The paper also reports that increasing the number of paths from 4 to 32 increased hardware resource consumption only marginally in the evaluated implementation.

These are prototype-level hardware results, and the exact comparison depends on the baseline implementation and synthesis configuration.

7.2 FPGA throughput and redundant retransmissions

#

In the reported FPGA experiments involving out-of-order traffic, the results were:

PSN-PATH achieved substantially higher effective throughput than MP-CX5 in this test while reducing the proportion of retransmissions that did not contribute new application data.

Falcon also reduced unnecessary retransmissions compared with MP-CX5, but its reported redundant retransmission ratio remained above that of PSN-PATH.

The key result is that faster packet delivery alone is not sufficient. Goodput improves when the transport delivers more useful data for the same amount of network activity.

7.3 Large-scale simulation results

#

Under high-load, lossy network conditions, the simulation experiments reported the following improvements compared with existing approaches:

The very large FCT improvement is a maximum measured under the evaluated simulation conditions, not an expected reduction for every production RDMA deployment.

As server counts increased from 128 to 1,024, PSN-PATH also showed less performance degradation than the compared schemes in the reported high-contention experiments.

These results support the paper’s central claim: when multipath reordering is mistaken for loss, unnecessary retransmissions can substantially reduce useful throughput, and improving the receiver’s interpretation of packet order can recover much of the lost performance.

7.4 Ablation studies: Both mechanisms matter

#

The authors also examined the effects of removing major PSN-PATH components.

These experiments indicate that fast loss recovery and dynamic path management address different parts of the performance problem.

Fast retransmission helps repair genuine losses sooner. Path switching reduces repeated exposure to links that have become unreliable or congested. Using both mechanisms provides the best results in the reported experiments.

⚖️ 8. Why Four Paths Can Be Better Than 32

#

More paths do not automatically produce better performance.

Increasing the number of paths can improve bandwidth utilization by distributing traffic across more network routes. However, each additional path can also introduce more variation in delivery latency and therefore more cross-path out-of-order arrivals.

PSN-PATH reduces the cost of handling that reordering, but it does not make path diversity free. Excessive paths can increase scheduling complexity, path-management work, and variability in packet arrival times.

The paper’s results indicate that increasing \(K\) from 1 to 4 improves bandwidth utilization. Increasing it further to 16 or 32, however, introduces enough additional path and reordering overhead to reduce performance under the evaluated conditions.

The authors therefore recommend \(K=4\) as the default configuration for the evaluated design.

This is not a universal rule for every data center. The ideal path count depends on network topology, available routes, workload behavior, congestion-control settings, and the cost of managing additional paths. Four paths represent the paper’s recommended balance between performance and resource overhead, rather than a guarantee that all multipath RDMA deployments should use the same number.

✅ 9. Key Takeaways for RDMA and Data Center Networking

#

PSN-PATH demonstrates how a small change in the interpretation of sequence numbers can produce substantial improvements in a complicated transport system.

The main engineering lessons are:

1. Do not equate every out-of-order packet with packet loss. Multipath transmission naturally creates variable arrival times, so reliable transports need a way to distinguish cross-path reordering from genuine loss.
2. Use path-local ordering to improve loss detection. Mapping PSNs to logical paths and checking each packet’s same-path predecessor provides stronger evidence of a real sequence gap.
3. Handle mapping changes explicitly. FPSN marks the start of a new mapping, preventing the receiver from applying an old path-ordering assumption to packets sent after a remapping event.
4. Separate loss recovery from path selection. Fast retransmission recovers missing data, while DT-Switch moves traffic away from paths that show repeated degradation.
5. Evaluate goodput and FCT, not just nominal bandwidth. The FPGA and simulation results show that unnecessary retransmissions can waste capacity and dominate application completion time.
6. Tune path count to the workload. More paths may improve parallelism, but additional reordering and management overhead can offset their benefits.

🏁 Conclusion

#

Multipath RDMA can unlock network parallelism in high-performance data centers, but variable path delays create a fundamental challenge for reliable transport protocols. When the receiver cannot tell whether a missing PSN reflects packet loss or a packet still traveling along a slower route, unnecessary retransmissions can waste bandwidth and inflate Flow Completion Time.

PSN-PATH addresses this problem by relating packet sequence numbers to logical paths. The Path Mapper assigns packets, the Path Recognizer checks ordering within each path, FPSN preserves the distinction between old and new mappings, and the Path Manager removes degraded paths when appropriate.

Implemented on an FPGA NIC and evaluated through large-scale simulation, the design reported significant reductions in redundant retransmissions and FCT, alongside improvements in effective throughput.

Its central contribution is not eliminating out-of-order delivery. It is making the transport aware of why packets arrive out of order, allowing the receiver to distinguish normal multipath behavior from genuine loss.

For lossy RDMA networks, that distinction can be the difference between realizing the benefits of multipath transmission and wasting much of the available bandwidth on unnecessary recovery traffic.