Meta Introduces MetaRoCE: Clean-Sheet RDMA for AI

Meta has announced the introduction of MetaRoCE, a clean-sheet RDMA transport protocol purpose-built to handle artificial intelligence workloads on commodity Ethernet infrastructure. This design intentionally breaks away from standard RoCE assumptions, which typically expect the underlying network to deliver every single frame in exact order.

With Meta scaling its infrastructure clusters to hundreds of thousands of GPUs across multiple data centers and regions, the network consistently acts as a critical bottleneck during training steps. Collective operations such as all-reduce and all-to-all synchronize thousands of accelerators, meaning that the slowest data transfer sets the pace for the entire job execution.

The core innovation of MetaRoCE inverts traditional paradigms by moving endpoint intelligence and shifting packet ordering, path selection, and error recovery directly into the NIC. Instead of relying on priority-based flow control, the network fabric decomposes into fine-grained logical paths equipped with real-time telemetry.

"MetaRoCE maintains ~86% throughput at 1% packet loss and continues delivering useful bandwidth even at 10% loss rates, converging gracefully rather than collapsing."

MarkTechPost

Performance evaluations conducted on a 64-node AMD GPU cluster utilizing RCCL collectives demonstrated that MetaRoCE delivered superior throughput and lower flow completion times compared directly against RoCEv2. Multiplane validation across topologies of up to 4,000 concurrent connections confirmed that performance scales linearly.