Altruistic Batching: Slowing Down the Network Stack to Go Faster

Networking stacks rely heavily on adaptive batching to amortize packet-processing costs: each stage processes all queued work upon invocation, up to a bound. This strategy works particularly well when the buildup of each queue reflects pressure in the overall client-to-server networking pipeline. Unfortunately, this assumption does not always hold in modern Linux network stacks: queues can be scheduled with different priorities, making local queue length a misleading signal of global contention.

We address this problem with altruistic batching, which makes batching decisions using congestion from the whole pipeline spanning both clients and servers, not only a stage’s local queue. When one stage is congested, other stages can altruistically defer work to form larger batches, amortizing overheads and freeing shared CPU cycles.

We implement a proof-of-concept altruistic batching in Linux as a userspace batching layer deployed at both clients and servers. The layer combines local receive-side congestion exposed via eBPF, with remote congestion signals piggybacked on messages from the peer. By creating larger batches at the right points, altruistic batching unlocks larger coalescing opportunities for existing kernel mechanisms such as GRO and GSO, improving throughput by 4.5× on an echo service and 2.1× on Memcached under a 750µs p99 SLO.