Intel engineers talk about ways to improve network throughput on multi-processor systems running Linux:
Multi-gigabit/s of networking traffic is pushing the limits of current symmetric multi-processor (SMP) systems. Network protocol stacks, in particular TCP/IP software implementations, are known for their inability to scale well in general-purpose monolithic operating systems for SMP. This article discusses how to improve network performance through the use of processes/threads and interrupts affinity. In specific, our experiments are based on the Redhat Linux-2.4.20 stack, where we explore the usage of the set_schedaffinity() series of programming interfaces, officially folded into the mainstream Linux-2.6 kernel]). This paper also highlights how interrupt affinity alone provides a throughput gain of up to 25%, and a combined process and interrupt affinity can achieve gains of 30%, for bulk data transfers. Finally, cache behavior was measured and we quantified the reduction in the number of level 2 (L2) and level 3 (L3) cache misses when affinity is used.
