Subject: Network performance forwarding tests on RT

From: Stephen Hemminger <[email-masked]>
Date: Wed, 5 Nov 2008 11:52:05 -0800

As an experiment, I rebuilt a version of Vyatta using 2.6.26-rt11 kernel.
This required some fixes to unionfs and aufs which I'll send to anyone who wants.

The performance of the RT PREEMPT kernel is worse than non-PREEMPT kernel.

Running RFC2544, frame loss test we the loss rate is worse on RT than non RT.
Ideally, there would be no loss, but on this platform, the best we have
seen is 70% loss at 64 bytes.

Size	2.6.26		2.6.26-rt11
64	80.5%		99%
128	67		99
256	43		92
512	0		54
1024	0		3
1280	0		0
1518	0		0

More importantly, with RT PREEMPT, the driver gets stuck and times out
under heavy load (see 99% loss above). It appears the change to network
scheduling related to NAPI doesn't work well under load.

I wouldn't recommend RT PREEMPT for heavy network loads.

Hardware: Celeron 430 @ 1.80GHz  Intel dual-port 82546GB NIC

Config:

CONFIG_NR_CPUS=8
CONFIG_SCHED_SMT=y
CONFIG_SCHED_MC=y
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT_DESKTOP is not set
CONFIG_PREEMPT_RT=y
CONFIG_PREEMPT=y
CONFIG_PREEMPT_SOFTIRQS=y
CONFIG_PREEMPT_HARDIRQS=y
CONFIG_PREEMPT_RCU=y
CONFIG_PREEMPT_RCU_BOOST=y