Subject: Re: gigabit trouble

From: Bart Alewijnse <[email-masked]>
Date: Sun, 1 Aug 2004 21:03:32 +0200

On Sat, 31 Jul 2004 23:18:36 +0200, Francois Romieu
<[email-masked]> wrote:
> > Right, both computers now run 2.6.8-rc2-mm1. It's better. Roughly
> > speaking, the top *benchmark* speeds, rouding slightly up, are:
> > udp: 33MB/s (using ~26Kints) one way, 12MB/s (9Kints/s) the other,
> > tcp: 22MB/s (16Kints)one way, 12MB/s (9Kints/s) the other.
> >
> > (The 9Kints ca be 12 when the packet size is 1K or 2K)
> 
> Can you give a try to a napi version of the module (it is a compile-time
> option) ?
> You may want higher {r/w}mem_max as well as renicing ksoftirqd with
> a strongly < 0 value on the celeron client.
That's with NAPI on both. I notice r/wmem has an effect, but it's not much.
Although on my new computer it seems moot as there's something weird
going on with interrupts with my botherboard.

> > Oddly enough, the slow direction is sending from my new computer
> > (XP1700, KT333, 1GB ram) to my old (both running linux and the
> > mentioned kernel). I'm fairly sure this is due to the fact that my
> 
> The r8169 driver has been reported to be faster on Rx than on Tx so your
> observation makes sense.

How does that make sense? When one side receives, the other sends, hrm?
They're two identical cards, it should be entirely symmetrical assuming
equal hardware - not faster on the years older hardware.

> [...]
> > I guess I'll have to settle for this. I'm just annoyed that the 'giga'
> > is basically a joke,  especially on my setup.
> 
> It should be possible to make things slightly better for the celeron box
> as a client. I'll cook up a patch.

The celeron box is the server, that's the entire point. Anyhow, it's
much faster in transmission than my new computer right now due to said
mobo problem, so I *want* it as the server.

> Would you be kind enough to do some ftp transfer test where the celeron
> box retrieves data and send the 'vmstat 1' output of the client during
> the test ?

Sure, but I can tell you right now with reasonable certaintly that my
new computer
won't top 9000interrupts/sec, i.e. 9000 packets per second, and therefore do
the 12MB/s at most, and probably less; interruptwise, my new computer
is the bottleneck, and I'm guessing UDP is faster because TCP is
limited by the amount of packets, or in the other direction ACKs, it
can send per second.

Transfers to the celeron are a relatively pointless measure because
of my new computer being horribly interrupt limited at the moment. 
That may have started when I installed the gbit card, but mostly because it
disturbs the mobo's precious and unguessable hardware balance. I'll try figuring
out if I can solve it, but basically it just involves swapping around
cards until it
works better, so that'ld take a while.

Attached are the vmstats for an ftp and nfs transfer, as measured from
my old computer (the nfs and ftp server). Both were going at a pretty
low speed, sub-9000 packets; this may have to do with drive
fragmentation, my hard drives are rather full at the moment.

Also, the logs on the client (my new computer) while sending data to
my old one. These were made at a different time, and I believe the
transfer rate was better (6MB for ftp, 9MB for nfs) than for the
-to-old- logs, which were worse, and varied more.

--Bart