Subject: Re: bizarre network timing problem

From: Rick Jones <[email-masked]>
Date: Tue, 06 Nov 2007 13:12:13 -0800

Felix von Leitner wrote:
> Thus spake Rick Jones ([email-masked]):
> 
>>Past performance is no guarantee of current correctness :)  And over an 
>>Ethernet, there will be a very different set of both timings and TCP 
>>segment sizes compared to loopback.
> 
>>My guess is that you will find setting the lo mtu to 1500 a very 
>>interesting experiment.
> 
> Setting the MTU on lo to 1500 eliminates the problem and gives me double
> digit MB/sec throughput.

I'm not in a position at the moment to test it as my IPoIB systems are offline, 
and not sure you are either, but I will note that with IPoIB bits circa OFED1.2 
the default MTU for IPoIB goes up to 65520 bytes.  If indeed the problem you 
were seeing was related to sub-mss sends and window probing and such, it might 
appear on IPoIB in addition to loopback.

rick jones