Subject: Re: under-performing bonded interfaces

From: Ben Greear <[email-masked]>
Date: Wed, 16 Nov 2011 16:01:15 -0800

On 11/16/2011 03:44 PM, Simon Chen wrote:
> Hello,
>
> I am bonding two 10G interfaces (ixgbe driver) under Debian 6.0.2. The
> bonded interface for some reason can only achieve 12Gbps aggregated
> throughput. If a single NIC is used, I can get close to 10Gbps.
>
> I've tried different bonding modes (balance-xor, 802.3ad, balance-alb,
> balance-tlb), and different xmit hashing policy (layer2, layer2+3,
> layer3+4). I've increased all types of kernel parameters for TCP. MTU
> on the physical and bonded interface is set to 8000 and 9000. The MTU
> on the switch is 9200+.
>
> Instead of nperf (a single server), I also tried my own TCP sender and
> receivers.
>
> All those done, still only 12Gbps... How can I really achieve close to 20Gbps?
>
> (I also tried cutting loose the switch in between, and also getting
> 12G, so not an issue with the switch.)

How much can you get if you run each of the NIC ports independently
w/out bonding?  Plz send the 'dmesg' messages about ixgbe (ie, how
many lanes, how many GT/s).  What is your processor?

Thanks,
Ben

-- 
Ben Greear <[email-masked]>
Candela Technologies Inc  http://www.candelatech.com