From: Vladimir Ivashchenko <hazard@francoudi•com>
To: Jarek Poplawski <jarkao2@gmail•com>
Cc: Eric Dumazet <dada1@cosmosbay•com>, netdev@vger•kernel.org
Subject: Re: HTB accuracy for high speed (and bonding)
Date: Sat, 23 May 2009 13:37:32 +0300 [thread overview]
Message-ID: <1243075052.27210.22.camel@hazard2.francoudi.com> (raw)
In-Reply-To: <20090521082805.GB2892@ami.dom.local>
> > > cls_flow, alas not enough documented. Here is some hint:
> > > http://markmail.org/message/h24627xkrxyqxn4k
> >
> > Can I balance only by destination IP using this approach?
> > Normal IP flow-based balancing is not good for me, I need
> > to ensure equality between destination hosts.
>
> Yes, you need to use flow "dst" key, I guess. (tc filter add flow
> help)
What is the number of DRR classes I need to create, a separate class for
each host? I have around 20000 hosts.
I figured out that WRR does what I want and its documented, so I'm using
a 2.6.27 kernel with WRR now.
I was still hitting a wall with bonding. I played with a lot of
combinations and could not find a way to make it scale to multiple
cores. Cores which handle incoming traffic would get hit to 0-20% idle.
So, I got rid of bonding completely and instead configured PBR on Cisco
+ Linux routing in such a way so that packet gets received and
transmitted using NICs connected to the same pair of cores with common
cache. 65-70% idle on all cores now, compared to 0-30% idle in worst
case scenarios before.
> - gso/tso or other non standard packets sizes - for exceeding the
> rate.
Just FYI, kernel 2.6.29.1, sub-classes with sfq divisor 1024, tso & gso
off, netdevice.h and tc_core.c patches applied:
class htb 1:2 root rate 775000Kbit ceil 775000Kbit burst 98328b cburst
98328b
Sent 64883444467 bytes 72261124 pkt (dropped 0, overlimits 0 requeues 0)
rate 821332Kbit 112572pps backlog 0b 0p requeues 0
lended: 21736738 borrowed: 0 giants: 0
In any case, exceeding the rate is not big of a problem for me.
Thanks a lot to everyone for their help.
--
Best Regards,
Vladimir Ivashchenko
Chief Technology Officer
PrimeTel PLC, Cyprus - www.prime-tel.com
Tel: +357 25 100100 Fax: +357 2210 2211
next prev parent reply other threads:[~2009-05-23 10:37 UTC|newest]
Thread overview: 104+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <298f5c050905150745p13dc226eia1ff50ffa8c4b300@mail.gmail.com>
2009-05-15 14:49 ` HTB accuracy for high speed Antonio Almeida
2009-05-15 18:12 ` Stephen Hemminger
2009-05-18 10:01 ` Antonio Almeida
2009-05-18 10:45 ` Jarek Poplawski
2009-05-18 12:27 ` Antonio Almeida
2009-05-18 12:32 ` Jarek Poplawski
2009-05-18 16:13 ` Stephen Hemminger
2009-05-18 18:03 ` Antonio Almeida
2009-05-18 22:02 ` Stephen Hemminger
2009-05-19 11:48 ` Antonio Almeida
2009-05-19 13:08 ` Antonio Almeida
2009-05-16 8:31 ` Jarek Poplawski
2009-05-18 10:39 ` Antonio Almeida
2009-05-18 11:14 ` Jarek Poplawski
2009-05-18 12:05 ` Antonio Almeida
2009-05-16 14:14 ` Jarek Poplawski
2009-05-18 14:36 ` Antonio Almeida
2009-05-18 23:14 ` Vladimir Ivashchenko
2009-05-18 23:27 ` Vladimir Ivashchenko
2009-05-19 11:03 ` Jarek Poplawski
2009-05-19 14:04 ` Vladimir Ivashchenko
2009-05-19 20:10 ` Jarek Poplawski
2009-05-20 22:07 ` Vladimir Ivashchenko
2009-05-20 22:46 ` Eric Dumazet
2009-05-21 7:20 ` Jarek Poplawski
2009-05-21 7:44 ` Vladimir Ivashchenko
2009-05-21 8:28 ` Jarek Poplawski
2009-05-21 9:07 ` Eric Dumazet
2009-05-21 9:22 ` Jarek Poplawski
2009-05-23 10:37 ` Vladimir Ivashchenko [this message]
2009-05-23 14:34 ` HTB accuracy for high speed (and bonding) Jarek Poplawski
2009-05-23 15:06 ` Vladimir Ivashchenko
2009-05-23 15:35 ` Jarek Poplawski
2009-05-23 15:53 ` Vladimir Ivashchenko
2009-05-23 16:02 ` Jarek Poplawski
2009-05-18 16:40 ` HTB accuracy for high speed Eric Dumazet
2009-05-18 17:23 ` Jarek Poplawski
2009-05-18 21:52 ` David Miller
2009-05-18 23:59 ` [PATCH] pkt_sched: gen_estimator: use 64 bits intermediate counters for bps Eric Dumazet
2009-05-19 2:27 ` David Miller
2009-05-19 7:02 ` Jarek Poplawski
2009-05-19 7:31 ` Eric Dumazet
2009-05-19 7:42 ` Jarek Poplawski
2009-05-19 7:57 ` Jarek Poplawski
2009-05-19 18:03 ` Eric Dumazet
2009-05-19 19:09 ` [PATCH] pkt_sched: gen_estimator: Fix signed integers right-shifts Jarek Poplawski
2009-05-26 5:47 ` David Miller
2009-05-19 8:18 ` [PATCH] pkt_sched: gen_estimator: use 64 bits intermediate counters for bps David Miller
2009-05-17 20:15 ` HTB accuracy for high speed Jarek Poplawski
2009-05-18 6:56 ` [PATCH iproute2] " Jarek Poplawski
2009-05-18 16:54 ` Antonio Almeida
2009-05-18 17:16 ` Antonio Almeida
2009-05-21 8:51 ` Jarek Poplawski
2009-05-22 17:42 ` Antonio Almeida
2009-05-23 7:32 ` Jarek Poplawski
2009-05-28 18:13 ` Antonio Almeida
2009-05-28 21:12 ` Jarek Poplawski
2009-05-29 17:02 ` Antonio Almeida
2009-05-29 17:28 ` Stephen Hemminger
2009-05-29 19:58 ` Jarek Poplawski
2009-05-29 19:46 ` Jarek Poplawski
2009-05-29 20:49 ` Stephen Hemminger
2009-05-29 20:59 ` Jarek Poplawski
2009-05-30 20:07 ` Jarek Poplawski
2009-06-02 10:12 ` Antonio Almeida
2009-06-02 11:45 ` Antonio Almeida
2009-06-02 12:36 ` Jarek Poplawski
2009-06-02 12:45 ` Patrick McHardy
2009-06-02 13:08 ` Jarek Poplawski
2009-06-02 13:20 ` Patrick McHardy
2009-06-02 21:37 ` Jarek Poplawski
2009-06-02 21:50 ` Jarek Poplawski
2009-06-03 7:06 ` Patrick McHardy
2009-06-03 7:40 ` Jarek Poplawski
2009-06-03 7:53 ` Patrick McHardy
2009-06-03 8:01 ` Jarek Poplawski
2009-06-03 8:29 ` Patrick McHardy
2009-06-03 8:45 ` Jarek Poplawski
2009-06-03 9:54 ` Jarek Poplawski
2009-06-03 10:01 ` Patrick McHardy
2009-06-03 10:05 ` Patrick McHardy
2009-06-03 10:06 ` Patrick McHardy
2009-06-03 10:27 ` Jarek Poplawski
2009-06-04 13:50 ` Antonio Almeida
[not found] ` <20090604193013.GA2755@ami.dom.local>
[not found] ` <4A282216.20203@trash.net>
[not found] ` <20090604194203.GB2755@ami.dom.local>
2009-06-09 5:25 ` Badalian Vyacheslav
2009-06-09 5:49 ` Jarek Poplawski
2009-06-04 4:53 ` David Miller
2009-06-04 7:50 ` Jarek Poplawski
2009-05-18 17:53 ` Jarek Poplawski
2009-05-18 18:23 ` Antonio Almeida
2009-05-18 18:32 ` Jarek Poplawski
2009-05-18 18:56 ` Antonio Almeida
2009-05-18 19:05 ` Jarek Poplawski
2009-05-19 10:55 ` Antonio Almeida
2009-05-19 11:04 ` Denys Fedoryschenko
2009-05-19 11:18 ` Jarek Poplawski
2009-05-19 11:21 ` Denys Fedoryschenko
2009-05-19 11:28 ` Jarek Poplawski
2009-05-19 14:31 ` Antonio Almeida
2009-05-19 11:09 ` Jarek Poplawski
2009-05-19 13:18 ` Jesper Dangaard Brouer
2009-05-19 19:35 ` Jarek Poplawski
2009-05-18 7:01 ` [PATCH iproute2 v2] " Jarek Poplawski
2009-05-17 20:29 ` Vladimir Ivashchenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1243075052.27210.22.camel@hazard2.francoudi.com \
--to=hazard@francoudi$(echo .)com \
--cc=dada1@cosmosbay$(echo .)com \
--cc=jarkao2@gmail$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox