From: "Paweł Staszewski" <pstaszewski@itcare•pl>
To: Eric Dumazet <eric.dumazet@gmail•com>
Cc: Changli Gao <xiaosuo@gmail•com>,
Benny Amorsen <benny+usenet@amorsen•dk>,
zhigang gong <zhigang.gong@gmail•com>,
netdev@vger•kernel.org
Subject: Re: Strange packet drops with heavy firewalling
Date: Tue, 13 Apr 2010 15:39:06 +0200 [thread overview]
Message-ID: <4BC473FA.6020903@itcare.pl> (raw)
In-Reply-To: <1271163184.16881.307.camel@edumazet-laptop>
W dniu 2010-04-13 14:53, Eric Dumazet pisze:
> Le mardi 13 avril 2010 à 14:33 +0200, Paweł Staszewski a écrit :
>
>> W dniu 2010-04-13 01:18, Changli Gao pisze:
>>
>>> On Tue, Apr 13, 2010 at 1:06 AM, Benny Amorsen<benny+usenet@amorsen•dk> wrote:
>>>
>>>
>>>> 99: 24 1306226 3 2 PCI-MSI-edge eth1-tx-0
>>>> 100: 15735 1648774 3 7 PCI-MSI-edge eth1-tx-1
>>>> 101: 8 11 9 1083022 PCI-MSI-edge eth1-tx-2
>>>> 102: 0 0 0 0 PCI-MSI-edge eth1-tx-3
>>>> 103: 18 15 6131 1095383 PCI-MSI-edge eth1-rx-0
>>>> 104: 217 32 46544 1335325 PCI-MSI-edge eth1-rx-1
>>>> 105: 154 1305595 218 16 PCI-MSI-edge eth1-rx-2
>>>> 106: 17 16 8229 1467509 PCI-MSI-edge eth1-rx-3
>>>> 107: 0 0 1 0 PCI-MSI-edge eth1
>>>> 108: 2 14 15 1003053 PCI-MSI-edge eth0-tx-0
>>>> 109: 8226 1668924 478 487 PCI-MSI-edge eth0-tx-1
>>>> 110: 3 1188874 17 12 PCI-MSI-edge eth0-tx-2
>>>> 111: 0 0 0 0 PCI-MSI-edge eth0-tx-3
>>>> 112: 203 185 5324 1015263 PCI-MSI-edge eth0-rx-0
>>>> 113: 4141 1600793 153 159 PCI-MSI-edge eth0-rx-1
>>>> 114: 16242 1210108 436 3124 PCI-MSI-edge eth0-rx-2
>>>> 115: 267 4173 19471 1321252 PCI-MSI-edge eth0-rx-3
>>>> 116: 0 1 0 0 PCI-MSI-edge eth0
>>>>
>>>>
>>>> irqbalanced seems to have picked CPU1 and CPU3 for all the interrupts,
>>>> which to my mind should cause the same problem as before (where CPU1 and
>>>> CPU3 was handling all packets). Yet the box clearly works much better
>>>> than before.
>>>>
>>>>
>>> irqbalanced? I don't think it can work properly. Try RPS in netdev and
>>> linux-next tree, and if cpu load isn't even, try this patch:
>>> http://patchwork.ozlabs.org/patch/49915/ .
>>>
>>>
>>>
>>>
>> Yes without irqbalance - and with irq affinity set by hand router will
>> work much better.
>>
>> But I don't think that RPS will help him - I make some tests with RPS
>> and AFFINITY - results in attached file.
>> Test router make traffic management (hfsc) for almost 9k users
>>
> Thanks for sharing Pawel.
>
> But obviously you are mixing apples and oranges.
>
> Are you aware that HFSC and other trafic shapers do serialize access to
> data structures ? If many cpus try to access these structures in //, you
> have a lot of cache line misses. HFSC is a real memory hog :(
>
>
Thanks Eric for explanation why RPS is useless for traffic management
routers.
> Benny do have firewalling (highly parallelized these days, iptables was
> well improved in this area), but no traffic control.
>
>
Hmm so maybe better choice for traffic management is use iptables for
"filter classification" instead of "u32 filters"- something like
iptables CLASSIFY target
> Anyway, Benny has now multiqueue devices, and therefore RPS will not
> help him. I suggested RPS before his move to multiqueue, and multiqueue
> is the most sensible way to improve things, when no central lock is
> used. Every cpu can really work in //.
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger•kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
>
prev parent reply other threads:[~2010-04-13 13:39 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-04-09 9:56 Strange packet drops with heavy firewalling Benny Amorsen
2010-04-09 11:47 ` Eric Dumazet
2010-04-09 12:33 ` Benny Amorsen
2010-04-09 13:29 ` Eric Dumazet
2010-04-12 6:20 ` Benny Amorsen
[not found] ` <q2v40c9f5b21004120116p766df82dj88c6af4e4cad55f@mail.gmail.com>
2010-04-12 14:44 ` Benny Lyne Amorsen
[not found] ` <p2x40c9f5b21004120833jd7a749cak6ea69cebd28f8352@mail.gmail.com>
2010-04-12 17:06 ` Benny Amorsen
2010-04-12 23:18 ` Changli Gao
2010-04-13 5:56 ` Eric Dumazet
2010-04-13 7:56 ` Benny Amorsen
2010-04-15 13:23 ` Benny Amorsen
2010-04-15 13:42 ` Eric Dumazet
2010-04-13 12:33 ` Paweł Staszewski
2010-04-13 12:53 ` Eric Dumazet
2010-04-13 13:39 ` Paweł Staszewski [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4BC473FA.6020903@itcare.pl \
--to=pstaszewski@itcare$(echo .)pl \
--cc=benny+usenet@amorsen$(echo .)dk \
--cc=eric.dumazet@gmail$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
--cc=xiaosuo@gmail$(echo .)com \
--cc=zhigang.gong@gmail$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox