From: Paolo Abeni <pabeni@redhat•com>
To: Eric Dumazet <eric.dumazet@gmail•com>, netdev@vger•kernel.org
Cc: "David S. Miller" <davem@davemloft•net>
Subject: Re: [PATCH net-next 2/2] udp: implement and use per cpu rx skbs cache
Date: Wed, 18 Apr 2018 19:15:12 +0200 [thread overview]
Message-ID: <1524071712.2599.60.camel@redhat.com> (raw)
In-Reply-To: <890db004-4dfe-7f77-61ee-1ac0d7d2a24c@gmail.com>
Hi,
On Wed, 2018-04-18 at 09:56 -0700, Eric Dumazet wrote:
>
> On 04/18/2018 03:22 AM, Paolo Abeni wrote:
> > This changeset extends the idea behind commit c8c8b127091b ("udp:
> > under rx pressure, try to condense skbs"), trading more BH cpu
> > time and memory bandwidth to decrease the load on the user space
> > receiver.
> >
> > At boot time we allocate a limited amount of skbs with small
> > data buffer, storing them in per cpu arrays. Such skbs are never
> > freed.
> >
> > At run time, under rx pressure, the BH tries to copy the current
> > skb contents into the cache - if the current cache skb is available,
> > and the ingress skb is small enough and without any head states.
> >
> > When using the cache skb, the ingress skb is dropped by the BH
> > - while still hot on cache - and the cache skb is inserted into
> > the rx queue, after increasing its usage count. Also, the cache
> > array index is moved to the next entry.
> >
> > The receive side is unmodified: in udp_rcvmsg() the usage skb
> > usage count is decreased and the skb is _not_ freed - since the
> > cache keeps usage > 0. Since skb->usage is hot in the cache of the
> > receiver at consume time - the receiver has just read skb->data,
> > which lies in the same cacheline - the whole skb_consume_udp() becomes
> > really cheap.
> >
> > UDP receive performances under flood improve as follow:
> >
> > NR RX queues Kpps Kpps Delta (%)
> > Before After
> >
> > 1 2252 2305 2
> > 2 2151 2569 19
> > 4 2033 2396 17
> > 8 1969 2329 18
> >
> > Overall performances of knotd DNS server under real traffic flood
> > improves as follow:
> >
> > Kpps Kpps Delta (%)
> > Before After
> >
> > 3777 3981 5
>
>
> It might be time for knotd DNS server to finally use SO_REUSEPORT instead of
> adding this bloat to the kernel ?
>
> Sorry, 5% improvement while you easily can get 300% improvement with no kernel change
> is not appealing to me :/
Thank you for the feedback.
Sorry for not being clear about it, but knotd is using SO_REUSEPORT and
the above tests are leveraging it.
That 5% is on top of that 300%.
Cheers,
Paolo
next prev parent reply other threads:[~2018-04-18 17:15 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-04-18 10:22 [PATCH net-next 0/2] UDP: introduce RX skb cache Paolo Abeni
2018-04-18 10:22 ` [PATCH net-next 1/2] udp: if the rx queue is full, free the skb in __udp_enqueue_schedule_skb() Paolo Abeni
2018-04-18 10:22 ` [PATCH net-next 2/2] udp: implement and use per cpu rx skbs cache Paolo Abeni
2018-04-18 16:56 ` Eric Dumazet
2018-04-18 17:15 ` Paolo Abeni [this message]
2018-04-18 19:21 ` Eric Dumazet
2018-04-19 7:40 ` Paolo Abeni
2018-04-19 13:47 ` Eric Dumazet
2018-04-20 13:48 ` Jesper Dangaard Brouer
2018-04-21 15:54 ` Willem de Bruijn
2018-04-21 16:45 ` Eric Dumazet
2018-04-22 11:22 ` Paolo Abeni
2018-04-23 8:52 ` Jesper Dangaard Brouer
2018-04-23 8:13 ` Tariq Toukan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1524071712.2599.60.camel@redhat.com \
--to=pabeni@redhat$(echo .)com \
--cc=davem@davemloft$(echo .)net \
--cc=eric.dumazet@gmail$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox