From: Shaohua Li <shli@fb•com>
To: Eric Dumazet <eric.dumazet@gmail•com>
Cc: <netdev@vger•kernel.org>, <davem@davemloft•net>,
<Kernel-team@fb•com>, <clm@fb•com>, <linux-mm@kvack•org>,
<dbavatar@gmail•com>, Eric Dumazet <edumazet@google•com>
Subject: Re: [RFC v2] net: use atomic allocation for order-3 page allocation
Date: Thu, 11 Jun 2015 16:32:35 -0700 [thread overview]
Message-ID: <20150611233235.GA667489@devbig257.prn2.facebook.com> (raw)
In-Reply-To: <1434063184.27504.60.camel@edumazet-glaptop2.roam.corp.google.com>
On Thu, Jun 11, 2015 at 03:53:04PM -0700, Eric Dumazet wrote:
> On Thu, 2015-06-11 at 15:27 -0700, Shaohua Li wrote:
> > We saw excessive direct memory compaction triggered by skb_page_frag_refill.
> > This causes performance issues and add latency. Commit 5640f7685831e0
> > introduces the order-3 allocation. According to the changelog, the order-3
> > allocation isn't a must-have but to improve performance. But direct memory
> > compaction has high overhead. The benefit of order-3 allocation can't
> > compensate the overhead of direct memory compaction.
> >
> > This patch makes the order-3 page allocation atomic. If there is no memory
> > pressure and memory isn't fragmented, the alloction will still success, so we
> > don't sacrifice the order-3 benefit here. If the atomic allocation fails,
> > direct memory compaction will not be triggered, skb_page_frag_refill will
> > fallback to order-0 immediately, hence the direct memory compaction overhead is
> > avoided. In the allocation failure case, kswapd is waken up and doing
> > compaction, so chances are allocation could success next time.
> >
> > The mellanox driver does similar thing, if this is accepted, we must fix
> > the driver too.
> >
> > V2: make the changelog clearer
> >
> > Cc: Eric Dumazet <edumazet@google•com>
> > Signed-off-by: Shaohua Li <shli@fb•com>
> > ---
> > net/core/sock.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/net/core/sock.c b/net/core/sock.c
> > index 292f422..e9855a4 100644
> > --- a/net/core/sock.c
> > +++ b/net/core/sock.c
> > @@ -1883,7 +1883,7 @@ bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t gfp)
> >
> > pfrag->offset = 0;
> > if (SKB_FRAG_PAGE_ORDER) {
> > - pfrag->page = alloc_pages(gfp | __GFP_COMP |
> > + pfrag->page = alloc_pages((gfp & ~__GFP_WAIT) | __GFP_COMP |
> > __GFP_NOWARN | __GFP_NORETRY,
> > SKB_FRAG_PAGE_ORDER);
> > if (likely(pfrag->page)) {
>
>
> OK, now what about alloc_skb_with_frags() ?
>
> This should have same problem right ?
Ok, looks similar, added. Didn't trigger this one though.
>From 940dde18f7f655377a4c30d5de54c9eff15ab5a5 Mon Sep 17 00:00:00 2001
Message-Id: <940dde18f7f655377a4c30d5de54c9eff15ab5a5.1434065353.git.shli@fb•com>
From: Shaohua Li <shli@fb•com>
Date: Thu, 11 Jun 2015 16:16:21 -0700
Subject: [RFC] net: use atomic allocation for order-3 page allocation
We saw excessive direct memory compaction triggered by skb_page_frag_refill.
This causes performance issues and add latency. Commit 5640f7685831e0
introduces the order-3 allocation. According to the changelog, the order-3
allocation isn't a must-have but to improve performance. But direct memory
compaction has high overhead. The benefit of order-3 allocation can't
compensate the overhead of direct memory compaction.
This patch makes the order-3 page allocation atomic. If there is no memory
pressure and memory isn't fragmented, the alloction will still success, so we
don't sacrifice the order-3 benefit here. If the atomic allocation fails,
direct memory compaction will not be triggered, skb_page_frag_refill will
fallback to order-0 immediately, hence the direct memory compaction overhead is
avoided. In the allocation failure case, kswapd is waken up and doing
compaction, so chances are allocation could success next time.
alloc_skb_with_frags is the same.
The mellanox driver does similar thing, if this is accepted, we must fix
the driver too.
V3: fix the same issue in alloc_skb_with_frags as pointed out by Eric
V2: make the changelog clearer
Cc: Eric Dumazet <edumazet@google•com>
Cc: Chris Mason <clm@fb•com>
Cc: Debabrata Banerjee <dbavatar@gmail•com>
Signed-off-by: Shaohua Li <shli@fb•com>
---
net/core/skbuff.c | 4 +++-
net/core/sock.c | 2 +-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 3cfff2a..9856c7a 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -4398,7 +4398,9 @@ struct sk_buff *alloc_skb_with_frags(unsigned long header_len,
while (order) {
if (npages >= 1 << order) {
- page = alloc_pages(gfp_mask |
+ gfp_t gfp = order > 0 ?
+ gfp_mask & ~__GFP_WAIT : gfp_mask;
+ page = alloc_pages(gfp |
__GFP_COMP |
__GFP_NOWARN |
__GFP_NORETRY,
diff --git a/net/core/sock.c b/net/core/sock.c
index 292f422..e9855a4 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -1883,7 +1883,7 @@ bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t gfp)
pfrag->offset = 0;
if (SKB_FRAG_PAGE_ORDER) {
- pfrag->page = alloc_pages(gfp | __GFP_COMP |
+ pfrag->page = alloc_pages((gfp & ~__GFP_WAIT) | __GFP_COMP |
__GFP_NOWARN | __GFP_NORETRY,
SKB_FRAG_PAGE_ORDER);
if (likely(pfrag->page)) {
--
1.8.1
next prev parent reply other threads:[~2015-06-11 23:32 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-11 20:24 [RFC] net: use atomic allocation for order-3 page allocation Shaohua Li
2015-06-11 20:48 ` Eric Dumazet
2015-06-11 21:16 ` Chris Mason
2015-06-11 21:22 ` Eric Dumazet
2015-06-11 21:45 ` Shaohua Li
2015-06-11 21:56 ` Eric Dumazet
2015-06-11 22:01 ` Shaohua Li
2015-06-11 22:18 ` Chris Mason
2015-06-11 22:55 ` Eric Dumazet
2015-06-11 21:35 ` Debabrata Banerjee
2015-06-11 22:18 ` David Miller
2015-06-12 9:25 ` Vlastimil Babka
2015-06-11 21:25 ` Debabrata Banerjee
2015-06-11 21:28 ` Debabrata Banerjee
2015-06-12 9:34 ` Vlastimil Babka
2015-06-11 22:53 ` [RFC v2] " Eric Dumazet
2015-06-11 23:32 ` Shaohua Li [this message]
2015-06-11 23:38 ` Eric Dumazet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150611233235.GA667489@devbig257.prn2.facebook.com \
--to=shli@fb$(echo .)com \
--cc=Kernel-team@fb$(echo .)com \
--cc=clm@fb$(echo .)com \
--cc=davem@davemloft$(echo .)net \
--cc=dbavatar@gmail$(echo .)com \
--cc=edumazet@google$(echo .)com \
--cc=eric.dumazet@gmail$(echo .)com \
--cc=linux-mm@kvack$(echo .)org \
--cc=netdev@vger$(echo .)kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox