From: Jason Wang <jasowang@redhat•com>
To: "Michael S. Tsirkin" <mst@redhat•com>
Cc: Eric Dumazet <eric.dumazet@gmail•com>,
davem@davemloft•net, hkchu@google•com,
netdev <netdev@vger•kernel.org>,
dwmw2@infradead•org
Subject: Re: [PATCH net-next] net: allow large number of tx queues
Date: Fri, 21 Jun 2013 14:41:48 +0800 [thread overview]
Message-ID: <51C3F5AC.2090008@redhat.com> (raw)
In-Reply-To: <20130620083530.GA23909@redhat.com>
On 06/20/2013 04:35 PM, Michael S. Tsirkin wrote:
> On Thu, Jun 20, 2013 at 01:15:51AM -0700, Eric Dumazet wrote:
>> From: Eric Dumazet <edumazet@google•com>
>>
>> On Wed, 2013-06-19 at 21:07 +0300, Michael S. Tsirkin wrote:
>>
>>> As we have explicit code to recover, maybe set __GFP_NOWARN?
>>> If we are out of memory, vzalloc will warn too, we don't
>>> need two warnings.
>> Yes, that's absolutely the right thing to do.
>>
>> Its yet not clear this patch is really needed, but here it is.
>>
>> Thanks
>>
>> [PATCH net-next] net: allow large number of tx queues
>>
>> netif_alloc_netdev_queues() uses kcalloc() to allocate memory
>> for the "struct netdev_queue *_tx" array.
>>
>> For large number of tx queues, kcalloc() might fail, so this
>> patch does a fallback to vzalloc().
>>
>> As vmalloc() adds overhead on a critical network path, add __GFP_REPEAT
>> to kzalloc() flags to do this fallback only when really needed.
>>
>> Signed-off-by: Eric Dumazet <edumazet@google•com>
> FWIW
>
> Acked-by: Michael S. Tsirkin <mst@redhat•com>
>
> This makes it possible to go up to 1K queues which should
> be enough, so we can revert the first chunk in
> edfb6a148ce62e5e19354a1dcd9a34e00815c2a1.
1K queues (about 80 pages) looks a little bit aggressive which means we
may always fall back to vmalloc()?
>
>
>> ---
>> net/core/dev.c | 26 +++++++++++++++++++-------
>> 1 file changed, 19 insertions(+), 7 deletions(-)
>>
>> diff --git a/net/core/dev.c b/net/core/dev.c
>> index fa007db..722f633 100644
>> --- a/net/core/dev.c
>> +++ b/net/core/dev.c
>> @@ -130,6 +130,7 @@
>> #include <linux/cpu_rmap.h>
>> #include <linux/static_key.h>
>> #include <linux/hashtable.h>
>> +#include <linux/vmalloc.h>
>>
>> #include "net-sysfs.h"
>>
>> @@ -5253,17 +5254,28 @@ static void netdev_init_one_queue(struct net_device *dev,
>> #endif
>> }
>>
>> +static void netif_free_tx_queues(struct net_device *dev)
>> +{
>> + if (is_vmalloc_addr(dev->_tx))
>> + vfree(dev->_tx);
>> + else
>> + kfree(dev->_tx);
>> +}
>> +
>> static int netif_alloc_netdev_queues(struct net_device *dev)
>> {
>> unsigned int count = dev->num_tx_queues;
>> struct netdev_queue *tx;
>> + size_t sz = count * sizeof(*tx);
>>
>> - BUG_ON(count < 1);
>> -
>> - tx = kcalloc(count, sizeof(struct netdev_queue), GFP_KERNEL);
>> - if (!tx)
>> - return -ENOMEM;
>> + BUG_ON(count < 1 || count > 0xffff);
>>
>> + tx = kzalloc(sz, GFP_KERNEL | __GFP_NOWARN | __GFP_REPEAT);
>> + if (!tx) {
>> + tx = vzalloc(sz);
>> + if (!tx)
>> + return -ENOMEM;
>> + }
>> dev->_tx = tx;
>>
>> netdev_for_each_tx_queue(dev, netdev_init_one_queue, NULL);
>> @@ -5811,7 +5823,7 @@ free_all:
>>
>> free_pcpu:
>> free_percpu(dev->pcpu_refcnt);
>> - kfree(dev->_tx);
>> + netif_free_tx_queues(dev);
>> #ifdef CONFIG_RPS
>> kfree(dev->_rx);
>> #endif
>> @@ -5836,7 +5848,7 @@ void free_netdev(struct net_device *dev)
>>
>> release_net(dev_net(dev));
>>
>> - kfree(dev->_tx);
>> + netif_free_tx_queues(dev);
>> #ifdef CONFIG_RPS
>> kfree(dev->_rx);
>> #endif
>>
next prev parent reply other threads:[~2013-06-21 6:42 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-06-19 5:40 [net-next rfc 0/3] increase the limit of tuntap queues Jason Wang
2013-06-19 5:40 ` [net-next rfc 1/3] net: avoid high order memory allocation for queues by using flex array Jason Wang
2013-06-19 6:31 ` Eric Dumazet
2013-06-19 7:14 ` Jason Wang
2013-06-19 9:11 ` Michael S. Tsirkin
2013-06-19 9:56 ` Eric Dumazet
2013-06-19 12:22 ` Michael S. Tsirkin
2013-06-19 15:40 ` Michael S. Tsirkin
2013-06-19 15:58 ` Eric Dumazet
2013-06-19 16:06 ` David Laight
2013-06-19 16:28 ` Eric Dumazet
2013-06-19 18:07 ` Michael S. Tsirkin
2013-06-20 8:15 ` [PATCH net-next] net: allow large number of tx queues Eric Dumazet
2013-06-20 8:35 ` Michael S. Tsirkin
2013-06-21 6:41 ` Jason Wang [this message]
2013-06-21 7:12 ` Eric Dumazet
2013-06-23 10:29 ` Michael S. Tsirkin
2013-06-24 6:57 ` David Miller
2013-06-20 5:14 ` [net-next rfc 1/3] net: avoid high order memory allocation for queues by using flex array Jason Wang
2013-06-20 6:05 ` Eric Dumazet
2013-06-19 5:40 ` [net-next rfc 2/3] tuntap: reduce the size of tun_struct " Jason Wang
2013-06-19 5:40 ` [net-next rfc 3/3] tuntap: increase the max queues to 16 Jason Wang
2013-06-19 6:34 ` Eric Dumazet
2013-06-19 7:15 ` Jason Wang
2013-06-19 19:16 ` Jerry Chu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=51C3F5AC.2090008@redhat.com \
--to=jasowang@redhat$(echo .)com \
--cc=davem@davemloft$(echo .)net \
--cc=dwmw2@infradead$(echo .)org \
--cc=eric.dumazet@gmail$(echo .)com \
--cc=hkchu@google$(echo .)com \
--cc=mst@redhat$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox