From: Jesper Dangaard Brouer <brouer@redhat•com>
To: David Miller <davem@davemloft•net>
Cc: ttoukan.linux@gmail•com, edumazet@google•com,
alexander.duyck@gmail•com, netdev@vger•kernel.org,
tariqt@mellanox•com, kafai@fb•com, saeedm@mellanox•com,
willemb@google•com, bblanco@plumgrid•com, ast@kernel•org,
eric.dumazet@gmail•com, linux-mm@kvack•org, brouer@redhat•com
Subject: Re: [PATCH v3 net-next 08/14] mlx4: use order-0 pages for RX
Date: Tue, 14 Feb 2017 20:38:22 +0100 [thread overview]
Message-ID: <20170214203822.72d41268@redhat.com> (raw)
In-Reply-To: <20170214.120426.2032015522492111544.davem@davemloft.net>
On Tue, 14 Feb 2017 12:04:26 -0500 (EST)
David Miller <davem@davemloft•net> wrote:
> From: Tariq Toukan <ttoukan.linux@gmail•com>
> Date: Tue, 14 Feb 2017 16:56:49 +0200
>
> > Internally, I already implemented "dynamic page-cache" and
> > "page-reuse" mechanisms in the driver, and together they totally
> > bridge the performance gap.
It sounds like you basically implemented a page_pool scheme...
> I worry about a dynamically growing page cache inside of drivers
> because it is invisible to the rest of the kernel.
Exactly, that is why I wanted a separate standardized thing, I call the
page_pool, which is part of the MM-tree and interacts with the page
allocator. E.g. it must implement/support a way the page allocator can
reclaim pages from it (admit I didn't implement this in RFC patches).
> It responds only to local needs.
Generally true, but a side effect of recycling these pages, result in
less fragmentation of the page allocator/buddy system.
> The price of the real page allocator comes partly because it can
> respond to global needs.
>
> If a driver consumes some unreasonable percentage of system memory, it
> is keeping that memory from being used from other parts of the system
> even if it would be better for networking to be slightly slower with
> less cache because that other thing that needs memory is more
> important.
(That is why I want to have OOM protection at device level, with the
recycle feedback from page pool we have this knowledge, and further I
wanted to allow blocking a specific RX queue[1])
[1] https://prototype-kernel.readthedocs.io/en/latest/vm/page_pool/design/memory_model_nic.html#userspace-delivery-and-oom
> I think this is one of the primary reasons that the MM guys severely
> chastise us when we build special purpose local caches into networking
> facilities.
>
> And the more I think about it the more I think they are right.
+1
> One path I see around all of this is full integration. Meaning that
> we can free pages into the page allocator which are still DMA mapped.
> And future allocations from that device are prioritized to take still
> DMA mapped objects.
I like this idea. Are you saying that this should be done per DMA
engine or per device?
If this is per device, it is almost the page_pool idea.
> Yes, we still need to make the page allocator faster, but this kind of
> work helps everyone not just 100GB ethernet NICs.
True. And Mel already have some generic improvements to the page
allocator queued for the next merge. And I have the responsibility to
get the bulking API into shape.
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack•org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack•org"> email@kvack•org </a>
next prev parent reply other threads:[~2017-02-14 19:38 UTC|newest]
Thread overview: 71+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-02-13 19:58 [PATCH v3 net-next 00/14] mlx4: order-0 allocations and page recycling Eric Dumazet
2017-02-13 19:58 ` [PATCH v3 net-next 01/14] mlx4: use __skb_fill_page_desc() Eric Dumazet
2017-02-13 19:58 ` [PATCH v3 net-next 02/14] mlx4: dma_dir is a mlx4_en_priv attribute Eric Dumazet
2017-02-13 19:58 ` [PATCH v3 net-next 03/14] mlx4: remove order field from mlx4_en_frag_info Eric Dumazet
2017-02-13 19:58 ` [PATCH v3 net-next 04/14] mlx4: get rid of frag_prefix_size Eric Dumazet
2017-02-13 19:58 ` [PATCH v3 net-next 05/14] mlx4: rx_headroom is a per port attribute Eric Dumazet
2017-02-13 19:58 ` [PATCH v3 net-next 06/14] mlx4: reduce rx ring page_cache size Eric Dumazet
2017-02-13 19:58 ` [PATCH v3 net-next 07/14] mlx4: removal of frag_sizes[] Eric Dumazet
2017-02-13 19:58 ` [PATCH v3 net-next 08/14] mlx4: use order-0 pages for RX Eric Dumazet
2017-02-13 20:51 ` Alexander Duyck
2017-02-13 21:09 ` Eric Dumazet
2017-02-13 23:16 ` Alexander Duyck
2017-02-13 23:22 ` Eric Dumazet
2017-02-13 23:26 ` Alexander Duyck
2017-02-13 23:29 ` Eric Dumazet
2017-02-13 23:47 ` Alexander Duyck
2017-02-14 0:22 ` Eric Dumazet
2017-02-14 0:34 ` Alexander Duyck
2017-02-14 0:46 ` Eric Dumazet
2017-02-14 0:47 ` Eric Dumazet
2017-02-14 0:57 ` Eric Dumazet
2017-02-14 1:32 ` Alexander Duyck
2017-02-14 12:12 ` Jesper Dangaard Brouer
2017-02-14 13:45 ` Eric Dumazet
2017-02-14 14:12 ` Eric Dumazet
2017-02-14 14:56 ` Tariq Toukan
2017-02-14 15:51 ` Eric Dumazet
2017-02-14 16:03 ` Eric Dumazet
2017-02-14 17:29 ` Tom Herbert
2017-02-15 16:42 ` Tariq Toukan
2017-02-15 16:57 ` Eric Dumazet
2017-02-16 13:08 ` Tariq Toukan
2017-02-16 15:47 ` Eric Dumazet
2017-02-16 17:05 ` Tom Herbert
2017-02-16 17:11 ` Eric Dumazet
2017-02-16 20:49 ` Saeed Mahameed
2017-02-16 19:03 ` David Miller
2017-02-16 21:06 ` Saeed Mahameed
2017-02-14 17:04 ` David Miller
2017-02-14 17:17 ` David Laight
2017-02-14 17:22 ` David Miller
2017-02-14 19:38 ` Jesper Dangaard Brouer [this message]
2017-02-14 19:59 ` David Miller
2017-02-14 17:29 ` Alexander Duyck
2017-02-14 18:46 ` Jesper Dangaard Brouer
2017-02-14 19:02 ` Eric Dumazet
2017-02-14 20:02 ` Jesper Dangaard Brouer
2017-02-14 21:56 ` Eric Dumazet
2017-02-14 19:06 ` Alexander Duyck
2017-02-14 19:50 ` Jesper Dangaard Brouer
2017-02-22 16:22 ` Eric Dumazet
2017-02-22 17:23 ` Alexander Duyck
2017-02-22 17:58 ` David Laight
2017-02-22 18:21 ` Eric Dumazet
2017-02-23 1:08 ` Alexander Duyck
2017-02-23 2:06 ` Eric Dumazet
2017-02-23 2:18 ` Alexander Duyck
2017-02-23 14:02 ` Tariq Toukan
2017-02-24 9:42 ` Jesper Dangaard Brouer
2017-03-12 14:57 ` Eric Dumazet
2017-03-12 15:29 ` Eric Dumazet
2017-03-12 15:49 ` Saeed Mahameed
2017-03-12 16:49 ` Eric Dumazet
2017-03-13 9:20 ` Saeed Mahameed
2017-02-13 19:58 ` [PATCH v3 net-next 09/14] mlx4: add page recycling in receive path Eric Dumazet
2017-02-13 19:58 ` [PATCH v3 net-next 10/14] mlx4: add rx_alloc_pages counter in ethtool -S Eric Dumazet
2017-02-13 19:58 ` [PATCH v3 net-next 11/14] mlx4: do not access rx_desc from mlx4_en_process_rx_cq() Eric Dumazet
2017-02-13 19:58 ` [PATCH v3 net-next 12/14] mlx4: factorize page_address() calls Eric Dumazet
2017-02-13 19:58 ` [PATCH v3 net-next 13/14] mlx4: make validate_loopback() more generic Eric Dumazet
2017-02-13 19:58 ` [PATCH v3 net-next 14/14] mlx4: remove duplicate code in mlx4_en_process_rx_cq() Eric Dumazet
2017-02-17 16:00 ` [PATCH v3 net-next 00/14] mlx4: order-0 allocations and page recycling David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170214203822.72d41268@redhat.com \
--to=brouer@redhat$(echo .)com \
--cc=alexander.duyck@gmail$(echo .)com \
--cc=ast@kernel$(echo .)org \
--cc=bblanco@plumgrid$(echo .)com \
--cc=davem@davemloft$(echo .)net \
--cc=edumazet@google$(echo .)com \
--cc=eric.dumazet@gmail$(echo .)com \
--cc=kafai@fb$(echo .)com \
--cc=linux-mm@kvack$(echo .)org \
--cc=netdev@vger$(echo .)kernel.org \
--cc=saeedm@mellanox$(echo .)com \
--cc=tariqt@mellanox$(echo .)com \
--cc=ttoukan.linux@gmail$(echo .)com \
--cc=willemb@google$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox