From: Stephen Hemminger <shemminger@vyatta•com>
To: Eric Dumazet <eric.dumazet@gmail•com>
Cc: Jesse Brandeburg <jesse.brandeburg@gmail•com>,
Jesper Dangaard Brouer <hawk@diku•dk>,
netdev@vger•kernel.org
Subject: Re: [RFC] skb align patch
Date: Mon, 21 Sep 2009 22:23:55 -0700 [thread overview]
Message-ID: <20090921222355.631467e0@nehalam> (raw)
In-Reply-To: <4AB84295.3050509@gmail.com>
On Tue, 22 Sep 2009 05:20:53 +0200
Eric Dumazet <eric.dumazet@gmail•com> wrote:
> Stephen Hemminger a écrit :
> > On Mon, 21 Sep 2009 08:13:20 +0200
> > Eric Dumazet <eric.dumazet@gmail•com> wrote:
> >
> >> Stephen Hemminger a écrit :
> >>> Based on the Intel suggestion that PCI-express overhead is
> >>> a significant cost.
> >>>
> >>> Would people doing performance please measure the impact of
> >>> changing SKB alignment (64 bit only).
> >> I had this idea some time ago when I hit a limit on bnx2 adapter
> >> (Giga bit link, BCM5708S), with small packets. pktgen was able
> >> to send ~500 Mbps 'only', or 700kps if I remember well.
> >> So I tried to align the pktgen build packet to a cache line,
> >> it gave no difference at all, but it was on a 32 bit kernel.
> >> (Thus my patch was for pktgen only, not a generic one as yours)
> >>
> >> Could you elaborate why this change could be useful on 64bit ?
> >>
> >
> > It is useful on all architecture where unaligned CPU access is
> > relatively cheap.
> >
> > The issue is that a unaligned DMA requires a read/modify/write
> > cache line access versus just a write access. I am not a bus
> > expert, but writes are probably more pipelined as well.
> >
>
> Oh I see, you want to optimize the rx (NIC has to do a DMA
> to write packet into host memory and this DMA could be a read
> /modify/write if address is not aligned, instead of a pure write),
> while I tried to align skb to optimize the pktgen tx
> (NIC has to do a DMA to read packet from host), and align the skb
> had no effect.
>
> Maybe we should separate the rx/tx, and try your idea only
> for skb allocated for rx.
>
> Also/Or we might try
> __builtin_prefetch (addr, 0, 0);
> to instruct cpu to commit to memory cache lines that are
> going to be modified by NIC.
Don't think it matters whether RX buffer has to read/modify/write
from cpu cache or memory on modern cache snooping architecures.
The cost is the PCI traffic.
--
next prev parent reply other threads:[~2009-09-22 5:23 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-09-20 21:22 [RFC] skb align patch Stephen Hemminger
2009-09-21 6:13 ` Eric Dumazet
2009-09-22 4:30 ` Stephen Hemminger
2009-09-22 3:20 ` Eric Dumazet
2009-09-22 5:23 ` Stephen Hemminger [this message]
2009-09-22 5:29 ` David Miller
2009-09-23 5:47 ` Thomas Graf
2009-09-25 22:18 ` Herbert Xu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20090921222355.631467e0@nehalam \
--to=shemminger@vyatta$(echo .)com \
--cc=eric.dumazet@gmail$(echo .)com \
--cc=hawk@diku$(echo .)dk \
--cc=jesse.brandeburg@gmail$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox