From: Jesper Dangaard Brouer <brouer@redhat•com>
To: Rick Jones <rick.jones2@hp•com>
Cc: "netdev@vger•kernel.org" <netdev@vger•kernel.org>,
Rick Jones <rick.jones2@hpe•com>,
brouer@redhat•com, Eric Dumazet <eric.dumazet@gmail•com>
Subject: Netperf UDP issue with connected sockets
Date: Wed, 16 Nov 2016 13:16:09 +0100 [thread overview]
Message-ID: <20161116131609.4e5726b4@redhat.com> (raw)
In-Reply-To: <1409757426.26422.41.camel@edumazet-glaptop2.roam.corp.google.com>
While optimizing the kernel RX path, I've run into an issue where I
cannot use netperf UDP_STREAM for testing, because the sender is
slower than receiver. Thus, it cannot show my receiver improvements
(as receiver have idle cycles).
Eric Dumazet previously told me[1] this was related to netperf need to
use connected socket for UDP. Netperf options "-- -n -N" should
enable connected UDP sockets, but it never worked! Options are
documented, but netperf seems to have a bug.
Called like:
netperf -H 198.18.50.1 -t UDP_STREAM -l 120 -- -m 1472 -n -N
Problem on sender-side is "__ip_select_ident".
Samples: 15K of event 'cycles', Event count (approx.): 16409681913
Overhead Command Shared Object Symbol
+ 11.18% netperf [kernel.vmlinux] [k] __ip_select_ident
+ 6.93% netperf [kernel.vmlinux] [k] _raw_spin_lock
+ 6.12% netperf [kernel.vmlinux] [k] copy_user_enhanced_fast_string
+ 4.31% netperf [kernel.vmlinux] [k] __ip_make_skb
+ 3.97% netperf [kernel.vmlinux] [k] fib_table_lookup
+ 3.51% netperf [mlx5_core] [k] mlx5e_sq_xmit
+ 2.43% netperf [kernel.vmlinux] [k] __ip_route_output_key_hash
+ 2.24% netperf netperf [.] send_omni_inner
+ 2.17% netperf netperf [.] send_data
[1] Subj: High perf top ip_idents_reserve doing netperf UDP_STREAM
- https://www.spinics.net/lists/netdev/msg294752.html
Not fixed in version 2.7.0.
- ftp://ftp.netperf.org/netperf/netperf-2.7.0.tar.gz
Used extra netperf configure compile options:
./configure --enable-histogram --enable-demo
It seems like some fix attempts exists in the SVN repository::
svn checkout http://www.netperf.org/svn/netperf2/trunk/ netperf2-svn
svn log -r709
# A quick stab at getting remote connect going for UDP_STREAM
svn diff -r708:709
Testing with SVN version, still show __ip_select_ident() in top#1.
(p.s. is netperf ever going to be converted from SVN to git?)
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
(old email below)
On Wed, 03 Sep 2014 08:17:06 -0700
Eric Dumazet <eric.dumazet@gmail•com> wrote:
> On Wed, 2014-09-03 at 16:59 +0200, Jesper Dangaard Brouer wrote:
> > Hi Eric,
> >
> > When doing:
> > super_netperf 120 -H 192.168.8.2 -t UDP_STREAM -l 100 -- -m 256
> >
> > I'm seeing function ip_idents_reserve() consuming most CPU. Could you
> > help, explain what is going on, and how I can avoid this?
> >
> > Perf top:
> > 11.67% [kernel] [k] ip_idents_reserve
> > 8.37% [kernel] [k] fib_table_lookup
> > 4.46% [kernel] [k] _raw_spin_lock
> > 3.21% [kernel] [k] copy_user_enhanced_fast_string
> > 2.92% [kernel] [k] sock_alloc_send_pskb
> > 2.88% [kernel] [k] udp_sendmsg
> >
>
> Because you use a single destination, all flows compete on a single
> atomic to get their next IP identifier.
>
> You can try to use netperf options (-- -N -n) so that netperf uses
> connected UDP sockets.
>
> In this case, the IP identifier generator is held in each socket.
next prev parent reply other threads:[~2016-11-16 12:16 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-03 14:59 High perf top ip_idents_reserve doing netperf UDP_STREAM Jesper Dangaard Brouer
2014-09-03 15:17 ` Eric Dumazet
2016-11-16 12:16 ` Jesper Dangaard Brouer [this message]
2016-11-16 17:46 ` Netperf UDP issue with connected sockets Rick Jones
2016-11-16 22:40 ` Jesper Dangaard Brouer
2016-11-16 22:50 ` Rick Jones
2016-11-17 0:34 ` Eric Dumazet
2016-11-17 8:16 ` Jesper Dangaard Brouer
2016-11-17 13:20 ` Eric Dumazet
2016-11-17 13:42 ` Jesper Dangaard Brouer
2016-11-17 14:17 ` Eric Dumazet
2016-11-17 14:57 ` Jesper Dangaard Brouer
2016-11-17 16:21 ` Eric Dumazet
2016-11-17 18:30 ` Jesper Dangaard Brouer
2016-11-17 18:51 ` Eric Dumazet
2016-11-17 21:19 ` Jesper Dangaard Brouer
2016-11-17 21:44 ` Eric Dumazet
2016-11-17 23:08 ` Rick Jones
2016-11-18 0:37 ` Julian Anastasov
2016-11-18 0:42 ` Rick Jones
2016-11-18 17:12 ` Jesper Dangaard Brouer
2016-11-21 16:03 ` Jesper Dangaard Brouer
2016-11-21 18:10 ` Eric Dumazet
2016-11-29 6:58 ` [WIP] net+mlx4: auto doorbell Eric Dumazet
2016-11-30 11:38 ` Jesper Dangaard Brouer
2016-11-30 15:56 ` Eric Dumazet
2016-11-30 19:17 ` Jesper Dangaard Brouer
2016-11-30 19:30 ` Eric Dumazet
2016-11-30 22:30 ` Jesper Dangaard Brouer
2016-11-30 22:40 ` Eric Dumazet
2016-12-01 0:27 ` Eric Dumazet
2016-12-01 1:16 ` Tom Herbert
2016-12-01 2:32 ` Eric Dumazet
2016-12-01 2:50 ` Eric Dumazet
2016-12-02 18:16 ` Eric Dumazet
2016-12-01 5:03 ` Tom Herbert
2016-12-01 19:24 ` Willem de Bruijn
2016-11-30 13:50 ` Saeed Mahameed
2016-11-30 15:44 ` Eric Dumazet
2016-11-30 16:27 ` Saeed Mahameed
2016-11-30 17:28 ` Eric Dumazet
2016-12-01 12:05 ` Jesper Dangaard Brouer
2016-12-01 14:24 ` Eric Dumazet
2016-12-01 16:04 ` Jesper Dangaard Brouer
2016-12-01 17:04 ` Eric Dumazet
2016-12-01 19:17 ` Jesper Dangaard Brouer
2016-12-01 20:11 ` Eric Dumazet
2016-12-01 20:20 ` David Miller
2016-12-01 22:10 ` Eric Dumazet
2016-12-02 14:23 ` Eric Dumazet
2016-12-01 21:32 ` Alexander Duyck
2016-12-01 22:04 ` Eric Dumazet
2016-11-17 17:34 ` Netperf UDP issue with connected sockets David Laight
2016-11-17 22:39 ` Alexander Duyck
2016-11-17 17:42 ` Rick Jones
2016-11-28 18:33 ` Rick Jones
2016-11-28 18:40 ` Rick Jones
2016-11-30 10:43 ` Jesper Dangaard Brouer
2016-11-30 17:42 ` Rick Jones
2016-11-30 18:11 ` David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20161116131609.4e5726b4@redhat.com \
--to=brouer@redhat$(echo .)com \
--cc=eric.dumazet@gmail$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
--cc=rick.jones2@hp$(echo .)com \
--cc=rick.jones2@hpe$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox