Subject: Re: [PATCH 2/2 net-next] net: move qdisc ingress filtering code where it belongs

From: Pablo Neira Ayuso <[email-masked]>
Date: Tue, 12 May 2015 14:55:26 +0200

Hi Eric,

On Mon, May 11, 2015 at 04:30:38PM -0700, Eric Dumazet wrote:
> On Mon, 2015-05-11 at 16:02 -0700, Alexei Starovoitov wrote:
> > On 5/11/15 7:35 AM, Eric Dumazet wrote:
> > >
> > > -static inline int deliver_skb(struct sk_buff *skb,
> > > +static noinline int deliver_skb(struct sk_buff *skb,
> > >   			      struct packet_type *pt_prev,
> > >   			      struct net_device *orig_dev)
> > 
> > have tried the above only and didn't see any difference
> > for simple 'netif_receive + drop in ip_rcv' test.
> > Not sure whether it's actually worth doing. I would leave it as-is.
> 
> Yes, this was probably too aggressive.

I tested this noinline patch and I got a bit less performance here in
my 32K i-cache testbed.

> unlikely() or static_key_false() are no moving code away enough, whole
> function including unused code pollutes icache.
>
> Code size increased a lot, while L1/L2 caches on cpu are about the same
> than 6 years ago.

OK, so that explains why I'm getting more performance with the patch
that move the qdisc ingress code using the indirection, since that
helped to move code away, right?

BTW, looking at the emails, Daniel said:

[...]
> After your patch set:
>
> Samples: 50K of event 'cycles:k', Event count (approx.): 45160667741
> +   40.49%  kpktgend_0  [kernel.kallsyms]  [k] __netif_receive_skb_core
> +   31.21%  kpktgend_0  [kernel.kallsyms]  [k] kfree_skb
> +    6.94%  kpktgend_0  [pktgen]           [k] pktgen_thread_worker
> +    6.63%  kpktgend_0  [kernel.kallsyms]  [k] netif_receive_skb_internal
> +    6.63%  kpktgend_0  [kernel.kallsyms]  [k] ip_rcv
> +    3.30%  kpktgend_0  [kernel.kallsyms]  [k] netif_receive_skb_sk
> +    3.30%  kpktgend_0  [kernel.kallsyms]  [k] __netif_receive_skb
> +    0.96%  kpktgend_0  [kernel.kallsyms]  [k] __local_bh_enable_ip
> +    0.37%  kpktgend_0  [kernel.kallsyms]  [k] kthread_should_stop
> +    0.03%  kpktgend_0  [kernel.kallsyms]  [k] _cond_resched
>
> For *all* three, I reliably get ~40.0 Mpps with the benchmark.

@Daniel, Alexei: Are you getting the same numbers with the indirection?

What's the i-cache size in your testbed?

Thanks.