From: Yang Yingliang <yangyingliang@huawei•com>
To: Eric Dumazet <eric.dumazet@gmail•com>
Cc: <netdev@vger•kernel.org>, <davem@davemloft•net>,
Ding Tianhong <dingtianhong@huawei•com>
Subject: Re: [PATCH RFC] net: decrease the length of backlog queue immediately after it's detached from sk
Date: Tue, 12 Apr 2016 20:31:24 +0800 [thread overview]
Message-ID: <570CEA9C.1070803@huawei.com> (raw)
In-Reply-To: <570C6483.5020502@huawei.com>
On 2016/4/12 10:59, Yang Yingliang wrote:
>
>
> On 2016/4/11 20:13, Eric Dumazet wrote:
>> On Mon, 2016-04-11 at 19:57 +0800, Yang Yingliang wrote:
>>>
>>> On 2016/4/8 22:44, Eric Dumazet wrote:
>>>> On Fri, 2016-04-08 at 19:18 +0800, Yang Yingliang wrote:
>>>>
>>>>> I expand tcp_adv_win_scale and tcp_rmem. It has no effect.
>>>>
>>>> Try :
>>>>
>>>> echo -2 >/proc/sys/net/ipv4/tcp_adv_win_scale
>>>>
>>>> And restart your flows.
>>>>
>>> cat /proc/sys/net/ipv4/tcp_rmem
>>> 10240 2097152 10485760
>>
>> What about leaving the default values ?
> I tried, it did not work.
>
>>
>> $ cat /proc/sys/net/ipv4/tcp_rmem
>> 4096 87380 6291456
>>
>>>
>>> echo 102400 20971520 104857600 > /proc/sys/net/ipv4/tcp_rmem
>>> echo -2 >/proc/sys/net/ipv4/tcp_adv_win_scale
>>>
>>> It seems has not effect.
>>>
>>
>> I have no idea what you did on the sender side to allow it to send more
>> than 1.5 MB then.
>
> We are doing performance test. The sender send 256KB per-block with 128
> threads to one socket. And the receiver uses 10Gb NIC to handle the
> data on ARM64. The data flow is driver->ip layer->tcp layer->iscsi.
>
> I added some debug messages and found handling backlog packets in
> __release_sock() cost about 11ms at most. This can cause backlog queue
> overflow. The sk_data_ready is re-assigned, it may cost time in our
> program. I will check it out.
>
I traced the cost cycles of handling backlog packets in
__release_sock().
16.97 ms to handling about 12MB backlog packets, of which 13.66ms to do
sk_data_ready.
The speed of handling packets in TCP is 5.65Gb/s which is smaller than
the NIC's bandwidth. So the packets will be dropped.
If the cost of sk_data_read cannot be reduced, do we have other choice
exclude dropping packets ?
next prev parent reply other threads:[~2016-04-12 12:43 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-03-30 5:16 [PATCH RFC] net: decrease the length of backlog queue immediately after it's detached from sk Yang Yingliang
2016-03-30 5:25 ` Eric Dumazet
2016-03-30 5:34 ` Eric Dumazet
2016-03-30 5:56 ` Yang Yingliang
2016-03-30 13:47 ` Eric Dumazet
2016-04-07 5:59 ` Yang Yingliang
2016-04-07 10:21 ` Eric Dumazet
2016-04-07 14:51 ` Eric Dumazet
2016-04-08 11:18 ` Yang Yingliang
2016-04-08 14:44 ` Eric Dumazet
2016-04-08 16:53 ` David Miller
2016-04-08 17:04 ` Eric Dumazet
2016-04-11 14:42 ` Yang Yingliang
2016-04-11 11:57 ` Yang Yingliang
2016-04-11 12:13 ` Eric Dumazet
2016-04-12 2:59 ` Yang Yingliang
2016-04-12 12:31 ` Yang Yingliang [this message]
2016-04-13 2:42 ` Eric Dumazet
2016-03-30 5:38 ` Yang Yingliang
2016-03-30 12:56 ` Sergei Shtylyov
2016-04-07 6:01 ` Yang Yingliang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=570CEA9C.1070803@huawei.com \
--to=yangyingliang@huawei$(echo .)com \
--cc=davem@davemloft$(echo .)net \
--cc=dingtianhong@huawei$(echo .)com \
--cc=eric.dumazet@gmail$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox