From: Nikolay Aleksandrov <nikolay@cumulusnetworks•com>
To: Frank Schreuder <fschreuder@transip•nl>,
Johan Schuijt <johan@transip•nl>
Cc: Eric Dumazet <eric.dumazet@gmail•com>,
"nikolay@redhat•com" <nikolay@redhat•com>,
"davem@davemloft•net" <davem@davemloft•net>,
"fw@strlen•de" <fw@strlen•de>,
"chutzpah@gentoo•org" <chutzpah@gentoo•org>,
Robin Geuze <robing@transip•nl>, netdev <netdev@vger•kernel.org>
Subject: Re: reproducable panic eviction work queue
Date: Mon, 20 Jul 2015 16:02:10 +0200 [thread overview]
Message-ID: <55ACFF62.6000006@cumulusnetworks.com> (raw)
In-Reply-To: <55ACEDE9.3090205@transip.nl>
On 07/20/2015 02:47 PM, Frank Schreuder wrote:
>
> On 7/18/2015 05:32 PM, Nikolay Aleksandrov wrote:
>> On 07/18/2015 05:28 PM, Johan Schuijt wrote:
>>> Thx for your looking into this!
>>>
>>>> Thank you for the report, I will try to reproduce this locally
>>>> Could you please post the full crash log ?
>>> Of course, please see attached file.
>>>
>>>> Also could you test
>>>> with a clean current kernel from Linus' tree or Dave's -net ?
>>> Will do.
>>>
>>>> These are available at:
>>>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
>>>> git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
>>>> respectively.
>>>>
>>>> One last question how many IRQs do you pin i.e. how many cores
>>>> do you actively use for receive ?
>>> This varies a bit across our systems, but we’ve managed to reproduce this with IRQs pinned on as many as 2,4,8 or 20 cores.
>>>
>>> I won’t have access to our test-setup till Monday again, so I’ll be testing 3 scenario’s then:
>>> - Your patch
>> -----
>>> - Linux tree
>>> - Dave’s -net tree
>> Just one of these two would be enough. I couldn't reproduce it here but
>> I don't have as many machines to test right now and had to improvise with VMs. :-)
>>
>>> I’ll make sure to keep you posted on all the results then. We have a kernel dump of the panic, so if you need me to extract any data from there just let me know! (Some instructions might be needed)
>>>
>>> - Johan
>>>
>> Great, thank you!
>>
> I'm able to reproduce this panic on the following kernel builds:
> - 3.18.7
> - 3.18.18
> - 3.18.18 + patch from Nikolay Aleksandrov
> - 4.1.0
>
> Would you happen to have any more suggestions we can try?
>
> Thanks,
> Frank
>
Unfortunately I was wrong about my theory because I mixed qp and qp_in, the new frag
doesn't make the chainlist if that codepath is hit so it couldn't mix the flags.
I'm still trying (unsuccessfully) to reproduce this, I've tried with up to 4 cores
and 4 different pinned irqs but no luck so far.
Anyway, I'll keep looking into this and will let you know if I get anywhere.
next prev parent reply other threads:[~2015-07-20 14:02 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <F8D94413-90A2-4F80-AAA2-7A6AB57DF314@transip.nl>
2015-07-18 8:56 ` reproducable panic eviction work queue Eric Dumazet
2015-07-18 9:01 ` Johan Schuijt
2015-07-18 10:02 ` Nikolay Aleksandrov
2015-07-18 13:31 ` Nikolay Aleksandrov
2015-07-18 15:28 ` Johan Schuijt
2015-07-18 15:30 ` Johan Schuijt
2015-07-18 15:32 ` Nikolay Aleksandrov
2015-07-20 12:47 ` Frank Schreuder
2015-07-20 14:02 ` Nikolay Aleksandrov [this message]
2015-07-20 14:30 ` Florian Westphal
2015-07-21 11:50 ` Frank Schreuder
2015-07-21 18:34 ` Florian Westphal
2015-07-22 8:09 ` Frank Schreuder
2015-07-22 8:17 ` Frank Schreuder
2015-07-22 9:11 ` Nikolay Aleksandrov
2015-07-22 10:55 ` Frank Schreuder
2015-07-22 13:58 ` Florian Westphal
2015-07-22 14:03 ` Nikolay Aleksandrov
2015-07-22 14:14 ` Nikolay Aleksandrov
2015-07-22 15:31 ` Frank Schreuder
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=55ACFF62.6000006@cumulusnetworks.com \
--to=nikolay@cumulusnetworks$(echo .)com \
--cc=chutzpah@gentoo$(echo .)org \
--cc=davem@davemloft$(echo .)net \
--cc=eric.dumazet@gmail$(echo .)com \
--cc=fschreuder@transip$(echo .)nl \
--cc=fw@strlen$(echo .)de \
--cc=johan@transip$(echo .)nl \
--cc=netdev@vger$(echo .)kernel.org \
--cc=nikolay@redhat$(echo .)com \
--cc=robing@transip$(echo .)nl \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox