Subject: Re: [PATCH 2.6.30-rc4] r8169: avoid losing MSI interrupts

From: [email-masked] (Eric W. Biederman)
Date: Sun, 23 Aug 2009 19:37:02 -0700

David Dillow <[email-masked]> writes:

> On Sat, 2009-08-22 at 05:07 -0700, Eric W. Biederman wrote:
>> [email-masked] (Eric W. Biederman) writes:
>> 
>> > David Dillow <[email-masked]> writes:
>> >
>> >>
>> >> Re-looking at the code, I'd guess that some IRQ status line is getting
>> >> stuck high, but I don't see why -- we should acknowledge all outstanding
>> >> interrupts each time through the loop, whether we care about them or
>> >> not.
>> >>
>> >> Could reproduce a problem with the following patch applied, and send the
>> >> full dmesg, please?
>> >
>> > Here is what I get.
>> >
>> > r8169 screaming irq status 00000085 mask 0000ffff event 0000803f napi 0000001d
>> 
>> And now that the machine has come out of it, that was followed by:
>> Looks like the soft lockup did not manage to trigger in this case.
>
> I need some more context, please. What is the network load through this
> NIC when you have the issues? Light, heavy? Can you give me more details
> about the machine? A full dmesg from boot until this happens would help
> quite a bit. At a minimum it would help answer which version of the chip
> we're dealing with and what the machine it is in looks like.

dmesg attached.

What seems to reproduce the problem is a download of about a gigabyte.
Which this machine does every hour or two.  The switch and the upstream
server are all 10Gig. So at least in bursts I expect I am saturating
the network adapter with traffic coming in as fast as it can come.

Last night I reverted your patch and the machine seems to be happy
and not having problems since then.

> Can you reproduce this with pci=nomsi? I'm assuming it the chip running
> in MSI mode.

It is.

> Also, can you reproduce it when booting UP (or maxcpus=1)? I'm thinking
> about a race between rtl8169_interrupt() and rtl8169_poll(), but it
> isn't jumping out at me.
>
> Also, I'm having connectivity troubles this weekend, so my response may
> be spotty. :(

No problem.

I haven't wrapped my head around the device specific bits but I suspect
we are simply receiving more packets while the interrupt handler is
running.

Eric