Subject: Re: [PATCH net] skge: dma_sync the whole receive buffer

From: poma <[email-masked]>
Date: Tue, 20 Aug 2013 05:28:09 +0200

On 19.08.2013 02:49, poma wrote:
> On 15.08.2013 17:41, Stephen Hemminger wrote:
>> On Wed, 14 Aug 2013 20:29:06 +0200
>> poma <[email-masked]> wrote:
>>
>>> On 14.08.2013 18:20, Stephen Hemminger wrote:
>>>> On Wed, 14 Aug 2013 12:20:03 +0200
>>>> poma <[email-masked]> wrote:
>>>>
>>>>> On 14.08.2013 03:00, Stephen Hemminger wrote:
>>>>>> On Tue, 13 Aug 2013 15:09:55 -0700 (PDT)
>>>>>> David Miller <[email-masked]> wrote:
>>>>>>
>>>>>>> From: Stephen Hemminger <[email-masked]>
>>>>>>> Date: Sat, 10 Aug 2013 15:02:07 -0700
>>>>>>>
>>>>>>>> The DMA sync should sync the whole receive buffer, not just
>>>>>>>> part of it. Fixes log messages dma_sync_check.
>>>>>>>>
>>>>>>>> Signed-off-by: Stephen Hemminger <[email-masked]>
>>>>>>>
>>>>>>> Applied, but I really suspect that your "check DMA mapping errors"
>>>>>>> patch has added a serious regression.  A regression much worse than
>>>>>>> the bug you were trying to fix with that change.
>>>>>>
>>>>>> Argh. The problem is deeper than that. Device got broken somewhere between
>>>>>> 3.2 and 3.4. My old Dlink card works on 3.2 but gets DMA errors on 3.4.
>>>>>> The config's are different though so checking that as well.
>>>>>>
>>>>>
>>>>> Can I help you with debugging?
>>>>> DGE-530T is rather solid device.
>>>>
>>>> Don't think it is a hardware problem.
>>>> The failure is when the board access the Receive ring PCI memory area.
>>>> This region is allocated with pci_alloc_consistent and therefore should
>>>> be available. Two possible issues are driver math issues, or hardware
>>>> problems with where the region is located. Some of these cards don't
>>>> really have full 64 bit PCI support.
>>>>
>>>> My board is:
>>>> 05:01.0 Ethernet controller: D-Link System Inc Gigabit Ethernet Adapter (rev 11)
>>>> 	Subsystem: D-Link System Inc DGE-530T Gigabit Ethernet Adapter
>>>> 	Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 18
>>>> 	Memory at f7d20000 (32-bit, non-prefetchable) [size=16K]
>>>> 	I/O ports at c000 [size=256]
>>>> 	Expansion ROM at f7d00000 [disabled] [size=128K]
>>>> 	Capabilities: [48] Power Management version 2
>>>> 	Capabilities: [50] Vital Product Data
>>>> 	Kernel driver in use: skge
>>>>
>>>>
>>>> What is your config?
>>>>
>>>
>>> 01:09.0 Ethernet controller: D-Link System Inc Gigabit Ethernet Adapter
>>> (rev 11)
>>> 	Subsystem: D-Link System Inc DGE-530T Gigabit Ethernet Adapter
>>> 	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 19
>>> 	Memory at fbffc000 (32-bit, non-prefetchable) [size=16K]
>>> 	I/O ports at b400 [size=256]
>>> 	[virtual] Expansion ROM at ec000000 [disabled] [size=128K]
>>> 	Capabilities: [48] Power Management version 2
>>> 	Capabilities: [50] Vital Product Data
>>> 	Kernel driver in use: skge
>>>
>>>
>>> poma
>>>
>>
>> In the course of debugging this, I moved the card to another slot
>> and all the problems went away. I suspect either card insertion or more likely
>> the crap consumer motherboards don't have full PCI support on some slots.
>>
>> There doesn't seem to be anyway to address this in software.
>>
> 
> 
> DGE-530T is further tested in the 3 available slots:
> 01:06.0 Ethernet controller: D-Link System Inc Gigabit Ethernet Adapter
> (rev 11)
> 01:07.0 Ethernet controller: D-Link System Inc Gigabit Ethernet Adapter
> (rev 11)
> 01:08.0 Ethernet controller: D-Link System Inc Gigabit Ethernet Adapter
> (rev 11)
> And the result is the same as in the slot:
> 01:09.0 Ethernet controller: D-Link System Inc Gigabit Ethernet Adapter
> (rev 11)
> warnings, oopses and kernel crashes.
> 
> However DGE-528T(RTL8110s) on the same bus runs without errors:
> 01:09.0 Ethernet controller: D-Link System Inc DGE-528T Gigabit Ethernet
> Adapter (rev 10)
> 	Subsystem: D-Link System Inc DGE-528T Gigabit Ethernet Adapter
> 	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 19
> 	I/O ports at cc00 [size=256]
> 	Memory at fbfff000 (32-bit, non-prefetchable) [size=256]
> 	[virtual] Expansion ROM at fbe00000 [disabled] [size=128K]
> 	Capabilities: [dc] Power Management version 2
> 	Kernel driver in use: r8169
> 
> Besides comparing the behavior of these two cards, e.g. NFS upload, I
> noticed an obvious difference in the data flow.
> Via DGE-528T transmission is steady, while via DGE-530T the traffic is
> at times interrupted and unstable.
> So it seems that the "WARNING: at lib/dma-debug.c:937 check_unmap…"
> isn't just a fun.
> 

In support of the validity of the device I made a test with the
2.6.32-358.14.1.el6.x86_64.debug kernel.
And everything worked as it should.

01:08.0 Ethernet controller: D-Link System Inc Gigabit Ethernet Adapter
(rev 11)
	Subsystem: D-Link System Inc DGE-530T Gigabit Ethernet Adapter
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 18
	Memory at fbff8000 (32-bit, non-prefetchable) [size=16K]
	I/O ports at cc00 [size=256]
	[virtual] Expansion ROM at fbe00000 [disabled] [size=128K]
	Capabilities: [48] Power Management version 2
	Capabilities: [50] Vital Product Data
	Kernel driver in use: skge
	Kernel modules: skge

filename:
/lib/modules/2.6.32-358.14.1.el6.x86_64.debug/kernel/drivers/net/skge.ko
version:        1.13
license:        GPL
author:         Stephen Hemminger <[email-masked]>
description:    SysKonnect Gigabit Ethernet driver
srcversion:     ADF6781C2E0D2D895F86279
alias:          pci:v00001737d00001032sv*sd00000015bc*sc*i*
alias:          pci:v00001737d00001064sv*sd*bc*sc*i*
alias:          pci:v00001371d0000434Esv*sd*bc*sc*i*
alias:          pci:v000011ABd00005005sv*sd*bc*sc*i*
alias:          pci:v000011ABd00004320sv*sd*bc*sc*i*
alias:          pci:v00001186d00004B01sv*sd*bc*sc*i*
alias:          pci:v00001186d00004C00sv*sd*bc*sc*i*
alias:          pci:v00001148d00004320sv*sd*bc*sc*i*
alias:          pci:v00001148d00004300sv*sd*bc*sc*i*
alias:          pci:v000010B7d000080EBsv*sd*bc*sc*i*
alias:          pci:v000010B7d00001700sv*sd*bc*sc*i*
depends:
vermagic:       2.6.32-358.14.1.el6.x86_64.debug SMP mod_unload modversions
parm:           debug:Debug level (0=none,...,16=all) (int)

Given all the tests and all written, something isn't right, at all.
Should I quote Shakespeare. :)

poma