Subject: Re: [PATCH net-next 00/10] net: lan966x: add support for PCIe FDMA

From: Herve Codina <[email-masked]>
Date: Wed, 8 Apr 2026 11:51:26 +0200

Hi Daniel,

On Tue, 7 Apr 2026 15:20:16 +0200
Daniel Machon <[email-masked]> wrote:

...
> 
> The following diff should fix the FDMA traffic issue, and the FDMA error splat,
> when reloading the lan966x-pci driver, by:
> 
> 1. Resetting the FDMA engine on PCI init()
> 
> 2. Clearing any rogue FDMA errors that may latch due to the soft reset by the
> reset driver.
> 
>   diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_fdma_pci.c                                                   
>   b/drivers/net/ethernet/microchip/lan966x/lan966x_fdma_pci.c                                                              
>   --- a/drivers/net/ethernet/microchip/lan966x/lan966x_fdma_pci.c                                                          
>   +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_fdma_pci.c                                                          
>   @@ -372,6 +372,9 @@ static int lan966x_fdma_pci_init(struct lan966x *lan966x)                                            
>         if (!lan966x->fdma)                                                                                                
>                 return 0;                                                                                                  
>                                                                                                                          
>   +     lan_wr(FDMA_CTRL_NRESET_SET(0), lan966x, FDMA_CTRL);                                                               
>   +     lan_wr(FDMA_CTRL_NRESET_SET(1), lan966x, FDMA_CTRL);                                                               
>   +                                                                                                                        
>         fdma_pci_atu_init(&lan966x->atu, lan966x->regs[TARGET_PCIE_DBI]);                                                  
>                                                             
>         lan966x->rx.lan966x = lan966x;                                                                                     
>   diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
>   b/drivers/net/ethernet/microchip/lan966x/lan966x_main.c                                                                  
>   --- a/drivers/net/ethernet/microchip/lan966x/lan966x_main.c                                                              
>   +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_main.c                                                              
>   @@ -1071,6 +1071,15 @@ static int lan966x_reset_switch(struct lan966x *lan966x)                                          
>                                                             
>         reset_control_reset(switch_reset);                                                                                 
>                                                                                                                            
>   +     /* When in PCI mode, the GCB soft reset issued by the reset
>   +      * controller can latch spurious bits in the FDMA error stickies.                                                  
>   +      * Clear them before request_irq hooks up the FDMA IRQ line,
>   +      * otherwise the handler fires immediately on probe.                                                               
>   +      */                                                 
>   +     lan_wr(lan_rd(lan966x, FDMA_ERRORS),   lan966x, FDMA_ERRORS);                                                      
>   +     lan_wr(lan_rd(lan966x, FDMA_INTR_ERR), lan966x, FDMA_INTR_ERR);                                                    
>   +     lan_wr(lan_rd(lan966x, FDMA_INTR_DB),  lan966x, FDMA_INTR_DB);                                                     
>   +                                                                                                                        
>         /* Don't reinitialize the switch core, if it is already initialized. In                                            
>          * case it is initialized twice, some pointers inside the queue system                                             
>          * in HW will get corrupted and then after a while the queue system gets                                           
>   diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h                                                       
>   b/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h                                                                  
>   --- a/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h                                                              
>   +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h                                                              
>   @@ -1010,6 +1010,15 @@ enum lan966x_target {                                                                             
>    #define FDMA_CH_CFG_CH_MEM_GET(x)\                                                                                      
>         FIELD_GET(FDMA_CH_CFG_CH_MEM, x)                                                                                   
>                                                             
>   +/*      FDMA:FDMA:FDMA_CTRL */                                                                                          
>   +#define FDMA_CTRL                 __REG(TARGET_FDMA, 0, 1, 8, 0, 1, 428, 424, 0, 1, 4)                                  
>   +                                                                                                                        
>   +#define FDMA_CTRL_NRESET                         BIT(0)                                                                 
>   +#define FDMA_CTRL_NRESET_SET(x)\                                                                                        
>   +     FIELD_PREP(FDMA_CTRL_NRESET, x)                                                                                    
>   +#define FDMA_CTRL_NRESET_GET(x)\                         
>   +     FIELD_GET(FDMA_CTRL_NRESET, x)                                                                                     
>   +                                                         
>    /*      FDMA:FDMA:FDMA_PORT_CTRL */                                                                                     
>    #define FDMA_PORT_CTRL(r)         __REG(TARGET_FDMA, 0, 1, 8, 0, 1, 428, 376, r, 2, 4)
> 
> Let me know if it works on your end.
> 
> (Btw. I have noticed another issue where TX stops working on lan966x-pci reload.
> It happens more rarely, but is unrelated to this patch series, as it also
> happens in register-based INJ/XTR mode. Whenever that happens, you will see
> "Flush timeout chip port" in the logs. This should also be fixed, but sent as a
> separate fix commit, I believe.)
> 

I have tested your proposed modification.

- From a cold boot, module unloading / re-loading:
  Tested Ok

- From a cold boot, module loading / no interface configuration / reboot:
  Tested Ok

- From a cold boot, module loading / interface configuration / reboot:
  Issue present.

- From a cold boot, module loading / interface configuration / module unloading / reboot:
  Tested Ok.

The interface configuration was the following command:
  ip link set eth2 up && ip addr add 192.168.32.20/16 dev eth2

With the interface configured and without unloading the lan966x_pci module
before the reboot, I have the already known FDMA error splat after the
reboot:
   [   18.908397] ------------[ cut here ]------------
   [   18.913226] Unexpected error: 64, error_type: 1073741824
   [   18.918704] WARNING: drivers/net/ethernet/microchip/lan966x/lan966x_fdma.c:558 at lan966x_fdma_irq_handler+0xe0/0x12c [lan966x_switch], CPU#0: kworker/u8:2/38
   ...

The issue is not present if I unload the lan966x_pci module before the
reboot. This let me think that something has been cleaned by the module
unloading.

On top of your proposed modification, I use the shutdown() hook in the
lan966x_driver:

    --- a/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
    +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
    @@ -1326,9 +1326,18 @@ static void lan966x_remove(struct platform_device *pdev)
            debugfs_remove_recursive(lan966x->debugfs_root);
     }
     
    +static void lan966x_shutdown(struct platform_device *pdev)
    +{
    +       struct lan966x *lan966x = platform_get_drvdata(pdev);
    +
    +       lan966x->ops->fdma_deinit(lan966x);
    +}
    +
    static struct platform_driver lan966x_driver = {
            .probe = lan966x_probe,
            .remove = lan966x_remove,
    +       .shutdown = lan966x_shutdown,
            .driver = {
                    .name = "lan966x-switch",
                    .of_match_table = lan966x_match,

With that done, the issue is no more present.

I am not sure that this is a correct fix but it can give some clues.

Feel free to ask for more tests or more details if you need to.

Best regards,
Hervé