Subject: Re: [RFC PATCH v5 01/16] RDMA/irdma: Add driver framework definitions

From: Jason Gunthorpe <[email-masked]>
Date: Thu, 23 Apr 2020 21:48:41 -0300

On Thu, Apr 23, 2020 at 11:54:18PM +0000, Saleem, Shiraz wrote:
> > Subject: Re: [RFC PATCH v5 01/16] RDMA/irdma: Add driver framework
> > definitions
> > 
> > On Thu, Apr 23, 2020 at 05:15:22PM +0000, Saleem, Shiraz wrote:
> > > > Subject: Re: [RFC PATCH v5 01/16] RDMA/irdma: Add driver framework
> > > > definitions
> > > >
> > > > On Thu, Apr 23, 2020 at 12:32:48AM +0000, Saleem, Shiraz wrote:
> > > >
> > > > > we have a split initialization design for gen2 and future products.
> > > > > phase1 is control path resource initialization in irdma_probe_dev
> > > > > and
> > > > > phase-2 is the rest of the resources with the ib registration at
> > > > > the end of irdma_open. irdma_close must de-register the ib device
> > > > > which will take care of ibdev free too. So it makes sense to keep
> > > > > allocation of the ib device in irdma_open.
> > > >
> > > > The best driver pattern is to allocate the ib_device at the very
> > > > start of probe() and use this to anchor all the device resources and memories.
> > > >
> > > > The whole close/open thing is really weird, you should get rid of it.
> > > maybe I missing something. But why is it weird?
> > 
> > Because the RDMA driver should exist as its own entity. It does not shutdown
> > unless the remove() method on is struct device_driver is closed.
> > So what exactly are open/cose supposed to be doing? I think it is a left over of
> > trying to re-implement the driver model.
> > 
> > > underlying configuration changes and reset management for the physical
> > > function need a light-weight mechanism which is realized with the
> > > close/open from netdev PCI drv --> rdma drv.
> > 
> > > Without a teardown and re-add of virtual device off the bus.
> > 
> > Yes, that is exactly right. If you have done something so disruptive that the
> > ib_device needs to be destroyed then you should unplug/replug the entire virtual
> > bus device, that is the correct and sane thing to do.
> 
> Well we have resources created in rdma driver probe which are used by any
> VF's regardless of the registration of the ib device on the PF.

Ugh, drivers that have the VF driver require the PF driver have a lot
of problems.

But, even so, with your new split design, resources held for a VF
belong in the core pci driver, not the rdma virtual bus device.

Jason