Subject: Re: [PATCH net-next 00/13] resource management using devlink reload

From: Michal Swiatkowski <[email-masked]>
Date: Tue, 15 Nov 2022 08:00:28 +0100

On Mon, Nov 14, 2022 at 07:09:36PM +0200, Leon Romanovsky wrote:
> On Mon, Nov 14, 2022 at 04:58:57PM +0000, Keller, Jacob E wrote:
> > 
> > 
> > > -----Original Message-----
> > > From: Samudrala, Sridhar <[email-masked]>
> > > Sent: Monday, November 14, 2022 7:31 AM
> > > To: Leon Romanovsky <[email-masked]>; Michal Swiatkowski
> > > <[email-masked]>
> > > Cc: [email-masked]; [email-masked]; [email-masked];
> > > [email-masked]; [email-masked]; [email-masked];
> > > [email-masked]; Nguyen, Anthony L <[email-masked]>; Lobakin,
> > > Alexandr <[email-masked]>; Drewek, Wojciech
> > > <[email-masked]>; Czapnik, Lukasz <[email-masked]>;
> > > Saleem, Shiraz <[email-masked]>; Brandeburg, Jesse
> > > <[email-masked]>; Ismail, Mustafa <[email-masked]>;
> > > Kitszel, Przemyslaw <[email-masked]>; Raczynski, Piotr
> > > <[email-masked]>; Keller, Jacob E <[email-masked]>; Ertman,
> > > David M <[email-masked]>; Kaliszczuk, Leszek
> > > <[email-masked]>
> > > Subject: Re: [PATCH net-next 00/13] resource management using devlink reload
> > > 
> > > On 11/14/2022 7:23 AM, Leon Romanovsky wrote:
> > > > On Mon, Nov 14, 2022 at 01:57:42PM +0100, Michal Swiatkowski wrote:
> > > >> Currently the default value for number of PF vectors is number of CPUs.
> > > >> Because of that there are cases when all vectors are used for PF
> > > >> and user can't create more VFs. It is hard to set default number of
> > > >> CPUs right for all different use cases. Instead allow user to choose
> > > >> how many vectors should be used for various features. After implementing
> > > >> subdevices this mechanism will be also used to set number of vectors
> > > >> for subfunctions.
> > > >>
> > > >> The idea is to set vectors for eth or VFs using devlink resource API.
> > > >> New value of vectors will be used after devlink reinit. Example
> > > >> commands:
> > > >> $ sudo devlink resource set pci/0000:31:00.0 path msix/msix_eth size 16
> > > >> $ sudo devlink dev reload pci/0000:31:00.0
> > > >> After reload driver will work with 16 vectors used for eth instead of
> > > >> num_cpus.
> > > > By saying "vectors", are you referring to MSI-X vectors?
> > > > If yes, you have specific interface for that.
> > > > https://lore.kernel.org/linux-pci/[email-masked]/
> > > 
> > > This patch series is exposing a resources API to split the device level MSI-X vectors
> > > across the different functions supported by the device (PF, RDMA, SR-IOV VFs
> > > and
> > > in future subfunctions). Today this is all hidden in a policy implemented within
> > > the PF driver.
> > > 
> > > The patch you are referring to seems to be providing an interface to change the
> > > msix count for a particular VF. This patch is providing a interface to set the total
> > > msix count for all the possible VFs from the available device level pool of
> > > msix-vectors.
> > > 
> > 
> > It looks like we should implement both: resources to configure the "pool" of available vectors for each VF, and the sysfs VF Interface to allow configuring individual VFs.
> 
> Yes, to be aligned with PCI spec and see coherent lspci output for VFs.
> 
> Thanks
> 

Thanks, I will implement this sysf interface to configure individual
VFs.

> > Thanks,
> > Jake