From: Jason Gunthorpe <jgg@mellanox•com>
To: Jakub Kicinski <jakub.kicinski@netronome•com>
Cc: Jiri Pirko <jiri@resnulli•us>,
"davem@davemloft•net" <davem@davemloft•net>,
"oss-drivers@netronome•com" <oss-drivers@netronome•com>,
"netdev@vger•kernel.org" <netdev@vger•kernel.org>,
Parav Pandit <parav@mellanox•com>
Subject: Re: [PATCH net-next 4/8] devlink: allow subports on devlink PCI ports
Date: Tue, 5 Mar 2019 22:11:48 +0000 [thread overview]
Message-ID: <20190305221144.GA1758@mellanox.com> (raw)
In-Reply-To: <20190304181107.1379e358@cakuba.netronome.com>
On Mon, Mar 04, 2019 at 06:11:07PM -0800, Jakub Kicinski wrote:
> > At least in RDMA we have drivers doing all combinations of this:
> > multiple ports per BDF, one port per BDF, and one composite RDMA
> > device formed by combining multiple BDFs worth of ports together.
>
> Right, last but not least we have the case where there is one port but
> multiple links (for NUMA, or just because 1 PCIe link can't really cope
> with 200Gbps). In that case which DBDF would the port go to? :(
> Do all internal info of the ASIC (health, regions, sbs) get registered
> twice?
This I don't know, at least for RDMA this configuration gets confusing
very fast and devlink is the least of the worries..
Personally I would advocate for a master/slave kind of arrangement
where the master BDF has a different PCI DID from the slaves. devlink
and other kernel objects hang off the master.
The slave port is then only used to carry selected NUMA aware data
path traffic and doesn't show in devlink.
Jason
next prev parent reply other threads:[~2019-03-05 22:11 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-26 18:24 [PATCH net-next 0/8] devlink: add PF and VF port flavours Jakub Kicinski
2019-02-26 18:24 ` [PATCH net-next 1/8] nfp: split devlink port init from registration Jakub Kicinski
2019-02-26 18:24 ` [PATCH net-next 2/8] devlink: add PF and VF port flavours Jakub Kicinski
2019-02-27 12:16 ` Jiri Pirko
2019-03-04 4:59 ` Parav Pandit
2019-03-04 7:30 ` Jiri Pirko
2019-03-20 17:29 ` Abodunrin, Akeem G
2019-03-21 12:26 ` Jiri Pirko
2019-02-27 12:23 ` Jiri Pirko
2019-02-27 12:41 ` Jiri Pirko
2019-02-27 17:23 ` Jakub Kicinski
2019-02-27 20:17 ` Jiri Pirko
2019-02-27 22:42 ` Jakub Kicinski
2019-02-28 8:44 ` Jiri Pirko
2019-02-28 16:08 ` Jakub Kicinski
2019-02-28 16:24 ` David Ahern
2019-02-26 18:24 ` [PATCH net-next 3/8] nfp: register devlink ports of all reprs Jakub Kicinski
2019-02-26 18:24 ` [PATCH net-next 4/8] devlink: allow subports on devlink PCI ports Jakub Kicinski
2019-02-27 12:37 ` Jiri Pirko
2019-02-27 18:30 ` Jakub Kicinski
2019-02-28 8:56 ` Jiri Pirko
2019-02-28 13:32 ` Jiri Pirko
2019-02-28 16:24 ` Jakub Kicinski
2019-03-01 7:25 ` Jiri Pirko
2019-03-01 16:04 ` Jakub Kicinski
2019-03-01 16:20 ` Jiri Pirko
2019-03-04 16:15 ` Jason Gunthorpe
2019-03-05 1:03 ` Jakub Kicinski
2019-03-05 1:30 ` Jason Gunthorpe
2019-03-05 2:11 ` Jakub Kicinski
2019-03-05 22:11 ` Jason Gunthorpe [this message]
2019-03-04 5:00 ` Parav Pandit
2019-02-26 18:24 ` [PATCH net-next 5/8] nfp: switch to devlink_port_get_phys_port_name() Jakub Kicinski
2019-02-26 18:24 ` [PATCH net-next 6/8] devlink: introduce port's peer netdevs Jakub Kicinski
2019-02-27 13:08 ` Jiri Pirko
2019-02-27 18:47 ` Jakub Kicinski
2019-02-28 9:00 ` Jiri Pirko
2019-02-28 16:36 ` Jakub Kicinski
2019-03-01 7:37 ` Jiri Pirko
2019-03-01 16:05 ` Jakub Kicinski
2019-03-04 5:07 ` Parav Pandit
2019-02-26 18:24 ` [PATCH net-next 7/8] nfp: expose PF " Jakub Kicinski
2019-02-26 18:24 ` [PATCH net-next 8/8] devlink: fix kdoc Jakub Kicinski
2019-02-27 13:13 ` Jiri Pirko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190305221144.GA1758@mellanox.com \
--to=jgg@mellanox$(echo .)com \
--cc=davem@davemloft$(echo .)net \
--cc=jakub.kicinski@netronome$(echo .)com \
--cc=jiri@resnulli$(echo .)us \
--cc=netdev@vger$(echo .)kernel.org \
--cc=oss-drivers@netronome$(echo .)com \
--cc=parav@mellanox$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox