From: Tom Talpey <tom-CLs1Zie5N5HQT0dZR+AlfA@public•gmane.org>
To: Michael Wang <yun.wang-EIkl63zCoXaH+58JC4qpiA@public•gmane.org>,
Roland Dreier <roland-DgEjT+Ai2ygdnm+yROfE0A@public•gmane.org>,
Sean Hefty <sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public•gmane.org>,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public•gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public•gmane.org,
linux-nfs-u79uwXL29TY76Z2rM5mHXA@public•gmane.org,
netdev-u79uwXL29TY76Z2rM5mHXA@public•gmane.org
Cc: Hal Rosenstock
<hal.rosenstock-Re5JQEeQqe8AvxtiuMwx3w@public•gmane.org>,
Tom Tucker
<tom-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public•gmane.org>,
Steve Wise
<swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public•gmane.org>,
Hoang-Nam Nguyen
<hnguyen-tA70FqPdS9bQT0dZR+AlfA@public•gmane.org>,
Christoph Raisch <raisch-tA70FqPdS9bQT0dZR+AlfA@public•gmane.org>,
Mike Marciniszyn
<infinipath-ral2JQCrhuEAvxtiuMwx3w@public•gmane.org>,
Eli Cohen <eli-VPRAkNaXOzVWk0Htik3J/w@public•gmane.org>,
Faisal Latif
<faisal.latif-ral2JQCrhuEAvxtiuMwx3w@public•gmane.org>,
Upinder Malhi <umalhi-FYB4Gu1CFyUAvxtiuMwx3w@public•gmane.org>,
Trond Myklebust
<trond.myklebust-7I+n7zu2hftEKMMhf/gKZA@public•gmane.org>,
"J. Bruce Fields"
<bfields-uC3wQj2KruNg9hUCZPvPmw@public•gmane.org>,
"David S. Miller" <davem-fT/PcQaiUtIeIZ0/mPfg9Q@public•gmane.org>,
Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public•gmane.org>,
PJ Waskiewicz
<pj.waskiewicz-PrFOU5CTvK1l57MIdRCFDg@public•gmane.org>,
Tatyana Nikolova
<Tatyana.E.Nikolova-ral2JQCrhuEAvxtiuMwx3w@public•gmane.org>,
Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public•gmane.org>,
Jack Morgenstein
<jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public•gmane.org>,
Haggai Eran <haggaie-VPRAkNaXOzVWk0Htik3J/w@public•gmane.org>,
Ilya Nelkenbaum <ilyan-VPRAkNaXOzVWk0Htik3J/w@public•gmane.org>,
Yann Droneaud <ydroneaud-RlY5vtjFyJ3QT0dZR+AlfA@public•gmane.org>,
Bart Van Assche <bvanassche-HInyCGIudOg@public•gmane.org>,
Shachar Raindel <raindel-VPRAkNaXOzVWk0Htik3J/w@public•gmane.org
Subject: Re: [PATCH v2 09/17] IB/Verbs: Use helper cap_read_multi_sge() and reform svc_rdma_accept()
Date: Tue, 07 Apr 2015 11:46:57 -0400 [thread overview]
Message-ID: <5523FBF1.80304@talpey.com> (raw)
In-Reply-To: <5523CEE4.5060901-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
On 4/7/2015 8:34 AM, Michael Wang wrote:
> /**
> + * cap_read_multi_sge - Check if the port of device has the capability
> + * RDMA Read Multiple Scatter-Gather Entries.
> + *
> + * @device: Device to be checked
> + * @port_num: Port number of the device
> + *
> + * Return 0 when port of the device don't support
> + * RDMA Read Multiple Scatter-Gather Entries.
> + */
> +static inline int cap_read_multi_sge(struct ib_device *device, u8 port_num)
> +{
> + return !rdma_transport_iwarp(device, port_num);
> +}
This just papers over the issue we discussed earlier. How *many*
entries does the device support? If a device supports one, or two,
is that enough? How does the upper layer know the limit?
This needs an explicit device attribute, to be fixed properly.
> +
> +/**
> * cap_ipoib - Check if the port of device has the capability
> * IP over Infiniband.
> *
> diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
> index e011027..604d035 100644
> --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
> +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
> @@ -118,8 +118,8 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp,
>
> static int rdma_read_max_sge(struct svcxprt_rdma *xprt, int sge_count)
> {
> - if (rdma_node_get_transport(xprt->sc_cm_id->device->node_type) ==
> - RDMA_TRANSPORT_IWARP)
> + if (!cap_read_multi_sge(xprt->sc_cm_id->device,
> + xprt->sc_cm_id->port_num))
> return 1;
> else
> return min_t(int, sge_count, xprt->sc_max_sge);
This is incorrect. The RDMA Read max is not at all the same as the
max_sge. It is a different operation, with a different set of work
request parameters.
In other words, the above same comment applies.
> diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c
> index 4e61880..e75175d 100644
> --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c
> +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c
> @@ -979,8 +979,8 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)
> /*
> * Determine if a DMA MR is required and if so, what privs are required
> */
> - switch (rdma_node_get_transport(newxprt->sc_cm_id->device->node_type)) {
> - case RDMA_TRANSPORT_IWARP:
> + if (rdma_transport_iwarp(newxprt->sc_cm_id->device,
> + newxprt->sc_cm_id->port_num)) {
> newxprt->sc_dev_caps |= SVCRDMA_DEVCAP_READ_W_INV;
Do I read this correctly that it is forcing the "read with invalidate"
capability to "on" for all iWARP devices? I don't think that is correct,
for the legacy devices you're also supporting.
> @@ -992,8 +992,8 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)
> dma_mr_acc = IB_ACCESS_LOCAL_WRITE;
> } else
> need_dma_mr = 0;
> - break;
> - case RDMA_TRANSPORT_IB:
> + } else if (rdma_ib_mgmt(newxprt->sc_cm_id->device,
> + newxprt->sc_cm_id->port_num)) {
> if (!(newxprt->sc_dev_caps & SVCRDMA_DEVCAP_FAST_REG)) {
> need_dma_mr = 1;
> dma_mr_acc = IB_ACCESS_LOCAL_WRITE;
Now I'm even more confused. How is the presence of IB management
related to needing a privileged lmr?
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public•gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2015-04-07 15:46 UTC|newest]
Thread overview: 79+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-04-07 12:25 [PATCH v2 00/17] IB/Verbs: IB Management Helpers Michael Wang
2015-04-07 12:31 ` [PATCH v2 04/17] IB/Verbs: Use management helper cap_ib_smi() for smi-check Michael Wang
[not found] ` <5523CCD5.6030401-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
2015-04-07 12:28 ` [PATCH 01/17] IB/Verbs: Implement new callback query_transport() for each HW Michael Wang
[not found] ` <5523CD62.1070102-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
2015-04-07 12:44 ` Michael Wang
2015-04-07 12:29 ` [PATCH v2 02/17] IB/Verbs: Implement raw management helpers Michael Wang
[not found] ` <5523CDA7.8090902-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
2015-04-07 21:25 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373A8FBE403-P5GAC/sN6hkd3b2yrw5b5LfspsVTdybXVpNB7YpNyf8@public.gmane.org>
2015-04-08 8:41 ` Michael Wang
2015-04-07 12:30 ` [PATCH v2 03/17] IB/Verbs: Use management helper cap_ib_mad() for mad-check Michael Wang
[not found] ` <5523CDDE.4050209-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
2015-04-07 17:26 ` Jason Gunthorpe
[not found] ` <20150407172612.GA15704-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-04-08 8:13 ` Michael Wang
2015-04-07 12:32 ` [PATCH v2 05/17] IB/Verbs: Use management helper cap_ib_cm() for cm-check Michael Wang
2015-04-07 12:32 ` [PATCH v2 06/17] IB/Verbs: Use management helper cap_ib_sa() for sa-check Michael Wang
2015-04-07 12:33 ` [PATCH v2 07/17] IB/Verbs: Use management helper cap_ib_mcast() for mcast-check Michael Wang
2015-04-07 12:34 ` [PATCH v2 08/17] IB/Verbs: Use management helper cap_ipoib() for ipoib-check Michael Wang
2015-04-07 12:37 ` [PATCH v2 13/17] IB/Verbs: Reform cma/ucma with management helpers Michael Wang
[not found] ` <5523CF74.8020004-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
2015-04-07 21:11 ` Steve Wise
2015-04-07 21:36 ` Hefty, Sean
2015-04-08 9:37 ` Michael Wang
2015-04-08 17:02 ` Hefty, Sean
2015-04-07 12:38 ` [PATCH v2 15/17] IB/Verbs: Reform cma_acquire_dev() " Michael Wang
2015-04-07 12:39 ` [PATCH v2 16/17] IB/Verbs: Cleanup rdma_node_get_transport() Michael Wang
2015-04-07 12:39 ` [PATCH v2 17/17] IB/Verbs: Move rdma_port_get_link_layer() to mlx4 head file Michael Wang
2015-04-08 11:38 ` [PATCH v2 00/17] IB/Verbs: IB Management Helpers Tom Talpey
[not found] ` <5525132F.1020004-CLs1Zie5N5HQT0dZR+AlfA@public.gmane.org>
2015-04-08 12:41 ` Michael Wang
[not found] ` <552521EE.5060206-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
2015-04-08 15:51 ` Jason Gunthorpe
[not found] ` <20150408155145.GA25178-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-04-08 16:05 ` Michael Wang
2015-04-07 12:34 ` [PATCH v2 09/17] IB/Verbs: Use helper cap_read_multi_sge() and reform svc_rdma_accept() Michael Wang
[not found] ` <5523CEE4.5060901-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
2015-04-07 15:46 ` Tom Talpey [this message]
[not found] ` <5523FBF1.80304-CLs1Zie5N5HQT0dZR+AlfA@public.gmane.org>
2015-04-07 16:05 ` Michael Wang
2015-04-07 17:42 ` Jason Gunthorpe
2015-04-07 12:35 ` [PATCH v2 10/17] IB/Verbs: Adopt management helpers for IB helpers Michael Wang
2015-04-07 18:40 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373A8FBE2A5-P5GAC/sN6hkd3b2yrw5b5LfspsVTdybXVpNB7YpNyf8@public.gmane.org>
2015-04-08 8:24 ` Michael Wang
[not found] ` <5523CF0A.5090206-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
2015-04-07 20:12 ` Jason Gunthorpe
[not found] ` <20150407201259.GA17147-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-04-07 20:16 ` Steve Wise
2015-04-08 8:28 ` Michael Wang
2015-04-09 5:36 ` ira.weiny
2015-04-07 20:18 ` Hefty, Sean
2015-04-07 12:36 ` [PATCH v2 11/17] IB/Verbs: Reform link_layer_show() and ib_uverbs_query_port() Michael Wang
[not found] ` <5523CF30.7000608-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
2015-04-07 18:49 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373A8FBE2FA-P5GAC/sN6hkd3b2yrw5b5LfspsVTdybXVpNB7YpNyf8@public.gmane.org>
2015-04-07 18:56 ` Steve Wise
2015-04-08 8:29 ` Michael Wang
2015-04-07 12:36 ` [PATCH v2 12/17] IB/Verbs: Use management helper cap_ib_cm_dev() for cm-device-check Michael Wang
2015-04-07 12:38 ` [PATCH v2 14/17] IB/Verbs: Reserve legacy transport type for 'struct rdma_dev_addr' Michael Wang
2015-04-07 12:42 ` [PATCH v2 01/17] IB/Verbs: Implement new callback query_transport() for each HW Michael Wang
[not found] ` <5523D098.3020007-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
2015-04-08 18:29 ` Doug Ledford
[not found] ` <1428517786.2980.180.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-04-08 18:41 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373A8FBF0E8-P5GAC/sN6hkd3b2yrw5b5LfspsVTdybXVpNB7YpNyf8@public.gmane.org>
2015-04-08 19:35 ` Jason Gunthorpe
2015-04-08 20:10 ` Jason Gunthorpe
2015-04-08 20:55 ` Tom Talpey
[not found] ` <20150408201015.GB28666-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-04-09 9:45 ` Michael Wang
2015-04-09 12:42 ` Michael Wang
2015-04-09 16:00 ` Jason Gunthorpe
[not found] ` <20150409160054.GB2035-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-04-10 8:19 ` Michael Wang
2015-04-09 14:34 ` Doug Ledford
2015-04-09 16:01 ` Jason Gunthorpe
2015-04-09 21:19 ` Doug Ledford
[not found] ` <1428614348.2980.287.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-04-09 21:36 ` Jason Gunthorpe
2015-04-10 7:46 ` Michael Wang
[not found] ` <55277FE8.50107-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
2015-04-10 16:48 ` Doug Ledford
2015-04-10 6:16 ` ira.weiny
2015-04-10 8:25 ` Michael Wang
2015-04-10 14:56 ` ira.weiny
[not found] ` <20150410061610.GA26288-W4f6Xiosr+yv7QzWx2u06xL4W9x8LtSr@public.gmane.org>
2015-04-10 16:15 ` Jason Gunthorpe
[not found] ` <20150410161551.GA26419-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-04-10 17:38 ` ira.weiny
[not found] ` <20150410173836.GE10675-W4f6Xiosr+yv7QzWx2u06xL4W9x8LtSr@public.gmane.org>
2015-04-10 17:49 ` Doug Ledford
2015-04-10 18:11 ` ira.weiny
2015-04-10 18:04 ` Jason Gunthorpe
[not found] ` <20150410180455.GA1277-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-04-10 18:24 ` Doug Ledford
2015-04-10 19:17 ` Jason Gunthorpe
[not found] ` <20150410191723.GC1277-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-04-10 21:06 ` ira.weiny
[not found] ` <20150410210639.GB19907-W4f6Xiosr+yv7QzWx2u06xL4W9x8LtSr@public.gmane.org>
2015-04-11 0:01 ` Tom Talpey
2015-04-10 20:38 ` ira.weiny
2015-04-10 7:48 ` ira.weiny
[not found] ` <20150410074805.GA11855-W4f6Xiosr+yv7QzWx2u06xL4W9x8LtSr@public.gmane.org>
2015-04-10 17:10 ` Doug Ledford
2015-04-10 17:36 ` Jason Gunthorpe
2015-04-13 7:40 ` Michael Wang
2015-04-10 17:50 ` Tom Talpey
[not found] ` <55280D51.20402-CLs1Zie5N5HQT0dZR+AlfA@public.gmane.org>
2015-04-10 18:17 ` Doug Ledford
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5523FBF1.80304@talpey.com \
--to=tom-cls1zie5n5hqt0dzr+alfa@public$(echo .)gmane.org \
--cc=Tatyana.E.Nikolova-ral2JQCrhuEAvxtiuMwx3w@public$(echo .)gmane.org \
--cc=bfields-uC3wQj2KruNg9hUCZPvPmw@public$(echo .)gmane.org \
--cc=bvanassche-HInyCGIudOg@public$(echo .)gmane.org \
--cc=davem-fT/PcQaiUtIeIZ0/mPfg9Q@public$(echo .)gmane.org \
--cc=eli-VPRAkNaXOzVWk0Htik3J/w@public$(echo .)gmane.org \
--cc=faisal.latif-ral2JQCrhuEAvxtiuMwx3w@public$(echo .)gmane.org \
--cc=haggaie-VPRAkNaXOzVWk0Htik3J/w@public$(echo .)gmane.org \
--cc=hal.rosenstock-Re5JQEeQqe8AvxtiuMwx3w@public$(echo .)gmane.org \
--cc=hnguyen-tA70FqPdS9bQT0dZR+AlfA@public$(echo .)gmane.org \
--cc=ilyan-VPRAkNaXOzVWk0Htik3J/w@public$(echo .)gmane.org \
--cc=infinipath-ral2JQCrhuEAvxtiuMwx3w@public$(echo .)gmane.org \
--cc=ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public$(echo .)gmane.org \
--cc=jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public$(echo .)gmane.org \
--cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public$(echo .)gmane.org \
--cc=linux-nfs-u79uwXL29TY76Z2rM5mHXA@public$(echo .)gmane.org \
--cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public$(echo .)gmane.org \
--cc=netdev-u79uwXL29TY76Z2rM5mHXA@public$(echo .)gmane.org \
--cc=ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public$(echo .)gmane.org \
--cc=pj.waskiewicz-PrFOU5CTvK1l57MIdRCFDg@public$(echo .)gmane.org \
--cc=raindel-VPRAkNaXOzVWk0Htik3J/w@public$(echo .)gmane.org \
--cc=raisch-tA70FqPdS9bQT0dZR+AlfA@public$(echo .)gmane.org \
--cc=roland-DgEjT+Ai2ygdnm+yROfE0A@public$(echo .)gmane.org \
--cc=sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public$(echo .)gmane.org \
--cc=swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public$(echo .)gmane.org \
--cc=tom-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public$(echo .)gmane.org \
--cc=trond.myklebust-7I+n7zu2hftEKMMhf/gKZA@public$(echo .)gmane.org \
--cc=umalhi-FYB4Gu1CFyUAvxtiuMwx3w@public$(echo .)gmane.org \
--cc=ydroneaud-RlY5vtjFyJ3QT0dZR+AlfA@public$(echo .)gmane.org \
--cc=yun.wang-EIkl63zCoXaH+58JC4qpiA@public$(echo .)gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox