From: Yann Droneaud <ydroneaud@opteya•com>
To: Haggai Eran <haggaie@mellanox•com>
Cc: Doug Ledford <dledford@redhat•com>,
linux-rdma@vger•kernel.org, netdev@vger•kernel.org,
Liran Liss <liranl@mellanox•com>,
Guy Shapiro <guysh@mellanox•com>,
Shachar Raindel <raindel@mellanox•com>,
Yotam Kenneth <yotamke@mellanox•com>
Subject: Re: [PATCH v4 for-next 00/12] Add network namespace support in the RDMA-CM
Date: Tue, 19 May 2015 16:30:26 +0200 [thread overview]
Message-ID: <1432045826.5304.6.camel@opteya.com> (raw)
In-Reply-To: <1431841868-28063-1-git-send-email-haggaie@mellanox.com>
Hi,
Le dimanche 17 mai 2015 à 08:50 +0300, Haggai Eran a écrit :
> Thanks again everyone for the review comments. I've updated the patch
> set
> accordingly. The main changes are in the first patch to use a read
> -write
> semaphore instead of an SRCU, and with the reference counting of
> shared
> ib_cm_ids.
> Please let me know if I missed anything, or if there are other issues
> with
> the series.
>
> Regards,
> Haggai
>
> Changes from v3:
> - Patch 1 and 3: use read-write semaphore instead of an SRCU.
> - Patch 5:
> * Use a direct reference count instead of a kref.
> * Instead of adding get/put pair for ib_cm_ids, just avoid
> destroying an
> id when it is still in use.
> * Squashes these two patches together, since the first one became
> too
> short:
> IB/cm: Reference count ib_cm_ids
> IB/cm: API to retrieve existing listening CM IDs
> - Rebase to Doug's to-be-rebased/for-4.2 branch.
>
> Changes from v2:
> - Add patch 1 to change device_mutex to an RCU.
> - Remove patch that fixed IPv4 connections to an IPv4/IPv6 listener.
> - Limit namespace related changes to RDMA CM and InfiniBand only.
> - Rebase on dledford/for-v4.2, with David Ahern's unaligned access
> patch.
> * Use Michael Wang's capability functions where needed.
> - Move the struct net argument to be the first in all functions, to
> match the
> networking core scheme.
> - Patch 2:
> * Remove unwanted braces.
> - Patch 4: check the return value of ib_find_cached_pkey.
> - Patch 8: verify the address family before calling cm_save_ib_info.
> - Patch 10: use generic_net instead of a custom radix tree for having
> per
> network namespace data.
> - Minor changes.
>
> Changes from v1:
> - Include patch 1 in this series.
> - Rebase for v4.1.
>
> Changes from v0:
> - Fix code review comments by Yann
> - Rebase on top of linux-3.19
>
> RDMA-CM uses IP based addressing and routing to setup RDMA
> connections between
> hosts. Currently, all of the IP interfaces and addresses used by the
> RDMA-CM
> must reside in the init_net namespace. This restricts the usage of
> containers
> with RDMA to only work with host network namespace (aka the kernel
> init_net NS
> instance).
>
> This patchset allows using network namespaces with the RDMA-CM.
>
> Each RDMA-CM id keeps a reference to a network namespace.
>
> This reference is based on the process network namespace at the time
> of the
> creation of the object or inherited from the listener.
>
> This network namespace is used to perform all IP and network related
> operations. Specifically, the local device lookup, as well as the
> remote GID
> address resolution are done in the context of the RDMA-CM object's
> namespace.
> This allows outgoing connections to reach the right target, even if
> the same
> IP address exists in multiple network namespaces. This can happen if
> each
> network namespace resides on a different P_Key.
>
> Additionally, the network namespace is used to split the listener
> service ID
> table. From the user point of view, each network namespace has a
> unique,
> completely independent table of service IDs. This allows running
> multiple
> instances of a single service on the same machine, using containers.
> To
> implement this, multiple RDMA CM IDs, belonging to different
> namespaces may
> now share their CM ID. When a request on such a CM ID arrives, the
> RDMA CM
> module finds out the correct namespaces and looks for the RDMA CM ID
> matching the request's parameters.
>
> The functionality introduced by this series would come into play when
> the
> transport is InfiniBand and IPoIB interfaces are assigned to each
> namespace.
> Multiple IPoIB interfaces can be created and assigned to different
> RDMA-CM
> capable containers, for example using pipework [1].
>
> Full support for RoCE will be introduced in a later stage.
>
How does this play with iWarp: as iWarp HCA are aware of IP addresses /
UDP/TCP ports, AFAIK, are those tied to namespace with this patchset or
will it be possible to use the iWarp HCA to access to address/port
resources tied to a different namespace ?
> The patches apply against Doug's tree for v4.2.
>
> The patchset is structured as follows:
>
> Patch 1 adds a read-write semaphore in addition to the device mutex
> in
> ib_core to allow traversing the client list without a deadlock in
> Patch 3.
>
> Patch 2 is a relatively trivial API extension, requiring the callers
> of certain ib_addr functions to provide a network namespace, as
> needed.
>
> Patches 3 and 4 adds the ability to lookup a network namespace
> according to
> the IP address, device and P_Key. It finds the matching IPoIB
> interfaces, and
> safely takes a reference on the network namespace before returning to
> the
> caller.
>
> Patches 5-6 make necessary changes to the CM layer, to allow sharing
> of a
> single CM ID between multiple RDMA CM IDs. This includes adding a
> reference
> count to ib_cm_id structs, add an API to either create a new CM ID or
> use
> an existing one, and expose the service ID to ib_cm clients.
>
> Patches 7-8 do some preliminary refactoring to the rdma_cm module.
> Patch 7
> refactors the logic that extracts the IP address from a connect
> request to
> allow reuse by the namespace lookup code further on. Patch 8 changes
> the
> way RDMA CM module creates CM IDs, to avoid relying on the
> compare_data
> feature of ib_cm. This feature associate a single compare_data struct
> per
> ib_cm_id, so it cannot be used when sharing CM IDs.
>
> Patches 9-12 add proper namespace support to the RDMA-CM module. This
> includes adding multiple port space tables, sharing ib_cm_ids between
> rdma_cm_ids, adding a network namespace parameter, and finally
> retrieving
> the namespace from the creating process.
>
Regards.
--
Yann Droneaud
OPTEYA
next prev parent reply other threads:[~2015-05-19 14:30 UTC|newest]
Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-05-17 5:50 [PATCH v4 for-next 00/12] Add network namespace support in the RDMA-CM Haggai Eran
2015-05-17 5:50 ` [PATCH v4 for-next 02/12] IB/addr: Pass network namespace as a parameter Haggai Eran
2015-05-17 5:50 ` [PATCH v4 for-next 03/12] IB/core: Find the network namespace matching connection parameters Haggai Eran
[not found] ` <1431841868-28063-4-git-send-email-haggaie-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-05-19 18:26 ` Jason Gunthorpe
[not found] ` <20150519182616.GF18675-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-05-20 14:48 ` Haggai Eran
2015-05-17 5:51 ` [PATCH v4 for-next 04/12] IB/ipoib: Return IPoIB devices " Haggai Eran
[not found] ` <1431841868-28063-5-git-send-email-haggaie-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-05-19 18:28 ` Jason Gunthorpe
[not found] ` <20150519182810.GG18675-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-05-20 15:17 ` Haggai Eran
2015-05-19 23:55 ` Jason Gunthorpe
[not found] ` <20150519235502.GB26634-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-05-21 5:33 ` Haggai Eran
[not found] ` <555D6E41.10606-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-05-21 5:48 ` Or Gerlitz
[not found] ` <CAJ3xEMjN+o=vC4abAeG5EuOo3Y1gSyh1qPDseA_aaYmoLWAunw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-05-21 6:33 ` Haggai Eran
[not found] ` <555D7C4A.2060708-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-05-21 10:31 ` Or Gerlitz
2015-05-21 17:43 ` Jason Gunthorpe
[not found] ` <20150521174336.GA6771-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-05-28 11:51 ` Haggai Eran
2015-05-28 15:45 ` Jason Gunthorpe
2015-05-21 5:48 ` Haggai Eran
2015-05-17 5:51 ` [PATCH v4 for-next 05/12] IB/cm: Share listening CM IDs Haggai Eran
2015-05-19 18:35 ` Jason Gunthorpe
[not found] ` <20150519183545.GH18675-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-05-19 22:35 ` Jason Gunthorpe
[not found] ` <20150519223502.GA26324-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-05-21 8:08 ` Haggai Eran
2015-05-21 17:54 ` Jason Gunthorpe
2015-05-21 7:07 ` Haggai Eran
[not found] ` <1431841868-28063-1-git-send-email-haggaie-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-05-17 5:50 ` [PATCH v4 for-next 01/12] IB/core: Add rwsem to allow reading device list or client list Haggai Eran
2015-05-17 5:51 ` [PATCH v4 for-next 06/12] IB/cm: Expose service ID in request events Haggai Eran
2015-05-17 5:51 ` [PATCH v4 for-next 07/12] IB/cma: Refactor RDMA IP CM private-data parsing code Haggai Eran
2015-05-17 5:51 ` [PATCH v4 for-next 08/12] IB/cma: Add compare_data checks to the RDMA CM module Haggai Eran
2015-05-17 5:51 ` [PATCH v4 for-next 12/12] IB/ucma: Take the network namespace from the process Haggai Eran
2015-05-17 5:51 ` [PATCH v4 for-next 09/12] IB/cma: Separate port allocation to network namespaces Haggai Eran
2015-05-17 5:51 ` [PATCH v4 for-next 10/12] IB/cma: Share CM IDs between namespaces Haggai Eran
2015-05-17 5:51 ` [PATCH v4 for-next 11/12] IB/cma: Add support for network namespaces Haggai Eran
2015-05-19 14:30 ` Yann Droneaud [this message]
2015-05-19 14:54 ` [PATCH v4 for-next 00/12] Add network namespace support in the RDMA-CM Haggai Eran
[not found] ` <555B4EBE.7010900-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-05-19 16:39 ` Parav Pandit
2015-05-19 18:01 ` Haggai Eran
[not found] ` <1432058488417.98688-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-05-19 18:42 ` Parav Pandit
2015-05-19 18:38 ` Jason Gunthorpe
[not found] ` <20150519183843.GI18675-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-05-19 18:44 ` Parav Pandit
[not found] ` <CAGgvQNTXAWkQWzBBrQfk39GaCQ2ck63AhgURpYFFBPTbkpx4kg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-05-19 19:20 ` Jason Gunthorpe
2015-05-26 13:34 ` Doug Ledford
[not found] ` <1432647280.28905.107.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-05-26 16:59 ` Jason Gunthorpe
2015-05-26 17:46 ` Doug Ledford
[not found] ` <1432662396.28905.157.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-05-26 18:47 ` Jason Gunthorpe
2015-05-28 13:22 ` Haggai Eran
2015-05-28 15:46 ` Jason Gunthorpe
[not found] ` <20150528154633.GB2962-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-06-03 10:07 ` Haggai Eran
2015-05-28 13:15 ` Haggai Eran
2015-05-26 17:55 ` Christian Benvenuti (benve)
2015-05-28 13:07 ` Haggai Eran
[not found] ` <55671309.6080303-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-05-28 14:07 ` Doug Ledford
[not found] ` <1432822057.114391.26.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-05-28 16:21 ` Or Gerlitz
[not found] ` <55674077.5040707-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-05-28 17:43 ` Jason Gunthorpe
[not found] ` <20150528174337.GA10448-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-05-28 18:22 ` Doug Ledford
[not found] ` <1432837360.114391.35.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-05-28 19:05 ` Or Gerlitz
[not found] ` <CAJ3xEMh2T5-56rFxWVdct2uAZYW1ZrKivWfS45V-mvhAfwyGaA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-05-28 21:55 ` Doug Ledford
[not found] ` <1432850150.114391.56.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-06-03 10:03 ` Haggai Eran
2015-06-03 16:14 ` Jason Gunthorpe
[not found] ` <20150603161447.GC12073-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-06-03 19:05 ` Or Gerlitz
2015-06-03 19:53 ` Jason Gunthorpe
[not found] ` <20150603195325.GC7902-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-06-03 20:07 ` Or Gerlitz
[not found] ` <CAJ3xEMiO+hEzOJ2oJ5G-mmBeaX4ZHvUyhNSAzsrRDui6dFjvCg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-06-03 21:45 ` Jason Gunthorpe
2015-06-04 9:41 ` Haggai Eran
2015-06-04 16:06 ` Jason Gunthorpe
2015-06-03 23:48 ` Jason Gunthorpe
[not found] ` <20150603234811.GA15128-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-06-04 6:24 ` Haggai Eran
[not found] ` <556FEF25.80409-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-06-04 16:40 ` Jason Gunthorpe
[not found] ` <20150604164058.GB27699-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-06-08 7:52 ` Haggai Eran
2015-06-08 16:53 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1432045826.5304.6.camel@opteya.com \
--to=ydroneaud@opteya$(echo .)com \
--cc=dledford@redhat$(echo .)com \
--cc=guysh@mellanox$(echo .)com \
--cc=haggaie@mellanox$(echo .)com \
--cc=linux-rdma@vger$(echo .)kernel.org \
--cc=liranl@mellanox$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
--cc=raindel@mellanox$(echo .)com \
--cc=yotamke@mellanox$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox