Subject: Re: [PATCH net-next V2 2/4] net/mlx5: Implement devlink port function cmds to control ipsec_crypto

From: Leon Romanovsky <[email-masked]>
Date: Sun, 23 Apr 2023 19:30:55 +0300

On Fri, Apr 21, 2023 at 01:48:59PM +0300, Dima Chumak wrote:
> Implement devlink port function commands to enable / disable IPsec
> crypto offloads.  This is used to control the IPsec capability of the
> device.
> 
> When ipsec_crypto is enabled for a VF, it prevents adding IPsec crypto
> offloads on the PF, because the two cannot be active simultaneously due
> to HW constraints. Conversely, if there are any active IPsec crypto
> offloads on the PF, it's not allowed to enable ipsec_crypto on a VF,
> until PF IPsec offloads are cleared.
> 
> Signed-off-by: Dima Chumak <[email-masked]>
> ---
> v1 -> v2:
>  - Fix build when CONFIG_XFRM is not set.
>  - Fix switchdev mode init for HW that doesn't have ipsec_offload
>    capability
>  - Perform additional capability checks to test if ipsec_crypto offload
>    is supported by the HW
> ---
>  .../ethernet/mellanox/mlx5/switchdev.rst      |   8 +
>  .../net/ethernet/mellanox/mlx5/core/Makefile  |   2 +-
>  .../net/ethernet/mellanox/mlx5/core/devlink.c |   6 +-
>  .../mellanox/mlx5/core/en_accel/ipsec.c       |  18 +
>  .../ethernet/mellanox/mlx5/core/esw/ipsec.c   | 329 ++++++++++++++++++
>  .../net/ethernet/mellanox/mlx5/core/eswitch.c |  29 ++
>  .../net/ethernet/mellanox/mlx5/core/eswitch.h |  23 ++
>  .../mellanox/mlx5/core/eswitch_offloads.c     | 105 ++++++
>  .../ethernet/mellanox/mlx5/core/lib/ipsec.h   |  41 +++
>  include/linux/mlx5/driver.h                   |   1 +
>  include/linux/mlx5/mlx5_ifc.h                 |   3 +
>  11 files changed, 563 insertions(+), 2 deletions(-)
>  create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/esw/ipsec.c
>  create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/lib/ipsec.h

<...>

> @@ -622,6 +624,7 @@ static int mlx5e_xfrm_add_state(struct xfrm_state *x,
>  	struct mlx5e_ipsec_sa_entry *sa_entry = NULL;
>  	struct net_device *netdev = x->xso.real_dev;
>  	struct mlx5e_ipsec *ipsec;
> +	struct mlx5_eswitch *esw;
>  	struct mlx5e_priv *priv;
>  	gfp_t gfp;
>  	int err;
> @@ -646,6 +649,11 @@ static int mlx5e_xfrm_add_state(struct xfrm_state *x,
>  	if (err)
>  		goto err_xfrm;
>  
> +	esw = priv->mdev->priv.eswitch;
> +	if (esw && mlx5_esw_vport_ipsec_offload_enabled(esw))
> +		return -EBUSY;
> +	mlx5_eswitch_ipsec_offloads_count_inc(priv->mdev);

<...>

> +void mlx5_esw_vport_ipsec_offload_enable(struct mlx5_eswitch *esw)
> +{
> +	esw->enabled_ipsec_vf_count++;
> +	WARN_ON(!esw->enabled_ipsec_vf_count);
> +}
> +
> +void mlx5_esw_vport_ipsec_offload_disable(struct mlx5_eswitch *esw)
> +{
> +	esw->enabled_ipsec_vf_count--;
> +	WARN_ON(esw->enabled_ipsec_vf_count == U16_MAX);
> +}
> +
> +bool mlx5_esw_vport_ipsec_offload_enabled(struct mlx5_eswitch *esw)
> +{
> +	return !!esw->enabled_ipsec_vf_count;
> +}

I afraid that without proper locking everything above is racy.

We are storing all offloaded SAs in Xarray DB, so you can simply check
if that DB is empty or not by calling to xa_empty(). However, it will be
not an easy task to make proper locking.

So I would expect to see here something close
to mlx5_eswitch_block_encap/mlx5_eswitch_unblock_encap, which take devlink and
eswitch locks in the right order.

Thanks