From: "Michael S. Tsirkin" <mst@redhat•com>
To: Leon Romanovsky <leon@kernel•org>
Cc: "Dragos Tatulea" <dtatulea@nvidia•com>,
"Saeed Mahameed" <saeedm@nvidia•com>,
"Tariq Toukan" <tariqt@nvidia•com>,
"David S. Miller" <davem@davemloft•net>,
"Eric Dumazet" <edumazet@google•com>,
"Jakub Kicinski" <kuba@kernel•org>,
"Paolo Abeni" <pabeni@redhat•com>,
"Jason Wang" <jasowang@redhat•com>,
"Xuan Zhuo" <xuanzhuo@linux•alibaba.com>,
"Eugenio Pérez" <eperezma@redhat•com>,
"Si-Wei Liu" <si-wei.liu@oracle•com>,
netdev@vger•kernel.org, linux-rdma@vger•kernel.org,
linux-kernel@vger•kernel.org, virtualization@lists•linux.dev
Subject: Re: [PATCH vhost 0/7] vdpa/mlx5: Parallelize device suspend/resume
Date: Sun, 4 Aug 2024 09:39:29 -0400 [thread overview]
Message-ID: <20240804093909-mutt-send-email-mst@kernel.org> (raw)
In-Reply-To: <20240804084839.GA22826@unreal>
On Sun, Aug 04, 2024 at 11:48:39AM +0300, Leon Romanovsky wrote:
> On Fri, Aug 02, 2024 at 09:14:28AM -0400, Michael S. Tsirkin wrote:
> > On Fri, Aug 02, 2024 at 10:20:17AM +0300, Dragos Tatulea wrote:
> > > This series parallelizes the mlx5_vdpa device suspend and resume
> > > operations through the firmware async API. The purpose is to reduce live
> > > migration downtime.
> > >
> > > The series starts with changing the VQ suspend and resume commands
> > > to the async API. After that, the switch is made to issue multiple
> > > commands of the same type in parallel.
> > >
> > > Finally, a bonus improvement is thrown in: keep the notifierd enabled
> > > during suspend but make it a NOP. Upon resume make sure that the link
> > > state is forwarded. This shaves around 30ms per device constant time.
> > >
> > > For 1 vDPA device x 32 VQs (16 VQPs), on a large VM (256 GB RAM, 32 CPUs
> > > x 2 threads per core), the improvements are:
> > >
> > > +-------------------+--------+--------+-----------+
> > > | operation | Before | After | Reduction |
> > > |-------------------+--------+--------+-----------|
> > > | mlx5_vdpa_suspend | 37 ms | 2.5 ms | 14x |
> > > | mlx5_vdpa_resume | 16 ms | 5 ms | 3x |
> > > +-------------------+--------+--------+-----------+
> > >
> > > Note for the maintainers:
> > > The first patch contains changes for mlx5_core. This must be applied
> > > into the mlx5-vhost tree [0] first. Once this patch is applied on
> > > mlx5-vhost, the change has to be pulled from mlx5-vdpa into the vhost
> > > tree and only then the remaining patches can be applied.
> >
> > Or maintainer just acks it and I apply directly.
>
> We can do it, but there is a potential to create a conflict between your tree
> and netdev for whole cycle, which will be a bit annoying. Easiest way to avoid
> this is to have a shared branch, but in august everyone is on vacation, so it
> will be probably fine to apply such patch directly.
>
> Thanks
We can let Linus do something, it's ok ;)
> >
> > Let me know when all this can happen.
> >
> > > [0] https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-vhost
> > >
> > > Dragos Tatulea (7):
> > > net/mlx5: Support throttled commands from async API
> > > vdpa/mlx5: Introduce error logging function
> > > vdpa/mlx5: Use async API for vq query command
> > > vdpa/mlx5: Use async API for vq modify commands
> > > vdpa/mlx5: Parallelize device suspend
> > > vdpa/mlx5: Parallelize device resume
> > > vdpa/mlx5: Keep notifiers during suspend but ignore
> > >
> > > drivers/net/ethernet/mellanox/mlx5/core/cmd.c | 21 +-
> > > drivers/vdpa/mlx5/core/mlx5_vdpa.h | 7 +
> > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 435 +++++++++++++-----
> > > 3 files changed, 333 insertions(+), 130 deletions(-)
> > >
> > > --
> > > 2.45.2
> >
> >
next prev parent reply other threads:[~2024-08-04 13:39 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-08-02 7:20 [PATCH vhost 0/7] vdpa/mlx5: Parallelize device suspend/resume Dragos Tatulea
2024-08-02 7:20 ` [PATCH mlx5-vhost 1/7] net/mlx5: Support throttled commands from async API Dragos Tatulea
2024-08-02 13:14 ` [PATCH vhost 0/7] vdpa/mlx5: Parallelize device suspend/resume Michael S. Tsirkin
2024-08-04 8:48 ` Leon Romanovsky
2024-08-04 13:39 ` Michael S. Tsirkin [this message]
2024-08-04 14:52 ` Leon Romanovsky
2024-08-16 9:13 ` Dragos Tatulea
2024-08-07 13:25 ` Eugenio Perez Martin
2024-08-07 14:54 ` Dragos Tatulea
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240804093909-mutt-send-email-mst@kernel.org \
--to=mst@redhat$(echo .)com \
--cc=davem@davemloft$(echo .)net \
--cc=dtatulea@nvidia$(echo .)com \
--cc=edumazet@google$(echo .)com \
--cc=eperezma@redhat$(echo .)com \
--cc=jasowang@redhat$(echo .)com \
--cc=kuba@kernel$(echo .)org \
--cc=leon@kernel$(echo .)org \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=linux-rdma@vger$(echo .)kernel.org \
--cc=netdev@vger$(echo .)kernel.org \
--cc=pabeni@redhat$(echo .)com \
--cc=saeedm@nvidia$(echo .)com \
--cc=si-wei.liu@oracle$(echo .)com \
--cc=tariqt@nvidia$(echo .)com \
--cc=virtualization@lists$(echo .)linux.dev \
--cc=xuanzhuo@linux$(echo .)alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox