From: Yury Norov <ynorov@nvidia•com>
To: Shradha Gupta <shradhagupta@linux•microsoft.com>
Cc: Dexuan Cui <decui@microsoft•com>, Wei Liu <wei.liu@kernel•org>,
Haiyang Zhang <haiyangz@microsoft•com>,
"K. Y. Srinivasan" <kys@microsoft•com>,
Andrew Lunn <andrew+netdev@lunn•ch>,
"David S. Miller" <davem@davemloft•net>,
Eric Dumazet <edumazet@google•com>,
Jakub Kicinski <kuba@kernel•org>, Paolo Abeni <pabeni@redhat•com>,
Konstantin Taranov <kotaranov@microsoft•com>,
Simon Horman <horms@kernel•org>,
Erni Sri Satya Vennela <ernis@linux•microsoft.com>,
Dipayaan Roy <dipayanroy@linux•microsoft.com>,
Shiraz Saleem <shirazsaleem@microsoft•com>,
Michael Kelley <mhklinux@outlook•com>,
Long Li <longli@microsoft•com>, Yury Norov <yury.norov@gmail•com>,
linux-hyperv@vger•kernel.org, linux-kernel@vger•kernel.org,
netdev@vger•kernel.org, Paul Rosswurm <paulros@microsoft•com>,
Shradha Gupta <shradhagupta@microsoft•com>,
Saurabh Singh Sengar <ssengar@microsoft•com>,
stable@vger•kernel.org
Subject: Re: [PATCH net v2] net: mana: Optimize irq affinity for low vcpu configs
Date: Sun, 10 May 2026 16:00:46 -0400 [thread overview]
Message-ID: <af4X_52txN28b9RV@yury> (raw)
In-Reply-To: <af15yfdotzVbK8Kb@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net>
On Thu, May 07, 2026 at 10:51:05PM -0700, Shradha Gupta wrote:
...
> > > We can definately get our throughput run results on other suggestions
> > > you have. And about that, I just needed a bit more clarity on what to
> > > test against. Are you suggesting, with irq_setup() intact and in use, we
> > > configure the non-mana IRQs to say CPU0 and capture the numbers?
> >
> > Can you try this:
> >
> > while(len--)
> > // Or cpu_online_mask or cpu_all_mask?
> > irq_set_affinity_and_hint(*irqs++, NULL);
> >
> > And compare it to the linear version under your vCPU scenario?
> >
> > Can you run your throughput test alone and on parallel with some
> > IRQ torture test?
> >
> > stress-ng --timer 4 --timeout 60s
> >
> > And maybe pin the stress test to the default CPU. Assuming it's 0:
> >
> > taskset -c 0 stress-ng --timer 4 --timeout 60s
> >
> > Unless the 'linear' version is significantly faster, I'd stick to the
> > above.
> >
> > Thanks,
> > Yury
>
> Hey Yury,
>
> We tried a few tests with your suggestion, and throughput seems to be
> the same compared to the linear distribution approach. We stressed out
> CPU0 in both the cases and the results were similar. No IRQ migration
> was observed in either case and no throughput drop.
>
> But one observation I had was that " irq_set_affinity_and_hint(*irqs++,
> NULL);" is essentially a no-op and we end up relying on the initial
> placement from pci_alloc_irq_vectors().
Yes you are, assuming you're not binding them before in your call chain.
> Even though in these tests we
> were not able to reproduce it, but with this distribution there is a
> chance we end up clustering the mana queue IRQs, while other vCPUs are
> not running any network load.
That sounds like an IRQ balancer bug which you're unable to reproduce.
> It's because the placement depends on
> system-wide IRQ state at allocation time.
I don't understand this point. The
irq_set_affinity_and_hint(*irqs++, NULL);
simply means: I trust system IRQ balancer to pick the best CPU for my
IRQ at runtime. It doesn't refer any "IRQ state at allocation time".
> The linear approach however gaurantees each queue IRQ lands on a
> distinct vCPU regardless of system state. Even after stressing the cpus
> using stress-ng, we did not observe any significant throughput drop.
If you just do nothing, it would lead to the same numbers, right? What
does that "non-significant throughput drop" mean? It sounds like the
linear approach is slightly worse.
--
So, as you can't demonstrate solid benefit for the 'linear' IRQ placement,
I would just stick to the no-affinity logic.
Thanks,
Yury
next prev parent reply other threads:[~2026-05-10 20:00 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-04-29 9:06 [PATCH net v2] net: mana: Optimize irq affinity for low vcpu configs Shradha Gupta
2026-05-01 9:12 ` Simon Horman
2026-05-01 16:22 ` Yury Norov
2026-05-02 14:37 ` Shradha Gupta
2026-05-02 17:15 ` Yury Norov
2026-05-05 6:15 ` Shradha Gupta
2026-05-05 15:43 ` Yury Norov
2026-05-08 5:51 ` Shradha Gupta
2026-05-10 20:00 ` Yury Norov [this message]
2026-05-18 7:04 ` Erni Sri Satya Vennela
2026-05-26 13:05 ` Shradha Gupta
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=af4X_52txN28b9RV@yury \
--to=ynorov@nvidia$(echo .)com \
--cc=andrew+netdev@lunn$(echo .)ch \
--cc=davem@davemloft$(echo .)net \
--cc=decui@microsoft$(echo .)com \
--cc=dipayanroy@linux$(echo .)microsoft.com \
--cc=edumazet@google$(echo .)com \
--cc=ernis@linux$(echo .)microsoft.com \
--cc=haiyangz@microsoft$(echo .)com \
--cc=horms@kernel$(echo .)org \
--cc=kotaranov@microsoft$(echo .)com \
--cc=kuba@kernel$(echo .)org \
--cc=kys@microsoft$(echo .)com \
--cc=linux-hyperv@vger$(echo .)kernel.org \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=longli@microsoft$(echo .)com \
--cc=mhklinux@outlook$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
--cc=pabeni@redhat$(echo .)com \
--cc=paulros@microsoft$(echo .)com \
--cc=shirazsaleem@microsoft$(echo .)com \
--cc=shradhagupta@linux$(echo .)microsoft.com \
--cc=shradhagupta@microsoft$(echo .)com \
--cc=ssengar@microsoft$(echo .)com \
--cc=stable@vger$(echo .)kernel.org \
--cc=wei.liu@kernel$(echo .)org \
--cc=yury.norov@gmail$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox