From: Yury Norov <yury.norov@gmail•com>
To: Souradeep Chakrabarti <schakrabarti@linux•microsoft.com>
Cc: kys@microsoft•com, haiyangz@microsoft•com, wei.liu@kernel•org,
decui@microsoft•com, davem@davemloft•net, edumazet@google•com,
kuba@kernel•org, pabeni@redhat•com, longli@microsoft•com,
leon@kernel•org, cai.huoqing@linux•dev,
ssengar@linux•microsoft.com, vkuznets@redhat•com,
tglx@linutronix•de, linux-hyperv@vger•kernel.org,
netdev@vger•kernel.org, linux-kernel@vger•kernel.org,
linux-rdma@vger•kernel.org, schakrabarti@microsoft•com,
paulros@microsoft•com
Subject: Re: [PATCH 4/4 V2 net-next] net: mana: Assigning IRQ affinity on HT cores
Date: Wed, 24 Jan 2024 07:20:27 -0800 [thread overview]
Message-ID: <ZbEqu68J3f9W8nIM@yury-ThinkPad> (raw)
In-Reply-To: <1705939259-2859-5-git-send-email-schakrabarti@linux.microsoft.com>
On Mon, Jan 22, 2024 at 08:00:59AM -0800, Souradeep Chakrabarti wrote:
> Existing MANA design assigns IRQ to every CPU, including sibling
> hyper-threads. This may cause multiple IRQs to be active simultaneously
> in the same core and may reduce the network performance.
>
> Improve the performance by assigning IRQ to non sibling CPUs in local
> NUMA node. The performance improvement we are getting using ntttcp with
> following patch is around 15 percent against existing design and
> approximately 11 percent, when trying to assign one IRQ in each core
> across NUMA nodes, if enough cores are present.
> The change will improve the performance for the system
> with high number of CPU, where number of CPUs in a node is more than
> 64 CPUs. Nodes with 64 CPUs or less than 64 CPUs will not be affected
> by this change.
>
> The performance study was done using ntttcp tool in Azure.
> The node had 2 nodes with 32 cores each, total 128 vCPU and number of channels
> were 32 for 32 RX rings.
>
> The below table shows a comparison between existing design and new
> design:
>
> IRQ node-num core-num CPU performance(%)
> 1 0 | 0 0 | 0 0 | 0-1 0
> 2 0 | 0 0 | 1 1 | 2-3 3
> 3 0 | 0 1 | 2 2 | 4-5 10
> 4 0 | 0 1 | 3 3 | 6-7 15
> 5 0 | 0 2 | 4 4 | 8-9 15
> ---
> ---
> 25 0 | 0 12| 24 24| 48-49 12
> ---
> 32 0 | 0 15| 31 31| 62-63 12
> 33 0 | 0 16| 0 32| 0-1 10
> ---
> 64 0 | 0 31| 31 63| 62-63 0
Did that omitted lines mean 5-24 : 15%, 25-31 : 12% and 33-63 : 10%?
Or that means that you didn't test those?
Would be nice to have full coverage...
Thanks,
Yury
next prev parent reply other threads:[~2024-01-24 15:20 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-22 16:00 [PATCH 0/4 V2 net-next] net: mana: Assigning IRQ affinity on HT cores Souradeep Chakrabarti
2024-01-22 16:00 ` [PATCH 1/4 V2 net-next] cpumask: add cpumask_weight_andnot() Souradeep Chakrabarti
2024-01-22 16:00 ` [PATCH 2/4 V2 net-next] cpumask: define cleanup function for cpumasks Souradeep Chakrabarti
2024-01-22 16:00 ` [PATCH 3/4 V2 net-next] net: mana: add a function to spread IRQs per CPUs Souradeep Chakrabarti
2024-01-22 16:00 ` [PATCH 4/4 V2 net-next] net: mana: Assigning IRQ affinity on HT cores Souradeep Chakrabarti
2024-01-24 1:03 ` Jakub Kicinski
2024-01-25 6:05 ` Souradeep Chakrabarti
2024-01-24 15:20 ` Yury Norov [this message]
2024-01-25 6:07 ` Souradeep Chakrabarti
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZbEqu68J3f9W8nIM@yury-ThinkPad \
--to=yury.norov@gmail$(echo .)com \
--cc=cai.huoqing@linux$(echo .)dev \
--cc=davem@davemloft$(echo .)net \
--cc=decui@microsoft$(echo .)com \
--cc=edumazet@google$(echo .)com \
--cc=haiyangz@microsoft$(echo .)com \
--cc=kuba@kernel$(echo .)org \
--cc=kys@microsoft$(echo .)com \
--cc=leon@kernel$(echo .)org \
--cc=linux-hyperv@vger$(echo .)kernel.org \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=linux-rdma@vger$(echo .)kernel.org \
--cc=longli@microsoft$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
--cc=pabeni@redhat$(echo .)com \
--cc=paulros@microsoft$(echo .)com \
--cc=schakrabarti@linux$(echo .)microsoft.com \
--cc=schakrabarti@microsoft$(echo .)com \
--cc=ssengar@linux$(echo .)microsoft.com \
--cc=tglx@linutronix$(echo .)de \
--cc=vkuznets@redhat$(echo .)com \
--cc=wei.liu@kernel$(echo .)org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox