From: Prarit Bhargava <prarit@redhat•com>
To: Alexander Duyck <alexander.h.duyck@intel•com>
Cc: nhorman@redhat•com, e1000-devel@lists•sourceforge.net,
netdev@vger•kernel.org, Bruce Allan <bruce.w.allan@intel•com>,
Jesse Brandeburg <jesse.brandeburg@intel•com>,
agospoda@redhat•com, John Ronciak <john.ronciak@intel•com>,
"David S. Miller" <davem@davemloft•net>
Subject: Re: [PATCH 1/2] ixgbe, make interrupt allocations NUMA aware
Date: Mon, 24 Feb 2014 14:39:58 -0500 [thread overview]
Message-ID: <530BA00E.4070802@redhat.com> (raw)
In-Reply-To: <530B9CE8.2070800@intel.com>
On 02/24/2014 02:26 PM, Alexander Duyck wrote:
> On 02/24/2014 10:51 AM, Prarit Bhargava wrote:
>> The ixgbe driver creates one queue/cpu on the system in order to spread
>> work out on all cpus rather than restricting work to a single cpu. This
>> model, while efficient, does not take into account the NUMA configuration
>> of the system.
>>
>> This patch introduces ixgbe_num_cpus() which returns
>> the number of online cpus if the adapter's PCI device has no NUMA
>> restrictions, and the number of cpus in the node if the PCI device is
>> allocated to a specific node.
>>
>> Signed-off-by: Prarit Bhargava <prarit@redhat•com>
>> Cc: Jeff Kirsher <jeffrey.t.kirsher@intel•com>
>> Cc: Jesse Brandeburg <jesse.brandeburg@intel•com>
>> Cc: Bruce Allan <bruce.w.allan@intel•com>
>> Cc: Carolyn Wyborny <carolyn.wyborny@intel•com>
>> Cc: Don Skidmore <donald.c.skidmore@intel•com>
>> Cc: Greg Rose <gregory.v.rose@intel•com>
>> Cc: Alex Duyck <alexander.h.duyck@intel•com>
>> Cc: John Ronciak <john.ronciak@intel•com>
>> Cc: Mitch Williams <mitch.a.williams@intel•com>
>> Cc: "David S. Miller" <davem@davemloft•net>
>> Cc: nhorman@redhat•com
>> Cc: agospoda@redhat•com
>> Cc: e1000-devel@lists•sourceforge.net
>> ---
>> drivers/net/ethernet/intel/ixgbe/ixgbe.h | 2 ++
>> drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c | 28 +++++++++++++++++++++---
>> drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 6 ++---
>> drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c | 5 +++--
>> 4 files changed, 33 insertions(+), 8 deletions(-)
>>
>
> [...]
>
>> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
>> index 18076c4..b68a6e9 100644
>> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
>> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
>> @@ -4953,13 +4953,13 @@ static int ixgbe_sw_init(struct ixgbe_adapter *adapter)
>> hw->subsystem_device_id = pdev->subsystem_device;
>>
>> /* Set common capability flags and settings */
>> - rss = min_t(int, IXGBE_MAX_RSS_INDICES, num_online_cpus());
>> + rss = min_t(int, IXGBE_MAX_RSS_INDICES, ixgbe_num_cpus(adapter));
>> adapter->ring_feature[RING_F_RSS].limit = rss;
>> adapter->flags2 |= IXGBE_FLAG2_RSC_CAPABLE;
>> adapter->flags2 |= IXGBE_FLAG2_RSC_ENABLED;
>> adapter->max_q_vectors = MAX_Q_VECTORS_82599;
>> adapter->atr_sample_rate = 20;
>> - fdir = min_t(int, IXGBE_MAX_FDIR_INDICES, num_online_cpus());
>> + fdir = min_t(int, IXGBE_MAX_FDIR_INDICES, ixgbe_num_cpus(adapter));
>> adapter->ring_feature[RING_F_FDIR].limit = fdir;
>> adapter->fdir_pballoc = IXGBE_FDIR_PBALLOC_64K;
>> #ifdef CONFIG_IXGBE_DCA
>
> This is the one bit I object to in this patch. The flow director queue
> count should be equal to the number of online CPUs, or at least as close
> to it as the hardware can get. Otherwise ATR is completely useless.
I'm reading up on ATR now and I see your point completely. I will remove this
chunk in V2. OOC, however, what about my concern with ATR & the location of the
PCI device (on a different root bridge)? Isn't that a concern with ATR or am I
missing something with the overall scheme of ATR?
P.
>
> Thanks,
>
> Alex
------------------------------------------------------------------------------
Flow-based real-time traffic analytics software. Cisco certified tool.
Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
Customize your own dashboards, set traffic alerts and generate reports.
Network behavioral analysis & security monitoring. All-in-one tool.
http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
_______________________________________________
E1000-devel mailing list
E1000-devel@lists•sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
next prev parent reply other threads:[~2014-02-24 19:39 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-02-24 18:51 [PATCH 0/2] ixgbe, fix numa issues Prarit Bhargava
2014-02-24 18:51 ` [PATCH 1/2] ixgbe, make interrupt allocations NUMA aware Prarit Bhargava
2014-02-24 19:26 ` Alexander Duyck
2014-02-24 19:39 ` Prarit Bhargava [this message]
2014-02-24 19:49 ` Alexander Duyck
2014-02-24 18:51 ` [PATCH 2/2] ixgbe, don't assume mapping of numa node cpus Prarit Bhargava
2014-02-24 19:39 ` Alexander Duyck
2014-02-25 17:27 ` Amir Vadai
2014-02-25 17:43 ` Prarit Bhargava
2014-02-24 19:23 ` [PATCH 0/2] ixgbe, fix numa issues Alexander Duyck
2014-02-24 19:34 ` Prarit Bhargava
2014-02-24 19:57 ` Alexander Duyck
2014-02-25 1:06 ` Prarit Bhargava
2014-02-25 10:21 ` David Laight
2014-02-25 11:00 ` Prarit Bhargava
2014-02-25 15:10 ` Alexander Duyck
2014-02-25 15:13 ` Prarit Bhargava
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=530BA00E.4070802@redhat.com \
--to=prarit@redhat$(echo .)com \
--cc=agospoda@redhat$(echo .)com \
--cc=alexander.h.duyck@intel$(echo .)com \
--cc=bruce.w.allan@intel$(echo .)com \
--cc=davem@davemloft$(echo .)net \
--cc=e1000-devel@lists$(echo .)sourceforge.net \
--cc=jesse.brandeburg@intel$(echo .)com \
--cc=john.ronciak@intel$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
--cc=nhorman@redhat$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox