* [PATCH] igb: add module param to set max-rss-queues.
@ 2017-03-24 20:58 greearb
2017-03-24 21:12 ` David Miller
2017-03-27 9:26 ` David Laight
0 siblings, 2 replies; 7+ messages in thread
From: greearb @ 2017-03-24 20:58 UTC (permalink / raw)
To: netdev; +Cc: Ben Greear
From: Ben Greear <greearb@candelatech•com>
In systems where you may have a very large number of network
adapters, certain drivers may consume an unfair amount of
IRQ resources. So, allow a module param that will limit the
number of IRQs at driver load time. This way, other drivers
(40G Ethernet, for instance), which probably will need the
multiple IRQs more, will not be starved of IRQ resources.
Signed-off-by: Ben Greear <greearb@candelatech•com>
---
drivers/net/ethernet/intel/igb/igb_main.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index f9f2874..fdb12e0 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -218,6 +218,10 @@ module_param(max_vfs, uint, 0);
MODULE_PARM_DESC(max_vfs, "Maximum number of virtual functions to allocate per physical function");
#endif /* CONFIG_PCI_IOV */
+static unsigned int max_rss_qs;
+module_param(max_rss_qs, uint, 0);
+MODULE_PARM_DESC(max_rss_qs, "Maximum number of RSS queues. Forcing lower will use less IRQ resources.");
+
static pci_ers_result_t igb_io_error_detected(struct pci_dev *,
pci_channel_state_t);
static pci_ers_result_t igb_io_slot_reset(struct pci_dev *);
@@ -2993,6 +2997,9 @@ static void igb_init_queue_configuration(struct igb_adapter *adapter)
break;
}
+ if (max_rss_qs && max_rss_qs < max_rss_queues)
+ max_rss_queues = max_rss_qs;
+
adapter->rss_queues = min_t(u32, max_rss_queues, num_online_cpus());
igb_set_flag_queue_pairs(adapter, max_rss_queues);
--
2.4.11
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] igb: add module param to set max-rss-queues.
2017-03-24 20:58 [PATCH] igb: add module param to set max-rss-queues greearb
@ 2017-03-24 21:12 ` David Miller
2017-03-24 21:20 ` Ben Greear
2017-03-27 9:26 ` David Laight
1 sibling, 1 reply; 7+ messages in thread
From: David Miller @ 2017-03-24 21:12 UTC (permalink / raw)
To: greearb; +Cc: netdev
From: greearb@candelatech•com
Date: Fri, 24 Mar 2017 13:58:47 -0700
> From: Ben Greear <greearb@candelatech•com>
>
> In systems where you may have a very large number of network
> adapters, certain drivers may consume an unfair amount of
> IRQ resources. So, allow a module param that will limit the
> number of IRQs at driver load time. This way, other drivers
> (40G Ethernet, for instance), which probably will need the
> multiple IRQs more, will not be starved of IRQ resources.
>
> Signed-off-by: Ben Greear <greearb@candelatech•com>
Sorry, no module params.
Use generic run-time facilities such as ethtool to configure
such things.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] igb: add module param to set max-rss-queues.
2017-03-24 21:12 ` David Miller
@ 2017-03-24 21:20 ` Ben Greear
2017-03-24 21:40 ` David Miller
2017-03-24 23:14 ` Stephen Hemminger
0 siblings, 2 replies; 7+ messages in thread
From: Ben Greear @ 2017-03-24 21:20 UTC (permalink / raw)
To: David Miller; +Cc: netdev
On 03/24/2017 02:12 PM, David Miller wrote:
> From: greearb@candelatech•com
> Date: Fri, 24 Mar 2017 13:58:47 -0700
>
>> From: Ben Greear <greearb@candelatech•com>
>>
>> In systems where you may have a very large number of network
>> adapters, certain drivers may consume an unfair amount of
>> IRQ resources. So, allow a module param that will limit the
>> number of IRQs at driver load time. This way, other drivers
>> (40G Ethernet, for instance), which probably will need the
>> multiple IRQs more, will not be starved of IRQ resources.
>>
>> Signed-off-by: Ben Greear <greearb@candelatech•com>
>
> Sorry, no module params.
>
> Use generic run-time facilities such as ethtool to configure
> such things.
You cannot call ethtool before module load time, and that is when
the IRQs are first acquired. It may be way more useful to give each
of 20 network adapters 2 irqs than have the first few grab 16 and the rest
get lumped into legacy crap.
Thanks,
Ben
--
Ben Greear <greearb@candelatech•com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] igb: add module param to set max-rss-queues.
2017-03-24 21:20 ` Ben Greear
@ 2017-03-24 21:40 ` David Miller
2017-03-24 23:14 ` Stephen Hemminger
1 sibling, 0 replies; 7+ messages in thread
From: David Miller @ 2017-03-24 21:40 UTC (permalink / raw)
To: greearb; +Cc: netdev
From: Ben Greear <greearb@candelatech•com>
Date: Fri, 24 Mar 2017 14:20:56 -0700
> On 03/24/2017 02:12 PM, David Miller wrote:
>> From: greearb@candelatech•com
>> Date: Fri, 24 Mar 2017 13:58:47 -0700
>>
>>> From: Ben Greear <greearb@candelatech•com>
>>>
>>> In systems where you may have a very large number of network
>>> adapters, certain drivers may consume an unfair amount of
>>> IRQ resources. So, allow a module param that will limit the
>>> number of IRQs at driver load time. This way, other drivers
>>> (40G Ethernet, for instance), which probably will need the
>>> multiple IRQs more, will not be starved of IRQ resources.
>>>
>>> Signed-off-by: Ben Greear <greearb@candelatech•com>
>>
>> Sorry, no module params.
>>
>> Use generic run-time facilities such as ethtool to configure
>> such things.
>
> You cannot call ethtool before module load time, and that is when
> the IRQs are first acquired. It may be way more useful to give each
> of 20 network adapters 2 irqs than have the first few grab 16 and the
> rest
> get lumped into legacy crap.
Sorry, you'll have to find a way to fix this without a module
parameter. I understand it might not be easy, but that's not
a reason to add countless driver private module parameters all
over the tree which is the worst user experience possible.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] igb: add module param to set max-rss-queues.
2017-03-24 21:20 ` Ben Greear
2017-03-24 21:40 ` David Miller
@ 2017-03-24 23:14 ` Stephen Hemminger
2017-03-25 1:35 ` Ben Greear
1 sibling, 1 reply; 7+ messages in thread
From: Stephen Hemminger @ 2017-03-24 23:14 UTC (permalink / raw)
To: Ben Greear; +Cc: David Miller, netdev
On Fri, 24 Mar 2017 14:20:56 -0700
Ben Greear <greearb@candelatech•com> wrote:
> On 03/24/2017 02:12 PM, David Miller wrote:
> > From: greearb@candelatech•com
> > Date: Fri, 24 Mar 2017 13:58:47 -0700
> >
> >> From: Ben Greear <greearb@candelatech•com>
> >>
> >> In systems where you may have a very large number of network
> >> adapters, certain drivers may consume an unfair amount of
> >> IRQ resources. So, allow a module param that will limit the
> >> number of IRQs at driver load time. This way, other drivers
> >> (40G Ethernet, for instance), which probably will need the
> >> multiple IRQs more, will not be starved of IRQ resources.
> >>
> >> Signed-off-by: Ben Greear <greearb@candelatech•com>
> >
> > Sorry, no module params.
> >
> > Use generic run-time facilities such as ethtool to configure
> > such things.
>
> You cannot call ethtool before module load time, and that is when
> the IRQs are first acquired. It may be way more useful to give each
> of 20 network adapters 2 irqs than have the first few grab 16 and the rest
> get lumped into legacy crap.
Almost all network devices do not acquire interrupts until device is brought up.
I.e request_irq is called from open not probe. This is done so that configuration
can be done and also so that unused ports don't consume interrupt space.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] igb: add module param to set max-rss-queues.
2017-03-24 23:14 ` Stephen Hemminger
@ 2017-03-25 1:35 ` Ben Greear
0 siblings, 0 replies; 7+ messages in thread
From: Ben Greear @ 2017-03-25 1:35 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: David Miller, netdev
On 03/24/2017 04:14 PM, Stephen Hemminger wrote:
> On Fri, 24 Mar 2017 14:20:56 -0700
> Ben Greear <greearb@candelatech•com> wrote:
>
>> On 03/24/2017 02:12 PM, David Miller wrote:
>>> From: greearb@candelatech•com
>>> Date: Fri, 24 Mar 2017 13:58:47 -0700
>>>
>>>> From: Ben Greear <greearb@candelatech•com>
>>>>
>>>> In systems where you may have a very large number of network
>>>> adapters, certain drivers may consume an unfair amount of
>>>> IRQ resources. So, allow a module param that will limit the
>>>> number of IRQs at driver load time. This way, other drivers
>>>> (40G Ethernet, for instance), which probably will need the
>>>> multiple IRQs more, will not be starved of IRQ resources.
>>>>
>>>> Signed-off-by: Ben Greear <greearb@candelatech•com>
>>>
>>> Sorry, no module params.
>>>
>>> Use generic run-time facilities such as ethtool to configure
>>> such things.
>>
>> You cannot call ethtool before module load time, and that is when
>> the IRQs are first acquired. It may be way more useful to give each
>> of 20 network adapters 2 irqs than have the first few grab 16 and the rest
>> get lumped into legacy crap.
>
> Almost all network devices do not acquire interrupts until device is brought up.
> I.e request_irq is called from open not probe. This is done so that configuration
> can be done and also so that unused ports don't consume interrupt space.
If I ever have to deal with this on stock kernels again I'll keep that in mind.
Thanks,
Ben
--
Ben Greear <greearb@candelatech•com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply [flat|nested] 7+ messages in thread
* RE: [PATCH] igb: add module param to set max-rss-queues.
2017-03-24 20:58 [PATCH] igb: add module param to set max-rss-queues greearb
2017-03-24 21:12 ` David Miller
@ 2017-03-27 9:26 ` David Laight
1 sibling, 0 replies; 7+ messages in thread
From: David Laight @ 2017-03-27 9:26 UTC (permalink / raw)
To: 'greearb@candelatech•com', netdev@vger•kernel.org
From: greearb@candelatech•com
> Sent: 24 March 2017 20:59
> From: Ben Greear <greearb@candelatech•com>
>
> In systems where you may have a very large number of network
> adapters, certain drivers may consume an unfair amount of
> IRQ resources. So, allow a module param that will limit the
> number of IRQs at driver load time. This way, other drivers
> (40G Ethernet, for instance), which probably will need the
> multiple IRQs more, will not be starved of IRQ resources.
ISTM that the functions for allocating MSI-X should allow a
driver to request additional interrupts after the original probe.
Then drivers than can use additional interrupts but get probed
early won't use up all the vectors before other drivers get
a chance.
David
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-03-27 9:27 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-03-24 20:58 [PATCH] igb: add module param to set max-rss-queues greearb
2017-03-24 21:12 ` David Miller
2017-03-24 21:20 ` Ben Greear
2017-03-24 21:40 ` David Miller
2017-03-24 23:14 ` Stephen Hemminger
2017-03-25 1:35 ` Ben Greear
2017-03-27 9:26 ` David Laight
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox