Subject: ksoftirqd and network processing

From: Anna Fischer <[email-masked]>
Date: Wed, 16 Mar 2016 20:57:33 +0100

Hi there,

I'd like to understand more in detail how ksoftirqd works in combination
with Linux kernel networking.

I'm doing IPSec processing on my Linux machine (kernel 4.4). On one
interface I receive encrypted packets which are decrypted and then
routed out plaintext (TCP) via another interface. Now I can
see that ksoftirqd runs at 100% CPU. Furthermore, I see that initially
it runs on core 0, then after a few seconds, it switches over to core 1
and runs at 100%, then switches over to core 2, and so on.

I have two questions:

1) I assume it switches cores because the kernel thread is scheduled on
different cores, depending on where the softirqs happen. Why does the
load never scale across cores? As far as I understand softirqs can run
concurrently on different cores. So why does it not scale?

2) Is it normal that ksoftirqd runs at such a high CPU load when doing
IPSec, if the core is a small core with small frequency (Intel Atoms,
for example)? And if so, is there any way to optimize this for IPSec?

3) When I pin RX queue interrupts of the NIC to certain cores, I can see
that interrupts are really only processed on that core. However, it does
not change anything about the ksoftirqd behavior. Should it not match,
how I route interrupts?

Many thanks,
Anna