From: Kiryl Shutsemau <kirill@shutemov•name>
To: Catalin Marinas <catalin.marinas@arm•com>,
Will Deacon <will@kernel•org>, James Morse <james.morse@arm•com>
Cc: Mark Rutland <mark.rutland@arm•com>,
Marc Zyngier <maz@kernel•org>,
Doug Anderson <dianders@chromium•org>,
Petr Mladek <pmladek@suse•com>,
Thomas Gleixner <tglx@linutronix•de>,
Andrew Morton <akpm@linux-foundation•org>,
Baoquan He <bhe@redhat•com>, Puranjay Mohan <puranjay@kernel•org>,
Usama Arif <usama.arif@linux•dev>,
Breno Leitao <leitao@debian•org>,
Julien Thierry <julien.thierry.kdev@gmail•com>,
Lecopzer Chen <lecopzer.chen@mediatek•com>,
Sumit Garg <sumit.garg@kernel•org>,
kernel-team@meta•com, kexec@lists•infradead.org,
linux-arm-kernel@lists•infradead.org,
linux-kernel@vger•kernel.org,
"Kiryl Shutsemau (Meta)" <kas@kernel•org>
Subject: [PATCH 2/4] drivers/firmware: add SDEI cross-CPU NMI service for arm64
Date: Wed, 3 Jun 2026 15:36:33 +0100 [thread overview]
Message-ID: <145b9e98b12a7d314fc4a203075f65c3a0c3a913.1780496779.git.kas@kernel.org> (raw)
In-Reply-To: <cover.1780496779.git.kas@kernel.org>
From: "Kiryl Shutsemau (Meta)" <kas@kernel•org>
Deliver an NMI-like event to an interrupt-masked arm64 CPU via the
standard SDEI software-signalled event (event 0), without the pseudo-NMI
hot-path cost: register a handler for event 0 and poke a target with
sdei_event_signal(0, mpidr).
First user is arch_trigger_cpumask_backtrace() (sysrq-l, RCU stalls,
hung-task/soft-lockup dumps), which otherwise rides an IPI that can't
reach a masked CPU. Falls back to the IPI path when SDEI is absent; no
watchdog backend yet, so the stock detector is untouched.
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel•org>
---
arch/arm64/include/asm/nmi.h | 24 ++++++
arch/arm64/kernel/smp.c | 9 +++
drivers/firmware/Kconfig | 19 +++++
drivers/firmware/Makefile | 1 +
drivers/firmware/sdei_nmi.c | 147 +++++++++++++++++++++++++++++++++++
5 files changed, 200 insertions(+)
create mode 100644 arch/arm64/include/asm/nmi.h
create mode 100644 drivers/firmware/sdei_nmi.c
diff --git a/arch/arm64/include/asm/nmi.h b/arch/arm64/include/asm/nmi.h
new file mode 100644
index 000000000000..ccdb75692e9d
--- /dev/null
+++ b/arch/arm64/include/asm/nmi.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ASM_NMI_H
+#define __ASM_NMI_H
+
+#include <linux/cpumask.h>
+
+/*
+ * Cross-CPU NMI provider hooks, consulted by the arm64 arch code before
+ * its regular-IRQ / pseudo-NMI IPI paths. The SDEI provider in
+ * drivers/firmware/sdei_nmi.c implements them when active; a future
+ * FEAT_NMI provider could slot in here too. The stubs let callers stay
+ * unconditional when ARM_SDEI_NMI is off.
+ */
+#ifdef CONFIG_ARM_SDEI_NMI
+bool sdei_nmi_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu);
+#else
+static inline bool sdei_nmi_trigger_cpumask_backtrace(const cpumask_t *mask,
+ int exclude_cpu)
+{
+ return false;
+}
+#endif
+
+#endif /* __ASM_NMI_H */
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 1aa324104afb..656b8417af72 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -45,6 +45,7 @@
#include <asm/daifflags.h>
#include <asm/kvm_mmu.h>
#include <asm/mmu_context.h>
+#include <asm/nmi.h>
#include <asm/numa.h>
#include <asm/processor.h>
#include <asm/smp_plat.h>
@@ -928,11 +929,19 @@ static void arm64_backtrace_ipi(cpumask_t *mask)
void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu)
{
/*
+ * Prefer the SDEI cross-CPU NMI provider when active: firmware
+ * dispatches the event out of EL3 and reaches CPUs that have
+ * interrupts locally masked, without the per-IRQ-mask cost that
+ * pseudo-NMI pays for the same reach. The plain IPI path below
+ * can't reach such a CPU unless pseudo-NMI is enabled.
+ *
* NOTE: though nmi_trigger_cpumask_backtrace() has "nmi_" in the name,
* nothing about it truly needs to be implemented using an NMI, it's
* just that it's _allowed_ to work with NMIs. If ipi_should_be_nmi()
* returned false our backtrace attempt will just use a regular IPI.
*/
+ if (sdei_nmi_trigger_cpumask_backtrace(mask, exclude_cpu))
+ return;
nmi_trigger_cpumask_backtrace(mask, exclude_cpu, arm64_backtrace_ipi);
}
diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig
index bbd2155d8483..6501087ff90d 100644
--- a/drivers/firmware/Kconfig
+++ b/drivers/firmware/Kconfig
@@ -36,6 +36,25 @@ config ARM_SDE_INTERFACE
standard for registering callbacks from the platform firmware
into the OS. This is typically used to implement RAS notifications.
+config ARM_SDEI_NMI
+ bool "SDEI-based cross-CPU NMI service (arm64)"
+ depends on ARM64 && ARM_SDE_INTERFACE
+ help
+ Provides SDEI-based cross-CPU NMI delivery for hooks that need
+ to reach interrupt-masked CPUs on silicon that lacks FEAT_NMI:
+
+ - arch_trigger_cpumask_backtrace() (sysrq-l, RCU stalls,
+ hardlockup_all_cpu_backtrace, soft-lockup secondary dumps,
+ hung-task auxiliary dumps)
+
+ The driver registers a handler for the SDEI software-signalled
+ event (event 0) and reaches a target CPU by signalling it with
+ SDEI_EVENT_SIGNAL. Firmware delivers the event out of EL3
+ regardless of the target's PSTATE.DAIF -- forced delivery into a
+ CPU wedged with interrupts locally masked.
+
+ If unsure, say N.
+
config EDD
tristate "BIOS Enhanced Disk Drive calls determine boot disk"
depends on X86
diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile
index 4ddec2820c96..48221fb8b385 100644
--- a/drivers/firmware/Makefile
+++ b/drivers/firmware/Makefile
@@ -4,6 +4,7 @@
#
obj-$(CONFIG_ARM_SCPI_PROTOCOL) += arm_scpi.o
obj-$(CONFIG_ARM_SDE_INTERFACE) += arm_sdei.o
+obj-$(CONFIG_ARM_SDEI_NMI) += sdei_nmi.o
obj-$(CONFIG_DMI) += dmi_scan.o
obj-$(CONFIG_DMI_SYSFS) += dmi-sysfs.o
obj-$(CONFIG_EDD) += edd.o
diff --git a/drivers/firmware/sdei_nmi.c b/drivers/firmware/sdei_nmi.c
new file mode 100644
index 000000000000..e5c3f28b3991
--- /dev/null
+++ b/drivers/firmware/sdei_nmi.c
@@ -0,0 +1,147 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * arm64 SDEI-based cross-CPU NMI service.
+ *
+ * Delivering an "NMI-shaped" event to an EL1 context that has locally
+ * masked interrupts, on silicon without FEAT_NMI, can be done two ways:
+ *
+ * - pseudo-NMI: mask "interrupts" via the GIC priority register
+ * (ICC_PMR_EL1) instead of PSTATE.DAIF, leaving a high-priority band
+ * deliverable. Functionally this works -- but it reimplements every
+ * local_irq_disable()/enable() and exception entry/exit as a PMR
+ * write plus synchronisation, a cost paid on that hot path forever,
+ * whether or not an NMI is ever delivered.
+ *
+ * - SDEI: leave interrupt masking as the cheap PSTATE.DAIF operation
+ * and have the firmware bounce an EL3-routed Group-0 SGI back to
+ * NS-EL1 as an event callback. The cost is a firmware round-trip,
+ * but only at the rare moment delivery is actually needed.
+ *
+ * This driver takes the second path: it keeps the IRQ-mask hot path
+ * free and pays only when it fires, which is what makes cross-CPU NMI
+ * affordable on hardware where the pseudo-NMI tax isn't, until FEAT_NMI
+ * makes NMI masking cheap in the architecture itself.
+ *
+ * Capabilities provided:
+ *
+ * - sdei_nmi_trigger_cpumask_backtrace() — override for arm64's
+ * arch_trigger_cpumask_backtrace(), so sysrq-l, RCU stall dumps,
+ * hardlockup_all_cpu_backtrace, soft-lockup/hung-task secondary
+ * dumps all reach interrupt-masked CPUs.
+ *
+ * Delivery uses the standard SDEI software-signalled event (event 0) and
+ * SDEI_EVENT_SIGNAL. We register a handler for event 0, enable it, and
+ * poke a target CPU with sdei_event_signal(0, mpidr): firmware makes
+ * event 0 pending on that PE and dispatches the handler NMI-like,
+ * regardless of the target's DAIF.
+ * Availability is simply whether event 0 registers and enables -- if SDEI
+ * and its software-signalled event are present we use it, otherwise the
+ * driver stays inert.
+ */
+
+#define pr_fmt(fmt) "sdei_nmi: " fmt
+
+#include <linux/arm_sdei.h>
+#include <linux/cpumask.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/nmi.h>
+#include <linux/printk.h>
+#include <linux/ptrace.h>
+#include <linux/smp.h>
+#include <linux/types.h>
+
+#include <asm/nmi.h>
+#include <asm/smp_plat.h>
+
+static bool sdei_nmi_available;
+
+#define SDEI_NMI_EVENT 0
+
+static int sdei_nmi_handler(u32 event, struct pt_regs *regs, void *arg)
+{
+ /*
+ * nmi_cpu_backtrace() no-ops unless this CPU's bit is set in the
+ * global backtrace mask (driven by nmi_trigger_cpumask_backtrace()),
+ * so a fire that reaches a CPU not being backtraced is harmless.
+ */
+ nmi_cpu_backtrace(regs);
+ return SDEI_EV_HANDLED;
+}
+
+static void sdei_nmi_fire(unsigned int target_cpu)
+{
+ int err = sdei_event_signal(SDEI_NMI_EVENT, cpu_logical_map(target_cpu));
+
+ if (err)
+ pr_warn("SDEI_EVENT_SIGNAL to CPU %u failed: %d\n",
+ target_cpu, err);
+}
+
+/*
+ * Raise callback for nmi_trigger_cpumask_backtrace(): signal event 0
+ * at every CPU still pending in @mask. The framework excludes the local
+ * CPU from @mask before calling us.
+ */
+static void sdei_nmi_raise_backtrace(cpumask_t *mask)
+{
+ unsigned int cpu;
+
+ for_each_cpu(cpu, mask)
+ sdei_nmi_fire(cpu);
+}
+
+/*
+ * Override hook for arch_trigger_cpumask_backtrace() (see
+ * arch/arm64/kernel/smp.c). Returns true when SDEI handled the request,
+ * which is the case whenever SDEI is active; on a false return the arch
+ * falls back to its regular-IRQ (or pseudo-NMI, if enabled) IPI.
+ *
+ * On a kernel built without paying the pseudo-NMI hot-path cost (the
+ * usual case for this driver's target), the IPI can't reach a CPU that
+ * has interrupts masked -- so the backtrace of the one CPU you care
+ * about comes back empty. SDEI is dispatched out of EL3 and lands
+ * regardless of the target's DAIF, without taxing the IRQ-mask path.
+ */
+bool sdei_nmi_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu)
+{
+ if (!sdei_nmi_available)
+ return false;
+
+ nmi_trigger_cpumask_backtrace(mask, exclude_cpu,
+ sdei_nmi_raise_backtrace);
+ return true;
+}
+
+/*
+ * device_initcall (after arch_initcall(sdei_init), so the SDEI subsystem
+ * is up): probe the firmware, register the event, and turn on the
+ * cross-CPU service. If the probe fails the driver stays inert and the
+ * override hooks decline, leaving the arch's own paths in place.
+ */
+static int __init sdei_nmi_init(void)
+{
+ int err;
+
+ err = sdei_event_register(SDEI_NMI_EVENT, sdei_nmi_handler, NULL);
+ if (err) {
+ pr_err("sdei_event_register(%u) failed: %d\n",
+ SDEI_NMI_EVENT, err);
+ return 0;
+ }
+
+ err = sdei_event_enable(SDEI_NMI_EVENT);
+ if (err) {
+ pr_err("sdei_event_enable(%u) failed: %d\n",
+ SDEI_NMI_EVENT, err);
+ sdei_event_unregister(SDEI_NMI_EVENT);
+ return 0;
+ }
+
+ sdei_nmi_available = true;
+ pr_info("using SDEI cross-CPU NMI (SDEI_EVENT_SIGNAL, event %u)\n",
+ SDEI_NMI_EVENT);
+
+ return 0;
+}
+device_initcall(sdei_nmi_init);
--
2.54.0
next prev parent reply other threads:[~2026-06-03 14:36 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-03 14:36 [PATCH 0/4] arm64: cross-CPU NMI via SDEI Kiryl Shutsemau
2026-06-03 14:36 ` [PATCH 1/4] firmware: arm_sdei: add SDEI_EVENT_SIGNAL support Kiryl Shutsemau
2026-06-03 14:36 ` Kiryl Shutsemau [this message]
2026-06-03 14:36 ` [PATCH 3/4] arm64: wire SDEI NMI into the hardlockup watchdog Kiryl Shutsemau
2026-06-03 14:36 ` [PATCH 4/4] arm64: route crash_smp_send_stop() last resort through SDEI Kiryl Shutsemau
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=145b9e98b12a7d314fc4a203075f65c3a0c3a913.1780496779.git.kas@kernel.org \
--to=kirill@shutemov$(echo .)name \
--cc=akpm@linux-foundation$(echo .)org \
--cc=bhe@redhat$(echo .)com \
--cc=catalin.marinas@arm$(echo .)com \
--cc=dianders@chromium$(echo .)org \
--cc=james.morse@arm$(echo .)com \
--cc=julien.thierry.kdev@gmail$(echo .)com \
--cc=kas@kernel$(echo .)org \
--cc=kernel-team@meta$(echo .)com \
--cc=kexec@lists$(echo .)infradead.org \
--cc=lecopzer.chen@mediatek$(echo .)com \
--cc=leitao@debian$(echo .)org \
--cc=linux-arm-kernel@lists$(echo .)infradead.org \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=mark.rutland@arm$(echo .)com \
--cc=maz@kernel$(echo .)org \
--cc=pmladek@suse$(echo .)com \
--cc=puranjay@kernel$(echo .)org \
--cc=sumit.garg@kernel$(echo .)org \
--cc=tglx@linutronix$(echo .)de \
--cc=usama.arif@linux$(echo .)dev \
--cc=will@kernel$(echo .)org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox