From: Leonardo Bras <leo.bras@arm•com>
To: Breno Leitao <leitao@debian•org>
Cc: Leonardo Bras <leo.bras@arm•com>,
Catalin Marinas <catalin.marinas@arm•com>,
Will Deacon <will@kernel•org>,
mark.rutland@arm•com, leo.yan@arm•com,
linux-arm-kernel@lists•infradead.org,
linux-kernel@vger•kernel.org, palmer@dabbelt•com,
paulmck@kernel•org, puranjay@kernel•org, usama.arif@linux•dev,
rmikey@meta•com, kernel-team@meta•com
Subject: Re: [PATCH v2] arm64/irqflags: __always_inline the arch_local_irq_*() helpers
Date: Tue, 21 Apr 2026 17:07:25 +0100 [thread overview]
Message-ID: <aeegvdLT_z9ESKYV@devkitleo> (raw)
In-Reply-To: <20260421-arm64_always_inline-v2-1-c59d1400514d@debian.org>
On Tue, Apr 21, 2026 at 08:58:57AM -0700, Breno Leitao wrote:
> The arch_local_irq_*() wrappers in <asm/irqflags.h> dispatch between two
> underlying primitives: the __daif_* path on most systems, and the
> __pmr_* path on builds that use GIC PMR-based masking (Pseudo-NMI). The
> leaf primitives are already __always_inline, but the wrappers themselves
> are plain "static inline".
>
> That is unsafe for noinstr callers: nothing prevents the compiler from
> emitting an out-of-line copy of e.g. arch_local_irq_disable(), and an
> out-of-line copy can be instrumented (ftrace, kcov, sanitizers), which
> breaks the noinstr contract on the entry/idle paths that rely on these
> helpers.
>
> x86 hit and fixed exactly this class of bug in commit 7a745be1cc90
> ("x86/entry: __always_inline irqflags for noinstr").
>
> Force-inline all of the arch_local_irq_*() wrappers so they cannot be
> emitted out-of-line:
>
> - arch_local_irq_enable()
> - arch_local_irq_disable()
> - arch_local_save_flags()
> - arch_irqs_disabled_flags()
> - arch_irqs_disabled()
> - arch_local_irq_save()
> - arch_local_irq_restore()
>
> The primary motivation is noinstr safety. There is a useful side effect
> for fleet-wide profiling: when the wrapper is emitted out-of-line,
> samples taken inside it during the post-WFI IRQ unmask in
> default_idle_call() are attributed to arch_local_irq_enable rather than
> default_idle_call(), and the FP-unwinder loses default_idle_call() from
> the chain.
>
> Signed-off-by: Breno Leitao <leitao@debian•org>
> ---
> Changes in v2:
> - Expand the functions that uses always_inline in arm64
> - Link to v1: https://patch.msgid.link/20260420-arm64_always_inline-v1-1-dba919cf46bc@debian.org
> ---
> arch/arm64/include/asm/irqflags.h | 14 +++++++-------
> 1 file changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
> index d4d7451c2c129..a8cb5a5c93b78 100644
> --- a/arch/arm64/include/asm/irqflags.h
> +++ b/arch/arm64/include/asm/irqflags.h
> @@ -40,7 +40,7 @@ static __always_inline void __pmr_local_irq_enable(void)
> barrier();
> }
>
> -static inline void arch_local_irq_enable(void)
> +static __always_inline void arch_local_irq_enable(void)
> {
> if (system_uses_irq_prio_masking()) {
> __pmr_local_irq_enable();
> @@ -68,7 +68,7 @@ static __always_inline void __pmr_local_irq_disable(void)
> barrier();
> }
>
> -static inline void arch_local_irq_disable(void)
> +static __always_inline void arch_local_irq_disable(void)
> {
> if (system_uses_irq_prio_masking()) {
> __pmr_local_irq_disable();
> @@ -90,7 +90,7 @@ static __always_inline unsigned long __pmr_local_save_flags(void)
> /*
> * Save the current interrupt enable state.
> */
> -static inline unsigned long arch_local_save_flags(void)
> +static __always_inline unsigned long arch_local_save_flags(void)
> {
> if (system_uses_irq_prio_masking()) {
> return __pmr_local_save_flags();
> @@ -109,7 +109,7 @@ static __always_inline bool __pmr_irqs_disabled_flags(unsigned long flags)
> return flags != GIC_PRIO_IRQON;
> }
>
> -static inline bool arch_irqs_disabled_flags(unsigned long flags)
> +static __always_inline bool arch_irqs_disabled_flags(unsigned long flags)
> {
> if (system_uses_irq_prio_masking()) {
> return __pmr_irqs_disabled_flags(flags);
> @@ -128,7 +128,7 @@ static __always_inline bool __pmr_irqs_disabled(void)
> return __pmr_irqs_disabled_flags(__pmr_local_save_flags());
> }
>
> -static inline bool arch_irqs_disabled(void)
> +static __always_inline bool arch_irqs_disabled(void)
> {
> if (system_uses_irq_prio_masking()) {
> return __pmr_irqs_disabled();
> @@ -160,7 +160,7 @@ static __always_inline unsigned long __pmr_local_irq_save(void)
> return flags;
> }
>
> -static inline unsigned long arch_local_irq_save(void)
> +static __always_inline unsigned long arch_local_irq_save(void)
> {
> if (system_uses_irq_prio_masking()) {
> return __pmr_local_irq_save();
> @@ -187,7 +187,7 @@ static __always_inline void __pmr_local_irq_restore(unsigned long flags)
> /*
> * restore saved IRQ state
> */
> -static inline void arch_local_irq_restore(unsigned long flags)
> +static __always_inline void arch_local_irq_restore(unsigned long flags)
> {
> if (system_uses_irq_prio_masking()) {
> __pmr_local_irq_restore(flags);
>
> ---
> base-commit: bee6ea30c48788e18348309f891ed8afbf7702ac
> change-id: 20260420-arm64_always_inline-6bc9dd3c17e6
>
> Best regards,
> --
> Breno Leitao <leitao@debian•org>
>
Looks correct to what was discussed in V1. FWIW:
Reviewed-by: Leonardo Bras <leo.bras@arm•com>
next prev parent reply other threads:[~2026-04-21 16:07 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-04-21 15:58 [PATCH v2] arm64/irqflags: __always_inline the arch_local_irq_*() helpers Breno Leitao
2026-04-21 16:07 ` Leonardo Bras [this message]
2026-04-23 16:45 ` Breno Leitao
2026-04-27 12:26 ` Catalin Marinas
2026-04-27 13:08 ` Mark Rutland
2026-04-27 14:01 ` [PATCH] arm64/daifflags: Make local_daif_*() helpers __always_inline Leonardo Bras
2026-05-19 10:39 ` Will Deacon
2026-05-19 11:38 ` Leonardo Bras
2026-04-27 13:44 ` [PATCH v2] arm64/irqflags: __always_inline the arch_local_irq_*() helpers Catalin Marinas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aeegvdLT_z9ESKYV@devkitleo \
--to=leo.bras@arm$(echo .)com \
--cc=catalin.marinas@arm$(echo .)com \
--cc=kernel-team@meta$(echo .)com \
--cc=leitao@debian$(echo .)org \
--cc=leo.yan@arm$(echo .)com \
--cc=linux-arm-kernel@lists$(echo .)infradead.org \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=mark.rutland@arm$(echo .)com \
--cc=palmer@dabbelt$(echo .)com \
--cc=paulmck@kernel$(echo .)org \
--cc=puranjay@kernel$(echo .)org \
--cc=rmikey@meta$(echo .)com \
--cc=usama.arif@linux$(echo .)dev \
--cc=will@kernel$(echo .)org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox