From: Vladimir Murzin <vladimir.murzin@arm•com>
To: Mark Rutland <mark.rutland@arm•com>,
linux-arm-kernel@lists•infradead.org, kvmarm@lists•linux.dev
Cc: broonie@kernel•org, catalin.marinas@arm•com, james.morse@arm•com,
maz@kernel•org, oupton@kernel•org, tabba@google•com,
will@kernel•org
Subject: Re: [PATCH 12/18] arm64: fpsimd: Move fpsimd save/restore inline
Date: Wed, 27 May 2026 15:49:18 +0100 [thread overview]
Message-ID: <c5ba3af9-08b2-4be2-b9f5-58a925f0b050@arm.com> (raw)
In-Reply-To: <20260521132556.584676-13-mark.rutland@arm.com>
Hi Mark,
On 5/21/26 14:25, Mark Rutland wrote:
> Currently the FPSIMD register save/restore sequences are written in
> out-of-line assembly routines. While this works, it's somewhat painful:
>
> * As KVM needs to be able to use the sequences in hyp code, separate
> assembly files are used for the regular kernel and KVM code. While the
> common logic is shared in assembly macros, this still requires some
> duplication, and has lead to some trivial divergence.
>
> * For historical reasons, the assembly macros take some register
> arguments as numerical indices (e.g. "fpsimd_save x0, 8" uses x0 and
> x8), which is simply confusing.
>
> * For historical reasons, the SVE save/restore code and FPSIMD
> save/restore code have distinct sequences for FPSR and FPCR. Ideally
> this logic would be shared.
>
> * The assembly sequences can't be instrumented, and so it's harder than
> necessary to catch memory safety issues.
>
> To handle the above, move the FPSIMD register save/restore sequences to
> inline assembly, and share the FPSR+FPCR save/restore with SVE.
>
> Neither GCC nor LLVM instrument memory arguments to inline assembly, so
> explicit instrumentation is added in the same manner as other assembly
> routines. This instrumentation is implicitly disabled by Kbuild for nVHE
> hyp code.
>
> Note that I've used the SVE sequence for restoring FPCR, which uses an
> unconditional write to FPCR. The plain FPSIMD assembly sequence used a
> conditional write to FPCR since 2014 in commit:
>
> 5959e25729a5 ("arm64: fpsimd: avoid restoring fpcr if the contents haven't change")
>
> ... but this was not followed for the SVE assembly implemented in 2017
> in commit:
>
> 1fc5dce78ad1 ("arm64/sve: Low-level SVE architectural state manipulation functions")
>
> ... so I've assumed that this doesn't actually matter in practice, and
> I've erred in favour of the simpler sequence.
>
> Signed-off-by: Mark Rutland <mark.rutland@arm•com>
> Cc: Catalin Marinas <catalin.marinas@arm•com>
> Cc: Fuad Tabba <tabba@google•com>
> Cc: James Morse <james.morse@arm•com>
> Cc: Marc Zyngier <maz@kernel•org>
> Cc: Mark Brown <broonie@kernel•org>
> Cc: Oliver Upton <oupton@kernel•org>
> Cc: Will Deacon <will@kernel•org>
> ---
> arch/arm64/include/asm/fpsimd.h | 68 ++++++++++++++++++++++++-
> arch/arm64/include/asm/fpsimdmacros.h | 59 ---------------------
> arch/arm64/include/asm/kvm_hyp.h | 2 -
> arch/arm64/kernel/entry-fpsimd.S | 20 --------
> arch/arm64/kvm/hyp/fpsimd.S | 10 ----
> arch/arm64/kvm/hyp/include/hyp/switch.h | 4 +-
> arch/arm64/kvm/hyp/nvhe/hyp-main.c | 4 +-
> 7 files changed, 70 insertions(+), 97 deletions(-)
>
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index 6fd5cdf5e5f17..19b373ad0ebf7 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -22,6 +22,8 @@
> #include <linux/stddef.h>
> #include <linux/types.h>
>
> +#define __FPSIMD_PREAMBLE ".arch_extension fp\n" \
> + ".arch_extension simd\n"
> #define __SVE_PREAMBLE ".arch_extension sve\n"
> #define __SME_PREAMBLE ".arch_extension sme\n"
>
> @@ -86,8 +88,70 @@ static inline void fpsimd_load_common(const struct user_fpsimd_state *state)
> write_sysreg_s(state->fpcr, SYS_FPCR);
> }
>
> -extern void fpsimd_save_state(struct user_fpsimd_state *state);
> -extern void fpsimd_load_state(struct user_fpsimd_state *state);
> +static inline void fpsimd_save_vregs(struct user_fpsimd_state *state)
> +{
> + instrument_write(state->vregs, sizeof(state->vregs));
> + asm volatile(
> + __FPSIMD_PREAMBLE
> + " stp q0, q1, [%[vregs], #16 * 0]\n"
> + " stp q2, q3, [%[vregs], #16 * 2]\n"
> + " stp q4, q5, [%[vregs], #16 * 4]\n"
> + " stp q6, q7, [%[vregs], #16 * 6]\n"
> + " stp q8, q9, [%[vregs], #16 * 8]\n"
> + " stp q10, q11, [%[vregs], #16 * 10]\n"
> + " stp q12, q13, [%[vregs], #16 * 12]\n"
> + " stp q14, q15, [%[vregs], #16 * 14]\n"
> + " stp q16, q17, [%[vregs], #16 * 16]\n"
> + " stp q18, q19, [%[vregs], #16 * 18]\n"
> + " stp q20, q21, [%[vregs], #16 * 20]\n"
> + " stp q22, q23, [%[vregs], #16 * 22]\n"
> + " stp q24, q25, [%[vregs], #16 * 24]\n"
> + " stp q26, q27, [%[vregs], #16 * 26]\n"
> + " stp q28, q29, [%[vregs], #16 * 28]\n"
> + " stp q30, q31, [%[vregs], #16 * 30]\n"
> + : "=Q" (state->vregs)
> + : [vregs] "r" (state->vregs)
Missing "memory" clobber here?
> + );
> +}
> +
> +static inline void fpsimd_load_vregs(const struct user_fpsimd_state *state)
> +{
> + instrument_read(state->vregs, sizeof(state->vregs));
> + asm volatile(
> + __FPSIMD_PREAMBLE
> + " ldp q0, q1, [%[vregs], #16 * 0]\n"
> + " ldp q2, q3, [%[vregs], #16 * 2]\n"
> + " ldp q4, q5, [%[vregs], #16 * 4]\n"
> + " ldp q6, q7, [%[vregs], #16 * 6]\n"
> + " ldp q8, q9, [%[vregs], #16 * 8]\n"
> + " ldp q10, q11, [%[vregs], #16 * 10]\n"
> + " ldp q12, q13, [%[vregs], #16 * 12]\n"
> + " ldp q14, q15, [%[vregs], #16 * 14]\n"
> + " ldp q16, q17, [%[vregs], #16 * 16]\n"
> + " ldp q18, q19, [%[vregs], #16 * 18]\n"
> + " ldp q20, q21, [%[vregs], #16 * 20]\n"
> + " ldp q22, q23, [%[vregs], #16 * 22]\n"
> + " ldp q24, q25, [%[vregs], #16 * 24]\n"
> + " ldp q26, q27, [%[vregs], #16 * 26]\n"
> + " ldp q28, q29, [%[vregs], #16 * 28]\n"
> + " ldp q30, q31, [%[vregs], #16 * 30]\n"
> + :
> + : "Q" (state->vregs),
> + [vregs] "r" (state->vregs)
Missing "memory" clobber here?
> + );
> +}
> +
[snip]
Cheers
Vladimir
next prev parent reply other threads:[~2026-05-27 14:49 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-05-21 13:25 [PATCH 00/18] arm64+KVM: FPSIMD/SVE/SME cleanups Mark Rutland
2026-05-21 13:25 ` [PATCH 01/18] KVM: arm64: Don't include <asm/fpsimdmacros.h> Mark Rutland
2026-05-26 14:18 ` Mark Brown
2026-05-27 10:10 ` Vladimir Murzin
2026-05-21 13:25 ` [PATCH 02/18] KVM: arm64: Don't override FFR save/restore argument Mark Rutland
2026-05-26 14:27 ` Mark Brown
2026-05-27 10:16 ` Vladimir Murzin
2026-05-21 13:25 ` [PATCH 03/18] KVM: arm64: pkvm: Save host FPMR in host cpu context Mark Rutland
2026-05-27 10:29 ` Vladimir Murzin
2026-05-21 13:25 ` [PATCH 04/18] KVM: arm64: pkvm: Remove struct cpu_sve_state Mark Rutland
2026-05-27 11:58 ` Vladimir Murzin
2026-05-27 16:02 ` Mark Rutland
2026-05-27 16:11 ` Vladimir Murzin
2026-05-28 15:09 ` Mark Rutland
2026-05-28 15:12 ` Vladimir Murzin
2026-05-21 13:25 ` [PATCH 05/18] arm64: fpsimd: Fold sve_init_regs() into do_sve_acc() Mark Rutland
2026-05-26 15:28 ` Mark Brown
2026-05-27 12:05 ` Vladimir Murzin
2026-05-21 13:25 ` [PATCH 06/18] arm64: fpsimd: Remove sve_set_vq() and sme_set_vq() Mark Rutland
2026-05-26 15:42 ` Mark Brown
2026-05-27 12:50 ` Vladimir Murzin
2026-05-21 13:25 ` [PATCH 07/18] arm64: fpsimd: Use assembler for SVE instructions Mark Rutland
2026-05-26 15:43 ` Mark Brown
2026-05-27 12:58 ` Vladimir Murzin
2026-05-27 16:10 ` Mark Rutland
2026-05-21 13:25 ` [PATCH 08/18] arm64: fpsimd: Use assembler for baseline SME instructions Mark Rutland
2026-05-26 15:45 ` Mark Brown
2026-05-27 13:06 ` Vladimir Murzin
2026-05-21 13:25 ` [PATCH 09/18] arm64: fpsimd: Move sve_get_vl() and sme_get_vl() inline Mark Rutland
2026-05-26 15:47 ` Mark Brown
2026-05-27 13:18 ` Vladimir Murzin
2026-05-21 13:25 ` [PATCH 10/18] arm64: sysreg: Add FPCR and FPSR Mark Rutland
2026-05-26 15:55 ` Mark Brown
2026-05-26 16:51 ` Mark Rutland
2026-05-26 16:54 ` Mark Brown
2026-05-21 13:25 ` [PATCH 11/18] arm64: fpsimd: Split FPSR/FPCR from SVE save/restore Mark Rutland
2026-05-26 16:28 ` Mark Brown
2026-05-27 13:51 ` Mark Rutland
2026-05-27 14:13 ` Mark Brown
2026-05-27 16:13 ` Mark Rutland
2026-05-27 13:44 ` Vladimir Murzin
2026-05-21 13:25 ` [PATCH 12/18] arm64: fpsimd: Move fpsimd save/restore inline Mark Rutland
2026-05-26 16:44 ` Mark Brown
2026-05-28 16:15 ` Mark Rutland
2026-05-28 16:39 ` Mark Brown
2026-05-27 14:49 ` Vladimir Murzin [this message]
2026-05-27 15:34 ` Mark Rutland
2026-05-27 16:13 ` Vladimir Murzin
2026-05-21 13:25 ` [PATCH 13/18] arm64: fpsimd: Use opaque type for SVE state Mark Rutland
2026-05-26 16:53 ` Mark Brown
2026-05-28 9:45 ` Vladimir Murzin
2026-05-28 16:25 ` Mark Rutland
2026-05-21 13:25 ` [PATCH 14/18] arm64: fpsimd: Use opaque type for SME state Mark Rutland
2026-05-26 16:56 ` Mark Brown
2026-05-28 9:51 ` Vladimir Murzin
2026-05-21 13:25 ` [PATCH 15/18] arm64: fpsimd: Move SVE save/restore inline Mark Rutland
2026-05-27 12:29 ` Mark Brown
2026-05-28 10:39 ` Vladimir Murzin
2026-05-21 13:25 ` [PATCH 16/18] arm64: fpsimd: Move sve_flush_live() inline Mark Rutland
2026-05-27 12:54 ` Mark Brown
2026-05-27 16:23 ` Mark Rutland
2026-05-28 10:49 ` Vladimir Murzin
2026-05-21 13:25 ` [PATCH 17/18] arm64: fpsimd: Move SME save/restore inline Mark Rutland
2026-05-26 14:08 ` Mark Rutland
2026-05-26 14:39 ` Vladimir Murzin
2026-05-26 15:28 ` Mark Rutland
2026-05-26 16:38 ` Mark Rutland
2026-05-27 9:00 ` Vladimir Murzin
2026-05-29 9:10 ` Mark Rutland
2026-05-28 12:30 ` Vladimir Murzin
2026-05-28 14:39 ` Mark Rutland
2026-05-21 13:25 ` [PATCH 18/18] arm64: fpsimd: Remove <asm/fpsimdmacros.h> Mark Rutland
2026-05-28 13:10 ` Vladimir Murzin
2026-05-27 8:07 ` [PATCH 00/18] arm64+KVM: FPSIMD/SVE/SME cleanups Marc Zyngier
2026-05-27 10:32 ` Mark Rutland
2026-05-27 14:36 ` Will Deacon
2026-05-28 13:21 ` Vladimir Murzin
2026-05-28 16:28 ` Mark Rutland
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c5ba3af9-08b2-4be2-b9f5-58a925f0b050@arm.com \
--to=vladimir.murzin@arm$(echo .)com \
--cc=broonie@kernel$(echo .)org \
--cc=catalin.marinas@arm$(echo .)com \
--cc=james.morse@arm$(echo .)com \
--cc=kvmarm@lists$(echo .)linux.dev \
--cc=linux-arm-kernel@lists$(echo .)infradead.org \
--cc=mark.rutland@arm$(echo .)com \
--cc=maz@kernel$(echo .)org \
--cc=oupton@kernel$(echo .)org \
--cc=tabba@google$(echo .)com \
--cc=will@kernel$(echo .)org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox