From: Uros Bizjak <ubizjak@gmail•com>
To: linux-alpha@vger•kernel.org, linux-kernel@vger•kernel.org,
loongarch@lists•linux.dev, linux-mips@vger•kernel.org,
linuxppc-dev@lists•ozlabs.org, linux-arch@vger•kernel.org,
linux-perf-users@vger•kernel.org
Cc: Mark Rutland <mark.rutland@arm•com>,
Ian Rogers <irogers@google•com>,
x86@kernel•org, Peter Zijlstra <peterz@infradead•org>,
Dave Hansen <dave.hansen@linux•intel.com>,
Jiaxun Yang <jiaxun.yang@flygoat•com>,
"H. Peter Anvin" <hpa@zytor•com>, WANG Xuerui <kernel@xen0n•name>,
Will Deacon <will@kernel•org>, Jun Yi <yijun@loongson•cn>,
Huacai Chen <chenhuacai@kernel•org>,
Uros Bizjak <ubizjak@gmail•com>,
Alexander Shishkin <alexander.shishkin@linux•intel.com>,
Ingo Molnar <mingo@redhat•com>, Matt Turner <mattst88@gmail•com>,
Arnd Bergmann <arnd@arndb•de>, Boqun Feng <boqun.feng@gmail•com>,
Richard Henderson <richard.henderson@linaro•org>,
Nicholas Piggin <npiggin@gmail•com>,
Ivan Kokshaysky <ink@jurassic•park.msu.ru>,
Arnaldo Carvalho de Melo <acme@kernel•org>,
Namhyung Kim <namhyung@kernel•org>,
Thomas Gleixner <tglx@linutronix•de>,
Thomas Bogendoerfer <tsbogend@alpha•franken.de>,
Jiri Olsa <jolsa@kernel•org>, Borislav Petkov <bp@alien8•de>
Subject: [PATCH 00/10] locking: Introduce local{,64}_try_cmpxchg
Date: Sun, 5 Mar 2023 21:56:18 +0100 [thread overview]
Message-ID: <20230305205628.27385-1-ubizjak@gmail.com> (raw)
Add generic and target specific support for local{,64}_try_cmpxchg
and wire up support for all targets that use local_t infrastructure.
The patch enables x86 targets to emit special instruction for
local_try_cmpxchg and also local64_try_cmpxchg for x86_64.
The last patch changes __perf_output_begin in events/ring_buffer
to use new locking primitive and improves code from
4b3: 48 8b 82 e8 00 00 00 mov 0xe8(%rdx),%rax
4ba: 48 8b b8 08 04 00 00 mov 0x408(%rax),%rdi
4c1: 8b 42 1c mov 0x1c(%rdx),%eax
4c4: 48 8b 4a 28 mov 0x28(%rdx),%rcx
4c8: 85 c0 test %eax,%eax
...
4ef: 48 89 c8 mov %rcx,%rax
4f2: 48 0f b1 7a 28 cmpxchg %rdi,0x28(%rdx)
4f7: 48 39 c1 cmp %rax,%rcx
4fa: 75 b7 jne 4b3 <...>
to
4b2: 48 8b 4a 28 mov 0x28(%rdx),%rcx
4b6: 48 8b 82 e8 00 00 00 mov 0xe8(%rdx),%rax
4bd: 48 8b b0 08 04 00 00 mov 0x408(%rax),%rsi
4c4: 8b 42 1c mov 0x1c(%rdx),%eax
4c7: 85 c0 test %eax,%eax
...
4d4: 48 89 c8 mov %rcx,%rax
4d7: 48 0f b1 72 28 cmpxchg %rsi,0x28(%rdx)
4dc: 0f 85 d0 00 00 00 jne 5b2 <...>
...
5b2: 48 89 c1 mov %rax,%rcx
5b5: e9 fc fe ff ff jmp 4b6 <...>
Please note that in addition to removed compare, the load from
0x28(%rdx) gets moved out of the loop and the code is rearranged
according to likely/unlikely tags in the source.
Cc: Richard Henderson <richard.henderson@linaro•org>
Cc: Ivan Kokshaysky <ink@jurassic•park.msu.ru>
Cc: Matt Turner <mattst88@gmail•com>
Cc: Huacai Chen <chenhuacai@kernel•org>
Cc: WANG Xuerui <kernel@xen0n•name>
Cc: Thomas Bogendoerfer <tsbogend@alpha•franken.de>
Cc: Michael Ellerman <mpe@ellerman•id.au>
Cc: Nicholas Piggin <npiggin@gmail•com>
Cc: Christophe Leroy <christophe.leroy@csgroup•eu>
Cc: Thomas Gleixner <tglx@linutronix•de>
Cc: Ingo Molnar <mingo@redhat•com>
Cc: Borislav Petkov <bp@alien8•de>
Cc: Dave Hansen <dave.hansen@linux•intel.com>
Cc: x86@kernel•org
Cc: "H. Peter Anvin" <hpa@zytor•com>
Cc: Arnd Bergmann <arnd@arndb•de>
Cc: Peter Zijlstra <peterz@infradead•org>
Cc: Arnaldo Carvalho de Melo <acme@kernel•org>
Cc: Mark Rutland <mark.rutland@arm•com>
Cc: Alexander Shishkin <alexander.shishkin@linux•intel.com>
Cc: Jiri Olsa <jolsa@kernel•org>
Cc: Namhyung Kim <namhyung@kernel•org>
Cc: Ian Rogers <irogers@google•com>
Cc: Will Deacon <will@kernel•org>
Cc: Boqun Feng <boqun.feng@gmail•com>
Cc: Jiaxun Yang <jiaxun.yang@flygoat•com>
Cc: Jun Yi <yijun@loongson•cn>
Uros Bizjak (10):
locking/atomic: Add missing cast to try_cmpxchg() fallbacks
locking/atomic: Add generic try_cmpxchg{,64}_local support
locking/alpha: Wire up local_try_cmpxchg
locking/loongarch: Wire up local_try_cmpxchg
locking/mips: Wire up local_try_cmpxchg
locking/powerpc: Wire up local_try_cmpxchg
locking/x86: Wire up local_try_cmpxchg
locking/generic: Wire up local{,64}_try_cmpxchg
locking/x86: Enable local{,64}_try_cmpxchg
perf/ring_buffer: use local_try_cmpxchg in __perf_output_begin
arch/alpha/include/asm/local.h | 2 ++
arch/loongarch/include/asm/local.h | 2 ++
arch/mips/include/asm/local.h | 2 ++
arch/powerpc/include/asm/local.h | 11 ++++++
arch/x86/include/asm/cmpxchg.h | 6 ++++
arch/x86/include/asm/local.h | 2 ++
include/asm-generic/local.h | 1 +
include/asm-generic/local64.h | 2 ++
include/linux/atomic/atomic-arch-fallback.h | 40 ++++++++++++++++-----
include/linux/atomic/atomic-instrumented.h | 20 ++++++++++-
kernel/events/ring_buffer.c | 5 +--
scripts/atomic/gen-atomic-fallback.sh | 6 +++-
scripts/atomic/gen-atomic-instrumented.sh | 2 +-
13 files changed, 87 insertions(+), 14 deletions(-)
--
2.39.2
next reply other threads:[~2023-03-05 20:57 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-05 20:56 Uros Bizjak [this message]
2023-03-05 20:56 ` [PATCH 01/10] locking/atomic: Add missing cast to try_cmpxchg() fallbacks Uros Bizjak
2023-03-24 14:13 ` Mark Rutland
2023-03-24 15:43 ` Uros Bizjak
2023-03-24 16:14 ` Mark Rutland
2023-03-24 16:32 ` Mark Rutland
2023-03-26 19:28 ` Uros Bizjak
2023-04-03 10:19 ` Mark Rutland
2023-04-04 12:24 ` Uros Bizjak
2023-04-04 13:19 ` Mark Rutland
2023-04-04 13:23 ` Uros Bizjak
2023-03-05 20:56 ` [PATCH 02/10] locking/atomic: Add generic try_cmpxchg{,64}_local support Uros Bizjak
2023-03-05 20:56 ` [PATCH 03/10] locking/alpha: Wire up local_try_cmpxchg Uros Bizjak
2023-03-05 20:56 ` [PATCH 04/10] locking/loongarch: " Uros Bizjak
2023-03-05 20:56 ` [PATCH 05/10] locking/mips: " Uros Bizjak
2023-03-05 20:56 ` [PATCH 06/10] locking/powerpc: " Uros Bizjak
2023-03-05 20:56 ` [PATCH 07/10] locking/x86: " Uros Bizjak
2023-03-05 20:56 ` [PATCH 08/10] locking/generic: Wire up local{,64}_try_cmpxchg Uros Bizjak
2023-03-05 20:56 ` [PATCH 09/10] locking/x86: Enable local{,64}_try_cmpxchg Uros Bizjak
2023-03-05 20:56 ` [PATCH 10/10] perf/ring_buffer: use local_try_cmpxchg in __perf_output_begin Uros Bizjak
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230305205628.27385-1-ubizjak@gmail.com \
--to=ubizjak@gmail$(echo .)com \
--cc=acme@kernel$(echo .)org \
--cc=alexander.shishkin@linux$(echo .)intel.com \
--cc=arnd@arndb$(echo .)de \
--cc=boqun.feng@gmail$(echo .)com \
--cc=bp@alien8$(echo .)de \
--cc=chenhuacai@kernel$(echo .)org \
--cc=dave.hansen@linux$(echo .)intel.com \
--cc=hpa@zytor$(echo .)com \
--cc=ink@jurassic$(echo .)park.msu.ru \
--cc=irogers@google$(echo .)com \
--cc=jiaxun.yang@flygoat$(echo .)com \
--cc=jolsa@kernel$(echo .)org \
--cc=kernel@xen0n$(echo .)name \
--cc=linux-alpha@vger$(echo .)kernel.org \
--cc=linux-arch@vger$(echo .)kernel.org \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=linux-mips@vger$(echo .)kernel.org \
--cc=linux-perf-users@vger$(echo .)kernel.org \
--cc=linuxppc-dev@lists$(echo .)ozlabs.org \
--cc=loongarch@lists$(echo .)linux.dev \
--cc=mark.rutland@arm$(echo .)com \
--cc=mattst88@gmail$(echo .)com \
--cc=mingo@redhat$(echo .)com \
--cc=namhyung@kernel$(echo .)org \
--cc=npiggin@gmail$(echo .)com \
--cc=peterz@infradead$(echo .)org \
--cc=richard.henderson@linaro$(echo .)org \
--cc=tglx@linutronix$(echo .)de \
--cc=tsbogend@alpha$(echo .)franken.de \
--cc=will@kernel$(echo .)org \
--cc=x86@kernel$(echo .)org \
--cc=yijun@loongson$(echo .)cn \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox