From: Bert Karwatzki <spasswolf@web•de>
To: Christian Brauner <brauner@kernel•org>
Cc: Bert Karwatzki <spasswolf@web•de>,
linux-kernel@vger•kernel.org, linux-next@vger•kernel.org,
linux-rt-devel@lists•linux.dev, linux-fsdevel@vger•kernel.org,
mjguzik@gmail•com, adobriyan@gmail•com, jack@suse•cz,
viro@zeniv•linux.org.uk,
Sebastian Andrzej Siewior <bigeasy@linutronix•de>,
Thomas Gleixner <tglx@linutronix•de>
Subject: context switch within RCU read-side critical section in next-20260518+ with PREEMPT_RT
Date: Thu, 21 May 2026 00:52:44 +0200 [thread overview]
Message-ID: <20260520225245.2962-1-spasswolf@web.de> (raw)
In-Reply-To: 20260425220844.1763933-3-mjguzik@gmail.com
Since version next-20260518 (with PREEMPT_RT) I noticed that my debian stable/trixie system
would sometimes hang when booting displaying the following error message. After about ~1min
booting continues to a rescue shell where I could save the dmesg output (The output shown
here is not from next-20260519 but from a step in the bisection).
[ 2.900440] [ T709] ------------[ cut here ]------------
[ 2.900441] [ T709] Voluntary context switch within RCU read-side critical section!
[ 2.900441] [ T709] WARNING: kernel/rcu/tree_plugin.h:332 at rcu_note_context_switch+0x2ac/0x460, CPU#4: systemd-fstab-g/709
[ 2.900447] [ T709] Modules linked in: efivarfs autofs4 ext4 mbcache jbd2 hid_generic usbhid hid amdgpu drm_client_lib i2c_algo_bit drm_buddy drm_ttm_helper ttm drm_exec drm_suballoc_helper mfd_core drm_panel_backlight_quirks gpu_sched xhci_pci amdxcp drm_display_helper xhci_hcd drm_kms_helper ahci libahci drm libata usbcore nvme scsi_mod nvme_core igc video i2c_piix4 cec nvme_keyring i2c_smbus usb_common scsi_common crc16 nvme_auth wmi gpio_amdpt gpio_generic
[ 2.900456] [ T709] CPU: 4 UID: 0 PID: 709 Comm: systemd-fstab-g Not tainted 7.1.0-rc4-bisect-02057-g134bedf6b3e5 #452 PREEMPT_RT
[ 2.900457] [ T709] Hardware name: ASUS System Product Name/ROG STRIX B850-F GAMING WIFI, BIOS 1627 02/05/2026
[ 2.900458] [ T709] RIP: 0010:rcu_note_context_switch+0x2ac/0x460
[ 2.900459] [ T709] Code: ef e8 58 56 87 00 48 8b 55 28 b9 01 00 00 00 4c 89 ef c6 45 11 00 48 89 c6 e8 e0 99 ff ff e9 cd fd ff ff 48 8d 3d 84 8a de 00 <67> 48 0f b9 3a e9 89 fd ff ff a9 a0 20 00 00 0f 85 df 00 00 00 f6
[ 2.900460] [ T709] RSP: 0018:ffffb538c1e3fb98 EFLAGS: 00010002
[ 2.900461] [ T709] RAX: 0000000000000001 RBX: ffff9bd494a0db00 RCX: 0000000000000000
[ 2.900462] [ T709] RDX: 0000000000000000 RSI: ffffffffb27ad182 RDI: ffffffffb2d29400
[ 2.900462] [ T709] RBP: ffff9be33f326b00 R08: ffffeba64492bec0 R09: ffff9bd491ed1100
[ 2.900462] [ T709] R10: 0000000000000001 R11: ffffeba64492bec0 R12: 0000000000000000
[ 2.900462] [ T709] R13: 0000000000000000 R14: ffff9bd494a0db00 R15: ffffb538c1e3fcc0
[ 2.900463] [ T709] FS: 00007f0367aaf9c0(0000) GS:ffff9be38c475000(0000) knlGS:0000000000000000
[ 2.900464] [ T709] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2.900464] [ T709] CR2: 00007f082baae1d4 CR3: 0000000111ff2000 CR4: 0000000000f50ef0
[ 2.900464] [ T709] PKRU: 55555554
[ 2.900465] [ T709] Call Trace:
[ 2.900466] [ T709] <TASK>
[ 2.900466] [ T709] ? __schedule+0x78/0xe50
[ 2.900469] [ T709] ? blk_finish_plug+0x23/0x40
[ 2.900472] [ T709] ? read_pages+0x17f/0x210
[ 2.900474] [ T709] ? schedule+0x22/0xd0
[ 2.900475] [ T709] ? io_schedule+0x41/0x60
[ 2.900476] [ T709] ? folio_wait_bit_common+0x10d/0x2f0
[ 2.900477] [ T709] ? filemap_invalidate_unlock_two+0x40/0x40
[ 2.900478] [ T709] ? filemap_fault+0x7a1/0xfc0
[ 2.900479] [ T709] ? __do_fault+0x30/0x90
[ 2.900480] [ T709] ? do_fault+0x3a9/0x5a0
[ 2.900481] [ T709] ? __handle_mm_fault+0x2c6/0x3a0
[ 2.900482] [ T709] ? handle_mm_fault+0xdc/0x2c0
[ 2.900483] [ T709] ? do_user_addr_fault+0x1e2/0x5f0
[ 2.900485] [ T709] ? exc_page_fault+0x49/0x70
[ 2.900486] [ T709] ? asm_exc_page_fault+0x26/0x30
[ 2.900487] [ T709] </TASK>
[ 2.900487] [ T709] ---[ end trace 0000000000000000 ]---
I bisected the error from v7.1-rc4 to next-20260519, declaring a
commit as GOOD when it survives 12 boots without displaying this error:
7.1.0-rc4-bisect-03214-ge373a2ca9f8f error on 5th boot BAD
7.1.0-rc4-bisect-01608-gbd4178634082 12 boots without error GOOD
7.1.0-rc4-bisect-02426-g6491cb7030d4 error on 2nd boot BAD
7.1.0-rc4-bisect-02025-gd97e5246790d 12 boots without error GOOD
7.1.0-rc1-bisect-00173-gd97d13c24d78 12 boots without error GOOD
7.1.0-rc4-bisect-02116-g44f7bcae0b3d error on 6th boot BAD
7.1.0-rc4-bisect-00045-g0a8e31d303d5 12 boots without error GOOD
7.0.0-bisect-10592-g2acc3b265f94 12 boots without error GOOD
7.1.0-rc4-bisect-02057-g134bedf6b3e5 error on 1st boot BAD
7.0.0-bisect-10570-g99bde1dfe878 12 boots without error GOOD
7.0.0-bisect-10616-g5d1aae9252b4 12 boots without error GOOD
7.1.0-rc4-bisect-00003-g24214ad405d1 12 boots without error GOOD
7.1.0-rc4-bisect-02055-g95185f3a36ec error on 4th boot BAD
This first bisect gives this bogus result:
commit 95185f3a36ec ("Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/hid/hid.git")
which shows there's a false negative in the bisection above.
So I retested some of the commits from above:
7.1.0-rc4-bisect-02025-gd97e5246790d error on 1st boot (13 boots in total) BAD
7.1.0-rc4-bisect-01608-gbd4178634082 12 boots without error (25 in total) GOOD
and started a bisection from bd4178634082 to d97e5246790d (with increased
number of boots required for a good commit)
7.1.0-rc4-bisect-00204-g43467cbc2260 18 boots without error GOOD
7.1.0-rc4-bisect-00333-geda8cb3fb0cb error on 3rd boot BAD
With the good and bad commits this close I took a look at
git log --oneline 43467cbc2260..eda8cb3fb0cb
and found exactly one RCU related commit:
dc651e25a6d2 ("fs: RCU-ify filesystems list")
So I reverted the this commit in next-20260519 (to get a clean revert I needed to
revert commit
36b3306779ea ("fs: cache the string generated by reading /proc/filesystems") first.
$ git log --oneline
c7321982a5d0 (HEAD -> rcu_critical_readside_bug) Revert "fs: RCU-ify filesystems list"
16ff8d6e7c28 Revert "fs: cache the string generated by reading /proc/filesystems"
6a50ba100ace (tag: next-20260519, origin/master, origin/HEAD, master) Add linux-next specific files for 20260519
With these reverts next-20260519 boots 30 times in a row without error, so
it appears that commit dc651e25a6d2 ("fs: RCU-ify filesystems list") causing the
error.
To see if this issue is PREEMPT_RT only I also tested next-20260519 *without* PREEMPT_RT
and got a different bug at my first boot (the second boot worked, the third failed again)
In the non-RT case there's no rescue shell so this error message is copied from a (bad) photo:
[ 2.823291][ T510] BUG: scheduling while atomic: sytemd-hiberna/510
[ 2.824837][ T504] /usr/lib/systemd/system-generators/systemd-hibernate-resume-generator terminated by signal SEGV
BUG: scheduling while atomic: sytemd-hiberna/510
Call Trace:
dump_stack_lvl
__schedule_bug.cold
[...]
asm_exc_page_fault
Code: unable to access opcode bytes at 0x7f243337e216
To see if the non-RT error is caused by the same commit as the RT error I tested
next-20260519 with the reverts and *without* PREEMPT_RT. With the reverts there was
no error in 20 boots. So the problem in the non-RT and RT case seem to be caused by
the same commits.
Hardware used:
$ cat /proc/cpuinfo
processor : 31
vendor_id : AuthenticAMD
cpu family : 26
model : 68
model name : AMD Ryzen 9 9950X 16-Core Processor
stepping : 0
microcode : 0xb404035
cpu MHz : 624.194
cache size : 1024 KB
physical id : 0
siblings : 32
core id : 15
cpu cores : 16
apicid : 31
initial apicid : 31
fpu : yes
fpu_exception : yes
cpuid level : 16
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good amd_lbr_v2 nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpuid_fault cpb cat_l3 cdp_l3 hw_pstate ssbd mba perfmon_v2 ibrs ibpb stibp ibrs_enhanced vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local user_shstk avx_vnni avx512_bf16 clzero irperf xsaveerptr rdpru wbnoinvd cppc arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif x2avic v_spec_ctrl vnmi avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid bus_lock_detect movdiri movdir64b overflow_recov succor smca fsrm avx512_vp2intersect flush_l1d amd_lbr_pmc_freeze
bugs : sysret_ss_attrs spectre_v1 spectre_v2 spec_store_bypass srso spectre_v2_user vmscape
bogomips : 8599.98
TLB size : 192 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]
$ lspci -nn
00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Root Complex [1022:14d8]
00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge IOMMU [1022:14d9]
00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Dummy Host Bridge [1022:14da]
00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge GPP Bridge [1022:14db]
00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge GPP Bridge [1022:14db]
00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Dummy Host Bridge [1022:14da]
00:02.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge GPP Bridge [1022:14db]
00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Dummy Host Bridge [1022:14da]
00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Dummy Host Bridge [1022:14da]
00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Dummy Host Bridge [1022:14da]
00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Internal GPP Bridge to Bus [C:A] [1022:14dd]
00:08.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Internal GPP Bridge to Bus [C:A] [1022:14dd]
00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 71)
00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 0 [1022:14e0]
00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 1 [1022:14e1]
00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 2 [1022:14e2]
00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 3 [1022:14e3]
00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 4 [1022:14e4]
00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 5 [1022:14e5]
00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 6 [1022:14e6]
00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 7 [1022:14e7]
01:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch [1002:1478] (rev 25)
02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch [1002:1479] (rev 25)
03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 44 [RX 9060 XT] [1002:7590] (rev c0)
03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 48 HDMI/DP Audio Controller [1002:ab40]
04:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD 9100 PRO [PM9E1] [144d:a810]
05:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Upstream Port [1022:43f4] (rev 01)
06:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port [1022:43f5] (rev 01)
06:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port [1022:43f5] (rev 01)
06:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port [1022:43f5] (rev 01)
06:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port [1022:43f5] (rev 01)
06:0c.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port [1022:43f5] (rev 01)
06:0d.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port [1022:43f5] (rev 01)
08:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-V [8086:125c] (rev 06)
09:00.0 Network controller [0280]: MEDIATEK Corp. MT7925 802.11be 160MHz 2x2 PCIe Wireless Network Adapter [Filogic 360] [14c3:7925]
0b:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] 800 Series Chipset USB 3.x XHCI Controller [1022:43fc] (rev 01)
0c:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset SATA Controller [1022:43f6] (rev 01)
0d:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Granite Ridge [Radeon Graphics] [1002:13c0] (rev c1)
0d:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Radeon High Definition Audio Controller [Rembrandt/Strix] [1002:1640]
0d:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 19h PSP/CCP [1022:1649]
0d:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge USB 3.1 xHCI [1022:15b6]
0d:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge USB 3.1 xHCI [1022:15b7]
0e:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge USB 2.0 xHCI [1022:15b8]
Bert Karwatzki
next reply other threads:[~2026-05-20 22:53 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-05-20 22:52 Bert Karwatzki [this message]
2026-05-21 8:37 ` context switch within RCU read-side critical section in next-20260518+ with PREEMPT_RT Thomas Gleixner
2026-05-21 8:53 ` Mateusz Guzik
2026-05-21 9:08 ` Sebastian Andrzej Siewior
2026-05-21 9:17 ` Mateusz Guzik
2026-05-21 9:09 ` Mateusz Guzik
2026-05-21 9:20 ` Bert Karwatzki
2026-05-21 9:25 ` Mateusz Guzik
2026-05-21 9:57 ` Bert Karwatzki
2026-05-21 10:17 ` Thomas Gleixner
2026-05-21 10:21 ` Bert Karwatzki
2026-05-21 10:33 ` Mateusz Guzik
2026-05-21 11:50 ` Bert Karwatzki
2026-05-21 12:01 ` Mateusz Guzik
2026-05-28 17:59 ` Bert Karwatzki
2026-05-29 17:20 ` Mateusz Guzik
2026-05-21 10:05 ` Thomas Gleixner
2026-05-21 10:13 ` Bert Karwatzki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260520225245.2962-1-spasswolf@web.de \
--to=spasswolf@web$(echo .)de \
--cc=adobriyan@gmail$(echo .)com \
--cc=bigeasy@linutronix$(echo .)de \
--cc=brauner@kernel$(echo .)org \
--cc=jack@suse$(echo .)cz \
--cc=linux-fsdevel@vger$(echo .)kernel.org \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=linux-next@vger$(echo .)kernel.org \
--cc=linux-rt-devel@lists$(echo .)linux.dev \
--cc=mjguzik@gmail$(echo .)com \
--cc=tglx@linutronix$(echo .)de \
--cc=viro@zeniv$(echo .)linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox