From: Valentin Schneider <vschneid@redhat•com>
To: paulmck@kernel•org
Cc: Chen Yu <yu.c.chen@intel•com>,
Peter Zijlstra <peterz@infradead•org>,
linux-kernel@vger•kernel.org, sfr@canb•auug.org.au,
linux-next@vger•kernel.org, kernel-team@meta•com,
Tomas Glozar <tglozar@redhat•com>
Subject: Re: [BUG almost bisected] Splat in dequeue_rt_stack() and build error
Date: Mon, 30 Sep 2024 22:44:24 +0200 [thread overview]
Message-ID: <xhsmhed50vplj.mognet@vschneid-thinkpadt14sgen2i.remote.csb> (raw)
In-Reply-To: <43d513c5-7620-481b-ab7e-30e76babbc80@paulmck-laptop>
On 30/09/24 12:09, Paul E. McKenney wrote:
> On Fri, Sep 13, 2024 at 11:00:39AM -0700, Paul E. McKenney wrote:
>> On Fri, Sep 13, 2024 at 06:55:34PM +0200, Valentin Schneider wrote:
>> > On 13/09/24 07:08, Paul E. McKenney wrote:
>> > > On Sun, Sep 08, 2024 at 09:32:18AM -0700, Paul E. McKenney wrote:
>> > >>
>> > >> Just following up...
>> > >>
>> > >> For whatever it is worth, on last night's run of next-20240906, I got
>> > >> nine failures out of 100 6-hour runs of rcutorture’s TREE03 scenario.
>> > >> These failures were often, but not always, shortly followed by a hard hang.
>> > >>
>> > >> The warning at line 1995 is the WARN_ON_ONCE(on_dl_rq(dl_se))
>> > >> in enqueue_dl_entity() and the warning at line 1971 is the
>> > >> WARN_ON_ONCE(!RB_EMPTY_NODE(&dl_se->rb_node)) in __enqueue_dl_entity().
>> > >>
>> > >> The pair of splats is shown below, in case it helps.
>> > >
>> > > Again following up...
>> > >
>> > > I am still seeing this on next-20240912, with six failures out of 100
>> > > 6-hour runs of rcutorture’s TREE03 scenario. Statistics suggests that
>> > > we not read much into the change in frequency.
>> > >
>> > > Please let me know if there are any diagnostic patches or options that
>> > > I should apply.
>> >
>> > Hey, sorry I haven't forgotten about this, I've just spread myself a bit
>> > too thin and also apparently I'm supposed to prepare some slides for next
>> > week, I'll get back to this soonish.
>>
>> I know that feeling! Just didn't want it to get lost.
>
> And Peter asked that I send along a reproducer, which I am finally getting
> around to doing:
>
> tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 12h --configs "100*TREE03" --trust-make
>
FYI Tomas (on Cc) has been working on getting pretty much this to run on
our infra, no hit so far.
How much of a pain would it be to record an ftrace trace while this runs?
I'm thinking sched_switch, sched_wakeup and function-tracing
dl_server_start() and dl_server_stop() would be a start.
AIUI this is running under QEMU so we'd need to record the trace within
that, I'm guessing we can (ab)use --bootargs to feed it tracing arguments,
but how do we get the trace out?
next prev parent reply other threads:[~2024-09-30 20:44 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-08-21 21:57 [BUG almost bisected] Splat in dequeue_rt_stack() and build error Paul E. McKenney
2024-08-22 23:01 ` Paul E. McKenney
2024-08-23 7:47 ` Peter Zijlstra
2024-08-23 12:46 ` Paul E. McKenney
2024-08-23 21:51 ` Paul E. McKenney
2024-08-24 6:54 ` Peter Zijlstra
2024-08-24 15:26 ` Paul E. McKenney
2024-08-25 2:10 ` Paul E. McKenney
2024-08-25 19:36 ` Paul E. McKenney
2024-08-26 11:44 ` Valentin Schneider
2024-08-26 16:31 ` Paul E. McKenney
2024-08-27 10:03 ` Valentin Schneider
2024-08-27 15:41 ` Valentin Schneider
2024-08-27 17:33 ` Paul E. McKenney
2024-08-27 18:35 ` Paul E. McKenney
2024-08-27 20:30 ` Valentin Schneider
2024-08-27 20:36 ` Paul E. McKenney
2024-08-28 12:35 ` Valentin Schneider
2024-08-28 13:03 ` Paul E. McKenney
2024-08-28 13:40 ` Paul E. McKenney
2024-08-28 13:44 ` Chen Yu
2024-08-28 14:32 ` Valentin Schneider
2024-08-28 16:35 ` Paul E. McKenney
2024-08-28 18:17 ` Valentin Schneider
2024-08-28 18:39 ` Paul E. McKenney
2024-08-29 10:28 ` Paul E. McKenney
2024-08-29 13:50 ` Valentin Schneider
2024-08-29 14:13 ` Paul E. McKenney
2024-09-08 16:32 ` Paul E. McKenney
2024-09-13 14:08 ` Paul E. McKenney
2024-09-13 16:55 ` Valentin Schneider
2024-09-13 18:00 ` Paul E. McKenney
2024-09-30 19:09 ` Paul E. McKenney
2024-09-30 20:44 ` Valentin Schneider [this message]
2024-10-01 10:10 ` Paul E. McKenney
2024-10-01 12:52 ` Valentin Schneider
2024-10-01 16:47 ` Paul E. McKenney
2024-10-02 9:01 ` Tomas Glozar
2024-10-02 12:07 ` Paul E. McKenney
2024-10-10 11:24 ` Tomas Glozar
2024-10-10 15:01 ` Paul E. McKenney
2024-10-10 23:28 ` Paul E. McKenney
2024-10-14 18:55 ` Paul E. McKenney
2024-10-21 19:25 ` Paul E. McKenney
2024-11-14 18:16 ` Paul E. McKenney
2024-12-15 18:31 ` Paul E. McKenney
2024-12-16 14:38 ` Tomas Glozar
2024-12-16 19:36 ` Paul E. McKenney
2024-12-17 16:42 ` Paul E. McKenney
2024-10-22 6:33 ` Tomas Glozar
2024-10-03 8:40 ` Peter Zijlstra
2024-10-03 8:47 ` Peter Zijlstra
2024-10-03 9:27 ` Peter Zijlstra
2024-10-03 12:28 ` Peter Zijlstra
2024-10-03 12:45 ` Paul E. McKenney
2024-10-03 14:22 ` Peter Zijlstra
2024-10-03 16:04 ` Paul E. McKenney
2024-10-03 18:50 ` Peter Zijlstra
2024-10-03 19:12 ` Paul E. McKenney
2024-10-04 13:22 ` Paul E. McKenney
2024-10-04 13:35 ` Peter Zijlstra
2024-10-06 20:44 ` Paul E. McKenney
2024-10-07 9:34 ` Peter Zijlstra
2024-10-08 11:11 ` Peter Zijlstra
2024-10-08 16:24 ` Paul E. McKenney
2024-10-08 22:34 ` Paul E. McKenney
2024-10-03 12:44 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xhsmhed50vplj.mognet@vschneid-thinkpadt14sgen2i.remote.csb \
--to=vschneid@redhat$(echo .)com \
--cc=kernel-team@meta$(echo .)com \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=linux-next@vger$(echo .)kernel.org \
--cc=paulmck@kernel$(echo .)org \
--cc=peterz@infradead$(echo .)org \
--cc=sfr@canb$(echo .)auug.org.au \
--cc=tglozar@redhat$(echo .)com \
--cc=yu.c.chen@intel$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox