From: "Paul E. McKenney" <paulmck@kernel•org>
To: Peter Zijlstra <peterz@infradead•org>,
vschneid@redhat•com, linux-kernel@vger•kernel.org,
sfr@canb•auug.org.au, linux-next@vger•kernel.org,
kernel-team@meta•com
Subject: Re: [BUG almost bisected] Splat in dequeue_rt_stack() and build error
Date: Sat, 24 Aug 2024 19:10:21 -0700 [thread overview]
Message-ID: <5a90aec0-9959-4bac-a479-d9c3c4dedd69@paulmck-laptop> (raw)
In-Reply-To: <1a1009b9-615f-44fc-8ef6-da3bbc773012@paulmck-laptop>
On Sat, Aug 24, 2024 at 08:26:57AM -0700, Paul E. McKenney wrote:
> On Sat, Aug 24, 2024 at 08:54:34AM +0200, Peter Zijlstra wrote:
> > On Fri, Aug 23, 2024 at 02:51:03PM -0700, Paul E. McKenney wrote:
> >
> > > > > Does the below help any? That's more or less what it was before Valentin
> > > > > asked me why it was weird like that :-)
> > > > >
> > > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > > > > index 6be618110885..5757dd50b02f 100644
> > > > > --- a/kernel/sched/fair.c
> > > > > +++ b/kernel/sched/fair.c
> > > > > @@ -13107,7 +13107,6 @@ static void switched_from_fair(struct rq *rq, struct task_struct *p)
> > > > > * and we cannot use DEQUEUE_DELAYED.
> > > > > */
> > > > > if (p->se.sched_delayed) {
> > > > > - dequeue_task(rq, p, DEQUEUE_NOCLOCK | DEQUEUE_SLEEP);
> > > > > p->se.sched_delayed = 0;
> > > > > p->se.rel_deadline = 0;
> > > > > if (sched_feat(DELAY_ZERO) && p->se.vlag > 0)
> > > >
> > > > Removing that line from 2e0199df252a still gets me the complaint about
> > > > __SCHED_FEAT_DELAY_ZERO being undefined. To my naive eyes, it appears
> > > > that this commit:
> > > >
> > > > 54a58a787791 ("sched/fair: Implement DELAY_ZERO")
> > > >
> > > > Need to be placed before 2e0199df252a. Of course, when I try it, I
> > > > get conflicts. So I took just this hunk:
> > > >
> > > > ------------------------------------------------------------------------
> > > >
> > > > diff --git a/kernel/sched/features.h b/kernel/sched/features.h
> > > > index 97fb2d4920898..6c5f5424614d4 100644
> > > > --- a/kernel/sched/features.h
> > > > +++ b/kernel/sched/features.h
> > > > @@ -28,6 +28,11 @@ SCHED_FEAT(NEXT_BUDDY, false)
> > > > */
> > > > SCHED_FEAT(CACHE_HOT_BUDDY, true)
> > > >
> > > > +/*
> > > > + * DELAY_ZERO clips the lag on dequeue (or wakeup) to 0.
> > > > + */
> > > > +SCHED_FEAT(DELAY_ZERO, true)
> > > > +
> > > > /*
> > > > * Allow wakeup-time preemption of the current task:
> > > > */
> > > >
> > > > ------------------------------------------------------------------------
> > > >
> > > > That makes the build error go away. Maybe even legitimately?
> >
> > Yep.
> >
> > > > Just to pick on the easy one, I took a look at the complaint about
> > > > cfs_rq being unused and the complaint about __SCHED_FEAT_DELAY_ZERO
> > > > being undefined. This variable was added here:
> > > >
> > > > 781773e3b680 ("sched/fair: Implement ENQUEUE_DELAYED")
> > > >
> > > > And its first use was added here:
> > > >
> > > > 54a58a787791 ("sched/fair: Implement DELAY_ZERO")
> > > >
> > > > Which matches my experience.
> > > >
> > > > So left to myself, I would run on these commits with the above hunk:
> > > >
> > > > 54a58a7877916 sched/fair: Implement DELAY_ZERO
> > > > 152e11f6df293 sched/fair: Implement delayed dequeue
> > > > e1459a50ba318 sched: Teach dequeue_task() about special task states
> > > > a1c446611e31c sched,freezer: Mark TASK_FROZEN special
> > > > 781773e3b6803 sched/fair: Implement ENQUEUE_DELAYED
> > > > f12e148892ede sched/fair: Prepare pick_next_task() for delayed dequeue
> > > > 2e0199df252a5 sched/fair: Prepare exit/cleanup paths for delayed_dequeue
> > > > e28b5f8bda017 sched/fair: Assert {set_next,put_prev}_entity() are properly balanced
> > > >
> > > > And where needed, remove the unused cfs_rq local variable.
> > > >
> > > > Would that likely work?
> >
> > Sounds about right.
> >
> > > >
> > > > In the meantime, SIGFOOD!
> > >
> > > Hearing no objections...
> >
> > Yeah, sorry, I'm on holidays with the kids and not glued to the screen
> > as per usual :-)
>
> No worries, and have a great holiday!!!
>
> > > Given two patches each of which might or might not need to be applied to a
> > > given commit, I chose to rebase as follows:
> > >
> > > e28b5f8bda017 sched/fair: Assert {set_next,put_prev}_entity() are properly balanced
> > > 8aed87410a695 EXP sched/fair: Provide DELAY_ZERO definition
> > > I took this from 54a58a7877916 sched/fair: Implement DELAY_ZERO.
> > > 49575c0087bc0 sched/fair: Prepare exit/cleanup paths for delayed_dequeue
> > > 14c3207fd2456 sched/fair: Prepare pick_next_task() for delayed dequeue
> > > be567af45dd04 sched/fair: Implement ENQUEUE_DELAYED
> > > I dropped the unused cfs_rq local variable from requeue_delayed_entity()
> > > ed28f7b3ac3f4 sched,freezer: Mark TASK_FROZEN special
> > > 48d541847b4a6 sched: Teach dequeue_task() about special task states
> > > ef3b9c5d038dc sched/fair: Implement delayed dequeue
> > > --- First bad commit with dequeue_rt_stack() failures.
> > > 876c99c058219 sched/fair: Implement DELAY_ZERO
> > > I added the cfs_rq local variable to requeue_delayed_entity()
> > >
> > > This is on -rcu branch peterz.2024.08.23b.
> > >
> > > I ran 50*TREE05 in a bisection, which converged on be567af45dd04, but only
> > > one run of the 50 had a complaint, and that was in enqueue_dl_entry(),
> >
> > Hmm, I have one other report about that. Hasn't made much sense yet --
> > then again, as per the above mentioned reason, I'm not able to put real
> > time in atm.
>
> I ran 1000*TREE03 on that same commit, no failures. Just started
> 5000*TREE03, and will let you know what happens. This will likely take
> better part of a day to complete.
>
> > > not the dequeue_rt_stack() that I have been chasing. I ran three
> > > additional 50*TREE05 runs on its predecessor (14c3207fd2456) with no
> > > failures. I then ran 50*TREE03 on each of ed28f7b3ac3f4, 48d541847b4a6,
> > > and ef3b9c5d038dc. Only this last ("ef3b9c5d038dc sched/fair: Implement
> > > delayed dequeue") had failure, and they were all the dequeue_rt_stack()
> > > failures I am chasing. One of the runs also hung.
> >
> > I'm a little confused now though; this is with the dequeue removed from
> > switched_from_fair() ?
>
> Ah!!! I thought that change was for the build issue, which I will
> admit puzzled me a bit.
>
> > Looking at your tree, 49575c0087bc0 still has that dequeue. Does the
> > dequeue_rt_stack() issue go away with that line removed?
>
> I will try it and let you know. Thank you for reminding me!
Preliminary results show that removing the dequeue from that commit or
just from next-20240823 at the very least greatly reduces the probability
of the problem occurring. I am doing an overnight run with that dequeue
removed from next-20240823 and will let you know how it goes.
Thanx, Paul
next prev parent reply other threads:[~2024-08-25 2:10 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-08-21 21:57 [BUG almost bisected] Splat in dequeue_rt_stack() and build error Paul E. McKenney
2024-08-22 23:01 ` Paul E. McKenney
2024-08-23 7:47 ` Peter Zijlstra
2024-08-23 12:46 ` Paul E. McKenney
2024-08-23 21:51 ` Paul E. McKenney
2024-08-24 6:54 ` Peter Zijlstra
2024-08-24 15:26 ` Paul E. McKenney
2024-08-25 2:10 ` Paul E. McKenney [this message]
2024-08-25 19:36 ` Paul E. McKenney
2024-08-26 11:44 ` Valentin Schneider
2024-08-26 16:31 ` Paul E. McKenney
2024-08-27 10:03 ` Valentin Schneider
2024-08-27 15:41 ` Valentin Schneider
2024-08-27 17:33 ` Paul E. McKenney
2024-08-27 18:35 ` Paul E. McKenney
2024-08-27 20:30 ` Valentin Schneider
2024-08-27 20:36 ` Paul E. McKenney
2024-08-28 12:35 ` Valentin Schneider
2024-08-28 13:03 ` Paul E. McKenney
2024-08-28 13:40 ` Paul E. McKenney
2024-08-28 13:44 ` Chen Yu
2024-08-28 14:32 ` Valentin Schneider
2024-08-28 16:35 ` Paul E. McKenney
2024-08-28 18:17 ` Valentin Schneider
2024-08-28 18:39 ` Paul E. McKenney
2024-08-29 10:28 ` Paul E. McKenney
2024-08-29 13:50 ` Valentin Schneider
2024-08-29 14:13 ` Paul E. McKenney
2024-09-08 16:32 ` Paul E. McKenney
2024-09-13 14:08 ` Paul E. McKenney
2024-09-13 16:55 ` Valentin Schneider
2024-09-13 18:00 ` Paul E. McKenney
2024-09-30 19:09 ` Paul E. McKenney
2024-09-30 20:44 ` Valentin Schneider
2024-10-01 10:10 ` Paul E. McKenney
2024-10-01 12:52 ` Valentin Schneider
2024-10-01 16:47 ` Paul E. McKenney
2024-10-02 9:01 ` Tomas Glozar
2024-10-02 12:07 ` Paul E. McKenney
2024-10-10 11:24 ` Tomas Glozar
2024-10-10 15:01 ` Paul E. McKenney
2024-10-10 23:28 ` Paul E. McKenney
2024-10-14 18:55 ` Paul E. McKenney
2024-10-21 19:25 ` Paul E. McKenney
2024-11-14 18:16 ` Paul E. McKenney
2024-12-15 18:31 ` Paul E. McKenney
2024-12-16 14:38 ` Tomas Glozar
2024-12-16 19:36 ` Paul E. McKenney
2024-12-17 16:42 ` Paul E. McKenney
2024-10-22 6:33 ` Tomas Glozar
2024-10-03 8:40 ` Peter Zijlstra
2024-10-03 8:47 ` Peter Zijlstra
2024-10-03 9:27 ` Peter Zijlstra
2024-10-03 12:28 ` Peter Zijlstra
2024-10-03 12:45 ` Paul E. McKenney
2024-10-03 14:22 ` Peter Zijlstra
2024-10-03 16:04 ` Paul E. McKenney
2024-10-03 18:50 ` Peter Zijlstra
2024-10-03 19:12 ` Paul E. McKenney
2024-10-04 13:22 ` Paul E. McKenney
2024-10-04 13:35 ` Peter Zijlstra
2024-10-06 20:44 ` Paul E. McKenney
2024-10-07 9:34 ` Peter Zijlstra
2024-10-08 11:11 ` Peter Zijlstra
2024-10-08 16:24 ` Paul E. McKenney
2024-10-08 22:34 ` Paul E. McKenney
2024-10-03 12:44 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5a90aec0-9959-4bac-a479-d9c3c4dedd69@paulmck-laptop \
--to=paulmck@kernel$(echo .)org \
--cc=kernel-team@meta$(echo .)com \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=linux-next@vger$(echo .)kernel.org \
--cc=peterz@infradead$(echo .)org \
--cc=sfr@canb$(echo .)auug.org.au \
--cc=vschneid@redhat$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox