From: Li Zhong <zhong@linux•vnet.ibm.com>
To: "Srivatsa S. Bhat" <srivatsa.bhat@linux•vnet.ibm.com>
Cc: Paul Mackerras <paulus@samba•org>,
PowerPC email list <linuxppc-dev@lists•ozlabs.org>,
Nikunj A Dadhania <nikunj@linux•vnet.ibm.com>
Subject: Re: [RFC PATCH powerpc] Set cpu sibling mask before online cpu
Date: Thu, 13 Jun 2013 17:42:35 +0800 [thread overview]
Message-ID: <1371116555.3704.15.camel@ThinkPad-T5421> (raw)
In-Reply-To: <51B6E0F4.6040008@linux.vnet.ibm.com>
On Tue, 2013-06-11 at 14:03 +0530, Srivatsa S. Bhat wrote:
> On 06/11/2013 12:30 PM, Benjamin Herrenschmidt wrote:
> > On Thu, 2013-05-16 at 18:20 +0800, Li Zhong wrote:
> >> It seems following race is possible:
> >>
> >
> > .../...
> >
> >> vdso_getcpu_init();
> >> #endif
> >> - notify_cpu_starting(cpu);
> >> - set_cpu_online(cpu, true);
> >> /* Update sibling maps */
> >> base = cpu_first_thread_sibling(cpu);
> >> for (i = 0; i < threads_per_core; i++) {
> >> - if (cpu_is_offline(base + i))
> >> + if (cpu_is_offline(base + i) && (cpu != base + i))
> >> continue;
> >> cpumask_set_cpu(cpu, cpu_sibling_mask(base + i));
> >> cpumask_set_cpu(base + i, cpu_sibling_mask(cpu));
> >> @@ -667,6 +665,10 @@ __cpuinit void start_secondary(void *unused)
> >> }
> >> of_node_put(l2_cache);
> >>
> >> + smp_wmb();
> >> + notify_cpu_starting(cpu);
> >> + set_cpu_online(cpu, true);
> >> +
> >
> > So we could have an online CPU with an empty sibling mask. Now we can
> > have a sibling that isn't online ... Is that ok ?
>
> I think it is OK. We do the same thing on x86 as well - we set up the
> sibling links before calling notify_cpu_starting() and setting the cpu
> in the cpu_online_mask. In fact, there is even a comment explicitly
> noting that order:
>
> arch/x86/kernel/smpboot.c:
> 220 /*
> 221 * This must be done before setting cpu_online_mask
> 222 * or calling notify_cpu_starting.
> 223 */
> 224 set_cpu_sibling_map(raw_smp_processor_id());
> 225 wmb();
> 226
> 227 notify_cpu_starting(cpuid);
> 228
> 229 /*
> 230 * Allow the master to continue.
> 231 */
> 232 cpumask_set_cpu(cpuid, cpu_callin_mask);
>
>
> So I agree with Li Zhong's solution.
>
> [Arch-specific CPU hotplug code consolidation efforts such as [1] would
> have weeded out such nasty bugs.. I guess we should revive that patchset
> sometime soon.]
>
Thank you both for the review and comments.
Good to know that it matches that of x86, and there is a patchset
consolidating the code. With the patches in [1], it seems we only need
the line to include the "to be onlined cpu" in this patch.
Thanks, Zhong
> Regards,
> Srivatsa S. Bhat
>
> [1]. https://lwn.net/Articles/500185/
>
next prev parent reply other threads:[~2013-06-13 9:42 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-05-16 10:20 [RFC PATCH powerpc] Set cpu sibling mask before online cpu Li Zhong
2013-06-11 7:00 ` Benjamin Herrenschmidt
2013-06-11 8:33 ` Srivatsa S. Bhat
2013-06-11 9:03 ` Benjamin Herrenschmidt
2013-06-13 9:42 ` Li Zhong [this message]
2013-06-11 8:35 ` Srivatsa S. Bhat
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1371116555.3704.15.camel@ThinkPad-T5421 \
--to=zhong@linux$(echo .)vnet.ibm.com \
--cc=linuxppc-dev@lists$(echo .)ozlabs.org \
--cc=nikunj@linux$(echo .)vnet.ibm.com \
--cc=paulus@samba$(echo .)org \
--cc=srivatsa.bhat@linux$(echo .)vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox