From: Michael Ellerman <mpe@ellerman•id.au>
To: Nathan Lynch <nathanl@linux•ibm.com>, linuxppc-dev@lists•ozlabs.org
Cc: tyreld@linux•ibm.com, ajd@linux•ibm.com, mmc@linux•vnet.ibm.com,
cforno12@linux•vnet.ibm.com, drt@linux•vnet.ibm.com,
brking@linux•ibm.com
Subject: Re: [PATCH 13/29] powerpc/pseries/mobility: use stop_machine for join/suspend
Date: Sat, 05 Dec 2020 22:03:02 +1100 [thread overview]
Message-ID: <87v9dgtt09.fsf@mpe.ellerman.id.au> (raw)
In-Reply-To: <87wnxx1rwv.fsf@linux.ibm.com>
Nathan Lynch <nathanl@linux•ibm.com> writes:
> Hi Michael,
>
> Michael Ellerman <mpe@ellerman•id.au> writes:
>> Nathan Lynch <nathanl@linux•ibm.com> writes:
>>> The partition suspend sequence as specified in the platform
>>> architecture requires that all active processor threads call
>>> H_JOIN, which:
>> ...
>>> diff --git a/arch/powerpc/platforms/pseries/mobility.c b/arch/powerpc/platforms/pseries/mobility.c
>>> index 1b8ae221b98a..44ca7d4e143d 100644
>>> --- a/arch/powerpc/platforms/pseries/mobility.c
>>> +++ b/arch/powerpc/platforms/pseries/mobility.c
>>> @@ -412,6 +414,128 @@ static int wait_for_vasi_session_suspending(u64 handle)
>> ...
>>
>>> +
>>> +static int do_join(void *arg)
>>> +{
>>> + atomic_t *counter = arg;
>>> + long hvrc;
>>> + int ret;
>>> +
>>> + /* Must ensure MSR.EE off for H_JOIN. */
>>> + hard_irq_disable();
>>
>> Didn't stop_machine() already do that for us?
>>
>> In the state machine in multi_cpu_stop().
>
> Yes, but I didn't want to rely on something that seems like an
> implementation detail of stop_machine(). I assumed it's benign and in
> keeping with hard_irq_disable()'s intended semantics to make multiple
> calls to it within a critical section.
OK. I think it's part of the contract of stop_machine() these days, but
you're right hard_irq_disable() can be called multiple times, so we may
as well leave it there as insurance/documentation.
>>> + hvrc = plpar_hcall_norets(H_JOIN);
>>> +
>>> + switch (hvrc) {
>>> + case H_CONTINUE:
>>> + /*
>>> + * All other CPUs are offline or in H_JOIN. This CPU
>>> + * attempts the suspend.
>>> + */
>>> + ret = do_suspend();
>>> + break;
>>> + case H_SUCCESS:
>>> + /*
>>> + * The suspend is complete and this cpu has received a
>>> + * prod.
>>> + */
>>> + ret = 0;
>>> + break;
>>> + case H_BAD_MODE:
>>> + case H_HARDWARE:
>>> + default:
>>> + ret = -EIO;
>>> + pr_err_ratelimited("H_JOIN error %ld on CPU %i\n",
>>> + hvrc, smp_processor_id());
>>> + break;
>>> + }
>>> +
>>> + if (atomic_inc_return(counter) == 1) {
>>> + pr_info("CPU %u waking all threads\n", smp_processor_id());
>>> + prod_others();
>>> + }
>>
>> Do we even need the counter? IIUC only one CPU receives H_CONTINUE. So
>> couldn't we just have that CPU do the prodding of others?
>
> CPUs may exit H_JOIN due to system reset interrupt at any time, and
> H_JOIN may return H_HARDWARE to a caller after other CPUs have entered
> the join state successfully. In these cases the counter ensures exactly
> one thread performs the prod sequence.
OK.
>>> + /*
>>> + * Execution may have been suspended for several seconds, so
>>> + * reset the watchdog.
>>> + */
>>> + touch_nmi_watchdog();
>>> + return ret;
>>> +}
>>> +
>>> +static int pseries_migrate_partition(u64 handle)
>>> +{
>>> + atomic_t counter = ATOMIC_INIT(0);
>>> + int ret;
>>> +
>>> + ret = wait_for_vasi_session_suspending(handle);
>>> + if (ret)
>>> + goto out;
>>
>> Direct return would be clearer IMHO.
>
> OK, I can change this.
Thanks.
cheers
next prev parent reply other threads:[~2020-12-05 11:04 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-30 1:17 [PATCH 00/29] partition suspend updates Nathan Lynch
2020-10-30 1:17 ` [PATCH 01/29] powerpc/rtas: move rtas_call_reentrant() out of pseries guards Nathan Lynch
2020-12-04 20:37 ` Nathan Lynch
2020-10-30 1:17 ` [PATCH 02/29] powerpc/rtas: prevent suspend-related sys_rtas use on LE Nathan Lynch
2020-10-30 3:45 ` Andrew Donnellan
2020-10-30 12:10 ` Nathan Lynch
2020-11-06 14:59 ` Andrew Donnellan
2020-11-06 16:44 ` Nathan Lynch
2020-10-30 1:17 ` [PATCH 03/29] powerpc/rtas: complete ibm,suspend-me status codes Nathan Lynch
2020-12-04 12:52 ` Michael Ellerman
2020-12-04 14:40 ` Nathan Lynch
2020-10-30 1:17 ` [PATCH 04/29] powerpc/rtas: rtas_ibm_suspend_me -> rtas_ibm_suspend_me_unsafe Nathan Lynch
2020-10-30 1:17 ` [PATCH 05/29] powerpc/rtas: add rtas_ibm_suspend_me() Nathan Lynch
2020-10-30 1:17 ` [PATCH 06/29] powerpc/rtas: add rtas_activate_firmware() Nathan Lynch
2020-10-30 1:17 ` [PATCH 07/29] powerpc/hvcall: add token and codes for H_VASI_SIGNAL Nathan Lynch
2020-10-30 1:17 ` [PATCH 08/29] powerpc/pseries/mobility: don't error on absence of ibm, update-nodes Nathan Lynch
2020-10-30 1:17 ` [PATCH 09/29] powerpc/pseries/mobility: add missing break to default case Nathan Lynch
2020-10-30 1:17 ` [PATCH 10/29] powerpc/pseries/mobility: error message improvements Nathan Lynch
2020-10-30 1:17 ` [PATCH 11/29] powerpc/pseries/mobility: use rtas_activate_firmware() on resume Nathan Lynch
2020-10-30 1:17 ` [PATCH 12/29] powerpc/pseries/mobility: extract VASI session polling logic Nathan Lynch
2020-12-04 12:51 ` Michael Ellerman
2020-12-04 14:46 ` Nathan Lynch
2020-10-30 1:17 ` [PATCH 13/29] powerpc/pseries/mobility: use stop_machine for join/suspend Nathan Lynch
2020-12-04 12:52 ` Michael Ellerman
2020-12-04 16:01 ` Nathan Lynch
2020-12-05 11:03 ` Michael Ellerman [this message]
2020-10-30 1:17 ` [PATCH 14/29] powerpc/pseries/mobility: signal suspend cancellation to platform Nathan Lynch
2020-10-30 1:17 ` [PATCH 15/29] powerpc/pseries/mobility: retry partition suspend after error Nathan Lynch
2020-10-30 1:17 ` [PATCH 16/29] powerpc/rtas: dispatch partition migration requests to pseries Nathan Lynch
2020-12-04 12:52 ` Michael Ellerman
2020-12-04 16:04 ` Nathan Lynch
2020-10-30 1:17 ` [PATCH 17/29] powerpc/rtas: remove rtas_ibm_suspend_me_unsafe() Nathan Lynch
2020-10-30 1:17 ` [PATCH 18/29] powerpc/pseries/hibernation: drop pseries_suspend_begin() from suspend ops Nathan Lynch
2020-10-30 1:17 ` [PATCH 19/29] powerpc/pseries/hibernation: pass stream id via function arguments Nathan Lynch
2020-10-30 1:17 ` [PATCH 20/29] powerpc/pseries/hibernation: remove pseries_suspend_cpu() Nathan Lynch
2020-10-30 1:17 ` [PATCH 21/29] powerpc/machdep: remove suspend_disable_cpu() Nathan Lynch
2020-10-30 1:17 ` [PATCH 22/29] powerpc/rtas: remove rtas_suspend_cpu() Nathan Lynch
2020-10-30 1:17 ` [PATCH 23/29] powerpc/pseries/hibernation: switch to rtas_ibm_suspend_me() Nathan Lynch
2020-10-30 1:18 ` [PATCH 24/29] powerpc/rtas: remove unused rtas_suspend_last_cpu() Nathan Lynch
2020-10-30 1:18 ` [PATCH 25/29] powerpc/pseries/hibernation: remove redundant cacheinfo update Nathan Lynch
2020-10-30 1:18 ` [PATCH 26/29] powerpc/pseries/hibernation: perform post-suspend fixups later Nathan Lynch
2020-10-30 1:18 ` [PATCH 27/29] powerpc/pseries/hibernation: remove prepare_late() callback Nathan Lynch
2020-10-30 1:18 ` [PATCH 28/29] powerpc/rtas: remove unused rtas_suspend_me_data Nathan Lynch
2020-10-30 1:18 ` [PATCH 29/29] powerpc/pseries/mobility: refactor node lookup during DT update Nathan Lynch
2020-11-20 16:09 ` Nathan Lynch
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87v9dgtt09.fsf@mpe.ellerman.id.au \
--to=mpe@ellerman$(echo .)id.au \
--cc=ajd@linux$(echo .)ibm.com \
--cc=brking@linux$(echo .)ibm.com \
--cc=cforno12@linux$(echo .)vnet.ibm.com \
--cc=drt@linux$(echo .)vnet.ibm.com \
--cc=linuxppc-dev@lists$(echo .)ozlabs.org \
--cc=mmc@linux$(echo .)vnet.ibm.com \
--cc=nathanl@linux$(echo .)ibm.com \
--cc=tyreld@linux$(echo .)ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox