From: Peter Zijlstra <peterz@infradead•org>
To: Nishanth Aravamudan <nacc@linux•vnet.ibm.com>
Cc: Boqun Feng <boqun.feng@linux•vnet.ibm.com>,
Srikar Dronamraju <srikar@linux•vnet.ibm.com>,
linux-kernel@vger•kernel.org, Ingo Molnar <mingo@redhat•com>,
Anton Blanchard <anton@samba•org>,
linuxppc-dev@lists•ozlabs.org,
Anshuman Khandual <khandual@linux•vnet.ibm.com>
Subject: Re: Topology updates and NUMA-level sched domains
Date: Tue, 7 Apr 2015 12:21:47 +0200 [thread overview]
Message-ID: <20150407102147.GJ23123@twins.programming.kicks-ass.net> (raw)
In-Reply-To: <20150406214558.GA38501@linux.vnet.ibm.com>
On Mon, Apr 06, 2015 at 02:45:58PM -0700, Nishanth Aravamudan wrote:
> Hi Peter,
>
> As you are very aware, I think, power has some odd NUMA topologies (and
> changes to the those topologies) at run-time. In particular, we can see
> a topology at boot:
>
> Node 0: all Cpus
> Node 7: no cpus
>
> Then we get a notification from the hypervisor that a core (or two) have
> moved from node 0 to node 7. This results in the:
> or a re-init API (which won't try to reallocate various bits), because
> the topology could be completely different now (e.g.,
> sched_domains_numa_distance will also be inaccurate now). Really, a
> topology update on power (not sure on s390x, but those are the only two
> archs that return a positive value from arch_update_cpu_topology() right
> now, afaics) is a lot like a hotplug event and we need to re-initialize
> any dependent structures.
>
> I'm just sending out feelers, as we can limp by with the above warning,
> it seems, but is less than ideal. Any help or insight you could provide
> would be greatly appreciated!
So I think (and ISTR having stated this before) that dynamic cpu<->node
maps are absolutely insane.
There is a ton of stuff that assumes the cpu<->node relation is a boot
time fixed one. Userspace being one of them. Per-cpu memory another.
You simply cannot do this without causing massive borkage.
So please come up with a coherent plan to deal with the entire problem
of dynamic cpu to memory relation and I might consider the scheduler
impact. But we're not going to hack around and maybe make it not crash
in a few corner cases while the entire thing is shite.
next prev parent reply other threads:[~2015-04-07 10:22 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-04-06 21:45 Topology updates and NUMA-level sched domains Nishanth Aravamudan
2015-04-07 10:21 ` Peter Zijlstra [this message]
2015-04-07 17:14 ` Nishanth Aravamudan
2015-04-07 19:41 ` Peter Zijlstra
2015-04-08 10:32 ` Brice Goglin
2015-04-08 10:52 ` Peter Zijlstra
2015-04-09 22:40 ` Nishanth Aravamudan
2015-04-09 22:37 ` Nishanth Aravamudan
2015-04-09 22:29 ` Nishanth Aravamudan
2015-04-10 8:31 ` Peter Zijlstra
2015-04-10 9:08 ` Peter Zijlstra
2015-04-10 19:50 ` Nishanth Aravamudan
2015-04-10 20:30 ` Nishanth Aravamudan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150407102147.GJ23123@twins.programming.kicks-ass.net \
--to=peterz@infradead$(echo .)org \
--cc=anton@samba$(echo .)org \
--cc=boqun.feng@linux$(echo .)vnet.ibm.com \
--cc=khandual@linux$(echo .)vnet.ibm.com \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=linuxppc-dev@lists$(echo .)ozlabs.org \
--cc=mingo@redhat$(echo .)com \
--cc=nacc@linux$(echo .)vnet.ibm.com \
--cc=srikar@linux$(echo .)vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox