From: David Hildenbrand <david@redhat•com>
To: linux-kernel@vger•kernel.org
Cc: Mark Rutland <mark.rutland@arm•com>,
Pankaj Gupta <pagupta@redhat•com>, Michal Hocko <mhocko@suse•com>,
linux-ia64@vger•kernel.org, linux-sh@vger•kernel.org,
Peter Zijlstra <peterz@infradead•org>,
Catalin Marinas <catalin.marinas@arm•com>,
Dave Hansen <dave.hansen@linux•intel.com>,
David Hildenbrand <david@redhat•com>,
Wei Yang <richard.weiyang@gmail•com>,
linux-mm@kvack•org, Pavel Tatashin <pavel.tatashin@microsoft•com>,
Rich Felker <dalias@libc•org>,
Alexander Potapenko <glider@google•com>,
"H. Peter Anvin" <hpa@zytor•com>,
Alexander Duyck <alexander.h.duyck@linux•intel.com>,
Ira Weiny <ira.weiny@intel•com>,
Thomas Gleixner <tglx@linutronix•de>, Qian Cai <cai@lca•pw>,
linux-s390@vger•kernel.org, Yu Zhao <yuzhao@google•com>,
Yoshinori Sato <ysato@users•sourceforge.jp>,
Jason Gunthorpe <jgg@ziepe•ca>,
"Aneesh Kumar K . V" <aneesh.kumar@linux•ibm.com>,
x86@kernel•org, "Matthew Wilcox \(Oracle\)" <willy@infradead•org>,
Mike Rapoport <rppt@linux•ibm.com>,
Halil Pasic <pasic@linux•ibm.com>,
Christian Borntraeger <borntraeger@de•ibm.com>,
Ingo Molnar <mingo@redhat•com>,
Gerald Schaefer <gerald.schaefer@de•ibm.com>,
Wei Yang <richardw.yang@linux•intel.com>,
Fenghua Yu <fenghua.yu@intel•com>,
Pavel Tatashin <pasha.tatashin@soleen•com>,
Vasily Gorbik <gor@linux•ibm.com>,
Anshuman Khandual <anshuman.khandual@arm•com>,
Vlastimil Babka <vbabka@suse•cz>,
Heiko Carstens <heiko.carstens@de•ibm.com>,
Will Deacon <will@kernel•org>,
Robin Murphy <robin.murphy@arm•com>,
Jun Yao <yaojun8558363@gmail•com>, Borislav Petkov <bp@alien8•de>,
Andy Lutomirski <luto@kernel•org>,
Dan Williams <dan.j.williams@intel•com>,
linux-arm-kernel@lists•infradead.org,
Oscar Salvador <osalvador@suse•de>,
Tony Luck <tony.luck@intel•com>,
Masahiro Yamada <yamada.masahiro@socionext•com>,
Greg Kroah-Hartman <gregkh@linuxfoundation•org>,
Steve Capper <steve.capper@arm•com>,
Mel Gorman <mgorman@techsingularity•net>,
Logan Gunthorpe <logang@deltatee•com>,
Tom Lendacky <thomas.lendacky@amd•com>,
Paul Mackerras <paulus@samba•org>,
Andrew Morton <akpm@linux-foundation•org>,
linuxppc-dev@lists•ozlabs.org
Subject: [PATCH v6 00/10] mm/memory_hotplug: Shrink zones before removing memory
Date: Sun, 6 Oct 2019 10:56:36 +0200 [thread overview]
Message-ID: <20191006085646.5768-1-david@redhat.com> (raw)
This series fixes the access of uninitialized memmaps when shrinking
zones/nodes and when removing memory. Also, it contains all fixes for
crashes that can be triggered when removing certain namespace using
memunmap_pages() - ZONE_DEVICE, reported by Aneesh.
We stop trying to shrink ZONE_DEVICE, as it's buggy, fixing it would be
more involved (we don't have SECTION_IS_ONLINE as an indicator), and
shrinking is only of limited use (set_zone_contiguous() cannot detect
the ZONE_DEVICE as contiguous).
We continue shrinking !ZONE_DEVICE zones, however, I reduced the amount of
code to a minimum. Shrinking is especially necessary to keep
zone->contiguous set where possible, especially, on memory unplug of
DIMMs at zone boundaries.
--------------------------------------------------------------------------
Zones are now properly shrunk when offlining memory blocks or when
onlining failed. This allows to properly shrink zones on memory unplug
even if the separate memory blocks of a DIMM were onlined to different
zones or re-onlined to a different zone after offlining.
Example:
:/# cat /proc/zoneinfo
Node 1, zone Movable
spanned 0
present 0
managed 0
:/# echo "online_movable" > /sys/devices/system/memory/memory41/state
:/# echo "online_movable" > /sys/devices/system/memory/memory43/state
:/# cat /proc/zoneinfo
Node 1, zone Movable
spanned 98304
present 65536
managed 65536
:/# echo 0 > /sys/devices/system/memory/memory43/online
:/# cat /proc/zoneinfo
Node 1, zone Movable
spanned 32768
present 32768
managed 32768
:/# echo 0 > /sys/devices/system/memory/memory41/online
:/# cat /proc/zoneinfo
Node 1, zone Movable
spanned 0
present 0
managed 0
--------------------------------------------------------------------------
I tested this with DIMMs on x86. I didn't test the ZONE_DEVICE part.
v4 -> v6:
- "mm/memunmap: Don't access uninitialized memmap in memunmap_pages()"
-- Minimize code changes, rephrase subject and description
- "mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span()"
-- Add ifdef to make it compile without ZONE_DEVICE
v4 -> v5:
- "mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span()"
-- Add more details why ZONE_DEVICE is special
- Include two patches from Aneesh
-- "mm/memunmap: Use the correct start and end pfn when removing pages
from zone"
-- "mm/memmap_init: Update variable name in memmap_init_zone"
v3 -> v4:
- Drop "mm/memremap: Get rid of memmap_init_zone_device()"
-- As Alexander noticed, it was messy either way
- Drop "mm/memory_hotplug: Exit early in __remove_pages() on BUGs"
- Drop "mm: Exit early in set_zone_contiguous() if already contiguous"
- Drop "mm/memory_hotplug: Optimize zone shrinking code when checking for
holes"
- Merged "mm/memory_hotplug: Remove pages from a zone before removing
memory" and "mm/memory_hotplug: Remove zone parameter from
__remove_pages()" into "mm/memory_hotplug: Shrink zones when offlining
memory"
- Added "mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone()"
- Stop shrinking ZONE_DEVICE
- Reshuffle patches, moving all fixes to the front. Add Fixes: tags.
- Change subject/description of various patches
- Minor changes (too many to mention)
Cc: Aneesh Kumar K.V <aneesh.kumar@linux•ibm.com>
Cc: Andrew Morton <akpm@linux-foundation•org>
Cc: Dan Williams <dan.j.williams@intel•com>
Cc: Michal Hocko <mhocko@suse•com>
Aneesh Kumar K.V (2):
mm/memunmap: Don't access uninitialized memmap in memunmap_pages()
mm/memmap_init: Update variable name in memmap_init_zone
David Hildenbrand (8):
mm/memory_hotplug: Don't access uninitialized memmaps in
shrink_pgdat_span()
mm/memory_hotplug: Don't access uninitialized memmaps in
shrink_zone_span()
mm/memory_hotplug: Shrink zones when offlining memory
mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone()
mm/memory_hotplug: We always have a zone in
find_(smallest|biggest)_section_pfn
mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
mm/memory_hotplug: Drop local variables in shrink_zone_span()
mm/memory_hotplug: Cleanup __remove_pages()
arch/arm64/mm/mmu.c | 4 +-
arch/ia64/mm/init.c | 4 +-
arch/powerpc/mm/mem.c | 3 +-
arch/s390/mm/init.c | 4 +-
arch/sh/mm/init.c | 4 +-
arch/x86/mm/init_32.c | 4 +-
arch/x86/mm/init_64.c | 4 +-
include/linux/memory_hotplug.h | 7 +-
mm/memory_hotplug.c | 186 ++++++++++++---------------------
mm/memremap.c | 13 ++-
mm/page_alloc.c | 8 +-
11 files changed, 90 insertions(+), 151 deletions(-)
--
2.21.0
next reply other threads:[~2019-10-06 10:44 UTC|newest]
Thread overview: 75+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-06 8:56 David Hildenbrand [this message]
2019-10-06 8:56 ` [PATCH v6 01/10] mm/memunmap: Don't access uninitialized memmap in memunmap_pages() David Hildenbrand
2019-10-06 19:58 ` Damian Tometzki
2019-10-06 20:13 ` David Hildenbrand
2019-10-14 9:05 ` David Hildenbrand
2019-10-06 8:56 ` [PATCH v6 02/10] mm/memmap_init: Update variable name in memmap_init_zone David Hildenbrand
2019-10-06 8:56 ` [PATCH v6 03/10] mm/memory_hotplug: Don't access uninitialized memmaps in shrink_pgdat_span() David Hildenbrand
2019-10-14 9:31 ` David Hildenbrand
2019-10-06 8:56 ` [PATCH v6 04/10] mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span() David Hildenbrand
2019-10-14 9:32 ` David Hildenbrand
2019-10-14 19:17 ` Andrew Morton
2019-11-19 14:16 ` David Hildenbrand
2019-11-19 20:44 ` Andrew Morton
2019-10-06 8:56 ` [PATCH v6 05/10] mm/memory_hotplug: Shrink zones when offlining memory David Hildenbrand
2019-10-14 9:39 ` David Hildenbrand
2019-10-14 19:16 ` Andrew Morton
2019-10-27 22:45 ` David Hildenbrand
2019-11-30 23:21 ` Andrew Morton
2019-11-30 23:43 ` David Hildenbrand
2019-12-18 17:08 ` David Hildenbrand
2019-12-18 20:16 ` Andrew Morton
2019-12-03 15:10 ` Oscar Salvador
2019-12-03 15:27 ` David Hildenbrand
2019-10-06 8:56 ` [PATCH v6 06/10] mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone() David Hildenbrand
2019-10-16 14:01 ` David Hildenbrand
2020-02-04 8:59 ` Oscar Salvador
2019-10-06 8:56 ` [PATCH v6 07/10] mm/memory_hotplug: We always have a zone in find_(smallest|biggest)_section_pfn David Hildenbrand
2020-02-04 9:06 ` Oscar Salvador
2020-02-05 8:57 ` Wei Yang
2020-02-05 8:59 ` David Hildenbrand
2020-02-05 9:26 ` Wei Yang
2019-10-06 8:56 ` [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span() David Hildenbrand
2020-02-04 9:13 ` Oscar Salvador
2020-02-04 9:20 ` David Hildenbrand
2020-02-04 14:25 ` Baoquan He
2020-02-04 14:42 ` David Hildenbrand
2020-02-05 12:43 ` Baoquan He
2020-02-05 13:20 ` David Hildenbrand
2020-02-05 13:34 ` Baoquan He
2020-02-05 13:38 ` David Hildenbrand
2020-02-05 14:12 ` Baoquan He
2020-02-05 14:16 ` David Hildenbrand
2020-02-05 14:26 ` Baoquan He
2020-02-05 9:59 ` Wei Yang
2020-02-05 14:48 ` Baoquan He
2020-02-05 22:56 ` Wei Yang
2020-02-05 23:08 ` Baoquan He
2020-02-05 23:26 ` Wei Yang
2020-02-05 23:30 ` Baoquan He
2020-02-05 23:34 ` Wei Yang
2020-02-05 14:54 ` David Laight
2020-02-05 14:55 ` David Hildenbrand
2019-10-06 8:56 ` [PATCH v6 09/10] mm/memory_hotplug: Drop local variables " David Hildenbrand
2020-02-04 9:26 ` Oscar Salvador
2020-02-04 9:29 ` David Hildenbrand
2020-02-05 10:07 ` Wei Yang
2019-10-06 8:56 ` [PATCH v6 10/10] mm/memory_hotplug: Cleanup __remove_pages() David Hildenbrand
2020-02-04 9:46 ` Oscar Salvador
2020-02-04 12:41 ` David Hildenbrand
2020-02-04 13:13 ` Segher Boessenkool
2020-02-04 13:38 ` David Hildenbrand
2020-02-05 12:51 ` Segher Boessenkool
2020-02-05 13:17 ` David Hildenbrand
2020-02-05 13:18 ` David Hildenbrand
2020-02-05 13:23 ` David Hildenbrand
2020-02-05 11:48 ` Wei Yang
2019-12-02 9:09 ` [PATCH v6 00/10] mm/memory_hotplug: Shrink zones before removing memory David Hildenbrand
2019-12-03 13:36 ` Oscar Salvador
2020-01-31 4:40 ` Andrew Morton
2020-01-31 9:18 ` David Hildenbrand
2020-01-31 10:03 ` Michal Hocko
2020-01-31 10:36 ` David Hildenbrand
2020-02-04 1:46 ` Andrew Morton
2020-02-04 8:45 ` David Hildenbrand
2020-02-04 9:51 ` Oscar Salvador
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191006085646.5768-1-david@redhat.com \
--to=david@redhat$(echo .)com \
--cc=akpm@linux-foundation$(echo .)org \
--cc=alexander.h.duyck@linux$(echo .)intel.com \
--cc=aneesh.kumar@linux$(echo .)ibm.com \
--cc=anshuman.khandual@arm$(echo .)com \
--cc=borntraeger@de$(echo .)ibm.com \
--cc=bp@alien8$(echo .)de \
--cc=cai@lca$(echo .)pw \
--cc=catalin.marinas@arm$(echo .)com \
--cc=dalias@libc$(echo .)org \
--cc=dan.j.williams@intel$(echo .)com \
--cc=dave.hansen@linux$(echo .)intel.com \
--cc=fenghua.yu@intel$(echo .)com \
--cc=gerald.schaefer@de$(echo .)ibm.com \
--cc=glider@google$(echo .)com \
--cc=gor@linux$(echo .)ibm.com \
--cc=gregkh@linuxfoundation$(echo .)org \
--cc=heiko.carstens@de$(echo .)ibm.com \
--cc=hpa@zytor$(echo .)com \
--cc=ira.weiny@intel$(echo .)com \
--cc=jgg@ziepe$(echo .)ca \
--cc=linux-arm-kernel@lists$(echo .)infradead.org \
--cc=linux-ia64@vger$(echo .)kernel.org \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=linux-mm@kvack$(echo .)org \
--cc=linux-s390@vger$(echo .)kernel.org \
--cc=linux-sh@vger$(echo .)kernel.org \
--cc=linuxppc-dev@lists$(echo .)ozlabs.org \
--cc=logang@deltatee$(echo .)com \
--cc=luto@kernel$(echo .)org \
--cc=mark.rutland@arm$(echo .)com \
--cc=mgorman@techsingularity$(echo .)net \
--cc=mhocko@suse$(echo .)com \
--cc=mingo@redhat$(echo .)com \
--cc=osalvador@suse$(echo .)de \
--cc=pagupta@redhat$(echo .)com \
--cc=pasha.tatashin@soleen$(echo .)com \
--cc=pasic@linux$(echo .)ibm.com \
--cc=paulus@samba$(echo .)org \
--cc=pavel.tatashin@microsoft$(echo .)com \
--cc=peterz@infradead$(echo .)org \
--cc=richard.weiyang@gmail$(echo .)com \
--cc=richardw.yang@linux$(echo .)intel.com \
--cc=robin.murphy@arm$(echo .)com \
--cc=rppt@linux$(echo .)ibm.com \
--cc=steve.capper@arm$(echo .)com \
--cc=tglx@linutronix$(echo .)de \
--cc=thomas.lendacky@amd$(echo .)com \
--cc=tony.luck@intel$(echo .)com \
--cc=vbabka@suse$(echo .)cz \
--cc=will@kernel$(echo .)org \
--cc=willy@infradead$(echo .)org \
--cc=x86@kernel$(echo .)org \
--cc=yamada.masahiro@socionext$(echo .)com \
--cc=yaojun8558363@gmail$(echo .)com \
--cc=ysato@users$(echo .)sourceforge.jp \
--cc=yuzhao@google$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox