From: David Hildenbrand <david@redhat.com> To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, x86@kernel.org, David Hildenbrand <david@redhat.com>, "Aneesh Kumar K . V" <aneesh.kumar@linux.ibm.com>, Andrew Morton <akpm@linux-foundation.org>, Dan Williams <dan.j.williams@intel.com>, Michal Hocko <mhocko@suse.com>, Alexander Duyck <alexander.h.duyck@linux.intel.com>, Alexander Potapenko <glider@google.com>, Andy Lutomirski <luto@kernel.org>, Anshuman Khandual <anshuman.khandual@arm.com>, Benjamin Herrenschmidt <benh@kernel.crashing.org>, Borislav Petkov <bp@alien8.de>, Catalin Marinas <catalin.marinas@arm.com>, Christian Borntraeger <borntraeger@de.ibm.com>, Christophe Leroy <christophe.leroy@c-s.fr>, Dave Hansen <dave.hansen@linux.intel.com>, Fenghua Yu <fenghua.yu@intel.com>, Gerald Schaefer <gerald.schaefer@de.ibm.com>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Halil Pasic <pasic@linux.ibm.com>, Heiko Carstens <heiko.carstens@de.ibm.com>, "H. Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>, Ira Weiny <ira.weiny@intel.com>, Jason Gunthorpe <jgg@ziepe.ca>, Jun Yao <yaojun8558363@gmail.com>, Logan Gunthorpe <logang@deltatee.com>, Mark Rutland <mark.rutland@arm.com>, Masahiro Yamada <yamada.masahiro@socionext.com>, "Matthew Wilcox (Oracle)" <willy@infradead.org>, Mel Gorman <mgorman@techsingularity.net>, Michael Ellerman <mpe@ellerman.id.au>, Mike Rapoport <rppt@linux.ibm.com>, Oscar Salvador <osalvador@suse.de>, Pankaj Gupta <pagupta@redhat.com>, Paul Mackerras <paulus@samba.org>, Pavel Tatashin <pasha.tatashin@soleen.com>, Pavel Tatashin <pavel.tatashin@microsoft.com>, Peter Zijlstra <peterz@infradead.org>, Qian Cai <cai@lca.pw>, Rich Felker <dalias@libc.org>, Robin Murphy <robin.murphy@arm.com>, Steve Capper <steve.capper@arm.com>, Thomas Gleixner <tglx@linutronix.de>, Tom Lendacky <thomas.lendacky@amd.com>, Tony Luck <tony.luck@intel.com>, Vasily Gorbik <gor@linux.ibm.com>, Vlastimil Babka <vbabka@suse.cz>, Wei Yang <richard.weiyang@gmail.com>, Wei Yang <richardw.yang@linux.intel.com>, Will Deacon <will@kernel.org>, Yoshinori Sato <ysato@users.sourceforge.jp>, Yu Zhao <yuzhao@google.com> Subject: [PATCH v6 00/10] mm/memory_hotplug: Shrink zones before removing memory Date: Sun, 6 Oct 2019 10:56:36 +0200 Message-ID: <20191006085646.5768-1-david@redhat.com> (raw) This series fixes the access of uninitialized memmaps when shrinking zones/nodes and when removing memory. Also, it contains all fixes for crashes that can be triggered when removing certain namespace using memunmap_pages() - ZONE_DEVICE, reported by Aneesh. We stop trying to shrink ZONE_DEVICE, as it's buggy, fixing it would be more involved (we don't have SECTION_IS_ONLINE as an indicator), and shrinking is only of limited use (set_zone_contiguous() cannot detect the ZONE_DEVICE as contiguous). We continue shrinking !ZONE_DEVICE zones, however, I reduced the amount of code to a minimum. Shrinking is especially necessary to keep zone->contiguous set where possible, especially, on memory unplug of DIMMs at zone boundaries. -------------------------------------------------------------------------- Zones are now properly shrunk when offlining memory blocks or when onlining failed. This allows to properly shrink zones on memory unplug even if the separate memory blocks of a DIMM were onlined to different zones or re-onlined to a different zone after offlining. Example: :/# cat /proc/zoneinfo Node 1, zone Movable spanned 0 present 0 managed 0 :/# echo "online_movable" > /sys/devices/system/memory/memory41/state :/# echo "online_movable" > /sys/devices/system/memory/memory43/state :/# cat /proc/zoneinfo Node 1, zone Movable spanned 98304 present 65536 managed 65536 :/# echo 0 > /sys/devices/system/memory/memory43/online :/# cat /proc/zoneinfo Node 1, zone Movable spanned 32768 present 32768 managed 32768 :/# echo 0 > /sys/devices/system/memory/memory41/online :/# cat /proc/zoneinfo Node 1, zone Movable spanned 0 present 0 managed 0 -------------------------------------------------------------------------- I tested this with DIMMs on x86. I didn't test the ZONE_DEVICE part. v4 -> v6: - "mm/memunmap: Don't access uninitialized memmap in memunmap_pages()" -- Minimize code changes, rephrase subject and description - "mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span()" -- Add ifdef to make it compile without ZONE_DEVICE v4 -> v5: - "mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span()" -- Add more details why ZONE_DEVICE is special - Include two patches from Aneesh -- "mm/memunmap: Use the correct start and end pfn when removing pages from zone" -- "mm/memmap_init: Update variable name in memmap_init_zone" v3 -> v4: - Drop "mm/memremap: Get rid of memmap_init_zone_device()" -- As Alexander noticed, it was messy either way - Drop "mm/memory_hotplug: Exit early in __remove_pages() on BUGs" - Drop "mm: Exit early in set_zone_contiguous() if already contiguous" - Drop "mm/memory_hotplug: Optimize zone shrinking code when checking for holes" - Merged "mm/memory_hotplug: Remove pages from a zone before removing memory" and "mm/memory_hotplug: Remove zone parameter from __remove_pages()" into "mm/memory_hotplug: Shrink zones when offlining memory" - Added "mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone()" - Stop shrinking ZONE_DEVICE - Reshuffle patches, moving all fixes to the front. Add Fixes: tags. - Change subject/description of various patches - Minor changes (too many to mention) Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Michal Hocko <mhocko@suse.com> Aneesh Kumar K.V (2): mm/memunmap: Don't access uninitialized memmap in memunmap_pages() mm/memmap_init: Update variable name in memmap_init_zone David Hildenbrand (8): mm/memory_hotplug: Don't access uninitialized memmaps in shrink_pgdat_span() mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span() mm/memory_hotplug: Shrink zones when offlining memory mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone() mm/memory_hotplug: We always have a zone in find_(smallest|biggest)_section_pfn mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span() mm/memory_hotplug: Drop local variables in shrink_zone_span() mm/memory_hotplug: Cleanup __remove_pages() arch/arm64/mm/mmu.c | 4 +- arch/ia64/mm/init.c | 4 +- arch/powerpc/mm/mem.c | 3 +- arch/s390/mm/init.c | 4 +- arch/sh/mm/init.c | 4 +- arch/x86/mm/init_32.c | 4 +- arch/x86/mm/init_64.c | 4 +- include/linux/memory_hotplug.h | 7 +- mm/memory_hotplug.c | 186 ++++++++++++--------------------- mm/memremap.c | 13 ++- mm/page_alloc.c | 8 +- 11 files changed, 90 insertions(+), 151 deletions(-) -- 2.21.0
next reply index Thread overview: 75+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-10-06 8:56 David Hildenbrand [this message] 2019-10-06 8:56 ` [PATCH v6 01/10] mm/memunmap: Don't access uninitialized memmap in memunmap_pages() David Hildenbrand 2019-10-06 19:58 ` Damian Tometzki 2019-10-06 20:13 ` David Hildenbrand 2019-10-14 9:05 ` David Hildenbrand 2019-10-06 8:56 ` [PATCH v6 02/10] mm/memmap_init: Update variable name in memmap_init_zone David Hildenbrand 2019-10-06 8:56 ` [PATCH v6 03/10] mm/memory_hotplug: Don't access uninitialized memmaps in shrink_pgdat_span() David Hildenbrand 2019-10-14 9:31 ` David Hildenbrand 2019-10-06 8:56 ` [PATCH v6 04/10] mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span() David Hildenbrand 2019-10-14 9:32 ` David Hildenbrand 2019-10-14 19:17 ` Andrew Morton 2019-11-19 14:16 ` David Hildenbrand 2019-11-19 20:44 ` Andrew Morton 2019-10-06 8:56 ` [PATCH v6 05/10] mm/memory_hotplug: Shrink zones when offlining memory David Hildenbrand 2019-10-14 9:39 ` David Hildenbrand 2019-10-14 19:16 ` Andrew Morton 2019-10-27 22:45 ` David Hildenbrand 2019-11-30 23:21 ` Andrew Morton 2019-11-30 23:43 ` David Hildenbrand 2019-12-18 17:08 ` David Hildenbrand 2019-12-18 20:16 ` Andrew Morton 2019-12-03 15:10 ` Oscar Salvador 2019-12-03 15:27 ` David Hildenbrand 2019-10-06 8:56 ` [PATCH v6 06/10] mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone() David Hildenbrand 2019-10-16 14:01 ` David Hildenbrand 2020-02-04 8:59 ` Oscar Salvador 2019-10-06 8:56 ` [PATCH v6 07/10] mm/memory_hotplug: We always have a zone in find_(smallest|biggest)_section_pfn David Hildenbrand 2020-02-04 9:06 ` Oscar Salvador 2020-02-05 8:57 ` Wei Yang 2020-02-05 8:59 ` David Hildenbrand 2020-02-05 9:26 ` Wei Yang 2019-10-06 8:56 ` [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span() David Hildenbrand 2020-02-04 9:13 ` Oscar Salvador 2020-02-04 9:20 ` David Hildenbrand 2020-02-04 14:25 ` Baoquan He 2020-02-04 14:42 ` David Hildenbrand 2020-02-05 12:43 ` Baoquan He 2020-02-05 13:20 ` David Hildenbrand 2020-02-05 13:34 ` Baoquan He 2020-02-05 13:38 ` David Hildenbrand 2020-02-05 14:12 ` Baoquan He 2020-02-05 14:16 ` David Hildenbrand 2020-02-05 14:26 ` Baoquan He 2020-02-05 9:59 ` Wei Yang 2020-02-05 14:48 ` Baoquan He 2020-02-05 22:56 ` Wei Yang 2020-02-05 23:08 ` Baoquan He 2020-02-05 23:26 ` Wei Yang 2020-02-05 23:30 ` Baoquan He 2020-02-05 23:34 ` Wei Yang 2020-02-05 14:54 ` David Laight 2020-02-05 14:55 ` David Hildenbrand 2019-10-06 8:56 ` [PATCH v6 09/10] mm/memory_hotplug: Drop local variables " David Hildenbrand 2020-02-04 9:26 ` Oscar Salvador 2020-02-04 9:29 ` David Hildenbrand 2020-02-05 10:07 ` Wei Yang 2019-10-06 8:56 ` [PATCH v6 10/10] mm/memory_hotplug: Cleanup __remove_pages() David Hildenbrand 2020-02-04 9:46 ` Oscar Salvador 2020-02-04 12:41 ` David Hildenbrand 2020-02-04 13:13 ` Segher Boessenkool 2020-02-04 13:38 ` David Hildenbrand 2020-02-05 12:51 ` Segher Boessenkool 2020-02-05 13:17 ` David Hildenbrand 2020-02-05 13:18 ` David Hildenbrand 2020-02-05 13:23 ` David Hildenbrand 2020-02-05 11:48 ` Wei Yang 2019-12-02 9:09 ` [PATCH v6 00/10] mm/memory_hotplug: Shrink zones before removing memory David Hildenbrand 2019-12-03 13:36 ` Oscar Salvador 2020-01-31 4:40 ` Andrew Morton 2020-01-31 9:18 ` David Hildenbrand 2020-01-31 10:03 ` Michal Hocko 2020-01-31 10:36 ` David Hildenbrand 2020-02-04 1:46 ` Andrew Morton 2020-02-04 8:45 ` David Hildenbrand 2020-02-04 9:51 ` Oscar Salvador
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191006085646.5768-1-david@redhat.com \ --to=david@redhat.com \ --cc=akpm@linux-foundation.org \ --cc=alexander.h.duyck@linux.intel.com \ --cc=aneesh.kumar@linux.ibm.com \ --cc=anshuman.khandual@arm.com \ --cc=benh@kernel.crashing.org \ --cc=borntraeger@de.ibm.com \ --cc=bp@alien8.de \ --cc=cai@lca.pw \ --cc=catalin.marinas@arm.com \ --cc=christophe.leroy@c-s.fr \ --cc=dalias@libc.org \ --cc=dan.j.williams@intel.com \ --cc=dave.hansen@linux.intel.com \ --cc=fenghua.yu@intel.com \ --cc=gerald.schaefer@de.ibm.com \ --cc=glider@google.com \ --cc=gor@linux.ibm.com \ --cc=gregkh@linuxfoundation.org \ --cc=heiko.carstens@de.ibm.com \ --cc=hpa@zytor.com \ --cc=ira.weiny@intel.com \ --cc=jgg@ziepe.ca \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-ia64@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-s390@vger.kernel.org \ --cc=linux-sh@vger.kernel.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=logang@deltatee.com \ --cc=luto@kernel.org \ --cc=mark.rutland@arm.com \ --cc=mgorman@techsingularity.net \ --cc=mhocko@suse.com \ --cc=mingo@redhat.com \ --cc=mpe@ellerman.id.au \ --cc=osalvador@suse.de \ --cc=pagupta@redhat.com \ --cc=pasha.tatashin@soleen.com \ --cc=pasic@linux.ibm.com \ --cc=paulus@samba.org \ --cc=pavel.tatashin@microsoft.com \ --cc=peterz@infradead.org \ --cc=richard.weiyang@gmail.com \ --cc=richardw.yang@linux.intel.com \ --cc=robin.murphy@arm.com \ --cc=rppt@linux.ibm.com \ --cc=steve.capper@arm.com \ --cc=tglx@linutronix.de \ --cc=thomas.lendacky@amd.com \ --cc=tony.luck@intel.com \ --cc=vbabka@suse.cz \ --cc=will@kernel.org \ --cc=willy@infradead.org \ --cc=x86@kernel.org \ --cc=yamada.masahiro@socionext.com \ --cc=yaojun8558363@gmail.com \ --cc=ysato@users.sourceforge.jp \ --cc=yuzhao@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-mm Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-mm/0 linux-mm/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-mm linux-mm/ https://lore.kernel.org/linux-mm \ linux-mm@kvack.org public-inbox-index linux-mm Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kvack.linux-mm AGPL code for this site: git clone https://public-inbox.org/public-inbox.git