[PATCH v1 0/5] mm/memory_hotplug: Consider all zones when removing memory

* [PATCH v1 0/5] mm/memory_hotplug: Consider all zones when removing memory
@ 2019-08-21 15:40 David Hildenbrand
  2019-08-21 15:40 ` [PATCH v1 1/5] mm/memory_hotplug: Exit early in __remove_pages() on BUGs David Hildenbrand
                   ` (4 more replies)
  0 siblings, 5 replies; 7+ messages in thread
From: David Hildenbrand @ 2019-08-21 15:40 UTC (permalink / raw)
  To: linux-kernel
  Cc: linux-mm, David Hildenbrand, Aneesh Kumar K.V, H. Peter Anvin,
	Matthew Wilcox (Oracle),
	Alexander Duyck, Andrew Morton, Andy Lutomirski,
	Anshuman Khandual, Benjamin Herrenschmidt, Borislav Petkov,
	Catalin Marinas, Christian Borntraeger, Christophe Leroy,
	Dan Williams, Dave Hansen, Fenghua Yu, Gerald Schaefer,
	Greg Kroah-Hartman, Halil Pasic, Heiko Carstens, Ingo Molnar,
	Ira Weiny, Jason Gunthorpe, Jun Yao, Logan Gunthorpe,
	Mark Rutland, Masahiro Yamada, Mel Gorman, Michael Ellerman,
	Michal Hocko, Mike Rapoport, Oscar Salvador, Paul Mackerras,
	Pavel Tatashin, Pavel Tatashin, Peter Zijlstra, Qian Cai,
	Rich Felker, Robin Murphy, Steve Capper, Thomas Gleixner,
	Tom Lendacky, Tony Luck, Vasily Gorbik, Vlastimil Babka,
	Wei Yang, Wei Yang, Will Deacon, Yoshinori Sato, Yu Zhao

Working on virtio-mem, I was able to trigger a kernel BUG (with debug
options enabled) when removing memory that was never onlined. As far as I
can see the same can also happen without debug configs, if we're unlucky
and the uninitialized memmap contains selected garbage :).

The root problem is that we should not try to derive the zone of memory we
are removing from the first PFN. The individual memory blocks of a DIMM
could be spanned by different ZONEs, multiple ZONES (after being offline and
re-onlined) or no ZONE at all (never onlined).

Let's process all applicable zones when removing memory so we're on the
safe side. In the long term, we want to resize the zones when offlining
memory (and before removing ZONE_DEVICE memory), however, that will require
more thought (and most probably a new SECTION_ACTIVE / pfn_active()
thingy). More details about that in patch #3.

Along with the fix, some related cleanups.

If we run into performance issues (doubt it) we could
- Pass the node along from remove_memory() and only consider zones of
  that node
- Remember zones that are worth calling "set_zone_contiguous()", right now
  we try to recompute it for all zones that are not contiguous.

--- snip ---

I gave this a quick test with a DIMM on x86-64:

Start with a NUMA-less node 1. Hotplug a DIMM (512MB) to Node 1.
1st memory block is not onlined. 2nd and 4th is onlined MOVABLE.
3rd is onlined NORMAL.

:/# echo "online_movable" > /sys/devices/system/memory/memory41/state
[...]
:/# echo "online_movable" > /sys/devices/system/memory/memory43/state
:/# echo "online_kernel" > /sys/devices/system/memory/memory42/state
:/# cat /sys/devices/system/memory/memory40/state
offline

:/# cat /proc/zoneinfo
Node 1, zone   Normal
 [...]
        spanned  32768
        present  32768
        managed  32768
 [...]
Node 1, zone  Movable
 [...]
        spanned  98304
        present  65536
        managed  65536
 [...]

Trigger hotunplug. If it succeeds (block 42 can be offlined):

:/# cat /proc/zoneinfo

Node 1, zone   Normal
  pages free     0
        min      0
        low      0
        high     0
        spanned  0
        present  0
        managed  0
        protection: (0, 0, 0, 0, 0)
Node 1, zone  Movable
  pages free     0
        min      0
        low      0
        high     0
        spanned  0
        present  0
        managed  0
        protection: (0, 0, 0, 0, 0)

So all zones were properly fixed up and we don't access the memmap of the
first, never-onlined memory block (garbage). I am no longer able to trigger
the BUG. I did a similar test with an already populated node.

David Hildenbrand (5):
  mm/memory_hotplug: Exit early in __remove_pages() on BUGs
  mm: Exit early in set_zone_contiguous() if already contiguous
  mm/memory_hotplug: Process all zones when removing memory
  mm/memory_hotplug: Cleanup __remove_pages()
  mm/memory_hotplug: Remove zone parameter from __remove_pages()

 arch/arm64/mm/mmu.c            |  4 +--
 arch/ia64/mm/init.c            |  4 +--
 arch/powerpc/mm/mem.c          |  3 +-
 arch/s390/mm/init.c            |  4 +--
 arch/sh/mm/init.c              |  4 +--
 arch/x86/mm/init_32.c          |  4 +--
 arch/x86/mm/init_64.c          |  4 +--
 include/linux/memory_hotplug.h |  4 +--
 mm/memory_hotplug.c            | 50 +++++++++++++++++++---------------
 mm/memremap.c                  |  3 +-
 mm/page_alloc.c                |  3 ++
 11 files changed, 41 insertions(+), 46 deletions(-)

-- 
2.21.0

^ permalink raw reply	[flat|nested] 7+ messages in thread