All of lore.kernel.org
 help / color / mirror / Atom feed
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org,
	"Aneesh Kumar K . V" <aneesh.kumar@linux.ibm.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Dan Williams <dan.j.williams@intel.com>,
	Michal Hocko <mhocko@suse.com>, Andy Lutomirski <luto@kernel.org>,
	Anshuman Khandual <anshuman.khandual@arm.com>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Borislav Petkov <bp@alien8.de>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Christian Borntraeger <borntraeger@de.ibm.com>,
	Christophe Leroy <christophe.leroy@c-s.fr>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Fenghua Yu <fenghua.yu@intel.com>,
	Gerald Schaefer <gerald.schaefer@de.ibm.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Halil Pasic <pasic@linux.ibm.com>,
	Heiko Carstens <heiko.carstens@de.ibm.com>,
	"H. Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>,
	Ira Weiny <ira.weiny@intel.com>, Jason Gunthorpe <jgg@ziepe.ca>,
	Jun Yao <yaojun8558363@gmail.com>,
	Logan Gunthorpe <logang@deltatee.com>,
	Mark Rutland <mark.rutland@arm.com>,
	Masahiro Yamada <yamada.masahiro@socionext.com>,
	"Matthew Wilcox (Oracle)" <willy@infradead.org>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Mike Rapoport <rppt@linux.ibm.com>,
	Oscar Salvador <osalvador@suse.de>,
	Paul Mackerras <paulus@samba.org>,
	Pavel Tatashin <pasha.tatashin@soleen.com>,
	Peter Zijlstra <peterz@infradead.org>, Qian Cai <cai@lca.pw>,
	Rich Felker <dalias@libc.org>,
	Robin Murphy <robin.murphy@arm.com>,
	Steve Capper <steve.capper@arm.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Tom Lendacky <thomas.lendacky@amd.com>,
	Tony Luck <tony.luck@intel.com>,
	Vasily Gorbik <gor@linux.ibm.com>,
	Wei Yang <richard.weiyang@gmail.com>,
	Wei Yang <richardw.yang@linux.intel.com>,
	Will Deacon <will@kernel.org>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Yu Zhao <yuzhao@google.com>
Subject: Re: [PATCH v4 0/8] mm/memory_hotplug: Shrink zones before removing memory
Date: Thu, 19 Sep 2019 15:58:43 +0200	[thread overview]
Message-ID: <76784385-c6fa-8c7e-2596-93b901ac5e4d@redhat.com> (raw)
In-Reply-To: <20190830091428.18399-1-david@redhat.com>

On 30.08.19 11:14, David Hildenbrand wrote:
> This series fixes the access of uninitialized memmaps when shrinking
> zones/nodes and when removing memory.
> 
> We stop trying to shrink ZONE_DEVICE, as it's buggy, fixing it would be
> more involved (we don't have SECTION_IS_ONLINE as an indicator), and
> shrinking is only of limited use (set_zone_contiguous() cannot detect
> the ZONE_DEVICE as contiguous). As far as I can tell, this should be fine
> for ZONE_DEVICE.
> 
> We continue shrinking zones, but I reduced the amount of code to a
> minimum. Shrinking is especially necessary to keep zone->contiguous set
> where possible, especially on memory unplug of DIMMs at zone boundaries.
> 
> --------------------------------------------------------------------------
> 
> Zones are now properly shrunk when offlining memory blocks or when
> onlining failed. This allows to properly shrink zones on memory unplug
> even if the separate memory blocks of a DIMM were onlined to different
> zones or re-onlined to a different zone after offlining.
> 
> Example:
> 
> :/# cat /proc/zoneinfo
> Node 1, zone  Movable
>         spanned  0
>         present  0
>         managed  0
> :/# echo "online_movable" > /sys/devices/system/memory/memory41/state
> :/# echo "online_movable" > /sys/devices/system/memory/memory43/state
> :/# cat /proc/zoneinfo
> Node 1, zone  Movable
>         spanned  98304
>         present  65536
>         managed  65536
> :/# echo 0 > /sys/devices/system/memory/memory43/online
> :/# cat /proc/zoneinfo
> Node 1, zone  Movable
>         spanned  32768
>         present  32768
>         managed  32768
> :/# echo 0 > /sys/devices/system/memory/memory41/online
> :/# cat /proc/zoneinfo
> Node 1, zone  Movable
>         spanned  0
>         present  0
>         managed  0
> 
> --------------------------------------------------------------------------
> 
> I tested this with DIMMs on x86, but didn't test the ZONE_DEVICE part yet.
> 
> 
> v3 -> v4:
> - Drop "mm/memremap: Get rid of memmap_init_zone_device()"
> -- As Alexander noticed, it was messy either way :)
> - Drop "mm/memory_hotplug: Exit early in __remove_pages() on BUGs"
> - Drop "mm: Exit early in set_zone_contiguous() if already contiguous"
> - Drop "mm/memory_hotplug: Optimize zone shrinking code when checking for
>   holes"
> - Merged "mm/memory_hotplug: Remove pages from a zone before removing
>   memory" and "mm/memory_hotplug: Remove zone parameter from
>   __remove_pages()" into "mm/memory_hotplug: Shrink zones when offlining
>   memory"
> - Added "mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone()"
> - Stop shrinking ZONE_DEVICE
> - Reshuffle patches, moving all fixes to the front. Add Fixes: tags.
> - Change subject/description of various patches
> - Minor changes (too many to mention)
> 
> 
> Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: Michal Hocko <mhocko@suse.com>
> 
> 
> David Hildenbrand (8):
>   mm/memory_hotplug: Don't access uninitialized memmaps in
>     shrink_pgdat_span()
>   mm/memory_hotplug: Don't access uninitialized memmaps in
>     shrink_zone_span()
>   mm/memory_hotplug: Shrink zones when offlining memory
>   mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone()
>   mm/memory_hotplug: We always have a zone in
>     find_(smallest|biggest)_section_pfn
>   mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
>   mm/memory_hotplug: Drop local variables in shrink_zone_span()
>   mm/memory_hotplug: Cleanup __remove_pages()
> 
>  arch/arm64/mm/mmu.c            |   4 +-
>  arch/ia64/mm/init.c            |   4 +-
>  arch/powerpc/mm/mem.c          |   3 +-
>  arch/s390/mm/init.c            |   4 +-
>  arch/sh/mm/init.c              |   4 +-
>  arch/x86/mm/init_32.c          |   4 +-
>  arch/x86/mm/init_64.c          |   4 +-
>  include/linux/memory_hotplug.h |   7 +-
>  mm/memory_hotplug.c            | 184 +++++++++++----------------------
>  mm/memremap.c                  |  10 +-
>  10 files changed, 80 insertions(+), 148 deletions(-)
> 

@Andrew, if there are no further comments, can we give this some testing?

-- 

Thanks,

David / dhildenb


  parent reply	other threads:[~2019-09-19 13:58 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-30  9:14 [PATCH v4 0/8] mm/memory_hotplug: Shrink zones before removing memory David Hildenbrand
2019-08-30  9:14 ` [PATCH v4 1/8] mm/memory_hotplug: Don't access uninitialized memmaps in shrink_pgdat_span() David Hildenbrand
2019-08-30  9:14 ` [PATCH v4 2/8] mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span() David Hildenbrand
2019-09-26  9:12   ` Aneesh Kumar K.V
2019-09-26  9:22     ` David Hildenbrand
2019-08-30  9:14 ` [PATCH v4 3/8] mm/memory_hotplug: Shrink zones when offlining memory David Hildenbrand
2019-08-30  9:14   ` David Hildenbrand
2019-08-30  9:14 ` [PATCH v4 4/8] mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone() David Hildenbrand
2019-09-26  9:10   ` Aneesh Kumar K.V
2019-09-26  9:14     ` David Hildenbrand
2019-08-30  9:14 ` [PATCH v4 5/8] mm/memory_hotplug: We always have a zone in find_(smallest|biggest)_section_pfn David Hildenbrand
2019-08-30  9:14 ` [PATCH v4 6/8] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span() David Hildenbrand
2019-08-30  9:14 ` [PATCH v4 7/8] mm/memory_hotplug: Drop local variables " David Hildenbrand
2019-08-30  9:14 ` [PATCH v4 8/8] mm/memory_hotplug: Cleanup __remove_pages() David Hildenbrand
2019-09-06  9:21 ` [PATCH v4 0/8] mm/memory_hotplug: Shrink zones before removing memory David Hildenbrand
2019-09-19 13:58 ` David Hildenbrand [this message]
2019-09-19 19:16   ` Andrew Morton
2019-09-20  8:16     ` David Hildenbrand
2019-09-26 12:25 ` [PATCH 1/2] mm/memunmap: Use the correct start and end pfn when removing pages from zone Aneesh Kumar K.V
2019-09-26 12:25   ` Aneesh Kumar K.V
2019-09-26 12:25   ` [PATCH 2/2] mm/memmap_init: Update variable name in memmap_init_zone Aneesh Kumar K.V
2019-09-26 12:25     ` Aneesh Kumar K.V
2019-09-26 12:56     ` David Hildenbrand
2019-09-26 12:56       ` David Hildenbrand
2019-09-26 13:38     ` Pankaj Gupta
2019-09-26 13:38       ` Pankaj Gupta
2019-09-26 12:43   ` [PATCH 1/2] mm/memunmap: Use the correct start and end pfn when removing pages from zone David Hildenbrand
2019-09-26 12:43     ` David Hildenbrand
2019-09-26 13:15     ` Aneesh Kumar K.V
2019-09-26 13:15       ` Aneesh Kumar K.V
2019-09-26 13:34   ` Pankaj Gupta
2019-09-26 13:34     ` Pankaj Gupta
2019-09-26 22:45   ` Andrew Morton
2019-09-26 22:45     ` Andrew Morton
2019-09-27  1:51     ` Aneesh Kumar K.V
2019-09-27  1:51       ` Aneesh Kumar K.V
2019-09-27  7:46       ` David Hildenbrand
2019-09-27  7:46         ` David Hildenbrand
2019-09-27 10:32         ` [PATCH] " Aneesh Kumar K.V
2019-09-27 10:32           ` Aneesh Kumar K.V
2019-09-27 10:38           ` David Hildenbrand
2019-09-27 10:38             ` David Hildenbrand
2019-09-27 10:36         ` [PATCH 1/2] " Aneesh Kumar K.V
2019-09-27 10:36           ` Aneesh Kumar K.V
2019-09-27 10:40           ` David Hildenbrand
2019-09-27 10:40             ` David Hildenbrand
2019-09-27 11:35             ` Aneesh Kumar K.V
2019-09-27 11:35               ` Aneesh Kumar K.V
2019-09-27 11:38               ` David Hildenbrand
2019-09-27 11:38                 ` David Hildenbrand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=76784385-c6fa-8c7e-2596-93b901ac5e4d@redhat.com \
    --to=david@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=aneesh.kumar@linux.ibm.com \
    --cc=anshuman.khandual@arm.com \
    --cc=benh@kernel.crashing.org \
    --cc=borntraeger@de.ibm.com \
    --cc=bp@alien8.de \
    --cc=cai@lca.pw \
    --cc=catalin.marinas@arm.com \
    --cc=christophe.leroy@c-s.fr \
    --cc=dalias@libc.org \
    --cc=dan.j.williams@intel.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=fenghua.yu@intel.com \
    --cc=gerald.schaefer@de.ibm.com \
    --cc=gor@linux.ibm.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=heiko.carstens@de.ibm.com \
    --cc=hpa@zytor.com \
    --cc=ira.weiny@intel.com \
    --cc=jgg@ziepe.ca \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=logang@deltatee.com \
    --cc=luto@kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mhocko@suse.com \
    --cc=mingo@redhat.com \
    --cc=mpe@ellerman.id.au \
    --cc=osalvador@suse.de \
    --cc=pasha.tatashin@soleen.com \
    --cc=pasic@linux.ibm.com \
    --cc=paulus@samba.org \
    --cc=peterz@infradead.org \
    --cc=richard.weiyang@gmail.com \
    --cc=richardw.yang@linux.intel.com \
    --cc=robin.murphy@arm.com \
    --cc=rppt@linux.ibm.com \
    --cc=steve.capper@arm.com \
    --cc=tglx@linutronix.de \
    --cc=thomas.lendacky@amd.com \
    --cc=tony.luck@intel.com \
    --cc=will@kernel.org \
    --cc=willy@infradead.org \
    --cc=yamada.masahiro@socionext.com \
    --cc=yaojun8558363@gmail.com \
    --cc=ysato@users.sourceforge.jp \
    --cc=yuzhao@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.