From: David Hildenbrand <david@redhat.com> To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, Andrew Morton <akpm@linux-foundation.org>, David Hildenbrand <david@redhat.com>, Alexander Duyck <alexander.h.duyck@linux.intel.com>, Alexander Potapenko <glider@google.com>, Alexey Kardashevskiy <aik@ozlabs.ru>, Allison Randal <allison@lohutok.net>, Anshuman Khandual <anshuman.khandual@arm.com>, Anshuman Khandual <khandual@linux.vnet.ibm.com>, Arun KS <arunks@codeaurora.org>, Benjamin Herrenschmidt <benh@kernel.crashing.org>, Christian Brauner <christian@brauner.io>, Dan Williams <dan.j.williams@intel.com>, David Howells <dhowells@redhat.com>, "Enrico Weigelt, metux IT consult" <info@metux.net>, Gao Xiang <xiang@kernel.org>, Geert Uytterhoeven <geert@linux-m68k.org>, Greg Hackmann <ghackmann@google.com>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Konstantin Khlebnikov <khlebnikov@yandex-team.ru>, Mel Gorman <mgorman@techsingularity.net>, Michael Ellerman <mpe@ellerman.id.au>, Michal Hocko <mhocko@suse.com>, Mike Rapoport <rppt@linux.vnet.ibm.com>, "Oliver O'Halloran" <oohall@gmail.com>, Oscar Salvador <osalvador@suse.de>, Paul Mackerras <paulus@samba.org>, Pavel Tatashin <pasha.tatashin@soleen.com>, Pingfan Liu <kernelfans@gmail.com>, Qian Cai <cai@lca.pw>, "Rafael J. Wysocki" <rafael@kernel.org>, Richard Fontana <rfontana@redhat.com>, Stephen Rothwell <sfr@canb.auug.org.au>, Thiago Jung Bauermann <bauerman@linux.ibm.com>, Thomas Gleixner <tglx@linutronix.de>, Todd Kjos <tkjos@google.com>, Vlastimil Babka <vbabka@suse.cz>, Wei Yang <richardw.yang@linux.intel.com> Subject: [PATCH v1 00/12] powerpc/pseries: CMM: Implement balloon compaction and remove isolate notifier Date: Thu, 31 Oct 2019 15:29:21 +0100 [thread overview] Message-ID: <20191031142933.10779-1-david@redhat.com> (raw) This is the follow-up of: https://lkml.org/lkml/2019/9/10/711 We can get rid of the memory isolate notifier by switching to balloon compaction in powerpc's CMM (Collaborative Memory Management). The memory isolate notifier was only necessary to allow to offline memory blocks that contain inflated/"loaned" pages - which also possible when the inflated pages are movable (via balloon compaction). Having movable pages implies that memory will also no longer get fragmented when CMM is active. While I do have access to a LPAR, it does not have CMM active and I have np clue how to enable it. Instead, I implemented a simple simulation mode. I did some tests and the whole infrastructure, including page migration, seems to work fine (e.g., I can still offline memory blocks that contain inflated pages). Of course, I cannot tell if HW will like the changes, especially: 1. I now use page_to_phys() to come up with the addresses to report to the hypervisor. Hope that's correct. 2. When migrating a page, I first inflate/"loan" the new page and then deflate the old page. I have no idea if HW accepts to set pages loaned if it didn't request a loan. I assume it does. 3. We might now inflate/deflate pages in parallel (of course, not the same page). I have no idea if HW likes that. It would be good if somebody could either point me at the spec of the hypervisor interface or verify directly. Also, it would be good if somebody could test with actual HW that has this feature. Cc: Alexander Duyck <alexander.h.duyck@linux.intel.com> Cc: Alexander Potapenko <glider@google.com> Cc: Alexey Kardashevskiy <aik@ozlabs.ru> Cc: Allison Randal <allison@lohutok.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Anshuman Khandual <anshuman.khandual@arm.com> Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com> Cc: Arun KS <arunks@codeaurora.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Christian Brauner <christian@brauner.io> Cc: Dan Williams <dan.j.williams@intel.com> Cc: David Hildenbrand <david@redhat.com> Cc: David Howells <dhowells@redhat.com> Cc: "Enrico Weigelt, metux IT consult" <info@metux.net> Cc: Gao Xiang <xiang@kernel.org> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Greg Hackmann <ghackmann@google.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Konstantin Khlebnikov <khlebnikov@yandex-team.ru> Cc: Mel Gorman <mgorman@techsingularity.net> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@linux.vnet.ibm.com> Cc: "Oliver O'Halloran" <oohall@gmail.com> Cc: Oscar Salvador <osalvador@suse.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Pavel Tatashin <pasha.tatashin@soleen.com> Cc: Pingfan Liu <kernelfans@gmail.com> Cc: Qian Cai <cai@lca.pw> Cc: "Rafael J. Wysocki" <rafael@kernel.org> Cc: Richard Fontana <rfontana@redhat.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Thiago Jung Bauermann <bauerman@linux.ibm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Todd Kjos <tkjos@google.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Wei Yang <richardw.yang@linux.intel.com> David Hildenbrand (12): powerpc/pseries: CMM: Implement release() function for sysfs device powerpc/pseries: CMM: Report errors when registering notifiers fails powerpc/pseries: CMM: Cleanup rc handling in cmm_init() powerpc/pseries: CMM: Drop page array powerpc/pseries: CMM: Use adjust_managed_page_count() insted of totalram_pages_* powerpc/pseries: CMM: Rip out memory isolate notifier powerpc/pseries: CMM: Convert loaned_pages to an atomic_long_t powerpc/pseries: CMM: Implement balloon compaction powerpc/pseries: CMM: Switch to balloon_page_alloc() powerpc/pseries: CMM: Simulation mode mm: remove the memory isolate notifier mm: remove "count" parameter from has_unmovable_pages() arch/powerpc/platforms/pseries/Kconfig | 1 + arch/powerpc/platforms/pseries/cmm.c | 430 +++++++++++-------------- drivers/base/memory.c | 19 -- include/linux/memory.h | 27 -- include/linux/page-isolation.h | 4 +- include/uapi/linux/magic.h | 1 + mm/memory_hotplug.c | 2 +- mm/page_alloc.c | 21 +- mm/page_isolation.c | 27 +- 9 files changed, 204 insertions(+), 328 deletions(-) -- 2.21.0
WARNING: multiple messages have this Message-ID (diff)
From: David Hildenbrand <david@redhat.com> To: linux-kernel@vger.kernel.org Cc: Michal Hocko <mhocko@suse.com>, David Hildenbrand <david@redhat.com>, Konstantin Khlebnikov <khlebnikov@yandex-team.ru>, Pingfan Liu <kernelfans@gmail.com>, David Howells <dhowells@redhat.com>, linux-mm@kvack.org, Alexander Potapenko <glider@google.com>, Alexander Duyck <alexander.h.duyck@linux.intel.com>, Thomas Gleixner <tglx@linutronix.de>, Paul Mackerras <paulus@samba.org>, Anshuman Khandual <khandual@linux.vnet.ibm.com>, Stephen Rothwell <sfr@canb.auug.org.au>, "Rafael J. Wysocki" <rafael@kernel.org>, Greg Hackmann <ghackmann@google.com>, Alexey Kardashevskiy <aik@ozlabs.ru>, Geert Uytterhoeven <geert@linux-m68k.org>, Allison Randal <allison@lohutok.net>, Gao Xiang <xiang@kernel.org>, Oliver O'Halloran <oohall@gmail.com>, Pavel Tatashin <pasha.tatashin@soleen.com>, Anshuman Khandual <anshuman.khandual@arm.com>, Vlastimil Babka <vbabka@suse.cz>, linuxppc-dev@lists.ozlabs.org, Mike Rapoport <rppt@linux.vnet.ibm.com>, Qian Cai <cai@lca.pw>, Dan Williams <dan.j.williams@intel.com>, Christian Brauner <christian@brauner.io>, Oscar Salvador <osalvador@suse.de>, Richard Fontana <rfontana@redhat.com>, Mel Gorman <mgorman@techsingularity.net>, Wei Yang <richardw.yang@linux.intel.com>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Arun KS <arunks@codeaurora.org>, Andrew Morton <akpm@linux-foundation.org>, "Enrico Weigelt, metux IT consult" <info@metux.net>, Thiago Jung Bauermann <bauerman@linux.ibm.com>, Todd Kjos <tkjos@google.com> Subject: [PATCH v1 00/12] powerpc/pseries: CMM: Implement balloon compaction and remove isolate notifier Date: Thu, 31 Oct 2019 15:29:21 +0100 [thread overview] Message-ID: <20191031142933.10779-1-david@redhat.com> (raw) This is the follow-up of: https://lkml.org/lkml/2019/9/10/711 We can get rid of the memory isolate notifier by switching to balloon compaction in powerpc's CMM (Collaborative Memory Management). The memory isolate notifier was only necessary to allow to offline memory blocks that contain inflated/"loaned" pages - which also possible when the inflated pages are movable (via balloon compaction). Having movable pages implies that memory will also no longer get fragmented when CMM is active. While I do have access to a LPAR, it does not have CMM active and I have np clue how to enable it. Instead, I implemented a simple simulation mode. I did some tests and the whole infrastructure, including page migration, seems to work fine (e.g., I can still offline memory blocks that contain inflated pages). Of course, I cannot tell if HW will like the changes, especially: 1. I now use page_to_phys() to come up with the addresses to report to the hypervisor. Hope that's correct. 2. When migrating a page, I first inflate/"loan" the new page and then deflate the old page. I have no idea if HW accepts to set pages loaned if it didn't request a loan. I assume it does. 3. We might now inflate/deflate pages in parallel (of course, not the same page). I have no idea if HW likes that. It would be good if somebody could either point me at the spec of the hypervisor interface or verify directly. Also, it would be good if somebody could test with actual HW that has this feature. Cc: Alexander Duyck <alexander.h.duyck@linux.intel.com> Cc: Alexander Potapenko <glider@google.com> Cc: Alexey Kardashevskiy <aik@ozlabs.ru> Cc: Allison Randal <allison@lohutok.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Anshuman Khandual <anshuman.khandual@arm.com> Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com> Cc: Arun KS <arunks@codeaurora.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Christian Brauner <christian@brauner.io> Cc: Dan Williams <dan.j.williams@intel.com> Cc: David Hildenbrand <david@redhat.com> Cc: David Howells <dhowells@redhat.com> Cc: "Enrico Weigelt, metux IT consult" <info@metux.net> Cc: Gao Xiang <xiang@kernel.org> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Greg Hackmann <ghackmann@google.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Konstantin Khlebnikov <khlebnikov@yandex-team.ru> Cc: Mel Gorman <mgorman@techsingularity.net> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@linux.vnet.ibm.com> Cc: "Oliver O'Halloran" <oohall@gmail.com> Cc: Oscar Salvador <osalvador@suse.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Pavel Tatashin <pasha.tatashin@soleen.com> Cc: Pingfan Liu <kernelfans@gmail.com> Cc: Qian Cai <cai@lca.pw> Cc: "Rafael J. Wysocki" <rafael@kernel.org> Cc: Richard Fontana <rfontana@redhat.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Thiago Jung Bauermann <bauerman@linux.ibm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Todd Kjos <tkjos@google.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Wei Yang <richardw.yang@linux.intel.com> David Hildenbrand (12): powerpc/pseries: CMM: Implement release() function for sysfs device powerpc/pseries: CMM: Report errors when registering notifiers fails powerpc/pseries: CMM: Cleanup rc handling in cmm_init() powerpc/pseries: CMM: Drop page array powerpc/pseries: CMM: Use adjust_managed_page_count() insted of totalram_pages_* powerpc/pseries: CMM: Rip out memory isolate notifier powerpc/pseries: CMM: Convert loaned_pages to an atomic_long_t powerpc/pseries: CMM: Implement balloon compaction powerpc/pseries: CMM: Switch to balloon_page_alloc() powerpc/pseries: CMM: Simulation mode mm: remove the memory isolate notifier mm: remove "count" parameter from has_unmovable_pages() arch/powerpc/platforms/pseries/Kconfig | 1 + arch/powerpc/platforms/pseries/cmm.c | 430 +++++++++++-------------- drivers/base/memory.c | 19 -- include/linux/memory.h | 27 -- include/linux/page-isolation.h | 4 +- include/uapi/linux/magic.h | 1 + mm/memory_hotplug.c | 2 +- mm/page_alloc.c | 21 +- mm/page_isolation.c | 27 +- 9 files changed, 204 insertions(+), 328 deletions(-) -- 2.21.0
next reply other threads:[~2019-10-31 14:30 UTC|newest] Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-10-31 14:29 David Hildenbrand [this message] 2019-10-31 14:29 ` [PATCH v1 00/12] powerpc/pseries: CMM: Implement balloon compaction and remove isolate notifier David Hildenbrand 2019-10-31 14:29 ` [PATCH v1 01/12] powerpc/pseries: CMM: Implement release() function for sysfs device David Hildenbrand 2019-10-31 14:29 ` David Hildenbrand 2019-11-14 9:08 ` Michael Ellerman 2019-11-14 9:08 ` Michael Ellerman 2019-11-14 12:21 ` David Hildenbrand 2019-11-14 12:21 ` David Hildenbrand 2019-11-20 10:35 ` David Hildenbrand 2019-11-20 10:35 ` David Hildenbrand 2019-10-31 14:29 ` [PATCH v1 02/12] powerpc/pseries: CMM: Report errors when registering notifiers fails David Hildenbrand 2019-10-31 14:29 ` David Hildenbrand 2019-10-31 14:29 ` [PATCH v1 03/12] powerpc/pseries: CMM: Cleanup rc handling in cmm_init() David Hildenbrand 2019-10-31 14:29 ` David Hildenbrand 2019-10-31 14:29 ` [PATCH v1 04/12] powerpc/pseries: CMM: Drop page array David Hildenbrand 2019-10-31 14:29 ` David Hildenbrand 2019-10-31 14:29 ` [PATCH v1 05/12] powerpc/pseries: CMM: Use adjust_managed_page_count() insted of totalram_pages_* David Hildenbrand 2019-10-31 14:29 ` David Hildenbrand 2019-10-31 14:29 ` [PATCH v1 06/12] powerpc/pseries: CMM: Rip out memory isolate notifier David Hildenbrand 2019-10-31 14:29 ` David Hildenbrand 2019-10-31 14:29 ` [PATCH v1 07/12] powerpc/pseries: CMM: Convert loaned_pages to an atomic_long_t David Hildenbrand 2019-10-31 14:29 ` David Hildenbrand 2019-10-31 14:29 ` [PATCH v1 08/12] powerpc/pseries: CMM: Implement balloon compaction David Hildenbrand 2019-10-31 14:29 ` David Hildenbrand 2019-10-31 14:35 ` David Hildenbrand 2019-10-31 14:35 ` David Hildenbrand 2019-11-12 10:46 ` Michael Ellerman 2019-11-12 10:46 ` Michael Ellerman 2019-11-12 11:12 ` David Hildenbrand 2019-11-12 11:12 ` David Hildenbrand 2019-11-13 11:08 ` Michael Ellerman 2019-11-13 11:08 ` Michael Ellerman 2019-10-31 14:29 ` [PATCH v1 09/12] powerpc/pseries: CMM: Switch to balloon_page_alloc() David Hildenbrand 2019-10-31 14:29 ` David Hildenbrand 2019-10-31 14:29 ` [PATCH v1 10/12] powerpc/pseries: CMM: Simulation mode David Hildenbrand 2019-10-31 14:29 ` David Hildenbrand 2019-10-31 14:29 ` [PATCH v1 11/12] mm: remove the memory isolate notifier David Hildenbrand 2019-10-31 14:29 ` David Hildenbrand 2019-10-31 14:29 ` [PATCH v1 12/12] mm: remove "count" parameter from has_unmovable_pages() David Hildenbrand 2019-10-31 14:29 ` David Hildenbrand
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191031142933.10779-1-david@redhat.com \ --to=david@redhat.com \ --cc=aik@ozlabs.ru \ --cc=akpm@linux-foundation.org \ --cc=alexander.h.duyck@linux.intel.com \ --cc=allison@lohutok.net \ --cc=anshuman.khandual@arm.com \ --cc=arunks@codeaurora.org \ --cc=bauerman@linux.ibm.com \ --cc=benh@kernel.crashing.org \ --cc=cai@lca.pw \ --cc=christian@brauner.io \ --cc=dan.j.williams@intel.com \ --cc=dhowells@redhat.com \ --cc=geert@linux-m68k.org \ --cc=ghackmann@google.com \ --cc=glider@google.com \ --cc=gregkh@linuxfoundation.org \ --cc=info@metux.net \ --cc=kernelfans@gmail.com \ --cc=khandual@linux.vnet.ibm.com \ --cc=khlebnikov@yandex-team.ru \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=mgorman@techsingularity.net \ --cc=mhocko@suse.com \ --cc=mpe@ellerman.id.au \ --cc=oohall@gmail.com \ --cc=osalvador@suse.de \ --cc=pasha.tatashin@soleen.com \ --cc=paulus@samba.org \ --cc=rafael@kernel.org \ --cc=rfontana@redhat.com \ --cc=richardw.yang@linux.intel.com \ --cc=rppt@linux.vnet.ibm.com \ --cc=sfr@canb.auug.org.au \ --cc=tglx@linutronix.de \ --cc=tkjos@google.com \ --cc=vbabka@suse.cz \ --cc=xiang@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.