From: Alexander Duyck <alexander.h.duyck@linux.intel.com> To: linux-mm@kvack.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org Cc: pavel.tatashin@microsoft.com, mhocko@suse.com, dave.hansen@intel.com, jglisse@redhat.com, rppt@linux.vnet.ibm.com, mingo@kernel.org, kirill.shutemov@linux.intel.com Subject: [PATCH v5 3/4] mm: Create non-atomic version of SetPageReserved for init use Date: Tue, 25 Sep 2018 13:20:47 -0700 [thread overview] Message-ID: <20180925202018.3576.11607.stgit@localhost.localdomain> (raw) In-Reply-To: <20180925200551.3576.18755.stgit@localhost.localdomain> It doesn't make much sense to use the atomic SetPageReserved at init time when we are using memset to clear the memory and manipulating the page flags via simple "&=" and "|=" operations in __init_single_page. This patch adds a non-atomic version __SetPageReserved that can be used during page init and shows about a 10% improvement in initialization times on the systems I have available for testing. On those systems I saw initialization times drop from around 35 seconds to around 32 seconds to initialize a 3TB block of persistent memory. I believe the main advantage of this is that it allows for more compiler optimization as the __set_bit operation can be reordered whereas the atomic version cannot. I tried adding a bit of documentation based on commit <f1dd2cd13c4> ("mm, memory_hotplug: do not associate hotadded memory to zones until online"). Ideally the reserved flag should be set earlier since there is a brief window where the page is initialization via __init_single_page and we have not set the PG_Reserved flag. I'm leaving that for a future patch set as that will require a more significant refactor. Acked-by: Michal Hocko <mhocko@suse.com> Reviewed-by: Pavel Tatashin <pavel.tatashin@microsoft.com> Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com> --- v4: Added comment about __set_bit vs set_bit to the patch description v5: No change include/linux/page-flags.h | 1 + mm/page_alloc.c | 9 +++++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 934f91ef3f54..50ce1bddaf56 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -303,6 +303,7 @@ static inline void page_init_poison(struct page *page, size_t size) PAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) __CLEARPAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) + __SETPAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) PAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) __CLEARPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) __SETPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 511447ac02cf..926ad3083b28 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1238,7 +1238,12 @@ void __meminit reserve_bootmem_region(phys_addr_t start, phys_addr_t end) /* Avoid false-positive PageTail() */ INIT_LIST_HEAD(&page->lru); - SetPageReserved(page); + /* + * no need for atomic set_bit because the struct + * page is not visible yet so nobody should + * access it yet. + */ + __SetPageReserved(page); } } } @@ -5512,7 +5517,7 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, page = pfn_to_page(pfn); __init_single_page(page, pfn, zone, nid); if (context == MEMMAP_HOTPLUG) - SetPageReserved(page); + __SetPageReserved(page); /* * Mark the block movable so that blocks are reserved for _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm
WARNING: multiple messages have this Message-ID (diff)
From: Alexander Duyck <alexander.h.duyck@linux.intel.com> To: linux-mm@kvack.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org Cc: pavel.tatashin@microsoft.com, mhocko@suse.com, dave.jiang@intel.com, dave.hansen@intel.com, jglisse@redhat.com, rppt@linux.vnet.ibm.com, dan.j.williams@intel.com, logang@deltatee.com, mingo@kernel.org, kirill.shutemov@linux.intel.com Subject: [PATCH v5 3/4] mm: Create non-atomic version of SetPageReserved for init use Date: Tue, 25 Sep 2018 13:20:47 -0700 [thread overview] Message-ID: <20180925202018.3576.11607.stgit@localhost.localdomain> (raw) In-Reply-To: <20180925200551.3576.18755.stgit@localhost.localdomain> It doesn't make much sense to use the atomic SetPageReserved at init time when we are using memset to clear the memory and manipulating the page flags via simple "&=" and "|=" operations in __init_single_page. This patch adds a non-atomic version __SetPageReserved that can be used during page init and shows about a 10% improvement in initialization times on the systems I have available for testing. On those systems I saw initialization times drop from around 35 seconds to around 32 seconds to initialize a 3TB block of persistent memory. I believe the main advantage of this is that it allows for more compiler optimization as the __set_bit operation can be reordered whereas the atomic version cannot. I tried adding a bit of documentation based on commit <f1dd2cd13c4> ("mm, memory_hotplug: do not associate hotadded memory to zones until online"). Ideally the reserved flag should be set earlier since there is a brief window where the page is initialization via __init_single_page and we have not set the PG_Reserved flag. I'm leaving that for a future patch set as that will require a more significant refactor. Acked-by: Michal Hocko <mhocko@suse.com> Reviewed-by: Pavel Tatashin <pavel.tatashin@microsoft.com> Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com> --- v4: Added comment about __set_bit vs set_bit to the patch description v5: No change include/linux/page-flags.h | 1 + mm/page_alloc.c | 9 +++++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 934f91ef3f54..50ce1bddaf56 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -303,6 +303,7 @@ static inline void page_init_poison(struct page *page, size_t size) PAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) __CLEARPAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) + __SETPAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) PAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) __CLEARPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) __SETPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 511447ac02cf..926ad3083b28 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1238,7 +1238,12 @@ void __meminit reserve_bootmem_region(phys_addr_t start, phys_addr_t end) /* Avoid false-positive PageTail() */ INIT_LIST_HEAD(&page->lru); - SetPageReserved(page); + /* + * no need for atomic set_bit because the struct + * page is not visible yet so nobody should + * access it yet. + */ + __SetPageReserved(page); } } } @@ -5512,7 +5517,7 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, page = pfn_to_page(pfn); __init_single_page(page, pfn, zone, nid); if (context == MEMMAP_HOTPLUG) - SetPageReserved(page); + __SetPageReserved(page); /* * Mark the block movable so that blocks are reserved for
next prev parent reply other threads:[~2018-09-25 20:21 UTC|newest] Thread overview: 144+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-09-25 20:18 [PATCH v5 0/4] Address issues slowing persistent memory initialization Alexander Duyck 2018-09-25 20:18 ` Alexander Duyck 2018-09-25 20:19 ` [PATCH v5 1/4] mm: Remove now defunct NO_BOOTMEM from depends list for deferred init Alexander Duyck 2018-09-25 20:19 ` Alexander Duyck 2018-09-25 21:05 ` Mike Rapoport 2018-09-25 21:05 ` Mike Rapoport 2018-09-25 20:20 ` [PATCH v5 2/4] mm: Provide kernel parameter to allow disabling page init poisoning Alexander Duyck 2018-09-25 20:20 ` Alexander Duyck 2018-09-25 20:26 ` Dave Hansen 2018-09-25 20:26 ` Dave Hansen 2018-09-25 20:38 ` Alexander Duyck 2018-09-25 20:38 ` Alexander Duyck 2018-09-25 22:14 ` Dave Hansen 2018-09-25 22:14 ` Dave Hansen 2018-09-25 22:14 ` Dave Hansen 2018-09-25 22:27 ` Alexander Duyck 2018-09-25 22:27 ` Alexander Duyck 2018-09-25 22:27 ` Alexander Duyck 2018-09-26 7:38 ` Michal Hocko 2018-09-26 7:38 ` Michal Hocko 2018-09-26 15:24 ` Alexander Duyck 2018-09-26 15:39 ` Michal Hocko 2018-09-26 15:39 ` Michal Hocko 2018-09-26 15:41 ` Dave Hansen 2018-09-26 15:41 ` Dave Hansen 2018-09-26 16:18 ` Alexander Duyck 2018-09-26 15:36 ` Dave Hansen 2018-09-26 22:36 ` Andrew Morton 2018-09-26 22:36 ` Andrew Morton 2018-09-25 20:20 ` Alexander Duyck [this message] 2018-09-25 20:20 ` [PATCH v5 3/4] mm: Create non-atomic version of SetPageReserved for init use Alexander Duyck 2018-09-25 20:21 ` [PATCH v5 4/4] mm: Defer ZONE_DEVICE page initialization to the point where we init pgmap Alexander Duyck 2018-09-25 20:21 ` Alexander Duyck 2018-09-26 7:55 ` Michal Hocko 2018-09-26 18:25 ` Alexander Duyck 2018-09-26 18:25 ` Alexander Duyck 2018-09-26 18:52 ` Dan Williams 2018-09-26 18:52 ` Dan Williams 2018-09-27 11:20 ` Michal Hocko 2018-09-27 11:20 ` Michal Hocko 2018-09-27 11:09 ` Michal Hocko 2018-09-27 11:09 ` Michal Hocko 2018-09-27 12:25 ` Oscar Salvador 2018-09-27 13:13 ` Michal Hocko 2018-09-27 14:50 ` Oscar Salvador 2018-09-27 14:50 ` Oscar Salvador 2018-09-27 14:50 ` Oscar Salvador 2018-09-27 15:41 ` David Hildenbrand 2018-09-27 15:41 ` David Hildenbrand 2018-09-28 8:12 ` Oscar Salvador 2018-09-28 8:12 ` Oscar Salvador 2018-09-28 8:44 ` Oscar Salvador 2018-09-28 8:44 ` Oscar Salvador 2018-09-28 15:50 ` Dan Williams 2018-09-28 15:50 ` Dan Williams 2018-09-27 12:32 ` Oscar Salvador 2018-10-08 21:01 ` Dan Williams 2018-10-08 21:01 ` Dan Williams 2018-10-08 21:38 ` Alexander Duyck 2018-10-08 21:38 ` Alexander Duyck 2018-10-08 22:00 ` Dan Williams 2018-10-08 22:00 ` Dan Williams 2018-10-08 22:00 ` Dan Williams 2018-10-08 22:07 ` Alexander Duyck 2018-10-08 22:07 ` Alexander Duyck 2018-10-08 22:36 ` Alexander Duyck 2018-10-08 22:36 ` Alexander Duyck 2018-10-08 22:59 ` Dan Williams 2018-10-08 23:34 ` [mm PATCH] memremap: Fix reference count for pgmap in devm_memremap_pages Alexander Duyck 2018-10-08 23:34 ` Alexander Duyck 2018-10-09 0:20 ` Dan Williams 2018-10-09 0:20 ` Dan Williams 2018-10-09 17:00 ` [PATCH v5 4/4] mm: Defer ZONE_DEVICE page initialization to the point where we init pgmap Yi Zhang 2018-10-09 17:00 ` Yi Zhang 2018-10-09 18:04 ` Dan Williams 2018-10-09 18:04 ` Dan Williams 2018-10-09 20:26 ` Alexander Duyck 2018-10-09 20:26 ` Alexander Duyck 2018-10-09 21:19 ` Dan Williams 2018-10-09 21:19 ` Dan Williams 2018-10-10 12:52 ` Yi Zhang 2018-10-10 12:52 ` Yi Zhang 2018-10-10 15:27 ` Alexander Duyck 2018-10-10 15:27 ` Alexander Duyck 2018-10-11 8:17 ` Yi Zhang 2018-10-11 8:17 ` Yi Zhang 2018-10-10 9:58 ` Michal Hocko 2018-10-10 16:39 ` Alexander Duyck 2018-10-10 16:39 ` Alexander Duyck 2018-10-10 17:24 ` Michal Hocko 2018-10-10 17:24 ` Michal Hocko 2018-10-10 17:39 ` Alexander Duyck 2018-10-10 17:39 ` Alexander Duyck 2018-10-10 17:53 ` Michal Hocko 2018-10-10 17:53 ` Michal Hocko 2018-10-10 18:13 ` Alexander Duyck 2018-10-10 18:13 ` Alexander Duyck 2018-10-10 18:52 ` Michal Hocko 2018-10-10 18:52 ` Michal Hocko 2018-10-11 8:55 ` Michal Hocko 2018-10-11 8:55 ` Michal Hocko 2018-10-11 17:38 ` Alexander Duyck 2018-10-11 18:22 ` Dan Williams 2018-10-11 18:22 ` Dan Williams 2018-10-17 7:52 ` Michal Hocko 2018-10-17 7:52 ` Michal Hocko 2018-10-17 15:02 ` Alexander Duyck 2018-10-17 15:02 ` Alexander Duyck 2018-10-29 14:12 ` Michal Hocko 2018-10-29 14:12 ` Michal Hocko 2018-10-29 15:59 ` Alexander Duyck 2018-10-29 15:59 ` Alexander Duyck 2018-10-29 15:59 ` Alexander Duyck 2018-10-29 16:35 ` Michal Hocko 2018-10-29 16:35 ` Michal Hocko 2018-10-29 17:01 ` Alexander Duyck 2018-10-29 17:24 ` Michal Hocko 2018-10-29 17:24 ` Michal Hocko 2018-10-29 17:34 ` Dan Williams 2018-10-29 17:34 ` Dan Williams 2018-10-29 17:45 ` Michal Hocko 2018-10-29 17:45 ` Michal Hocko 2018-10-29 17:42 ` Alexander Duyck 2018-10-29 17:42 ` Alexander Duyck 2018-10-29 18:18 ` Michal Hocko 2018-10-29 18:18 ` Michal Hocko 2018-10-29 19:59 ` Alexander Duyck 2018-10-29 19:59 ` Alexander Duyck 2018-10-30 6:29 ` Michal Hocko 2018-10-30 6:29 ` Michal Hocko 2018-10-30 6:55 ` Dan Williams 2018-10-30 8:17 ` Michal Hocko 2018-10-30 8:17 ` Michal Hocko 2018-10-30 15:57 ` Dan Williams 2018-10-30 8:05 ` Oscar Salvador 2018-10-29 15:49 ` Dan Williams 2018-10-29 15:49 ` Dan Williams 2018-10-29 15:56 ` Michal Hocko 2018-10-10 18:18 ` Dan Williams 2018-10-10 18:18 ` Dan Williams 2018-10-11 8:39 ` Yi Zhang 2018-10-11 8:39 ` Yi Zhang 2018-10-11 15:38 ` Alexander Duyck 2018-10-11 15:38 ` Alexander Duyck
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180925202018.3576.11607.stgit@localhost.localdomain \ --to=alexander.h.duyck@linux.intel.com \ --cc=akpm@linux-foundation.org \ --cc=dave.hansen@intel.com \ --cc=jglisse@redhat.com \ --cc=kirill.shutemov@linux.intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-nvdimm@lists.01.org \ --cc=mhocko@suse.com \ --cc=mingo@kernel.org \ --cc=pavel.tatashin@microsoft.com \ --cc=rppt@linux.vnet.ibm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.