From: Alexandru Elisei <alexandru.elisei@arm.com> To: catalin.marinas@arm.com, will@kernel.org, oliver.upton@linux.dev, maz@kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, arnd@arndb.de, akpm@linux-foundation.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mhiramat@kernel.org, rppt@kernel.org, hughd@google.com Cc: pcc@google.com, steven.price@arm.com, anshuman.khandual@arm.com, vincenzo.frascino@arm.com, david@redhat.com, eugenis@google.com, kcc@google.com, hyesoo.yu@samsung.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH RFC 06/37] mm: page_alloc: Allocate from movable pcp lists only if ALLOC_FROM_METADATA Date: Wed, 23 Aug 2023 14:13:19 +0100 [thread overview] Message-ID: <20230823131350.114942-7-alexandru.elisei@arm.com> (raw) In-Reply-To: <20230823131350.114942-1-alexandru.elisei@arm.com> pcp lists keep MIGRATE_METADATA pages on the MIGRATE_MOVABLE list. Make sure pages from the movable list are allocated only when the ALLOC_FROM_METADATA alloc flag is set, as otherwise the page allocator could end up allocating a metadata page when that page cannot be used. __alloc_pages_bulk() sidesteps rmqueue() and calls __rmqueue_pcplist() directly. Add a check for the flag before calling __rmqueue_pcplist(), and fallback to __alloc_pages() if the check is false. Note that CMA isn't a problem for __alloc_pages_bulk(): an allocation can always use CMA pages if the requested migratetype is MIGRATE_MOVABLE, which is not the case with MIGRATE_METADATA pages. Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> --- mm/page_alloc.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 829134a4dfa8..a693e23c4733 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2845,11 +2845,16 @@ struct page *rmqueue(struct zone *preferred_zone, if (likely(pcp_allowed_order(order))) { /* - * MIGRATE_MOVABLE pcplist could have the pages on CMA area and - * we need to skip it when CMA area isn't allowed. + * PCP lists keep MIGRATE_CMA/MIGRATE_METADATA pages on the same + * movable list. Make sure it's allowed to allocate both type of + * pages before allocating from the movable list. */ - if (!IS_ENABLED(CONFIG_CMA) || alloc_flags & ALLOC_CMA || - migratetype != MIGRATE_MOVABLE) { + bool movable_allowed = (!IS_ENABLED(CONFIG_CMA) || + (alloc_flags & ALLOC_CMA)) && + (!IS_ENABLED(CONFIG_MEMORY_METADATA) || + (alloc_flags & ALLOC_FROM_METADATA)); + + if (migratetype != MIGRATE_MOVABLE || movable_allowed) { page = rmqueue_pcplist(preferred_zone, zone, order, migratetype, alloc_flags); if (likely(page)) @@ -4388,6 +4393,14 @@ unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid, goto out; gfp = alloc_gfp; + /* + * pcp lists puts MIGRATE_METADATA on the MIGRATE_MOVABLE list, don't + * use pcp if allocating metadata pages is not allowed. + */ + if (metadata_storage_enabled() && ac.migratetype == MIGRATE_MOVABLE && + !(alloc_flags & ALLOC_FROM_METADATA)) + goto failed; + /* Find an allowed local zone that meets the low watermark. */ for_each_zone_zonelist_nodemask(zone, z, ac.zonelist, ac.highest_zoneidx, ac.nodemask) { unsigned long mark; -- 2.41.0
WARNING: multiple messages have this Message-ID (diff)
From: Alexandru Elisei <alexandru.elisei@arm.com> To: catalin.marinas@arm.com, will@kernel.org, oliver.upton@linux.dev, maz@kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, arnd@arndb.de, akpm@linux-foundation.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mhiramat@kernel.org, rppt@kernel.org, hughd@google.com Cc: pcc@google.com, steven.price@arm.com, anshuman.khandual@arm.com, vincenzo.frascino@arm.com, david@redhat.com, eugenis@google.com, kcc@google.com, hyesoo.yu@samsung.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH RFC 06/37] mm: page_alloc: Allocate from movable pcp lists only if ALLOC_FROM_METADATA Date: Wed, 23 Aug 2023 14:13:19 +0100 [thread overview] Message-ID: <20230823131350.114942-7-alexandru.elisei@arm.com> (raw) In-Reply-To: <20230823131350.114942-1-alexandru.elisei@arm.com> pcp lists keep MIGRATE_METADATA pages on the MIGRATE_MOVABLE list. Make sure pages from the movable list are allocated only when the ALLOC_FROM_METADATA alloc flag is set, as otherwise the page allocator could end up allocating a metadata page when that page cannot be used. __alloc_pages_bulk() sidesteps rmqueue() and calls __rmqueue_pcplist() directly. Add a check for the flag before calling __rmqueue_pcplist(), and fallback to __alloc_pages() if the check is false. Note that CMA isn't a problem for __alloc_pages_bulk(): an allocation can always use CMA pages if the requested migratetype is MIGRATE_MOVABLE, which is not the case with MIGRATE_METADATA pages. Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> --- mm/page_alloc.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 829134a4dfa8..a693e23c4733 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2845,11 +2845,16 @@ struct page *rmqueue(struct zone *preferred_zone, if (likely(pcp_allowed_order(order))) { /* - * MIGRATE_MOVABLE pcplist could have the pages on CMA area and - * we need to skip it when CMA area isn't allowed. + * PCP lists keep MIGRATE_CMA/MIGRATE_METADATA pages on the same + * movable list. Make sure it's allowed to allocate both type of + * pages before allocating from the movable list. */ - if (!IS_ENABLED(CONFIG_CMA) || alloc_flags & ALLOC_CMA || - migratetype != MIGRATE_MOVABLE) { + bool movable_allowed = (!IS_ENABLED(CONFIG_CMA) || + (alloc_flags & ALLOC_CMA)) && + (!IS_ENABLED(CONFIG_MEMORY_METADATA) || + (alloc_flags & ALLOC_FROM_METADATA)); + + if (migratetype != MIGRATE_MOVABLE || movable_allowed) { page = rmqueue_pcplist(preferred_zone, zone, order, migratetype, alloc_flags); if (likely(page)) @@ -4388,6 +4393,14 @@ unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid, goto out; gfp = alloc_gfp; + /* + * pcp lists puts MIGRATE_METADATA on the MIGRATE_MOVABLE list, don't + * use pcp if allocating metadata pages is not allowed. + */ + if (metadata_storage_enabled() && ac.migratetype == MIGRATE_MOVABLE && + !(alloc_flags & ALLOC_FROM_METADATA)) + goto failed; + /* Find an allowed local zone that meets the low watermark. */ for_each_zone_zonelist_nodemask(zone, z, ac.zonelist, ac.highest_zoneidx, ac.nodemask) { unsigned long mark; -- 2.41.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2023-08-23 13:15 UTC|newest] Thread overview: 136+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-08-23 13:13 [PATCH RFC 00/37] Add support for arm64 MTE dynamic tag storage reuse Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 01/37] mm: page_alloc: Rename gfp_to_alloc_flags_cma -> gfp_to_alloc_flags_fast Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 02/37] arm64: mte: Rework naming for tag manipulation functions Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 03/37] arm64: mte: Rename __GFP_ZEROTAGS to __GFP_TAGGED Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 04/37] mm: Add MIGRATE_METADATA allocation policy Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei [not found] ` <CGME20231012013834epcas2p28ff3162673294077caef3b0794b69e72@epcas2p2.samsung.com> 2023-10-12 1:28 ` Hyesoo Yu 2023-10-12 1:28 ` Hyesoo Yu 2023-10-16 12:40 ` Alexandru Elisei 2023-10-16 12:40 ` Alexandru Elisei 2023-10-23 7:52 ` Hyesoo Yu 2023-10-23 7:52 ` Hyesoo Yu 2023-08-23 13:13 ` [PATCH RFC 05/37] mm: Add memory statistics for the " Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei [this message] 2023-08-23 13:13 ` [PATCH RFC 06/37] mm: page_alloc: Allocate from movable pcp lists only if ALLOC_FROM_METADATA Alexandru Elisei [not found] ` <CGME20231012013524epcas2p4b50f306e3e4d0b937b31f978022844e5@epcas2p4.samsung.com> 2023-10-12 1:25 ` Hyesoo Yu 2023-10-12 1:25 ` Hyesoo Yu 2023-10-16 12:41 ` Alexandru Elisei 2023-10-16 12:41 ` Alexandru Elisei 2023-10-17 10:26 ` Catalin Marinas 2023-10-17 10:26 ` Catalin Marinas 2023-10-23 7:16 ` Hyesoo Yu 2023-10-23 7:16 ` Hyesoo Yu 2023-10-23 10:50 ` Catalin Marinas 2023-10-23 10:50 ` Catalin Marinas 2023-10-23 11:55 ` David Hildenbrand 2023-10-23 11:55 ` David Hildenbrand 2023-10-23 17:08 ` Catalin Marinas 2023-10-23 17:08 ` Catalin Marinas 2023-10-23 17:22 ` David Hildenbrand 2023-10-23 17:22 ` David Hildenbrand 2023-08-23 13:13 ` [PATCH RFC 07/37] mm: page_alloc: Bypass pcp when freeing MIGRATE_METADATA pages Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 08/37] mm: compaction: Account for free metadata pages in __compact_finished() Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 09/37] mm: compaction: Handle metadata pages as source for direct compaction Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 10/37] mm: compaction: Do not use MIGRATE_METADATA to replace pages with metadata Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 11/37] mm: migrate/mempolicy: Allocate metadata-enabled destination page Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 12/37] mm: gup: Don't allow longterm pinning of MIGRATE_METADATA pages Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 13/37] arm64: mte: Reserve tag storage memory Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 14/37] arm64: mte: Expose tag storage pages to the MIGRATE_METADATA freelist Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 15/37] arm64: mte: Make tag storage depend on ARCH_KEEP_MEMBLOCK Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 16/37] arm64: mte: Move tag storage to MIGRATE_MOVABLE when MTE is disabled Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 17/37] arm64: mte: Disable dynamic tag storage management if HW KASAN is enabled Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei [not found] ` <CGME20231012014514epcas2p3ca99a067f3044c5753309a08cd0b05c4@epcas2p3.samsung.com> 2023-10-12 1:35 ` Hyesoo Yu 2023-10-12 1:35 ` Hyesoo Yu 2023-10-16 12:42 ` Alexandru Elisei 2023-10-16 12:42 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 18/37] arm64: mte: Check that tag storage blocks are in the same zone Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 19/37] mm: page_alloc: Manage metadata storage on page allocation Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 20/37] mm: compaction: Reserve metadata storage in compaction_alloc() Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-11-21 4:49 ` Peter Collingbourne 2023-11-21 4:49 ` Peter Collingbourne 2023-11-21 11:54 ` Alexandru Elisei 2023-11-21 11:54 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 21/37] mm: khugepaged: Handle metadata-enabled VMAs Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 22/37] mm: shmem: Allocate metadata storage for in-memory filesystems Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 23/37] mm: Teach vma_alloc_folio() about metadata-enabled VMAs Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 24/37] mm: page_alloc: Teach alloc_contig_range() about MIGRATE_METADATA Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 25/37] arm64: mte: Manage tag storage on page allocation Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 26/37] arm64: mte: Perform CMOs for tag blocks on tagged page allocation/free Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 27/37] arm64: mte: Reserve tag block for the zero page Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 28/37] mm: sched: Introduce PF_MEMALLOC_ISOLATE Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 29/37] mm: arm64: Define the PAGE_METADATA_NONE page protection Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 30/37] mm: mprotect: arm64: Set PAGE_METADATA_NONE for mprotect(PROT_MTE) Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 31/37] mm: arm64: Set PAGE_METADATA_NONE in set_pte_at() if missing metadata storage Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 32/37] mm: Call arch_swap_prepare_to_restore() before arch_swap_restore() Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 33/37] arm64: mte: swap/copypage: Handle tag restoring when missing tag storage Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 34/37] arm64: mte: Handle fatal signal in reserve_metadata_storage() Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 35/37] mm: hugepage: Handle PAGE_METADATA_NONE faults for huge pages Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 36/37] KVM: arm64: Disable MTE is tag storage is enabled Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-23 13:13 ` [PATCH RFC 37/37] arm64: mte: Enable tag storage management Alexandru Elisei 2023-08-23 13:13 ` Alexandru Elisei 2023-08-24 7:50 ` [PATCH RFC 00/37] Add support for arm64 MTE dynamic tag storage reuse David Hildenbrand 2023-08-24 7:50 ` David Hildenbrand 2023-08-24 10:44 ` Catalin Marinas 2023-08-24 10:44 ` Catalin Marinas 2023-08-24 11:06 ` David Hildenbrand 2023-08-24 11:06 ` David Hildenbrand 2023-08-24 11:25 ` David Hildenbrand 2023-08-24 11:25 ` David Hildenbrand 2023-08-24 15:24 ` Catalin Marinas 2023-08-24 15:24 ` Catalin Marinas 2023-09-06 11:23 ` Alexandru Elisei 2023-09-06 11:23 ` Alexandru Elisei 2023-09-11 11:52 ` Catalin Marinas 2023-09-11 11:52 ` Catalin Marinas 2023-09-11 12:29 ` David Hildenbrand 2023-09-11 12:29 ` David Hildenbrand 2023-09-13 15:29 ` Catalin Marinas 2023-09-13 15:29 ` Catalin Marinas [not found] ` <CGME20231025031004epcas2p485a0b7a9247bc61d54064d7f7bdd1e89@epcas2p4.samsung.com> 2023-10-25 2:59 ` Hyesoo Yu 2023-10-25 2:59 ` Hyesoo Yu 2023-10-25 8:47 ` Alexandru Elisei 2023-10-25 8:47 ` Alexandru Elisei 2023-10-25 8:52 ` Hyesoo Yu 2023-10-25 8:52 ` Hyesoo Yu 2023-10-27 11:04 ` Catalin Marinas 2023-10-27 11:04 ` Catalin Marinas 2023-09-13 8:11 ` Kuan-Ying Lee (李冠穎) 2023-09-13 8:11 ` Kuan-Ying Lee (李冠穎) 2023-09-14 17:37 ` Catalin Marinas 2023-09-14 17:37 ` Catalin Marinas
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20230823131350.114942-7-alexandru.elisei@arm.com \ --to=alexandru.elisei@arm.com \ --cc=akpm@linux-foundation.org \ --cc=anshuman.khandual@arm.com \ --cc=arnd@arndb.de \ --cc=bristot@redhat.com \ --cc=bsegall@google.com \ --cc=catalin.marinas@arm.com \ --cc=david@redhat.com \ --cc=dietmar.eggemann@arm.com \ --cc=eugenis@google.com \ --cc=hughd@google.com \ --cc=hyesoo.yu@samsung.com \ --cc=james.morse@arm.com \ --cc=juri.lelli@redhat.com \ --cc=kcc@google.com \ --cc=kvmarm@lists.linux.dev \ --cc=linux-arch@vger.kernel.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-trace-kernel@vger.kernel.org \ --cc=maz@kernel.org \ --cc=mgorman@suse.de \ --cc=mhiramat@kernel.org \ --cc=mingo@redhat.com \ --cc=oliver.upton@linux.dev \ --cc=pcc@google.com \ --cc=peterz@infradead.org \ --cc=rostedt@goodmis.org \ --cc=rppt@kernel.org \ --cc=steven.price@arm.com \ --cc=suzuki.poulose@arm.com \ --cc=vincent.guittot@linaro.org \ --cc=vincenzo.frascino@arm.com \ --cc=vschneid@redhat.com \ --cc=will@kernel.org \ --cc=yuzenghui@huawei.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.