From: andrey.konovalov@linux.dev To: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Konovalov <andreyknvl@gmail.com>, Marco Elver <elver@google.com>, Alexander Potapenko <glider@google.com>, Dmitry Vyukov <dvyukov@google.com>, Andrey Ryabinin <ryabinin.a.a@gmail.com>, kasan-dev@googlegroups.com, linux-mm@kvack.org, Vincenzo Frascino <vincenzo.frascino@arm.com>, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, linux-arm-kernel@lists.infradead.org, Peter Collingbourne <pcc@google.com>, Evgenii Stepanov <eugenis@google.com>, linux-kernel@vger.kernel.org, Andrey Konovalov <andreyknvl@google.com> Subject: [PATCH v6 10/39] kasan, page_alloc: merge kasan_alloc_pages into post_alloc_hook Date: Mon, 24 Jan 2022 19:02:18 +0100 [thread overview] Message-ID: <5ac7e0b30f5cbb177ec363ddd7878a3141289592.1643047180.git.andreyknvl@google.com> (raw) In-Reply-To: <cover.1643047180.git.andreyknvl@google.com> From: Andrey Konovalov <andreyknvl@google.com> Currently, the code responsible for initializing and poisoning memory in post_alloc_hook() is scattered across two locations: kasan_alloc_pages() hook for HW_TAGS KASAN and post_alloc_hook() itself. This is confusing. This and a few following patches combine the code from these two locations. Along the way, these patches do a step-by-step restructure the many performed checks to make them easier to follow. Replace the only caller of kasan_alloc_pages() with its implementation. As kasan_has_integrated_init() is only true when CONFIG_KASAN_HW_TAGS is enabled, moving the code does no functional changes. Also move init and init_tags variables definitions out of kasan_has_integrated_init() clause in post_alloc_hook(), as they have the same values regardless of what the if condition evaluates to. This patch is not useful by itself but makes the simplifications in the following patches easier to follow. Signed-off-by: Andrey Konovalov <andreyknvl@google.com> --- Changes v2->v3: - Update patch description. --- include/linux/kasan.h | 9 --------- mm/kasan/common.c | 2 +- mm/kasan/hw_tags.c | 22 ---------------------- mm/page_alloc.c | 20 +++++++++++++++----- 4 files changed, 16 insertions(+), 37 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index a8bfe9f157c9..b88ca6b97ba3 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -95,8 +95,6 @@ static inline bool kasan_hw_tags_enabled(void) return kasan_enabled(); } -void kasan_alloc_pages(struct page *page, unsigned int order, gfp_t flags); - #else /* CONFIG_KASAN_HW_TAGS */ static inline bool kasan_enabled(void) @@ -109,13 +107,6 @@ static inline bool kasan_hw_tags_enabled(void) return false; } -static __always_inline void kasan_alloc_pages(struct page *page, - unsigned int order, gfp_t flags) -{ - /* Only available for integrated init. */ - BUILD_BUG(); -} - #endif /* CONFIG_KASAN_HW_TAGS */ static inline bool kasan_has_integrated_init(void) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index a0082fad48b1..d9079ec11f31 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -538,7 +538,7 @@ void * __must_check __kasan_kmalloc_large(const void *ptr, size_t size, return NULL; /* - * The object has already been unpoisoned by kasan_alloc_pages() for + * The object has already been unpoisoned by kasan_unpoison_pages() for * alloc_pages() or by kasan_krealloc() for krealloc(). */ diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index c643740b8599..76cf2b6229c7 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -192,28 +192,6 @@ void __init kasan_init_hw_tags(void) kasan_stack_collection_enabled() ? "on" : "off"); } -void kasan_alloc_pages(struct page *page, unsigned int order, gfp_t flags) -{ - /* - * This condition should match the one in post_alloc_hook() in - * page_alloc.c. - */ - bool init = !want_init_on_free() && want_init_on_alloc(flags); - bool init_tags = init && (flags & __GFP_ZEROTAGS); - - if (flags & __GFP_SKIP_KASAN_POISON) - SetPageSkipKASanPoison(page); - - if (init_tags) { - int i; - - for (i = 0; i != 1 << order; ++i) - tag_clear_highpage(page + i); - } else { - kasan_unpoison_pages(page, order, init); - } -} - #if IS_ENABLED(CONFIG_KASAN_KUNIT_TEST) void kasan_enable_tagging_sync(void) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 868480d463c7..abed862d889d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2398,6 +2398,9 @@ static bool check_new_pages(struct page *page, unsigned int order) inline void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags) { + bool init = !want_init_on_free() && want_init_on_alloc(gfp_flags); + bool init_tags = init && (gfp_flags & __GFP_ZEROTAGS); + set_page_private(page, 0); set_page_refcounted(page); @@ -2413,15 +2416,22 @@ inline void post_alloc_hook(struct page *page, unsigned int order, /* * As memory initialization might be integrated into KASAN, - * kasan_alloc_pages and kernel_init_free_pages must be + * KASAN unpoisoning and memory initializion code must be * kept together to avoid discrepancies in behavior. */ if (kasan_has_integrated_init()) { - kasan_alloc_pages(page, order, gfp_flags); - } else { - bool init = !want_init_on_free() && want_init_on_alloc(gfp_flags); - bool init_tags = init && (gfp_flags & __GFP_ZEROTAGS); + if (gfp_flags & __GFP_SKIP_KASAN_POISON) + SetPageSkipKASanPoison(page); + + if (init_tags) { + int i; + for (i = 0; i != 1 << order; ++i) + tag_clear_highpage(page + i); + } else { + kasan_unpoison_pages(page, order, init); + } + } else { kasan_unpoison_pages(page, order, init); if (init_tags) { -- 2.25.1
WARNING: multiple messages have this Message-ID (diff)
From: andrey.konovalov@linux.dev To: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Konovalov <andreyknvl@gmail.com>, Marco Elver <elver@google.com>, Alexander Potapenko <glider@google.com>, Dmitry Vyukov <dvyukov@google.com>, Andrey Ryabinin <ryabinin.a.a@gmail.com>, kasan-dev@googlegroups.com, linux-mm@kvack.org, Vincenzo Frascino <vincenzo.frascino@arm.com>, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, linux-arm-kernel@lists.infradead.org, Peter Collingbourne <pcc@google.com>, Evgenii Stepanov <eugenis@google.com>, linux-kernel@vger.kernel.org, Andrey Konovalov <andreyknvl@google.com> Subject: [PATCH v6 10/39] kasan, page_alloc: merge kasan_alloc_pages into post_alloc_hook Date: Mon, 24 Jan 2022 19:02:18 +0100 [thread overview] Message-ID: <5ac7e0b30f5cbb177ec363ddd7878a3141289592.1643047180.git.andreyknvl@google.com> (raw) In-Reply-To: <cover.1643047180.git.andreyknvl@google.com> From: Andrey Konovalov <andreyknvl@google.com> Currently, the code responsible for initializing and poisoning memory in post_alloc_hook() is scattered across two locations: kasan_alloc_pages() hook for HW_TAGS KASAN and post_alloc_hook() itself. This is confusing. This and a few following patches combine the code from these two locations. Along the way, these patches do a step-by-step restructure the many performed checks to make them easier to follow. Replace the only caller of kasan_alloc_pages() with its implementation. As kasan_has_integrated_init() is only true when CONFIG_KASAN_HW_TAGS is enabled, moving the code does no functional changes. Also move init and init_tags variables definitions out of kasan_has_integrated_init() clause in post_alloc_hook(), as they have the same values regardless of what the if condition evaluates to. This patch is not useful by itself but makes the simplifications in the following patches easier to follow. Signed-off-by: Andrey Konovalov <andreyknvl@google.com> --- Changes v2->v3: - Update patch description. --- include/linux/kasan.h | 9 --------- mm/kasan/common.c | 2 +- mm/kasan/hw_tags.c | 22 ---------------------- mm/page_alloc.c | 20 +++++++++++++++----- 4 files changed, 16 insertions(+), 37 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index a8bfe9f157c9..b88ca6b97ba3 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -95,8 +95,6 @@ static inline bool kasan_hw_tags_enabled(void) return kasan_enabled(); } -void kasan_alloc_pages(struct page *page, unsigned int order, gfp_t flags); - #else /* CONFIG_KASAN_HW_TAGS */ static inline bool kasan_enabled(void) @@ -109,13 +107,6 @@ static inline bool kasan_hw_tags_enabled(void) return false; } -static __always_inline void kasan_alloc_pages(struct page *page, - unsigned int order, gfp_t flags) -{ - /* Only available for integrated init. */ - BUILD_BUG(); -} - #endif /* CONFIG_KASAN_HW_TAGS */ static inline bool kasan_has_integrated_init(void) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index a0082fad48b1..d9079ec11f31 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -538,7 +538,7 @@ void * __must_check __kasan_kmalloc_large(const void *ptr, size_t size, return NULL; /* - * The object has already been unpoisoned by kasan_alloc_pages() for + * The object has already been unpoisoned by kasan_unpoison_pages() for * alloc_pages() or by kasan_krealloc() for krealloc(). */ diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index c643740b8599..76cf2b6229c7 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -192,28 +192,6 @@ void __init kasan_init_hw_tags(void) kasan_stack_collection_enabled() ? "on" : "off"); } -void kasan_alloc_pages(struct page *page, unsigned int order, gfp_t flags) -{ - /* - * This condition should match the one in post_alloc_hook() in - * page_alloc.c. - */ - bool init = !want_init_on_free() && want_init_on_alloc(flags); - bool init_tags = init && (flags & __GFP_ZEROTAGS); - - if (flags & __GFP_SKIP_KASAN_POISON) - SetPageSkipKASanPoison(page); - - if (init_tags) { - int i; - - for (i = 0; i != 1 << order; ++i) - tag_clear_highpage(page + i); - } else { - kasan_unpoison_pages(page, order, init); - } -} - #if IS_ENABLED(CONFIG_KASAN_KUNIT_TEST) void kasan_enable_tagging_sync(void) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 868480d463c7..abed862d889d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2398,6 +2398,9 @@ static bool check_new_pages(struct page *page, unsigned int order) inline void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags) { + bool init = !want_init_on_free() && want_init_on_alloc(gfp_flags); + bool init_tags = init && (gfp_flags & __GFP_ZEROTAGS); + set_page_private(page, 0); set_page_refcounted(page); @@ -2413,15 +2416,22 @@ inline void post_alloc_hook(struct page *page, unsigned int order, /* * As memory initialization might be integrated into KASAN, - * kasan_alloc_pages and kernel_init_free_pages must be + * KASAN unpoisoning and memory initializion code must be * kept together to avoid discrepancies in behavior. */ if (kasan_has_integrated_init()) { - kasan_alloc_pages(page, order, gfp_flags); - } else { - bool init = !want_init_on_free() && want_init_on_alloc(gfp_flags); - bool init_tags = init && (gfp_flags & __GFP_ZEROTAGS); + if (gfp_flags & __GFP_SKIP_KASAN_POISON) + SetPageSkipKASanPoison(page); + + if (init_tags) { + int i; + for (i = 0; i != 1 << order; ++i) + tag_clear_highpage(page + i); + } else { + kasan_unpoison_pages(page, order, init); + } + } else { kasan_unpoison_pages(page, order, init); if (init_tags) { -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2022-01-24 18:03 UTC|newest] Thread overview: 113+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-01-24 18:02 [PATCH v6 00/39] kasan, vmalloc, arm64: add vmalloc tagging support for SW/HW_TAGS andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 01/39] kasan, page_alloc: deduplicate should_skip_kasan_poison andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 02/39] kasan, page_alloc: move tag_clear_highpage out of kernel_init_free_pages andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 03/39] kasan, page_alloc: merge kasan_free_pages into free_pages_prepare andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 04/39] kasan, page_alloc: simplify kasan_poison_pages call site andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 05/39] kasan, page_alloc: init memory of skipped pages on free andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 06/39] kasan: drop skip_kasan_poison variable in free_pages_prepare andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 07/39] mm: clarify __GFP_ZEROTAGS comment andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 08/39] kasan: only apply __GFP_ZEROTAGS when memory is zeroed andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 09/39] kasan, page_alloc: refactor init checks in post_alloc_hook andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov [this message] 2022-01-24 18:02 ` [PATCH v6 10/39] kasan, page_alloc: merge kasan_alloc_pages into post_alloc_hook andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 11/39] kasan, page_alloc: combine tag_clear_highpage calls in post_alloc_hook andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 12/39] kasan, page_alloc: move SetPageSkipKASanPoison " andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 13/39] kasan, page_alloc: move kernel_init_free_pages " andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 14/39] kasan, page_alloc: rework kasan_unpoison_pages call site andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 15/39] kasan: clean up metadata byte definitions andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:02 ` [PATCH v6 16/39] kasan: define KASAN_VMALLOC_INVALID for SW_TAGS andrey.konovalov 2022-01-24 18:02 ` andrey.konovalov 2022-01-24 18:04 ` [PATCH v6 17/39] kasan, x86, arm64, s390: rename functions for modules shadow andrey.konovalov 2022-01-24 18:04 ` andrey.konovalov 2022-01-24 18:04 ` [PATCH v6 18/39] kasan, vmalloc: drop outdated VM_KASAN comment andrey.konovalov 2022-01-24 18:04 ` andrey.konovalov 2022-01-24 18:04 ` [PATCH v6 19/39] kasan: reorder vmalloc hooks andrey.konovalov 2022-01-24 18:04 ` andrey.konovalov 2022-01-24 18:04 ` [PATCH v6 20/39] kasan: add wrappers for " andrey.konovalov 2022-01-24 18:04 ` andrey.konovalov 2022-01-24 18:04 ` [PATCH v6 21/39] kasan, vmalloc: reset tags in vmalloc functions andrey.konovalov 2022-01-24 18:04 ` andrey.konovalov 2022-01-24 18:04 ` [PATCH v6 22/39] kasan, fork: reset pointer tags of vmapped stacks andrey.konovalov 2022-01-24 18:04 ` andrey.konovalov 2022-01-24 18:04 ` [PATCH v6 23/39] kasan, arm64: " andrey.konovalov 2022-01-24 18:04 ` andrey.konovalov 2022-01-24 18:04 ` [PATCH v6 24/39] kasan, vmalloc: add vmalloc tagging for SW_TAGS andrey.konovalov 2022-01-24 18:04 ` andrey.konovalov 2022-01-24 18:04 ` [PATCH v6 25/39] kasan, vmalloc, arm64: mark vmalloc mappings as pgprot_tagged andrey.konovalov 2022-01-24 18:04 ` andrey.konovalov 2022-01-24 18:05 ` [PATCH v6 26/39] kasan, vmalloc: unpoison VM_ALLOC pages after mapping andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-01-24 18:05 ` [PATCH v6 27/39] kasan, mm: only define ___GFP_SKIP_KASAN_POISON with HW_TAGS andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-03-23 11:48 ` Vlastimil Babka 2022-03-23 11:48 ` Vlastimil Babka 2022-03-23 13:02 ` Sebastian Andrzej Siewior 2022-03-23 13:02 ` Sebastian Andrzej Siewior 2022-03-23 13:19 ` Vlastimil Babka 2022-03-23 13:19 ` Vlastimil Babka 2022-03-23 13:36 ` Andrey Konovalov 2022-03-23 13:36 ` Andrey Konovalov 2022-03-23 13:57 ` Vlastimil Babka 2022-03-23 13:57 ` Vlastimil Babka 2022-03-23 15:11 ` Matthew Wilcox 2022-03-23 15:11 ` Matthew Wilcox 2022-03-25 21:13 ` Andrew Morton 2022-03-25 21:13 ` Andrew Morton 2022-01-24 18:05 ` [PATCH v6 28/39] kasan, page_alloc: allow skipping unpoisoning for HW_TAGS andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-01-24 18:05 ` [PATCH v6 29/39] kasan, page_alloc: allow skipping memory init " andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-01-24 18:05 ` [PATCH v6 30/39] kasan, vmalloc: add vmalloc tagging " andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-01-25 3:17 ` kernel test robot 2022-01-24 18:05 ` [PATCH v6 31/39] kasan, vmalloc: only tag normal vmalloc allocations andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-01-25 4:19 ` kernel test robot 2022-01-25 7:56 ` kernel test robot 2022-03-08 15:17 ` Vasily Gorbik 2022-03-08 15:17 ` Vasily Gorbik 2022-03-08 15:30 ` Andrey Konovalov 2022-03-08 15:30 ` Andrey Konovalov 2022-03-08 15:48 ` Vasily Gorbik 2022-03-08 15:48 ` Vasily Gorbik 2022-01-24 18:05 ` [PATCH v6 32/39] kasan, arm64: don't tag executable " andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-01-24 18:05 ` [PATCH v6 33/39] kasan: mark kasan_arg_stacktrace as __initdata andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-01-24 18:05 ` [PATCH v6 34/39] kasan: clean up feature flags for HW_TAGS mode andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-01-24 18:05 ` [PATCH v6 35/39] kasan: add kasan.vmalloc command line flag andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-01-24 18:05 ` [PATCH v6 36/39] kasan: allow enabling KASAN_VMALLOC and SW/HW_TAGS andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-01-24 18:05 ` [PATCH v6 37/39] arm64: select KASAN_VMALLOC for SW/HW_TAGS modes andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-01-24 18:05 ` [PATCH v6 38/39] kasan: documentation updates andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-01-24 18:05 ` [PATCH v6 39/39] kasan: improve vmalloc tests andrey.konovalov 2022-01-24 18:05 ` andrey.konovalov 2022-01-24 18:09 ` [PATCH v6 00/39] kasan, vmalloc, arm64: add vmalloc tagging support for SW/HW_TAGS Marco Elver 2022-01-24 18:09 ` Marco Elver 2022-01-24 18:32 ` Andrey Konovalov 2022-01-24 18:32 ` Andrey Konovalov 2022-04-28 14:13 ` Qian Cai 2022-04-28 14:13 ` Qian Cai 2022-04-28 15:28 ` Andrey Konovalov 2022-04-28 15:28 ` Andrey Konovalov 2022-04-28 16:12 ` Qian Cai 2022-04-28 16:12 ` Qian Cai
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=5ac7e0b30f5cbb177ec363ddd7878a3141289592.1643047180.git.andreyknvl@google.com \ --to=andrey.konovalov@linux.dev \ --cc=akpm@linux-foundation.org \ --cc=andreyknvl@gmail.com \ --cc=andreyknvl@google.com \ --cc=catalin.marinas@arm.com \ --cc=dvyukov@google.com \ --cc=elver@google.com \ --cc=eugenis@google.com \ --cc=glider@google.com \ --cc=kasan-dev@googlegroups.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mark.rutland@arm.com \ --cc=pcc@google.com \ --cc=ryabinin.a.a@gmail.com \ --cc=vincenzo.frascino@arm.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.