From: Marco Elver <elver@google.com> To: Andrey Konovalov <andreyknvl@google.com> Cc: Catalin Marinas <catalin.marinas@arm.com>, Vincenzo Frascino <vincenzo.frascino@arm.com>, Dmitry Vyukov <dvyukov@google.com>, Alexander Potapenko <glider@google.com>, Andrew Morton <akpm@linux-foundation.org>, Will Deacon <will.deacon@arm.com>, Andrey Ryabinin <aryabinin@virtuozzo.com>, Peter Collingbourne <pcc@google.com>, Evgenii Stepanov <eugenis@google.com>, Branislav Rankov <Branislav.Rankov@arm.com>, Kevin Brodsky <kevin.brodsky@arm.com>, kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 08/12] kasan, mm: optimize krealloc poisoning Date: Wed, 3 Feb 2021 15:34:59 +0100 [thread overview] Message-ID: <YBq0k2p5eudcY6bD@elver.google.com> (raw) In-Reply-To: <431c6cfa0ac8fb2b33d7ab561a64aa84c844d1a0.1612208222.git.andreyknvl@google.com> On Mon, Feb 01, 2021 at 08:43PM +0100, Andrey Konovalov wrote: > Currently, krealloc() always calls ksize(), which unpoisons the whole > object including the redzone. This is inefficient, as kasan_krealloc() > repoisons the redzone for objects that fit into the same buffer. > > This patch changes krealloc() instrumentation to use uninstrumented > __ksize() that doesn't unpoison the memory. Instead, kasan_kreallos() > is changed to unpoison the memory excluding the redzone. > > For objects that don't fit into the old allocation, this patch disables > KASAN accessibility checks when copying memory into a new object instead > of unpoisoning it. > > Signed-off-by: Andrey Konovalov <andreyknvl@google.com> > --- > mm/kasan/common.c | 12 ++++++++++-- > mm/slab_common.c | 20 ++++++++++++++------ > 2 files changed, 24 insertions(+), 8 deletions(-) > > diff --git a/mm/kasan/common.c b/mm/kasan/common.c > index 9c64a00bbf9c..a51d6ea580b0 100644 > --- a/mm/kasan/common.c > +++ b/mm/kasan/common.c > @@ -476,7 +476,7 @@ static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, > > /* > * The object has already been unpoisoned by kasan_slab_alloc() for > - * kmalloc() or by ksize() for krealloc(). > + * kmalloc() or by kasan_krealloc() for krealloc(). > */ > > /* > @@ -526,7 +526,7 @@ void * __must_check __kasan_kmalloc_large(const void *ptr, size_t size, > > /* > * The object has already been unpoisoned by kasan_alloc_pages() for > - * alloc_pages() or by ksize() for krealloc(). > + * alloc_pages() or by kasan_krealloc() for krealloc(). > */ > > /* > @@ -554,8 +554,16 @@ void * __must_check __kasan_krealloc(const void *object, size_t size, gfp_t flag > if (unlikely(object == ZERO_SIZE_PTR)) > return (void *)object; > > + /* > + * Unpoison the object's data. > + * Part of it might already have been unpoisoned, but it's unknown > + * how big that part is. > + */ > + kasan_unpoison(object, size); > + > page = virt_to_head_page(object); > > + /* Piggy-back on kmalloc() instrumentation to poison the redzone. */ > if (unlikely(!PageSlab(page))) > return __kasan_kmalloc_large(object, size, flags); > else > diff --git a/mm/slab_common.c b/mm/slab_common.c > index dad70239b54c..821f657d38b5 100644 > --- a/mm/slab_common.c > +++ b/mm/slab_common.c > @@ -1140,19 +1140,27 @@ static __always_inline void *__do_krealloc(const void *p, size_t new_size, > void *ret; > size_t ks; > > - if (likely(!ZERO_OR_NULL_PTR(p)) && !kasan_check_byte(p)) > - return NULL; > - > - ks = ksize(p); > + /* Don't use instrumented ksize to allow precise KASAN poisoning. */ > + if (likely(!ZERO_OR_NULL_PTR(p))) { > + if (!kasan_check_byte(p)) > + return NULL; > + ks = __ksize(p); > + } else > + ks = 0; > This unfortunately broke KFENCE: https://syzkaller.appspot.com/bug?extid=e444e1006d07feef0ef3 + various other false positives. We need to use ksize() here, as __ksize() is unaware of KFENCE. Or somehow add the same check here that ksize() uses to get the real object size. > + /* If the object still fits, repoison it precisely. */ > if (ks >= new_size) { > p = kasan_krealloc((void *)p, new_size, flags); > return (void *)p; > } > > ret = kmalloc_track_caller(new_size, flags); > - if (ret && p) > - memcpy(ret, p, ks); > + if (ret && p) { > + /* Disable KASAN checks as the object's redzone is accessed. */ > + kasan_disable_current(); > + memcpy(ret, kasan_reset_tag(p), ks); > + kasan_enable_current(); > + } > > return ret; > } > -- > 2.30.0.365.g02bc693789-goog >
WARNING: multiple messages have this Message-ID (diff)
From: Marco Elver <elver@google.com> To: Andrey Konovalov <andreyknvl@google.com> Cc: linux-arm-kernel@lists.infradead.org, Branislav Rankov <Branislav.Rankov@arm.com>, Catalin Marinas <catalin.marinas@arm.com>, Kevin Brodsky <kevin.brodsky@arm.com>, Will Deacon <will.deacon@arm.com>, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, Alexander Potapenko <glider@google.com>, Evgenii Stepanov <eugenis@google.com>, Andrey Ryabinin <aryabinin@virtuozzo.com>, Andrew Morton <akpm@linux-foundation.org>, Vincenzo Frascino <vincenzo.frascino@arm.com>, Peter Collingbourne <pcc@google.com>, Dmitry Vyukov <dvyukov@google.com> Subject: Re: [PATCH 08/12] kasan, mm: optimize krealloc poisoning Date: Wed, 3 Feb 2021 15:34:59 +0100 [thread overview] Message-ID: <YBq0k2p5eudcY6bD@elver.google.com> (raw) In-Reply-To: <431c6cfa0ac8fb2b33d7ab561a64aa84c844d1a0.1612208222.git.andreyknvl@google.com> On Mon, Feb 01, 2021 at 08:43PM +0100, Andrey Konovalov wrote: > Currently, krealloc() always calls ksize(), which unpoisons the whole > object including the redzone. This is inefficient, as kasan_krealloc() > repoisons the redzone for objects that fit into the same buffer. > > This patch changes krealloc() instrumentation to use uninstrumented > __ksize() that doesn't unpoison the memory. Instead, kasan_kreallos() > is changed to unpoison the memory excluding the redzone. > > For objects that don't fit into the old allocation, this patch disables > KASAN accessibility checks when copying memory into a new object instead > of unpoisoning it. > > Signed-off-by: Andrey Konovalov <andreyknvl@google.com> > --- > mm/kasan/common.c | 12 ++++++++++-- > mm/slab_common.c | 20 ++++++++++++++------ > 2 files changed, 24 insertions(+), 8 deletions(-) > > diff --git a/mm/kasan/common.c b/mm/kasan/common.c > index 9c64a00bbf9c..a51d6ea580b0 100644 > --- a/mm/kasan/common.c > +++ b/mm/kasan/common.c > @@ -476,7 +476,7 @@ static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, > > /* > * The object has already been unpoisoned by kasan_slab_alloc() for > - * kmalloc() or by ksize() for krealloc(). > + * kmalloc() or by kasan_krealloc() for krealloc(). > */ > > /* > @@ -526,7 +526,7 @@ void * __must_check __kasan_kmalloc_large(const void *ptr, size_t size, > > /* > * The object has already been unpoisoned by kasan_alloc_pages() for > - * alloc_pages() or by ksize() for krealloc(). > + * alloc_pages() or by kasan_krealloc() for krealloc(). > */ > > /* > @@ -554,8 +554,16 @@ void * __must_check __kasan_krealloc(const void *object, size_t size, gfp_t flag > if (unlikely(object == ZERO_SIZE_PTR)) > return (void *)object; > > + /* > + * Unpoison the object's data. > + * Part of it might already have been unpoisoned, but it's unknown > + * how big that part is. > + */ > + kasan_unpoison(object, size); > + > page = virt_to_head_page(object); > > + /* Piggy-back on kmalloc() instrumentation to poison the redzone. */ > if (unlikely(!PageSlab(page))) > return __kasan_kmalloc_large(object, size, flags); > else > diff --git a/mm/slab_common.c b/mm/slab_common.c > index dad70239b54c..821f657d38b5 100644 > --- a/mm/slab_common.c > +++ b/mm/slab_common.c > @@ -1140,19 +1140,27 @@ static __always_inline void *__do_krealloc(const void *p, size_t new_size, > void *ret; > size_t ks; > > - if (likely(!ZERO_OR_NULL_PTR(p)) && !kasan_check_byte(p)) > - return NULL; > - > - ks = ksize(p); > + /* Don't use instrumented ksize to allow precise KASAN poisoning. */ > + if (likely(!ZERO_OR_NULL_PTR(p))) { > + if (!kasan_check_byte(p)) > + return NULL; > + ks = __ksize(p); > + } else > + ks = 0; > This unfortunately broke KFENCE: https://syzkaller.appspot.com/bug?extid=e444e1006d07feef0ef3 + various other false positives. We need to use ksize() here, as __ksize() is unaware of KFENCE. Or somehow add the same check here that ksize() uses to get the real object size. > + /* If the object still fits, repoison it precisely. */ > if (ks >= new_size) { > p = kasan_krealloc((void *)p, new_size, flags); > return (void *)p; > } > > ret = kmalloc_track_caller(new_size, flags); > - if (ret && p) > - memcpy(ret, p, ks); > + if (ret && p) { > + /* Disable KASAN checks as the object's redzone is accessed. */ > + kasan_disable_current(); > + memcpy(ret, kasan_reset_tag(p), ks); > + kasan_enable_current(); > + } > > return ret; > } > -- > 2.30.0.365.g02bc693789-goog > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-02-03 14:37 UTC|newest] Thread overview: 89+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-02-01 19:43 [PATCH 00/12] kasan: optimizations and fixes for HW_TAGS Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` [PATCH 01/12] kasan, mm: don't save alloc stacks twice Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-02 16:06 ` Marco Elver 2021-02-02 16:06 ` Marco Elver 2021-02-02 18:01 ` Andrey Konovalov 2021-02-02 18:01 ` Andrey Konovalov 2021-02-02 18:01 ` Andrey Konovalov 2021-02-02 18:40 ` Marco Elver 2021-02-02 18:40 ` Marco Elver 2021-02-02 18:40 ` Marco Elver 2021-02-01 19:43 ` [PATCH 02/12] kasan, mm: optimize kmalloc poisoning Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-02 16:25 ` Marco Elver 2021-02-02 16:25 ` Marco Elver 2021-02-02 17:15 ` Andrey Konovalov 2021-02-02 17:15 ` Andrey Konovalov 2021-02-02 17:15 ` Andrey Konovalov 2021-02-02 17:39 ` Marco Elver 2021-02-02 17:39 ` Marco Elver 2021-02-02 17:39 ` Marco Elver 2021-02-01 19:43 ` [PATCH 03/12] kasan: optimize large " Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-02 16:57 ` Marco Elver 2021-02-02 16:57 ` Marco Elver 2021-02-01 19:43 ` [PATCH 04/12] kasan: clean up setting free info in kasan_slab_free Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-02 17:03 ` Marco Elver 2021-02-02 17:03 ` Marco Elver 2021-02-01 19:43 ` [PATCH 05/12] kasan: unify large kfree checks Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-03 12:13 ` Marco Elver 2021-02-03 12:13 ` Marco Elver 2021-02-01 19:43 ` [PATCH 06/12] kasan: rework krealloc tests Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-03 14:48 ` Marco Elver 2021-02-03 14:48 ` Marco Elver 2021-02-01 19:43 ` [PATCH 07/12] kasan, mm: remove krealloc side-effect Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-03 15:10 ` Marco Elver 2021-02-03 15:10 ` Marco Elver 2021-02-01 19:43 ` [PATCH 08/12] kasan, mm: optimize krealloc poisoning Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-03 14:34 ` Marco Elver [this message] 2021-02-03 14:34 ` Marco Elver 2021-02-01 19:43 ` [PATCH 09/12] kasan: ensure poisoning size alignment Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-03 15:31 ` Marco Elver 2021-02-03 15:31 ` Marco Elver 2021-02-01 19:43 ` [PATCH 10/12] arm64: kasan: simplify and inline MTE functions Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 22:44 ` Andrew Morton 2021-02-01 22:44 ` Andrew Morton 2021-02-04 12:39 ` Vincenzo Frascino 2021-02-04 12:39 ` Vincenzo Frascino 2021-02-02 15:42 ` Catalin Marinas 2021-02-02 15:42 ` Catalin Marinas 2021-02-02 18:04 ` Andrey Konovalov 2021-02-02 18:04 ` Andrey Konovalov 2021-02-02 18:04 ` Andrey Konovalov 2021-02-04 12:37 ` Vincenzo Frascino 2021-02-04 12:37 ` Vincenzo Frascino 2021-02-01 19:43 ` [PATCH 11/12] kasan: always inline HW_TAGS helper functions Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-03 15:51 ` Marco Elver 2021-02-03 15:51 ` Marco Elver 2021-02-01 19:43 ` [PATCH 12/12] arm64: kasan: export MTE symbols for KASAN tests Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-01 19:43 ` Andrey Konovalov 2021-02-02 10:46 ` Will Deacon 2021-02-02 10:46 ` Will Deacon 2021-02-02 13:42 ` Andrey Konovalov 2021-02-02 13:42 ` Andrey Konovalov 2021-02-02 13:42 ` Andrey Konovalov 2021-02-02 15:43 ` Catalin Marinas 2021-02-02 15:43 ` Catalin Marinas
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=YBq0k2p5eudcY6bD@elver.google.com \ --to=elver@google.com \ --cc=Branislav.Rankov@arm.com \ --cc=akpm@linux-foundation.org \ --cc=andreyknvl@google.com \ --cc=aryabinin@virtuozzo.com \ --cc=catalin.marinas@arm.com \ --cc=dvyukov@google.com \ --cc=eugenis@google.com \ --cc=glider@google.com \ --cc=kasan-dev@googlegroups.com \ --cc=kevin.brodsky@arm.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=pcc@google.com \ --cc=vincenzo.frascino@arm.com \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.