* [PATCH v2 0/2] slab: Provide full coverage for __alloc_size attribute @ 2022-11-18 3:51 Kees Cook 2022-11-18 3:51 ` [PATCH v2 1/2] slab: Clean up SLOB vs kmalloc() definition Kees Cook 2022-11-18 3:51 ` [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations Kees Cook 0 siblings, 2 replies; 8+ messages in thread From: Kees Cook @ 2022-11-18 3:51 UTC (permalink / raw) To: Vlastimil Babka Cc: Kees Cook, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Roman Gushchin, Hyeonggon Yoo, linux-kernel, linux-mm, linux-hardening Hi, These patches work around a deficiency in GCC (>=11) and Clang (<16) where the __alloc_size attribute does not apply to inlines. :( https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96503 This manifests as reduced overflow detection coverage for many allocation sites under CONFIG_FORTIFY_SOURCE=y, where the allocation size was not actually being propagated to __builtin_dynamic_object_size(). The problem was in two halves: the trace wrapper (now fixed in -next), and const-0 special-casing (covered here). Thanks, -Kees v2: - drop trace wrapper refactoring -- handled differently now (vbabka) - drop kunit/fortify test patch since it depends on other changes - rebase v1: https://lore.kernel.org/all/20221101222520.never.109-kees@kernel.org Kees Cook (2): slab: Clean up SLOB vs kmalloc() definition slab: Remove special-casing of const 0 size allocations include/linux/slab.h | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) -- 2.34.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 1/2] slab: Clean up SLOB vs kmalloc() definition 2022-11-18 3:51 [PATCH v2 0/2] slab: Provide full coverage for __alloc_size attribute Kees Cook @ 2022-11-18 3:51 ` Kees Cook 2022-11-18 11:29 ` Hyeonggon Yoo 2022-11-18 3:51 ` [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations Kees Cook 1 sibling, 1 reply; 8+ messages in thread From: Kees Cook @ 2022-11-18 3:51 UTC (permalink / raw) To: Vlastimil Babka Cc: Kees Cook, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Roman Gushchin, Hyeonggon Yoo, linux-mm, linux-kernel, linux-hardening As already done for kmalloc_node(), clean up the #ifdef usage in the definition of kmalloc() so that the SLOB-only version is an entirely separate and much more readable function. Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Christoph Lameter <cl@linux.com> Cc: Pekka Enberg <penberg@kernel.org> Cc: David Rientjes <rientjes@google.com> Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: linux-mm@kvack.org Signed-off-by: Kees Cook <keescook@chromium.org> --- include/linux/slab.h | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 1c670c16c737..9033937c758e 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -558,15 +558,15 @@ void *kmalloc_large_node(size_t size, gfp_t flags, int node) __assume_page_align * Try really hard to succeed the allocation but fail * eventually. */ +#ifndef CONFIG_SLOB static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags) { if (__builtin_constant_p(size)) { -#ifndef CONFIG_SLOB unsigned int index; -#endif + if (size > KMALLOC_MAX_CACHE_SIZE) return kmalloc_large(size, flags); -#ifndef CONFIG_SLOB + index = kmalloc_index(size); if (!index) @@ -575,10 +575,18 @@ static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags) return kmalloc_trace( kmalloc_caches[kmalloc_type(flags)][index], flags, size); -#endif } return __kmalloc(size, flags); } +#else +static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags) +{ + if (__builtin_constant_p(size) && size > KMALLOC_MAX_CACHE_SIZE) + return kmalloc_large(size, flags); + + return __kmalloc(size, flags); +} +#endif #ifndef CONFIG_SLOB static __always_inline __alloc_size(1) void *kmalloc_node(size_t size, gfp_t flags, int node) -- 2.34.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/2] slab: Clean up SLOB vs kmalloc() definition 2022-11-18 3:51 ` [PATCH v2 1/2] slab: Clean up SLOB vs kmalloc() definition Kees Cook @ 2022-11-18 11:29 ` Hyeonggon Yoo 0 siblings, 0 replies; 8+ messages in thread From: Hyeonggon Yoo @ 2022-11-18 11:29 UTC (permalink / raw) To: Kees Cook Cc: Vlastimil Babka, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Roman Gushchin, linux-mm, linux-kernel, linux-hardening On Thu, Nov 17, 2022 at 07:51:58PM -0800, Kees Cook wrote: > As already done for kmalloc_node(), clean up the #ifdef usage in the > definition of kmalloc() so that the SLOB-only version is an entirely > separate and much more readable function. > > Cc: Vlastimil Babka <vbabka@suse.cz> > Cc: Christoph Lameter <cl@linux.com> > Cc: Pekka Enberg <penberg@kernel.org> > Cc: David Rientjes <rientjes@google.com> > Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> > Cc: Andrew Morton <akpm@linux-foundation.org> > Cc: Roman Gushchin <roman.gushchin@linux.dev> > Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> > Cc: linux-mm@kvack.org > Signed-off-by: Kees Cook <keescook@chromium.org> > --- > include/linux/slab.h | 16 ++++++++++++---- > 1 file changed, 12 insertions(+), 4 deletions(-) > > diff --git a/include/linux/slab.h b/include/linux/slab.h > index 1c670c16c737..9033937c758e 100644 > --- a/include/linux/slab.h > +++ b/include/linux/slab.h > @@ -558,15 +558,15 @@ void *kmalloc_large_node(size_t size, gfp_t flags, int node) __assume_page_align > * Try really hard to succeed the allocation but fail > * eventually. > */ > +#ifndef CONFIG_SLOB > static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags) > { > if (__builtin_constant_p(size)) { > -#ifndef CONFIG_SLOB > unsigned int index; > -#endif > + > if (size > KMALLOC_MAX_CACHE_SIZE) > return kmalloc_large(size, flags); > -#ifndef CONFIG_SLOB > + > index = kmalloc_index(size); > > if (!index) > @@ -575,10 +575,18 @@ static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags) > return kmalloc_trace( > kmalloc_caches[kmalloc_type(flags)][index], > flags, size); > -#endif > } > return __kmalloc(size, flags); > } > +#else > +static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags) > +{ > + if (__builtin_constant_p(size) && size > KMALLOC_MAX_CACHE_SIZE) > + return kmalloc_large(size, flags); > + > + return __kmalloc(size, flags); > +} > +#endif > > #ifndef CONFIG_SLOB > static __always_inline __alloc_size(1) void *kmalloc_node(size_t size, gfp_t flags, int node) > -- > 2.34.1 Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> -- Thanks, Hyeonggon ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations 2022-11-18 3:51 [PATCH v2 0/2] slab: Provide full coverage for __alloc_size attribute Kees Cook 2022-11-18 3:51 ` [PATCH v2 1/2] slab: Clean up SLOB vs kmalloc() definition Kees Cook @ 2022-11-18 3:51 ` Kees Cook 2022-11-18 11:34 ` Vlastimil Babka 2022-11-18 11:36 ` Hyeonggon Yoo 1 sibling, 2 replies; 8+ messages in thread From: Kees Cook @ 2022-11-18 3:51 UTC (permalink / raw) To: Vlastimil Babka Cc: Kees Cook, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Roman Gushchin, Hyeonggon Yoo, linux-mm, linux-kernel, linux-hardening Passing a constant-0 size allocation into kmalloc() or kmalloc_node() does not need to be a fast-path operation, so the static return value can be removed entirely. This is in preparation for making sure that all paths through the inlines result in a full extern function call, where __alloc_size() hints will actually be seen[1] by GCC. (A constant return value of 0 means the "0" allocation size won't be propagated by the inline.) [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96503 Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Christoph Lameter <cl@linux.com> Cc: Pekka Enberg <penberg@kernel.org> Cc: David Rientjes <rientjes@google.com> Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: linux-mm@kvack.org Signed-off-by: Kees Cook <keescook@chromium.org> --- include/linux/slab.h | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 9033937c758e..84be05208418 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -561,17 +561,13 @@ void *kmalloc_large_node(size_t size, gfp_t flags, int node) __assume_page_align #ifndef CONFIG_SLOB static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags) { - if (__builtin_constant_p(size)) { + if (__builtin_constant_p(size) && size) { unsigned int index; if (size > KMALLOC_MAX_CACHE_SIZE) return kmalloc_large(size, flags); index = kmalloc_index(size); - - if (!index) - return ZERO_SIZE_PTR; - return kmalloc_trace( kmalloc_caches[kmalloc_type(flags)][index], flags, size); @@ -591,17 +587,13 @@ static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags) #ifndef CONFIG_SLOB static __always_inline __alloc_size(1) void *kmalloc_node(size_t size, gfp_t flags, int node) { - if (__builtin_constant_p(size)) { + if (__builtin_constant_p(size) && size) { unsigned int index; if (size > KMALLOC_MAX_CACHE_SIZE) return kmalloc_large_node(size, flags, node); index = kmalloc_index(size); - - if (!index) - return ZERO_SIZE_PTR; - return kmalloc_node_trace( kmalloc_caches[kmalloc_type(flags)][index], flags, node, size); -- 2.34.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations 2022-11-18 3:51 ` [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations Kees Cook @ 2022-11-18 11:34 ` Vlastimil Babka 2022-11-18 17:06 ` Kees Cook 2022-11-18 11:36 ` Hyeonggon Yoo 1 sibling, 1 reply; 8+ messages in thread From: Vlastimil Babka @ 2022-11-18 11:34 UTC (permalink / raw) To: Kees Cook Cc: Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Roman Gushchin, Hyeonggon Yoo, linux-mm, linux-kernel, linux-hardening On 11/18/22 04:51, Kees Cook wrote: > Passing a constant-0 size allocation into kmalloc() or kmalloc_node() > does not need to be a fast-path operation, so the static return value > can be removed entirely. This is in preparation for making sure that > all paths through the inlines result in a full extern function call, So with the kmalloc_trace() already solved, we could now say it's not "in preparation", but simply "makes sure", right? I can correct that while picking this patch. > where __alloc_size() hints will actually be seen[1] by GCC. (A constant > return value of 0 means the "0" allocation size won't be propagated by > the inline.) > > [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96503 > > Cc: Vlastimil Babka <vbabka@suse.cz> > Cc: Christoph Lameter <cl@linux.com> > Cc: Pekka Enberg <penberg@kernel.org> > Cc: David Rientjes <rientjes@google.com> > Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> > Cc: Andrew Morton <akpm@linux-foundation.org> > Cc: Roman Gushchin <roman.gushchin@linux.dev> > Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> > Cc: linux-mm@kvack.org > Signed-off-by: Kees Cook <keescook@chromium.org> > --- > include/linux/slab.h | 12 ++---------- > 1 file changed, 2 insertions(+), 10 deletions(-) > > diff --git a/include/linux/slab.h b/include/linux/slab.h > index 9033937c758e..84be05208418 100644 > --- a/include/linux/slab.h > +++ b/include/linux/slab.h > @@ -561,17 +561,13 @@ void *kmalloc_large_node(size_t size, gfp_t flags, int node) __assume_page_align > #ifndef CONFIG_SLOB > static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags) > { > - if (__builtin_constant_p(size)) { > + if (__builtin_constant_p(size) && size) { > unsigned int index; > > if (size > KMALLOC_MAX_CACHE_SIZE) > return kmalloc_large(size, flags); > > index = kmalloc_index(size); > - > - if (!index) > - return ZERO_SIZE_PTR; > - > return kmalloc_trace( > kmalloc_caches[kmalloc_type(flags)][index], > flags, size); > @@ -591,17 +587,13 @@ static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags) > #ifndef CONFIG_SLOB > static __always_inline __alloc_size(1) void *kmalloc_node(size_t size, gfp_t flags, int node) > { > - if (__builtin_constant_p(size)) { > + if (__builtin_constant_p(size) && size) { > unsigned int index; > > if (size > KMALLOC_MAX_CACHE_SIZE) > return kmalloc_large_node(size, flags, node); > > index = kmalloc_index(size); > - > - if (!index) > - return ZERO_SIZE_PTR; > - > return kmalloc_node_trace( > kmalloc_caches[kmalloc_type(flags)][index], > flags, node, size); ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations 2022-11-18 11:34 ` Vlastimil Babka @ 2022-11-18 17:06 ` Kees Cook 2022-11-21 9:28 ` Vlastimil Babka 0 siblings, 1 reply; 8+ messages in thread From: Kees Cook @ 2022-11-18 17:06 UTC (permalink / raw) To: Vlastimil Babka Cc: Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Roman Gushchin, Hyeonggon Yoo, linux-mm, linux-kernel, linux-hardening On Fri, Nov 18, 2022 at 12:34:01PM +0100, Vlastimil Babka wrote: > On 11/18/22 04:51, Kees Cook wrote: > > Passing a constant-0 size allocation into kmalloc() or kmalloc_node() > > does not need to be a fast-path operation, so the static return value > > can be removed entirely. This is in preparation for making sure that > > all paths through the inlines result in a full extern function call, > > So with the kmalloc_trace() already solved, we could now say it's not "in > preparation", but simply "makes sure", right? I can correct that while > picking this patch. Yeah, good point. I missed this when updating the commit logs. Thanks! -- Kees Cook ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations 2022-11-18 17:06 ` Kees Cook @ 2022-11-21 9:28 ` Vlastimil Babka 0 siblings, 0 replies; 8+ messages in thread From: Vlastimil Babka @ 2022-11-21 9:28 UTC (permalink / raw) To: Kees Cook Cc: Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Roman Gushchin, Hyeonggon Yoo, linux-mm, linux-kernel, linux-hardening On 11/18/22 18:06, Kees Cook wrote: > On Fri, Nov 18, 2022 at 12:34:01PM +0100, Vlastimil Babka wrote: >> On 11/18/22 04:51, Kees Cook wrote: >> > Passing a constant-0 size allocation into kmalloc() or kmalloc_node() >> > does not need to be a fast-path operation, so the static return value >> > can be removed entirely. This is in preparation for making sure that >> > all paths through the inlines result in a full extern function call, >> >> So with the kmalloc_trace() already solved, we could now say it's not "in >> preparation", but simply "makes sure", right? I can correct that while >> picking this patch. > > Yeah, good point. I missed this when updating the commit logs. Thanks! Great, pushed to slab/for-6.2/alloc_size ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations 2022-11-18 3:51 ` [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations Kees Cook 2022-11-18 11:34 ` Vlastimil Babka @ 2022-11-18 11:36 ` Hyeonggon Yoo 1 sibling, 0 replies; 8+ messages in thread From: Hyeonggon Yoo @ 2022-11-18 11:36 UTC (permalink / raw) To: Kees Cook Cc: Vlastimil Babka, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Roman Gushchin, linux-mm, linux-kernel, linux-hardening On Thu, Nov 17, 2022 at 07:51:59PM -0800, Kees Cook wrote: > Passing a constant-0 size allocation into kmalloc() or kmalloc_node() > does not need to be a fast-path operation, so the static return value > can be removed entirely. This is in preparation for making sure that > all paths through the inlines result in a full extern function call, > where __alloc_size() hints will actually be seen[1] by GCC. (A constant > return value of 0 means the "0" allocation size won't be propagated by > the inline.) > > [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96503 > > Cc: Vlastimil Babka <vbabka@suse.cz> > Cc: Christoph Lameter <cl@linux.com> > Cc: Pekka Enberg <penberg@kernel.org> > Cc: David Rientjes <rientjes@google.com> > Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> > Cc: Andrew Morton <akpm@linux-foundation.org> > Cc: Roman Gushchin <roman.gushchin@linux.dev> > Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> > Cc: linux-mm@kvack.org > Signed-off-by: Kees Cook <keescook@chromium.org> > --- > include/linux/slab.h | 12 ++---------- > 1 file changed, 2 insertions(+), 10 deletions(-) > > diff --git a/include/linux/slab.h b/include/linux/slab.h > index 9033937c758e..84be05208418 100644 > --- a/include/linux/slab.h > +++ b/include/linux/slab.h > @@ -561,17 +561,13 @@ void *kmalloc_large_node(size_t size, gfp_t flags, int node) __assume_page_align > #ifndef CONFIG_SLOB > static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags) > { > - if (__builtin_constant_p(size)) { > + if (__builtin_constant_p(size) && size) { > unsigned int index; > > if (size > KMALLOC_MAX_CACHE_SIZE) > return kmalloc_large(size, flags); > > index = kmalloc_index(size); > - > - if (!index) > - return ZERO_SIZE_PTR; > - > return kmalloc_trace( > kmalloc_caches[kmalloc_type(flags)][index], > flags, size); > @@ -591,17 +587,13 @@ static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags) > #ifndef CONFIG_SLOB > static __always_inline __alloc_size(1) void *kmalloc_node(size_t size, gfp_t flags, int node) > { > - if (__builtin_constant_p(size)) { > + if (__builtin_constant_p(size) && size) { > unsigned int index; > > if (size > KMALLOC_MAX_CACHE_SIZE) > return kmalloc_large_node(size, flags, node); > > index = kmalloc_index(size); > - > - if (!index) > - return ZERO_SIZE_PTR; > - > return kmalloc_node_trace( > kmalloc_caches[kmalloc_type(flags)][index], > flags, node, size); > -- > 2.34.1 Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> -- Thanks, Hyeonggon ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2022-11-21 9:28 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-11-18 3:51 [PATCH v2 0/2] slab: Provide full coverage for __alloc_size attribute Kees Cook 2022-11-18 3:51 ` [PATCH v2 1/2] slab: Clean up SLOB vs kmalloc() definition Kees Cook 2022-11-18 11:29 ` Hyeonggon Yoo 2022-11-18 3:51 ` [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations Kees Cook 2022-11-18 11:34 ` Vlastimil Babka 2022-11-18 17:06 ` Kees Cook 2022-11-21 9:28 ` Vlastimil Babka 2022-11-18 11:36 ` Hyeonggon Yoo
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.