* [PATCH v1 1/2] mm: slub: init_on_free=1 should wipe freelist ptr for bulk allocations
@ 2019-10-04 13:25 Alexander Potapenko
2019-10-04 13:25 ` [PATCH v1 2/2] lib/test_meminit: add a kmem_cache_alloc_bulk() test Alexander Potapenko
2019-10-04 18:45 ` [PATCH v1 1/2] mm: slub: init_on_free=1 should wipe freelist ptr for bulk allocations kbuild test robot
0 siblings, 2 replies; 3+ messages in thread
From: Alexander Potapenko @ 2019-10-04 13:25 UTC (permalink / raw)
To: Andrew Morton, Christoph Lameter
Cc: Alexander Potapenko, Thibaut Sautereau, Kees Cook, Laura Abbott,
linux-mm, kernel-hardening
slab_alloc_node() already zeroed out the freelist pointer if
init_on_free was on.
Thibaut Sautereau noticed that the same needs to be done for
kmem_cache_alloc_bulk(), which performs the allocations separately.
kmem_cache_alloc_bulk() is currently used in two places in the kernel,
so this change is unlikely to have a major performance impact.
SLAB doesn't require a similar change, as auto-initialization makes the
allocator store the freelist pointers off-slab.
Reported-by: Thibaut Sautereau <thibaut@sautereau.fr>
Reported-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Alexander Potapenko <glider@google.com>
Fixes: 6471384af2a6 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options")
To: Andrew Morton <akpm@linux-foundation.org>
To: Christoph Lameter <cl@linux.com>
Cc: Laura Abbott <labbott@redhat.com>
Cc: linux-mm@kvack.org
Cc: kernel-hardening@lists.openwall.com
---
mm/slub.c | 21 +++++++++++++++------
1 file changed, 15 insertions(+), 6 deletions(-)
diff --git a/mm/slub.c b/mm/slub.c
index 8834563cdb4b..fe90bed40eb3 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2669,6 +2669,16 @@ static void *__slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node,
return p;
}
+/*
+ * If the object has been wiped upon free, make sure it's fully initialized by
+ * zeroing out freelist pointer.
+ */
+static __always_inline maybe_wipe_obj_freeptr(struct kmem_cache *s, void *obj)
+{
+ if (unlikely(slab_want_init_on_free(s)) && obj)
+ memset((void *)((char *)obj + s->offset), 0, sizeof(void *));
+}
+
/*
* Inlined fastpath so that allocation functions (kmalloc, kmem_cache_alloc)
* have the fastpath folded into their functions. So no function call
@@ -2757,12 +2767,8 @@ static __always_inline void *slab_alloc_node(struct kmem_cache *s,
prefetch_freepointer(s, next_object);
stat(s, ALLOC_FASTPATH);
}
- /*
- * If the object has been wiped upon free, make sure it's fully
- * initialized by zeroing out freelist pointer.
- */
- if (unlikely(slab_want_init_on_free(s)) && object)
- memset(object + s->offset, 0, sizeof(void *));
+
+ maybe_wipe_obj_freeptr(s, object);
if (unlikely(slab_want_init_on_alloc(gfpflags, s)) && object)
memset(object, 0, s->object_size);
@@ -3176,10 +3182,13 @@ int kmem_cache_alloc_bulk(struct kmem_cache *s, gfp_t flags, size_t size,
goto error;
c = this_cpu_ptr(s->cpu_slab);
+ maybe_wipe_obj_freeptr(s, p[i]);
+
continue; /* goto for-loop */
}
c->freelist = get_freepointer(s, object);
p[i] = object;
+ maybe_wipe_obj_freeptr(s, p[i]);
}
c->tid = next_tid(c->tid);
local_irq_enable();
--
2.23.0.581.g78d2f28ef7-goog
^ permalink raw reply related [flat|nested] 3+ messages in thread
* [PATCH v1 2/2] lib/test_meminit: add a kmem_cache_alloc_bulk() test
2019-10-04 13:25 [PATCH v1 1/2] mm: slub: init_on_free=1 should wipe freelist ptr for bulk allocations Alexander Potapenko
@ 2019-10-04 13:25 ` Alexander Potapenko
2019-10-04 18:45 ` [PATCH v1 1/2] mm: slub: init_on_free=1 should wipe freelist ptr for bulk allocations kbuild test robot
1 sibling, 0 replies; 3+ messages in thread
From: Alexander Potapenko @ 2019-10-04 13:25 UTC (permalink / raw)
To: Andrew Morton, Christoph Lameter
Cc: Alexander Potapenko, Kees Cook, linux-mm, kernel-hardening
Make sure allocations from kmem_cache_alloc_bulk()/kmem_cache_free_bulk()
are properly initialized.
Signed-off-by: Alexander Potapenko <glider@google.com>
Cc: Kees Cook <keescook@chromium.org>
To: Andrew Morton <akpm@linux-foundation.org>
To: Christoph Lameter <cl@linux.com>
Cc: linux-mm@kvack.org
Cc: kernel-hardening@lists.openwall.com
---
lib/test_meminit.c | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/lib/test_meminit.c b/lib/test_meminit.c
index 9729f271d150..9742e5cb853a 100644
--- a/lib/test_meminit.c
+++ b/lib/test_meminit.c
@@ -297,6 +297,32 @@ static int __init do_kmem_cache_rcu_persistent(int size, int *total_failures)
return 1;
}
+static int __init do_kmem_cache_size_bulk(int size, int *total_failures)
+{
+ struct kmem_cache *c;
+ int i, iter, maxiter = 1024;
+ int num, bytes;
+ bool fail = false;
+ void *objects[10];
+
+ c = kmem_cache_create("test_cache", size, size, 0, NULL);
+ for (iter = 0; (iter < maxiter) && !fail; iter++) {
+ num = kmem_cache_alloc_bulk(c, GFP_KERNEL, ARRAY_SIZE(objects),
+ objects);
+ for (i = 0; i < num; i++) {
+ bytes = count_nonzero_bytes(objects[i], size);
+ if (bytes)
+ fail = true;
+ fill_with_garbage(objects[i], size);
+ }
+
+ if (num)
+ kmem_cache_free_bulk(c, num, objects);
+ }
+ *total_failures += fail;
+ return 1;
+}
+
/*
* Test kmem_cache allocation by creating caches of different sizes, with and
* without constructors, with and without SLAB_TYPESAFE_BY_RCU.
@@ -318,6 +344,7 @@ static int __init test_kmemcache(int *total_failures)
num_tests += do_kmem_cache_size(size, ctor, rcu, zero,
&failures);
}
+ num_tests += do_kmem_cache_size_bulk(size, &failures);
}
REPORT_FAILURES_IN_FN();
*total_failures += failures;
--
2.23.0.581.g78d2f28ef7-goog
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH v1 1/2] mm: slub: init_on_free=1 should wipe freelist ptr for bulk allocations
2019-10-04 13:25 [PATCH v1 1/2] mm: slub: init_on_free=1 should wipe freelist ptr for bulk allocations Alexander Potapenko
2019-10-04 13:25 ` [PATCH v1 2/2] lib/test_meminit: add a kmem_cache_alloc_bulk() test Alexander Potapenko
@ 2019-10-04 18:45 ` kbuild test robot
1 sibling, 0 replies; 3+ messages in thread
From: kbuild test robot @ 2019-10-04 18:45 UTC (permalink / raw)
To: Alexander Potapenko
Cc: kbuild-all, Andrew Morton, Christoph Lameter,
Alexander Potapenko, Thibaut Sautereau, Kees Cook, Laura Abbott,
linux-mm, kernel-hardening
[-- Attachment #1: Type: text/plain, Size: 1745 bytes --]
Hi Alexander,
I love your patch! Perhaps something to improve:
[auto build test WARNING on mmotm/master]
url: https://github.com/0day-ci/linux/commits/Alexander-Potapenko/mm-slub-init_on_free-1-should-wipe-freelist-ptr-for-bulk-allocations/20191005-012134
base: git://git.cmpxchg.org/linux-mmotm.git master
config: sparc64-allmodconfig (attached as .config)
compiler: sparc64-linux-gcc (GCC) 7.4.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=7.4.0 make.cross ARCH=sparc64
If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> mm/slub.c:2676:24: warning: return type defaults to 'int' [-Wreturn-type]
static __always_inline maybe_wipe_obj_freeptr(struct kmem_cache *s, void *obj)
^~~~~~~~~~~~~~~~~~~~~~
mm/slub.c: In function 'maybe_wipe_obj_freeptr':
>> mm/slub.c:2680:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
vim +/int +2676 mm/slub.c
2671
2672 /*
2673 * If the object has been wiped upon free, make sure it's fully initialized by
2674 * zeroing out freelist pointer.
2675 */
> 2676 static __always_inline maybe_wipe_obj_freeptr(struct kmem_cache *s, void *obj)
2677 {
2678 if (unlikely(slab_want_init_on_free(s)) && obj)
2679 memset((void *)((char *)obj + s->offset), 0, sizeof(void *));
> 2680 }
2681
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 59039 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-10-04 18:46 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-04 13:25 [PATCH v1 1/2] mm: slub: init_on_free=1 should wipe freelist ptr for bulk allocations Alexander Potapenko
2019-10-04 13:25 ` [PATCH v1 2/2] lib/test_meminit: add a kmem_cache_alloc_bulk() test Alexander Potapenko
2019-10-04 18:45 ` [PATCH v1 1/2] mm: slub: init_on_free=1 should wipe freelist ptr for bulk allocations kbuild test robot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).