From: Catalin Marinas <catalin.marinas@arm.com>
To: linux-mm@kvack.org
Cc: linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
Catalin Marinas <catalin.marinas@arm.com>,
Geert Uytterhoeven <geert@linux-m68k.org>,
Christoph Lameter <cl@linux.com>,
Pekka Enberg <penberg@kernel.org>,
David Rientjes <rientjes@google.com>,
Joonsoo Kim <iamjoonsoo.kim@lge.com>,
Andrew Morton <akpm@linux-foundation.org>
Subject: [PATCH] mm: slab: Only move management objects off-slab for sizes larger than KMALLOC_MIN_SIZE
Date: Thu, 5 Nov 2015 11:50:35 +0000 [thread overview]
Message-ID: <1446724235-31400-1-git-send-email-catalin.marinas@arm.com> (raw)
In-Reply-To: <20151105043155.GA20374@js1304-P5Q-DELUXE>
Commit 8fc9cf420b36 ("slab: make more slab management structure off the
slab") enables off-slab management objects for sizes starting with
PAGE_SIZE >> 5. This means 128 bytes for a 4KB page configuration.
However, on systems with a KMALLOC_MIN_SIZE of 128 (arm64 in 4.4), such
optimisation does not make sense since the slab management allocation
would take 128 bytes anyway (even though freelist_size is 32) with the
additional overhead of another allocation.
This patch introduces an OFF_SLAB_MIN_SIZE macro which takes
KMALLOC_MIN_SIZE into account. It also solves a slab bug on arm64 where
the first kmalloc_cache to be initialised after slab_early_init = 0,
"kmalloc-128", fails to allocate off-slab management objects from the
same "kmalloc-128" cache.
Fixes: 8fc9cf420b36 ("slab: make more slab management structure off the slab")
Cc: <stable@vger.kernel.org> # 3.15+
Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
---
mm/slab.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/mm/slab.c b/mm/slab.c
index 4fcc5dd8d5a6..419b649b395e 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -282,6 +282,7 @@ static void kmem_cache_node_init(struct kmem_cache_node *parent)
#define CFLGS_OFF_SLAB (0x80000000UL)
#define OFF_SLAB(x) ((x)->flags & CFLGS_OFF_SLAB)
+#define OFF_SLAB_MIN_SIZE (max_t(size_t, PAGE_SIZE >> 5, KMALLOC_MIN_SIZE + 1))
#define BATCHREFILL_LIMIT 16
/*
@@ -2212,7 +2213,7 @@ __kmem_cache_create (struct kmem_cache *cachep, unsigned long flags)
* it too early on. Always use on-slab management when
* SLAB_NOLEAKTRACE to avoid recursive calls into kmemleak)
*/
- if ((size >= (PAGE_SIZE >> 5)) && !slab_early_init &&
+ if (size >= OFF_SLAB_MIN_SIZE && !slab_early_init &&
!(flags & SLAB_NOLEAKTRACE))
/*
* Size is large, assume best to place the slab management obj
@@ -2276,7 +2277,7 @@ __kmem_cache_create (struct kmem_cache *cachep, unsigned long flags)
/*
* This is a possibility for one of the kmalloc_{dma,}_caches.
* But since we go off slab only for object size greater than
- * PAGE_SIZE/8, and kmalloc_{dma,}_caches get created
+ * OFF_SLAB_MIN_SIZE, and kmalloc_{dma,}_caches get created
* in ascending order,this should not happen at all.
* But leave a BUG_ON for some lucky dude.
*/
next prev parent reply other threads:[~2015-11-05 11:50 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-09-22 17:59 [PATCH] arm64: Increase the max granular size Robert Richter
2015-09-22 18:29 ` Will Deacon
2015-09-25 14:45 ` Robert Richter
2015-09-25 16:31 ` Tirumalesh Chalamarla
2015-10-10 17:39 ` Timur Tabi
2015-10-12 9:16 ` Will Deacon
2015-10-16 19:57 ` Timur Tabi
2015-10-28 19:09 ` Catalin Marinas
2015-11-03 11:07 ` Geert Uytterhoeven
2015-11-03 12:05 ` Catalin Marinas
2015-11-03 14:38 ` Catalin Marinas
2015-11-03 14:55 ` Geert Uytterhoeven
2015-11-03 18:50 ` Catalin Marinas
2015-11-03 23:33 ` Christoph Lameter
2015-11-04 12:36 ` Catalin Marinas
2015-11-04 13:53 ` Christoph Lameter
2015-11-04 14:54 ` Catalin Marinas
2015-11-04 15:28 ` Christoph Lameter
2015-11-04 15:39 ` Catalin Marinas
2015-11-05 4:31 ` Joonsoo Kim
2015-11-05 11:50 ` Catalin Marinas [this message]
2015-11-05 13:31 ` [PATCH] mm: slab: Only move management objects off-slab for sizes larger than KMALLOC_MIN_SIZE Andrew Morton
2015-11-05 16:08 ` Catalin Marinas
2015-11-06 13:00 ` Geert Uytterhoeven
2015-11-05 17:39 ` Christoph Lameter
2015-11-05 4:40 ` [PATCH] arm64: Increase the max granular size Joonsoo Kim
2015-11-05 10:32 ` Catalin Marinas
2015-11-05 11:45 ` Joonsoo Kim
2015-11-05 12:17 ` Catalin Marinas
2015-11-09 7:41 ` Joonsoo Kim
2015-11-09 18:36 ` Catalin Marinas
2015-11-10 0:19 ` Joonsoo Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1446724235-31400-1-git-send-email-catalin.marinas@arm.com \
--to=catalin.marinas@arm.com \
--cc=akpm@linux-foundation.org \
--cc=cl@linux.com \
--cc=geert@linux-m68k.org \
--cc=iamjoonsoo.kim@lge.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=penberg@kernel.org \
--cc=rientjes@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).