From: Catalin Marinas <catalin.marinas@arm.com>
To: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Robert Richter <rric@kernel.org>,
Linux-sh list <linux-sh@vger.kernel.org>,
Will Deacon <will.deacon@arm.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
Robert Richter <rrichter@cavium.com>,
Tirumalesh Chalamarla <tchalamarla@cavium.com>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
Joonsoo Kim <js1304@gmail.com>, Christoph Lameter <cl@linux.com>
Subject: Re: [PATCH] arm64: Increase the max granular size
Date: Tue, 3 Nov 2015 18:50:50 +0000 [thread overview]
Message-ID: <20151103185050.GJ7637@e104818-lin.cambridge.arm.com> (raw)
In-Reply-To: <CAMuHMdWk0fPzTSKhoCuS4wsOU1iddhKJb2SOpjo=a_9vCm_KXQ@mail.gmail.com>
On Tue, Nov 03, 2015 at 03:55:29PM +0100, Geert Uytterhoeven wrote:
> On Tue, Nov 3, 2015 at 3:38 PM, Catalin Marinas <catalin.marinas@arm.com> wrote:
> > On Tue, Nov 03, 2015 at 12:05:05PM +0000, Catalin Marinas wrote:
> >> On Tue, Nov 03, 2015 at 12:07:06PM +0100, Geert Uytterhoeven wrote:
> >> > On Wed, Oct 28, 2015 at 8:09 PM, Catalin Marinas
> >> > <catalin.marinas@arm.com> wrote:
> >> > > On Tue, Sep 22, 2015 at 07:59:48PM +0200, Robert Richter wrote:
> >> > >> From: Tirumalesh Chalamarla <tchalamarla@cavium.com>
> >> > >>
> >> > >> Increase the standard cacheline size to avoid having locks in the same
> >> > >> cacheline.
> >> > >>
> >> > >> Cavium's ThunderX core implements cache lines of 128 byte size. With
> >> > >> current granulare size of 64 bytes (L1_CACHE_SHIFT=6) two locks could
> >> > >> share the same cache line leading a performance degradation.
> >> > >> Increasing the size fixes that.
> >> > >>
> >> > >> Increasing the size has no negative impact to cache invalidation on
> >> > >> systems with a smaller cache line. There is an impact on memory usage,
> >> > >> but that's not too important for arm64 use cases.
> >> > >>
> >> > >> Signed-off-by: Tirumalesh Chalamarla <tchalamarla@cavium.com>
> >> > >> Signed-off-by: Robert Richter <rrichter@cavium.com>
> >> > >
> >> > > Applied. Thanks.
> >> >
> >> > This patch causes a BUG() on r8a7795/salvator-x, for which support is not
> >> > yet upstream.
> >> >
> >> > My config (attached) uses SLAB. If I switch to SLUB, it works.
> >> > The arm64 defconfig works, even if I switch from SLUB to SLAB.
> >> [...]
> >> > ------------[ cut here ]------------
> >> > kernel BUG at mm/slab.c:2283!
> >> > Internal error: Oops - BUG: 0 [#1] SMP
> >> [...]
> >> > Call trace:
> >> > [<ffffffc00014f9b4>] __kmem_cache_create+0x21c/0x280
> >> > [<ffffffc00068be50>] create_boot_cache+0x4c/0x80
> >> > [<ffffffc00068bed8>] create_kmalloc_cache+0x54/0x88
> >> > [<ffffffc00068bfc0>] create_kmalloc_caches+0x50/0xf4
> >> > [<ffffffc00068db08>] kmem_cache_init+0x104/0x118
> >> > [<ffffffc00067d7d8>] start_kernel+0x218/0x33c
> >>
> >> I haven't managed to reproduce this on a Juno kernel.
> >
> > I now managed to reproduce it with your config (slightly adapted to
> > allow Juno). I'll look into it.
>
> Good to hear that!
>
> BTW, I see this:
>
> freelist_size = 32
> cache_line_size() = 64
>
> It seems like the value returned by cache_line_size() in
> arch/arm64/include/asm/cache.h disagrees with L1_CACHE_SHIFT == 7:
>
> static inline int cache_line_size(void)
> {
> u32 cwg = cache_type_cwg();
> return cwg ? 4 << cwg : L1_CACHE_BYTES;
> }
>
> Making cache_line_size() always return L1_CACHE_BYTES doesn't help.
(cc'ing Jonsoo and Christoph; summary: slab failure with L1_CACHE_BYTES
of 128 and sizeof(kmem_cache_node) of 152)
If I revert commit 8fc9cf420b36 ("slab: make more slab management
structure off the slab") it works but I still need to figure out how
slab indices are calculated. The size_index[] array is overridden so
that 0..15 are 7 and 16..23 are 8. But the kmalloc_caches[7] has never
been populated, hence the BUG_ON. Another option may be to change
kmalloc_size and kmalloc_index to cope with KMALLOC_MIN_SIZE of 128.
I'll do some more investigation tomorrow.
--
Catalin
next prev parent reply other threads:[~2015-11-03 18:50 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-09-22 17:59 [PATCH] arm64: Increase the max granular size Robert Richter
2015-09-22 18:29 ` Will Deacon
2015-09-25 14:45 ` Robert Richter
2015-09-25 16:31 ` Tirumalesh Chalamarla
2015-10-10 17:39 ` Timur Tabi
2015-10-12 9:16 ` Will Deacon
2015-10-16 19:57 ` Timur Tabi
2015-10-28 19:09 ` Catalin Marinas
2015-11-03 11:07 ` Geert Uytterhoeven
2015-11-03 12:05 ` Catalin Marinas
2015-11-03 14:38 ` Catalin Marinas
2015-11-03 14:55 ` Geert Uytterhoeven
2015-11-03 18:50 ` Catalin Marinas [this message]
2015-11-03 23:33 ` Christoph Lameter
2015-11-04 12:36 ` Catalin Marinas
2015-11-04 13:53 ` Christoph Lameter
2015-11-04 14:54 ` Catalin Marinas
2015-11-04 15:28 ` Christoph Lameter
2015-11-04 15:39 ` Catalin Marinas
2015-11-05 4:31 ` Joonsoo Kim
2015-11-05 11:50 ` [PATCH] mm: slab: Only move management objects off-slab for sizes larger than KMALLOC_MIN_SIZE Catalin Marinas
2015-11-05 13:31 ` Andrew Morton
2015-11-05 16:08 ` Catalin Marinas
2015-11-06 13:00 ` Geert Uytterhoeven
2015-11-05 17:39 ` Christoph Lameter
2015-11-05 4:40 ` [PATCH] arm64: Increase the max granular size Joonsoo Kim
2015-11-05 10:32 ` Catalin Marinas
2015-11-05 11:45 ` Joonsoo Kim
2015-11-05 12:17 ` Catalin Marinas
2015-11-09 7:41 ` Joonsoo Kim
2015-11-09 18:36 ` Catalin Marinas
2015-11-10 0:19 ` Joonsoo Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20151103185050.GJ7637@e104818-lin.cambridge.arm.com \
--to=catalin.marinas@arm.com \
--cc=cl@linux.com \
--cc=geert@linux-m68k.org \
--cc=js1304@gmail.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sh@vger.kernel.org \
--cc=rric@kernel.org \
--cc=rrichter@cavium.com \
--cc=tchalamarla@cavium.com \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).