From: Catalin Marinas <catalin.marinas@arm.com> To: Joonsoo Kim <iamjoonsoo.kim@lge.com> Cc: Robert Richter <rric@kernel.org>, Will Deacon <will.deacon@arm.com>, linux-kernel@vger.kernel.org, Robert Richter <rrichter@cavium.com>, Tirumalesh Chalamarla <tchalamarla@cavium.com>, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH] arm64: Increase the max granular size Date: Thu, 5 Nov 2015 10:32:15 +0000 [thread overview] Message-ID: <20151105103214.GP7637@e104818-lin.cambridge.arm.com> (raw) In-Reply-To: <20151105044014.GB20374@js1304-P5Q-DELUXE> On Thu, Nov 05, 2015 at 01:40:14PM +0900, Joonsoo Kim wrote: > On Tue, Sep 22, 2015 at 07:59:48PM +0200, Robert Richter wrote: > > From: Tirumalesh Chalamarla <tchalamarla@cavium.com> > > > > Increase the standard cacheline size to avoid having locks in the same > > cacheline. > > > > Cavium's ThunderX core implements cache lines of 128 byte size. With > > current granulare size of 64 bytes (L1_CACHE_SHIFT=6) two locks could > > share the same cache line leading a performance degradation. > > Increasing the size fixes that. > > Beside, slab-side bug, I don't think this argument is valid. > Even if this change is applied, statically allocated spinlock could > share the same cache line. The benchmarks didn't show any difference with or without this patch applied. What convinced me to apply it was this email: http://lkml.kernel.org/g/CAOZdJXUiRMAguDV+HEJqPg57MyBNqEcTyaH+ya=U93NHb-pdJA@mail.gmail.com On ARM we have a notion of cache writeback granule (CWG) which tells us "the maximum size of memory that can be overwritten as a result of the eviction of a cache entry that has had a memory location in it modified". What we actually needed was ARCH_DMA_MINALIGN to be 128 (currently defined to the L1_CACHE_BYTES value). However, this wouldn't have fixed the KMALLOC_MIN_SIZE, unless we somehow generate different kmalloc_caches[] and kmalloc_dma_caches[] and probably introduce a size_dma_index[]. > If two locks should not share the same cache line, you'd better to use > compiler attribute such as ____cacheline_aligned_in_smp in appropriate > place. We could decouple SMP_CACHE_BYTES from L1_CACHE_BYTES but see above for the other issue we had to solve. -- Catalin
WARNING: multiple messages have this Message-ID (diff)
From: catalin.marinas@arm.com (Catalin Marinas) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH] arm64: Increase the max granular size Date: Thu, 5 Nov 2015 10:32:15 +0000 [thread overview] Message-ID: <20151105103214.GP7637@e104818-lin.cambridge.arm.com> (raw) In-Reply-To: <20151105044014.GB20374@js1304-P5Q-DELUXE> On Thu, Nov 05, 2015 at 01:40:14PM +0900, Joonsoo Kim wrote: > On Tue, Sep 22, 2015 at 07:59:48PM +0200, Robert Richter wrote: > > From: Tirumalesh Chalamarla <tchalamarla@cavium.com> > > > > Increase the standard cacheline size to avoid having locks in the same > > cacheline. > > > > Cavium's ThunderX core implements cache lines of 128 byte size. With > > current granulare size of 64 bytes (L1_CACHE_SHIFT=6) two locks could > > share the same cache line leading a performance degradation. > > Increasing the size fixes that. > > Beside, slab-side bug, I don't think this argument is valid. > Even if this change is applied, statically allocated spinlock could > share the same cache line. The benchmarks didn't show any difference with or without this patch applied. What convinced me to apply it was this email: http://lkml.kernel.org/g/CAOZdJXUiRMAguDV+HEJqPg57MyBNqEcTyaH+ya=U93NHb-pdJA at mail.gmail.com On ARM we have a notion of cache writeback granule (CWG) which tells us "the maximum size of memory that can be overwritten as a result of the eviction of a cache entry that has had a memory location in it modified". What we actually needed was ARCH_DMA_MINALIGN to be 128 (currently defined to the L1_CACHE_BYTES value). However, this wouldn't have fixed the KMALLOC_MIN_SIZE, unless we somehow generate different kmalloc_caches[] and kmalloc_dma_caches[] and probably introduce a size_dma_index[]. > If two locks should not share the same cache line, you'd better to use > compiler attribute such as ____cacheline_aligned_in_smp in appropriate > place. We could decouple SMP_CACHE_BYTES from L1_CACHE_BYTES but see above for the other issue we had to solve. -- Catalin
next prev parent reply other threads:[~2015-11-05 10:32 UTC|newest] Thread overview: 85+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-09-22 17:59 [PATCH] arm64: Increase the max granular size Robert Richter 2015-09-22 17:59 ` Robert Richter 2015-09-22 18:29 ` Will Deacon 2015-09-22 18:29 ` Will Deacon 2015-09-25 14:45 ` Robert Richter 2015-09-25 14:45 ` Robert Richter 2015-09-25 16:31 ` Tirumalesh Chalamarla 2015-09-25 16:31 ` Tirumalesh Chalamarla 2015-10-10 17:39 ` Timur Tabi 2015-10-10 17:39 ` Timur Tabi 2015-10-12 9:16 ` Will Deacon 2015-10-12 9:16 ` Will Deacon 2015-10-16 19:57 ` Timur Tabi 2015-10-16 19:57 ` Timur Tabi 2015-10-28 19:09 ` Catalin Marinas 2015-10-28 19:09 ` Catalin Marinas 2015-11-03 11:07 ` Geert Uytterhoeven 2015-11-03 12:05 ` Catalin Marinas 2015-11-03 12:05 ` Catalin Marinas 2015-11-03 12:05 ` Catalin Marinas 2015-11-03 14:38 ` Catalin Marinas 2015-11-03 14:38 ` Catalin Marinas 2015-11-03 14:38 ` Catalin Marinas 2015-11-03 14:55 ` Geert Uytterhoeven 2015-11-03 14:55 ` Geert Uytterhoeven 2015-11-03 14:55 ` Geert Uytterhoeven 2015-11-03 18:50 ` Catalin Marinas 2015-11-03 18:50 ` Catalin Marinas 2015-11-03 18:50 ` Catalin Marinas 2015-11-03 23:33 ` Christoph Lameter 2015-11-03 23:33 ` Christoph Lameter 2015-11-03 23:33 ` Christoph Lameter 2015-11-04 12:36 ` Catalin Marinas 2015-11-04 12:36 ` Catalin Marinas 2015-11-04 12:36 ` Catalin Marinas 2015-11-04 12:36 ` Catalin Marinas 2015-11-04 13:53 ` Christoph Lameter 2015-11-04 13:53 ` Christoph Lameter 2015-11-04 13:53 ` Christoph Lameter 2015-11-04 13:53 ` Christoph Lameter 2015-11-04 14:54 ` Catalin Marinas 2015-11-04 14:54 ` Catalin Marinas 2015-11-04 14:54 ` Catalin Marinas 2015-11-04 14:54 ` Catalin Marinas 2015-11-04 15:28 ` Christoph Lameter 2015-11-04 15:28 ` Christoph Lameter 2015-11-04 15:28 ` Christoph Lameter 2015-11-04 15:28 ` Christoph Lameter 2015-11-04 15:39 ` Catalin Marinas 2015-11-04 15:39 ` Catalin Marinas 2015-11-04 15:39 ` Catalin Marinas 2015-11-04 15:39 ` Catalin Marinas 2015-11-05 4:31 ` Joonsoo Kim 2015-11-05 4:31 ` Joonsoo Kim 2015-11-05 4:31 ` Joonsoo Kim 2015-11-05 4:31 ` Joonsoo Kim 2015-11-05 11:50 ` [PATCH] mm: slab: Only move management objects off-slab for sizes larger than KMALLOC_MIN_SIZE Catalin Marinas 2015-11-05 11:50 ` Catalin Marinas 2015-11-05 11:50 ` Catalin Marinas 2015-11-05 13:31 ` Andrew Morton 2015-11-05 13:31 ` Andrew Morton 2015-11-05 13:31 ` Andrew Morton 2015-11-05 16:08 ` Catalin Marinas 2015-11-05 16:08 ` Catalin Marinas 2015-11-05 16:08 ` Catalin Marinas 2015-11-06 13:00 ` Geert Uytterhoeven 2015-11-06 13:00 ` Geert Uytterhoeven 2015-11-06 13:00 ` Geert Uytterhoeven 2015-11-05 17:39 ` Christoph Lameter 2015-11-05 17:39 ` Christoph Lameter 2015-11-05 17:39 ` Christoph Lameter 2015-11-05 4:40 ` [PATCH] arm64: Increase the max granular size Joonsoo Kim 2015-11-05 4:40 ` Joonsoo Kim 2015-11-05 10:32 ` Catalin Marinas [this message] 2015-11-05 10:32 ` Catalin Marinas 2015-11-05 11:45 ` Joonsoo Kim 2015-11-05 11:45 ` Joonsoo Kim 2015-11-05 12:17 ` Catalin Marinas 2015-11-05 12:17 ` Catalin Marinas 2015-11-09 7:41 ` Joonsoo Kim 2015-11-09 7:41 ` Joonsoo Kim 2015-11-09 18:36 ` Catalin Marinas 2015-11-09 18:36 ` Catalin Marinas 2015-11-10 0:19 ` Joonsoo Kim 2015-11-10 0:19 ` Joonsoo Kim
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20151105103214.GP7637@e104818-lin.cambridge.arm.com \ --to=catalin.marinas@arm.com \ --cc=iamjoonsoo.kim@lge.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=rric@kernel.org \ --cc=rrichter@cavium.com \ --cc=tchalamarla@cavium.com \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.