From: "Chalamarla, Tirumalesh" <Tirumalesh.Chalamarla@caviumnetworks.com> To: Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will.deacon@arm.com> Cc: Ganesh Mahendran <opensource.ganesh@gmail.com>, "stable@vger.kernel.org" <stable@vger.kernel.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org> Subject: Re: [PATCH] Revert "arm64: Increase the max granular size" Date: Mon, 21 Mar 2016 17:39:20 +0000 [thread overview] Message-ID: <C1489C4D-9980-4889-B2F0-B071E58FD733@caviumnetworks.com> (raw) In-Reply-To: <20160321173317.GF25466@e104818-lin.cambridge.arm.com> On 3/21/16, 10:33 AM, "Catalin Marinas" <catalin.marinas@arm.com> wrote: >On Mon, Mar 21, 2016 at 05:23:01PM +0000, Will Deacon wrote: >> On Mon, Mar 21, 2016 at 05:14:03PM +0000, Catalin Marinas wrote: >> > diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h >> > index 5082b30bc2c0..4b5d7b27edaf 100644 >> > --- a/arch/arm64/include/asm/cache.h >> > +++ b/arch/arm64/include/asm/cache.h >> > @@ -18,17 +18,17 @@ >> > >> > #include <asm/cachetype.h> >> > >> > -#define L1_CACHE_SHIFT 7 >> > +#define L1_CACHE_SHIFT 6 >> > #define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT) >> > >> > /* >> > * Memory returned by kmalloc() may be used for DMA, so we must make >> > - * sure that all such allocations are cache aligned. Otherwise, >> > - * unrelated code may cause parts of the buffer to be read into the >> > - * cache before the transfer is done, causing old data to be seen by >> > - * the CPU. >> > + * sure that all such allocations are aligned to the maximum *known* >> > + * cache line size on ARMv8 systems. Otherwise, unrelated code may cause >> > + * parts of the buffer to be read into the cache before the transfer is >> > + * done, causing old data to be seen by the CPU. >> > */ >> > -#define ARCH_DMA_MINALIGN L1_CACHE_BYTES >> > +#define ARCH_DMA_MINALIGN (128) >> >> Does this actually fix the reported iperf regression? My assumption was >> that ARCH_DMA_MINALIGN is the problem, but I could be wrong. > >I can't tell. But since I haven't seen any better explanation in this >thread yet, I hope that at least someone would try this patch and come >back with numbers. > >For networking, SKB_DATA_ALIGN() uses SMP_CACHE_BYTES (== L1_CACHE_BYTES). >I think (hope) this alignment is not meant for non-coherent DMA, >otherwise using SMP_CACHE_BYTES wouldn't make sense. As I see the problem, may be it is because of fewer number of buffers available because of this alignment requirement. But that should be fixed by making slab alignment a run time thing. I may be totally wrong. We are still coming up with a decent benchmark that shows degradation. > >-- >Catalin
WARNING: multiple messages have this Message-ID (diff)
From: Tirumalesh.Chalamarla@caviumnetworks.com (Chalamarla, Tirumalesh) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH] Revert "arm64: Increase the max granular size" Date: Mon, 21 Mar 2016 17:39:20 +0000 [thread overview] Message-ID: <C1489C4D-9980-4889-B2F0-B071E58FD733@caviumnetworks.com> (raw) In-Reply-To: <20160321173317.GF25466@e104818-lin.cambridge.arm.com> On 3/21/16, 10:33 AM, "Catalin Marinas" <catalin.marinas@arm.com> wrote: >On Mon, Mar 21, 2016 at 05:23:01PM +0000, Will Deacon wrote: >> On Mon, Mar 21, 2016 at 05:14:03PM +0000, Catalin Marinas wrote: >> > diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h >> > index 5082b30bc2c0..4b5d7b27edaf 100644 >> > --- a/arch/arm64/include/asm/cache.h >> > +++ b/arch/arm64/include/asm/cache.h >> > @@ -18,17 +18,17 @@ >> > >> > #include <asm/cachetype.h> >> > >> > -#define L1_CACHE_SHIFT 7 >> > +#define L1_CACHE_SHIFT 6 >> > #define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT) >> > >> > /* >> > * Memory returned by kmalloc() may be used for DMA, so we must make >> > - * sure that all such allocations are cache aligned. Otherwise, >> > - * unrelated code may cause parts of the buffer to be read into the >> > - * cache before the transfer is done, causing old data to be seen by >> > - * the CPU. >> > + * sure that all such allocations are aligned to the maximum *known* >> > + * cache line size on ARMv8 systems. Otherwise, unrelated code may cause >> > + * parts of the buffer to be read into the cache before the transfer is >> > + * done, causing old data to be seen by the CPU. >> > */ >> > -#define ARCH_DMA_MINALIGN L1_CACHE_BYTES >> > +#define ARCH_DMA_MINALIGN (128) >> >> Does this actually fix the reported iperf regression? My assumption was >> that ARCH_DMA_MINALIGN is the problem, but I could be wrong. > >I can't tell. But since I haven't seen any better explanation in this >thread yet, I hope that at least someone would try this patch and come >back with numbers. > >For networking, SKB_DATA_ALIGN() uses SMP_CACHE_BYTES (== L1_CACHE_BYTES). >I think (hope) this alignment is not meant for non-coherent DMA, >otherwise using SMP_CACHE_BYTES wouldn't make sense. As I see the problem, may be it is because of fewer number of buffers available because of this alignment requirement. But that should be fixed by making slab alignment a run time thing. I may be totally wrong. We are still coming up with a decent benchmark that shows degradation. > >-- >Catalin
next prev parent reply other threads:[~2016-03-21 17:39 UTC|newest] Thread overview: 91+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-03-16 9:32 [PATCH] Revert "arm64: Increase the max granular size" Ganesh Mahendran 2016-03-16 9:32 ` Ganesh Mahendran 2016-03-16 10:07 ` Will Deacon 2016-03-16 10:07 ` Will Deacon 2016-03-16 13:06 ` Timur Tabi 2016-03-16 13:06 ` Timur Tabi 2016-03-16 14:03 ` Mark Rutland 2016-03-16 14:03 ` Mark Rutland 2016-03-16 14:35 ` Will Deacon 2016-03-16 14:35 ` Will Deacon 2016-03-16 14:54 ` Mark Rutland 2016-03-16 14:54 ` Mark Rutland 2016-03-16 14:18 ` Catalin Marinas 2016-03-16 14:18 ` Catalin Marinas 2016-03-16 15:26 ` Timur Tabi 2016-03-16 15:26 ` Timur Tabi 2016-03-17 14:27 ` Catalin Marinas 2016-03-17 14:27 ` Catalin Marinas 2016-03-17 14:49 ` Timur Tabi 2016-03-17 14:49 ` Timur Tabi 2016-03-17 15:37 ` Catalin Marinas 2016-03-17 15:37 ` Catalin Marinas 2016-03-17 16:03 ` Marc Zyngier 2016-03-17 16:03 ` Marc Zyngier 2016-03-17 18:07 ` Andrew Pinski 2016-03-17 18:07 ` Andrew Pinski 2016-03-17 18:34 ` Timur Tabi 2016-03-17 18:34 ` Timur Tabi 2016-03-17 18:37 ` Catalin Marinas 2016-03-17 18:37 ` Catalin Marinas 2016-03-18 21:05 ` Chalamarla, Tirumalesh 2016-03-18 21:05 ` Chalamarla, Tirumalesh 2016-03-18 21:05 ` Chalamarla, Tirumalesh 2016-03-21 1:56 ` Ganesh Mahendran 2016-03-21 1:56 ` Ganesh Mahendran 2016-03-21 1:56 ` Ganesh Mahendran 2016-03-21 17:14 ` Catalin Marinas 2016-03-21 17:14 ` Catalin Marinas 2016-03-21 17:14 ` Catalin Marinas 2016-03-21 17:23 ` Will Deacon 2016-03-21 17:23 ` Will Deacon 2016-03-21 17:23 ` Will Deacon 2016-03-21 17:33 ` Catalin Marinas 2016-03-21 17:33 ` Catalin Marinas 2016-03-21 17:33 ` Catalin Marinas 2016-03-21 17:39 ` Chalamarla, Tirumalesh [this message] 2016-03-21 17:39 ` Chalamarla, Tirumalesh 2016-03-21 17:39 ` Chalamarla, Tirumalesh [not found] ` <CAPub14-sFgx=oCHzJPb9h9b_V0rbn5UAMDNJ-yTkjhz38JPqMQ@mail.gmail.com> [not found] ` <10fef112-37f1-0a1b-b5af-435acd032f01@codeaurora.org> 2017-04-06 7:22 ` Imran Khan 2017-04-06 7:22 ` Imran Khan 2017-04-06 7:22 ` Imran Khan 2017-04-06 15:58 ` Catalin Marinas 2017-04-06 15:58 ` Catalin Marinas 2017-04-07 2:06 ` Ganesh Mahendran 2017-04-07 2:06 ` Ganesh Mahendran 2017-04-07 8:59 ` Catalin Marinas 2017-04-07 8:59 ` Catalin Marinas 2017-04-12 5:13 ` Imran Khan 2017-04-12 5:13 ` Imran Khan 2017-04-12 14:00 ` Chalamarla, Tirumalesh 2017-04-12 14:00 ` Chalamarla, Tirumalesh 2017-04-12 14:00 ` Chalamarla, Tirumalesh 2017-04-17 7:35 ` Imran Khan 2017-04-17 7:35 ` Imran Khan 2017-04-17 7:35 ` Imran Khan 2017-04-17 10:38 ` Sunil Kovvuri 2017-04-17 10:38 ` Sunil Kovvuri 2017-04-17 10:38 ` Sunil Kovvuri 2017-04-18 14:48 ` Catalin Marinas 2017-04-18 14:48 ` Catalin Marinas 2017-04-18 14:48 ` Catalin Marinas 2017-04-18 17:05 ` Sunil Kovvuri 2017-04-18 17:05 ` Sunil Kovvuri 2017-04-18 17:05 ` Sunil Kovvuri 2017-04-19 12:01 ` Catalin Marinas 2017-04-19 12:01 ` Catalin Marinas 2017-04-19 12:01 ` Catalin Marinas 2017-04-19 13:11 ` Sunil Kovvuri 2017-04-19 13:11 ` Sunil Kovvuri 2017-04-19 13:11 ` Sunil Kovvuri 2017-04-25 6:42 ` Ding Tianhong 2017-04-25 6:42 ` Ding Tianhong 2017-04-25 6:42 ` Ding Tianhong 2017-04-18 18:21 ` Chalamarla, Tirumalesh 2017-04-18 18:21 ` Chalamarla, Tirumalesh 2017-04-18 18:21 ` Chalamarla, Tirumalesh 2017-04-11 4:40 ` Jon Masters 2017-04-11 4:40 ` Jon Masters 2017-04-11 4:40 ` Jon Masters -- strict thread matches above, loose matches on Subject: below -- 2016-03-16 9:37 Ganesh Mahendran 2016-03-16 9:27 Ganesh Mahendran
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=C1489C4D-9980-4889-B2F0-B071E58FD733@caviumnetworks.com \ --to=tirumalesh.chalamarla@caviumnetworks.com \ --cc=catalin.marinas@arm.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=opensource.ganesh@gmail.com \ --cc=stable@vger.kernel.org \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.