From: Catalin Marinas <catalin.marinas@arm.com> To: Linus Torvalds <torvalds@linux-foundation.org>, Christoph Hellwig <hch@lst.de>, Robin Murphy <robin.murphy@arm.com> Cc: Arnd Bergmann <arnd@arndb.de>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Will Deacon <will@kernel.org>, Marc Zyngier <maz@kernel.org>, Andrew Morton <akpm@linux-foundation.org>, Herbert Xu <herbert@gondor.apana.org.au>, Ard Biesheuvel <ardb@kernel.org>, Isaac Manjarres <isaacmanjarres@google.com>, Saravana Kannan <saravanak@google.com>, Alasdair Kergon <agk@redhat.com>, Daniel Vetter <daniel@ffwll.ch>, Joerg Roedel <joro@8bytes.org>, Mark Brown <broonie@kernel.org>, Mike Snitzer <snitzer@kernel.org>, "Rafael J. Wysocki" <rafael@kernel.org>, Jonathan Cameron <jic23@kernel.org>, linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 04/17] mm/slab: Limit kmalloc() minimum alignment to dma_get_cache_alignment() Date: Wed, 31 May 2023 16:48:23 +0100 [thread overview] Message-ID: <20230531154836.1366225-5-catalin.marinas@arm.com> (raw) In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> Do not create kmalloc() caches which are not aligned to dma_get_cache_alignment(). There is no functional change since for current architectures defining ARCH_DMA_MINALIGN, ARCH_KMALLOC_MINALIGN equals ARCH_DMA_MINALIGN (and dma_get_cache_alignment()). On architectures without a specific ARCH_DMA_MINALIGN, dma_get_cache_alignment() is 1, so no change to the kmalloc() caches. If an architecture selects ARCH_HAS_DMA_CACHE_LINE_SIZE (introduced previously), the kmalloc() caches will be aligned to a cache line size. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Christoph Hellwig <hch@lst.de> Cc: Robin Murphy <robin.murphy@arm.com> --- mm/slab_common.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7f069159aee2..7c6475847fdf 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -17,6 +17,7 @@ #include <linux/cpu.h> #include <linux/uaccess.h> #include <linux/seq_file.h> +#include <linux/dma-mapping.h> #include <linux/proc_fs.h> #include <linux/debugfs.h> #include <linux/kasan.h> @@ -862,9 +863,18 @@ void __init setup_kmalloc_cache_index_table(void) } } +static unsigned int __kmalloc_minalign(void) +{ + return dma_get_cache_alignment(); +} + void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { + unsigned int minalign = __kmalloc_minalign(); + unsigned int aligned_size = kmalloc_info[idx].size; + int aligned_idx = idx; + if ((KMALLOC_RECLAIM != KMALLOC_NORMAL) && (type == KMALLOC_RECLAIM)) { flags |= SLAB_RECLAIM_ACCOUNT; } else if (IS_ENABLED(CONFIG_MEMCG_KMEM) && (type == KMALLOC_CGROUP)) { @@ -877,9 +887,17 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) flags |= SLAB_CACHE_DMA; } - kmalloc_caches[type][idx] = create_kmalloc_cache( - kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags); + if (minalign > ARCH_KMALLOC_MINALIGN) { + aligned_size = ALIGN(aligned_size, minalign); + aligned_idx = __kmalloc_index(aligned_size, false); + } + + if (!kmalloc_caches[type][aligned_idx]) + kmalloc_caches[type][aligned_idx] = create_kmalloc_cache( + kmalloc_info[aligned_idx].name[type], + aligned_size, flags); + if (idx != aligned_idx) + kmalloc_caches[type][idx] = kmalloc_caches[type][aligned_idx]; /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for
WARNING: multiple messages have this Message-ID (diff)
From: Catalin Marinas <catalin.marinas@arm.com> To: Linus Torvalds <torvalds@linux-foundation.org>, Christoph Hellwig <hch@lst.de>, Robin Murphy <robin.murphy@arm.com> Cc: Arnd Bergmann <arnd@arndb.de>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Will Deacon <will@kernel.org>, Marc Zyngier <maz@kernel.org>, Andrew Morton <akpm@linux-foundation.org>, Herbert Xu <herbert@gondor.apana.org.au>, Ard Biesheuvel <ardb@kernel.org>, Isaac Manjarres <isaacmanjarres@google.com>, Saravana Kannan <saravanak@google.com>, Alasdair Kergon <agk@redhat.com>, Daniel Vetter <daniel@ffwll.ch>, Joerg Roedel <joro@8bytes.org>, Mark Brown <broonie@kernel.org>, Mike Snitzer <snitzer@kernel.org>, "Rafael J. Wysocki" <rafael@kernel.org>, Jonathan Cameron <jic23@kernel.org>, linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 04/17] mm/slab: Limit kmalloc() minimum alignment to dma_get_cache_alignment() Date: Wed, 31 May 2023 16:48:23 +0100 [thread overview] Message-ID: <20230531154836.1366225-5-catalin.marinas@arm.com> (raw) In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> Do not create kmalloc() caches which are not aligned to dma_get_cache_alignment(). There is no functional change since for current architectures defining ARCH_DMA_MINALIGN, ARCH_KMALLOC_MINALIGN equals ARCH_DMA_MINALIGN (and dma_get_cache_alignment()). On architectures without a specific ARCH_DMA_MINALIGN, dma_get_cache_alignment() is 1, so no change to the kmalloc() caches. If an architecture selects ARCH_HAS_DMA_CACHE_LINE_SIZE (introduced previously), the kmalloc() caches will be aligned to a cache line size. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Christoph Hellwig <hch@lst.de> Cc: Robin Murphy <robin.murphy@arm.com> --- mm/slab_common.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7f069159aee2..7c6475847fdf 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -17,6 +17,7 @@ #include <linux/cpu.h> #include <linux/uaccess.h> #include <linux/seq_file.h> +#include <linux/dma-mapping.h> #include <linux/proc_fs.h> #include <linux/debugfs.h> #include <linux/kasan.h> @@ -862,9 +863,18 @@ void __init setup_kmalloc_cache_index_table(void) } } +static unsigned int __kmalloc_minalign(void) +{ + return dma_get_cache_alignment(); +} + void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { + unsigned int minalign = __kmalloc_minalign(); + unsigned int aligned_size = kmalloc_info[idx].size; + int aligned_idx = idx; + if ((KMALLOC_RECLAIM != KMALLOC_NORMAL) && (type == KMALLOC_RECLAIM)) { flags |= SLAB_RECLAIM_ACCOUNT; } else if (IS_ENABLED(CONFIG_MEMCG_KMEM) && (type == KMALLOC_CGROUP)) { @@ -877,9 +887,17 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) flags |= SLAB_CACHE_DMA; } - kmalloc_caches[type][idx] = create_kmalloc_cache( - kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags); + if (minalign > ARCH_KMALLOC_MINALIGN) { + aligned_size = ALIGN(aligned_size, minalign); + aligned_idx = __kmalloc_index(aligned_size, false); + } + + if (!kmalloc_caches[type][aligned_idx]) + kmalloc_caches[type][aligned_idx] = create_kmalloc_cache( + kmalloc_info[aligned_idx].name[type], + aligned_size, flags); + if (idx != aligned_idx) + kmalloc_caches[type][idx] = kmalloc_caches[type][aligned_idx]; /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2023-05-31 15:49 UTC|newest] Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-05-31 15:48 [PATCH v6 00/17] mm, dma, arm64: Reduce ARCH_KMALLOC_MINALIGN to 8 Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-05-31 15:48 ` [PATCH v6 01/17] mm/slab: Decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-06-09 12:32 ` Vlastimil Babka 2023-06-09 12:32 ` Vlastimil Babka 2023-06-09 13:44 ` Catalin Marinas 2023-06-09 13:44 ` Catalin Marinas 2023-06-09 13:57 ` Catalin Marinas 2023-06-09 13:57 ` Catalin Marinas 2023-06-09 14:13 ` Vlastimil Babka 2023-06-09 14:13 ` Vlastimil Babka 2023-05-31 15:48 ` [PATCH v6 02/17] dma: Allow dma_get_cache_alignment() to be overridden by the arch code Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-05-31 15:48 ` [PATCH v6 03/17] mm/slab: Simplify create_kmalloc_cache() args and make it static Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-06-09 13:03 ` Vlastimil Babka 2023-06-09 13:03 ` Vlastimil Babka 2023-05-31 15:48 ` Catalin Marinas [this message] 2023-05-31 15:48 ` [PATCH v6 04/17] mm/slab: Limit kmalloc() minimum alignment to dma_get_cache_alignment() Catalin Marinas 2023-06-09 14:33 ` Vlastimil Babka 2023-06-09 14:33 ` Vlastimil Babka 2023-05-31 15:48 ` [PATCH v6 05/17] drivers/base: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-05-31 15:48 ` [PATCH v6 06/17] drivers/gpu: " Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-05-31 15:48 ` [PATCH v6 07/17] drivers/usb: " Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-05-31 15:48 ` [PATCH v6 08/17] drivers/spi: " Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-05-31 15:48 ` [PATCH v6 09/17] dm-crypt: " Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-05-31 15:48 ` [PATCH v6 10/17] iio: core: " Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-06-02 11:19 ` Jonathan Cameron 2023-06-02 11:19 ` Jonathan Cameron 2023-05-31 15:48 ` [PATCH v6 11/17] arm64: Allow kmalloc() caches aligned to the smaller cache_line_size() Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-05-31 15:48 ` [PATCH v6 12/17] scatterlist: Add dedicated config for DMA flags Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-05-31 15:48 ` [PATCH v6 13/17] dma-mapping: Name SG DMA flag helpers consistently Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-05-31 15:48 ` [PATCH v6 14/17] dma-mapping: Force bouncing if the kmalloc() size is not cache-line-aligned Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-05-31 15:48 ` [PATCH v6 15/17] iommu/dma: Force bouncing if the size is not cacheline-aligned Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-06-09 11:52 ` Robin Murphy 2023-06-09 11:52 ` Robin Murphy 2023-05-31 15:48 ` [PATCH v6 16/17] mm: slab: Reduce the kmalloc() minimum alignment if DMA bouncing possible Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-06-09 14:39 ` Vlastimil Babka 2023-06-09 14:39 ` Vlastimil Babka 2023-05-31 15:48 ` [PATCH v6 17/17] arm64: Enable ARCH_WANT_KMALLOC_DMA_BOUNCE for arm64 Catalin Marinas 2023-05-31 15:48 ` Catalin Marinas 2023-06-08 5:45 ` [PATCH v6 00/17] mm, dma, arm64: Reduce ARCH_KMALLOC_MINALIGN to 8 Isaac Manjarres 2023-06-08 5:45 ` Isaac Manjarres 2023-06-08 8:05 ` Ard Biesheuvel 2023-06-08 8:05 ` Ard Biesheuvel 2023-06-08 21:29 ` Isaac Manjarres 2023-06-08 21:29 ` Isaac Manjarres 2023-06-09 8:11 ` Petr Tesařík 2023-06-09 8:11 ` Petr Tesařík 2023-06-12 7:44 ` Tomonori Fujita 2023-06-12 7:47 ` Christoph Hellwig 2023-06-14 23:55 ` Isaac Manjarres 2023-06-14 23:55 ` Isaac Manjarres
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20230531154836.1366225-5-catalin.marinas@arm.com \ --to=catalin.marinas@arm.com \ --cc=agk@redhat.com \ --cc=akpm@linux-foundation.org \ --cc=ardb@kernel.org \ --cc=arnd@arndb.de \ --cc=broonie@kernel.org \ --cc=daniel@ffwll.ch \ --cc=gregkh@linuxfoundation.org \ --cc=hch@lst.de \ --cc=herbert@gondor.apana.org.au \ --cc=iommu@lists.linux.dev \ --cc=isaacmanjarres@google.com \ --cc=jic23@kernel.org \ --cc=joro@8bytes.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-mm@kvack.org \ --cc=maz@kernel.org \ --cc=rafael@kernel.org \ --cc=robin.murphy@arm.com \ --cc=saravanak@google.com \ --cc=snitzer@kernel.org \ --cc=torvalds@linux-foundation.org \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.