From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 68296C433F5 for ; Wed, 6 Apr 2022 07:30:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6zt2fWeHIBTKmc3thhn0FV7F0Dt/q0O0c1kLQbM65ac=; b=ELg/zGb3BvmH3/ LepWaTMUfLUN//b7b0sQoHjNaQ/nDcKlhrkOM/u2OHW3STzThrGCaQQlGTiIRUkyvThb/glzjMsdQ 8ZtrtD7kURzhUSYJvqJQmRN6K6MvG/wXbhuTHqZvo/avsCk9j/MECTwpMleL9y8hv5gYZ/ry8BPvm 34oyEMM6ks8N/3yjkXip6fmVR8Ahn8B5HuxCJeSGjOoRMwdFe3LdyGx5JS+Y+63wWCOWNhlwWyA1S sDYlWxgksV5gmGQs5Ac50T+xyB/27yEmx/hj/MrVmdQMy9YwiX/wii4xXazsOlV4/q1SHrgOYS6wz B0sVXVeMtIInENvODRbw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nc06p-004TXZ-CJ; Wed, 06 Apr 2022 07:29:43 +0000 Received: from mout.kundenserver.de ([217.72.192.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nc06k-004TVU-IK for linux-arm-kernel@lists.infradead.org; Wed, 06 Apr 2022 07:29:40 +0000 Received: from mail-wr1-f50.google.com ([209.85.221.50]) by mrelayeu.kundenserver.de (mreue109 [213.165.67.113]) with ESMTPSA (Nemesis) id 1MZkxd-1nWlQV0deh-00WrFd for ; Wed, 06 Apr 2022 09:29:36 +0200 Received: by mail-wr1-f50.google.com with SMTP id d29so1786331wra.10 for ; Wed, 06 Apr 2022 00:29:36 -0700 (PDT) X-Gm-Message-State: AOAM533dPC2QkdZGoxbnZaIfvy3LupWcbA3Tf3tqA+2zIG0lp+UllWcd 8aDx/aEGUad1CQyAyO8ipcqYoso+5BE5K4GOia0= X-Google-Smtp-Source: ABdhPJzyKGu3dTz7c5wCMIgY8RrIo/iPoFC4jg/w0bCaH/HFgYzcAZoBaAjqJoBHz9Ym+MjmIi1iL6FIKFnTMAWf6Vc= X-Received: by 2002:a05:6000:178c:b0:204:648:b4c4 with SMTP id e12-20020a056000178c00b002040648b4c4mr5325019wrg.219.1649230175681; Wed, 06 Apr 2022 00:29:35 -0700 (PDT) MIME-Version: 1.0 References: <20220405135758.774016-1-catalin.marinas@arm.com> <20220405135758.774016-2-catalin.marinas@arm.com> In-Reply-To: From: Arnd Bergmann Date: Wed, 6 Apr 2022 09:29:19 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH 01/10] mm/slab: Decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN To: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds , Linux-MM , Linux ARM , Linux Kernel Mailing List , Christoph Lameter , Vlastimil Babka , Joonsoo Kim , David Rientjes , Pekka Enberg , roman.gushchin@linux.dev X-Provags-ID: V03:K1:Thxy80hvGd3VOrqvRCQ3Nd5yOdIjynLqA3U9qtf6+FpAC0hT3jV KCOmEUpD2dQgtPkCmFqN/UJ5Ujuq5ljHYkQOAQwENXFDzTgmMKO1jkiwfLL1qXkbF89+lUr ve/qgxKqUdXgHjjiqnu7pX46wiQlc+80oNHV9sgNDLG2RP8g8YEx86FMrpbnxpv6jUkmj5d gZVDfeuCmeV6U9X8bnBlg== X-UI-Out-Filterresults: notjunk:1;V03:K0:7XEzL1S1qvE=:W1GN0bsEp0vBaVo62vBq6i rpm505bpRhrWRs9c33hsnmlLQb4qulVPE70g+iudnSC8Y+NjeRDTmjUWDqd+vXWBzH6CzwJPb V6LxEyhTELcjUWo28UNjS2sQxGyAjSEZTwYlbBTTk3sC/BABSJD1bH7OGv8kkSqKQyM+FncWn uk4cp6XF/1oycPl/dnUpW8aKLfplUmgcD6T6g1lHygA8L8X5LpIeqGkFel7TE48jg+yPQjK2C ATmgDxx9h2ywVayQidPRNd5tQCS5eGNI0NgevC7nIeCQGUJVhxqu8YaWuNcg/plwfk4lAiP+R lRBLmK2M1ax9eOooPvWsIZNTLmaT9iGqy7b54pSFOKbQB8ZSJIJeSzM47l7juy1BbDpz0pjQm nDIBr41c+1ST/COp1wlwnr6/FSdVF7lkKBayRU0Q9RV4wBjeAE6HeZ/CkiuFhYtyUMRJXjfIt 4f2HHxu0aFOtMu1TB+vsf4X2Yz9q4QpYKLpLMQvv27Y3tOgd0wAASigtZpSChSKvZ59uLT6wE 0tlFALkLb7QTKT+lke3s+tWHdukNdZXm5SgLPIsi7wWbOIW3Y+yizvKvXLiIrGOpa/gMb5JJh RgkSyqHNxq5n6lomWGgcXVHhQskxIIdSPYBBbrLFG6as9hmVj6fZF4izHxbHxjS+5Sy2D3VKc P/31Ycq8NhntdDXotT9MeC8lf51Av5nHhV0s+MQ1uJt7KDVoHHQ6H7LdHodFLx8/dD4WHkVn4 DKClBr9t8bxZ+SaBwB04lvjpMYjV9O3qH9DuNDaQRkZ0SNs8REkt1KuK6kJpd1DTr2Vr7NrG7 bUMV2CJL1LUNKDGuQXBhcxtrfWsAolo0lavT5mbKqeyoKSlIAE= X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220406_002938_938743_1CC6247D X-CRM114-Status: GOOD ( 28.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Apr 6, 2022 at 1:59 AM Hyeonggon Yoo <42.hyeyoo@gmail.com> wrote: > > On Tue, Apr 05, 2022 at 02:57:49PM +0100, Catalin Marinas wrote: > > In preparation for supporting a dynamic kmalloc() minimum alignment, > > allow architectures to define ARCH_KMALLOC_MINALIGN independently of > > ARCH_DMA_MINALIGN. In addition, always define ARCH_DMA_MINALIGN even if > > an architecture does not override it. > > > > [ +Cc slab maintainer/reviewers ] > > I get why you want to set minimum alignment of kmalloc() dynamically. > That's because cache line size can be different and we cannot statically > know that, right? > > But I don't get why you are trying to decouple ARCH_KMALLOC_MINALIGN > from ARCH_DMA_MINALIGN. kmalloc'ed buffer is always supposed to be DMA-safe. > > I'm afraid this series may break some archs/drivers. > > in Documentation/dma-api-howto.rst: > > 2) ARCH_DMA_MINALIGN > > > > Architectures must ensure that kmalloc'ed buffer is > > DMA-safe. Drivers and subsystems depend on it. If an architecture > > isn't fully DMA-coherent (i.e. hardware doesn't ensure that data in > > the CPU cache is identical to data in main memory), > > ARCH_DMA_MINALIGN must be set so that the memory allocator > > makes sure that kmalloc'ed buffer doesn't share a cache line with > > the others. See arch/arm/include/asm/cache.h as an example. > > > > Note that ARCH_DMA_MINALIGN is about DMA memory alignment > > constraints. You don't need to worry about the architecture data > > alignment constraints (e.g. the alignment constraints about 64-bit > > objects). > > If I'm missing something, please let me know :) It helps in two ways: - you can start with a relatively large hardcoded ARCH_DMA_MINALIGN of 128 or 256 bytes, depending on what the largest possible line size is for any machine you want to support, and then drop that down to 32 or 64 bytes based on runtime detection. This should always be safe, and it means a very sizable chunk of wasted memory can be recovered. - On systems that are fully cache coherent, there is no need to align kmallloc() allocations for DMA safety at all, on these, we can drop the size even below the cache line. This does not apply on most of the cheaper embedded or mobile SoCs, but it helps a lot on the machines you'd find in a data center. Arnd _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD9FDC433EF for ; Wed, 6 Apr 2022 11:01:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350420AbiDFLDI (ORCPT ); Wed, 6 Apr 2022 07:03:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44914 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241614AbiDFLCR (ORCPT ); Wed, 6 Apr 2022 07:02:17 -0400 Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.17.13]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0A20528D10 for ; Wed, 6 Apr 2022 00:29:37 -0700 (PDT) Received: from mail-wr1-f45.google.com ([209.85.221.45]) by mrelayeu.kundenserver.de (mreue109 [213.165.67.113]) with ESMTPSA (Nemesis) id 1Mw9Lu-1nsZaW0HZ6-00s6Zi for ; Wed, 06 Apr 2022 09:29:36 +0200 Received: by mail-wr1-f45.google.com with SMTP id m30so1847161wrb.1 for ; Wed, 06 Apr 2022 00:29:36 -0700 (PDT) X-Gm-Message-State: AOAM532P/fXUxY7BOSmVH0Q7fTc8Mn3OTAWZhdzEpjf6x/jNxBHoXUyH WMjNQBKtDmaY+PcNTBUV2FDMqSD8SNNB6pw1u4k= X-Google-Smtp-Source: ABdhPJzyKGu3dTz7c5wCMIgY8RrIo/iPoFC4jg/w0bCaH/HFgYzcAZoBaAjqJoBHz9Ym+MjmIi1iL6FIKFnTMAWf6Vc= X-Received: by 2002:a05:6000:178c:b0:204:648:b4c4 with SMTP id e12-20020a056000178c00b002040648b4c4mr5325019wrg.219.1649230175681; Wed, 06 Apr 2022 00:29:35 -0700 (PDT) MIME-Version: 1.0 References: <20220405135758.774016-1-catalin.marinas@arm.com> <20220405135758.774016-2-catalin.marinas@arm.com> In-Reply-To: From: Arnd Bergmann Date: Wed, 6 Apr 2022 09:29:19 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH 01/10] mm/slab: Decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN To: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds , Linux-MM , Linux ARM , Linux Kernel Mailing List , Christoph Lameter , Vlastimil Babka , Joonsoo Kim , David Rientjes , Pekka Enberg , roman.gushchin@linux.dev Content-Type: text/plain; charset="UTF-8" X-Provags-ID: V03:K1:So8h0q9ZQICwqajKA7j398ySPx4gnTPbhPGpZie+33Zigmexopl wSZmniBKQ0uIGXvbmFrT9K+Y10U4VM+UL6jXszHDMaveFq3nGa3woJDbH3+KLk9N8qU7FWW HWLS84Ip/+/TSIwnuKhOBrm9kA77xV7hK4SFos1xlP+0qm9E/zhBKv5t3fHiIhRz7E7T1iJ SC4KVpTMICvFF7Fa2Z/UA== X-UI-Out-Filterresults: notjunk:1;V03:K0:easS4jZt73c=:R0u25Ml8TVTrQg81cUg1Kd UbE1c4mBHm5Ub9TSNuIFb1Q+cNHVGZtAQI7t0Ajf4q3SA8dk+YiHiFBE6tWhCCBXkblq4D2Zf 0oA3RPzv/WARltUFvidfFzHDgx8Htwxqf893dEE8xvRcfY3B+cM3z5i4clWXash6h2KCZo2ga dhmgry5dvnhb1UjI+A/pxv6NL/aameEgGnuGrPt6/WNvuzWn8E3nHopWY32ruwx4yRjk3rV4l uGM65pBTfuZP86pAGBtnIgRld74YIjD6u6thEEdcC3G5Ly1417/B7Dw1MVfv1NzFbBbfEhEiU vQ2RXYqmEtDx8oKSTzE3POV1KkDPZaAACH57jBiD8Z6Xi5M0v+vnOGJNJHSr+8Is2hT8VFON/ 3KbNE45omKqeVSPTPVw8l4oKXtMYDgk69eH4Nm+KW00aTxnctVuPJfeCNjIr8PtQ2hbvzLtSc q5GVnNr/oO0w9/OWiLvNSXh62PzeM2U5oVk5hri3F1470byaRK+/BdXIphxpCDOpm1yk0XrAX 2redUoIRjS8pdplL5q5HhivImHlLVG0p6n9IkDIKH8BdU4kew/4yBWikMtnJLHpwbOElZSejE 3LuOUV3/KDfSb1a/gv6NU7QIKrtDVXp58w6t8bHe2NihfUtD2LOORPgKNTQQm3IoeKqpRZ1hl wbzrmKPOc1PHOfqCnWVJud2PSSIZuhxTYVZx3aoNICarJgqf6M0CAh6wCwQa/YZ+CalKjAJ3T IEiEU3JGzuQJAkBSLqnLh91YReh/dWOO8sY9x1V2Jlp5TLTS40CAWsUSs3YWf4QNr13aaZwTK ThJo3/VIr5FFMgMt5D9vEf8kQzZx2cxFcJBsfM2BkvSMleEeDA= Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Apr 6, 2022 at 1:59 AM Hyeonggon Yoo <42.hyeyoo@gmail.com> wrote: > > On Tue, Apr 05, 2022 at 02:57:49PM +0100, Catalin Marinas wrote: > > In preparation for supporting a dynamic kmalloc() minimum alignment, > > allow architectures to define ARCH_KMALLOC_MINALIGN independently of > > ARCH_DMA_MINALIGN. In addition, always define ARCH_DMA_MINALIGN even if > > an architecture does not override it. > > > > [ +Cc slab maintainer/reviewers ] > > I get why you want to set minimum alignment of kmalloc() dynamically. > That's because cache line size can be different and we cannot statically > know that, right? > > But I don't get why you are trying to decouple ARCH_KMALLOC_MINALIGN > from ARCH_DMA_MINALIGN. kmalloc'ed buffer is always supposed to be DMA-safe. > > I'm afraid this series may break some archs/drivers. > > in Documentation/dma-api-howto.rst: > > 2) ARCH_DMA_MINALIGN > > > > Architectures must ensure that kmalloc'ed buffer is > > DMA-safe. Drivers and subsystems depend on it. If an architecture > > isn't fully DMA-coherent (i.e. hardware doesn't ensure that data in > > the CPU cache is identical to data in main memory), > > ARCH_DMA_MINALIGN must be set so that the memory allocator > > makes sure that kmalloc'ed buffer doesn't share a cache line with > > the others. See arch/arm/include/asm/cache.h as an example. > > > > Note that ARCH_DMA_MINALIGN is about DMA memory alignment > > constraints. You don't need to worry about the architecture data > > alignment constraints (e.g. the alignment constraints about 64-bit > > objects). > > If I'm missing something, please let me know :) It helps in two ways: - you can start with a relatively large hardcoded ARCH_DMA_MINALIGN of 128 or 256 bytes, depending on what the largest possible line size is for any machine you want to support, and then drop that down to 32 or 64 bytes based on runtime detection. This should always be safe, and it means a very sizable chunk of wasted memory can be recovered. - On systems that are fully cache coherent, there is no need to align kmallloc() allocations for DMA safety at all, on these, we can drop the size even below the cache line. This does not apply on most of the cheaper embedded or mobile SoCs, but it helps a lot on the machines you'd find in a data center. Arnd