From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDA21C433B4 for ; Mon, 17 May 2021 14:18:57 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A4D306194C for ; Mon, 17 May 2021 14:18:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A4D306194C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=bYydXQWSoRrC68ID7BW7u9uiCYS6yDCSzIu7VHtE7co=; b=YRq+9idtesIrrpNPf0qaGx91c 4awu7EDloM6PzBlcR345YNkaE9DlDfNi0X1MGk8RX1zpl8mdBgvRItG2S7rwStRu0IKgRsIqOs/mL D+Uh8tNzKNd/r4hinv1BwRoM2opNXv2v5nbbi2KYbqYKJMSVyhbCzz3il51ts6L6CW/cT4Iz1YMEO sSMaS+zj/bybkfqqK0TGFSYBYLoNCE3Gyy5atpLEBSgsxHtkYXbuBv/4ZhzC1oYzWB+LG8evFbjwb ziItdNx17cHp23wnqTyiJe6Dj7CiPZpMB+Y8Gbeq6YUBQdPtQBNmc4HdxDXMfEkhCNIGm1m8JdwV9 /gTjiwZsw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lie2U-00FD9I-HA; Mon, 17 May 2021 14:16:10 +0000 Received: from bombadil.infradead.org ([198.137.202.133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lie1l-00FD2S-23 for linux-arm-kernel@desiato.infradead.org; Mon, 17 May 2021 14:15:25 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=In-Reply-To:Content-Type:MIME-Version :References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=FGhkVcC6PK6uVFlsJMfiI7twn5Pn48pd0ktc+sxkW0Y=; b=CJKzh9rI1AXw2fRUyWGe7ZOcCd L48vV37vYgobS52OJM1C2J38CzlVSNUDiEjeR3eq25OqbKtDYH6LB2lm42HdvICBZRHxHwx5PgP/u MeiCHOYriNNoq+6zBfFffTzO8RwHERfSVHKtQr2Dc/Xzuk2KBlup0eDV6d75b4VCVMxKj7lS86bsp XurpjEvS+BSXeKNeK45J2YYhCCWB0BVnHAodL53BgjJkeXODRQTHFgUPxHnu1LJ7SZzVppg+t6WBt BPgRShSkqcjXAuEd3AXekVFdGG2z9R4wBEfg1yZycyLrSUFweEN01ZRt2EeALbPGKa4PEwLUsmBWT +dJzD0xQ==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lie1e-00DqLO-Uf for linux-arm-kernel@lists.infradead.org; Mon, 17 May 2021 14:15:23 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 778BC6186A; Mon, 17 May 2021 14:15:17 +0000 (UTC) Date: Mon, 17 May 2021 15:15:15 +0100 From: Catalin Marinas To: Ard Biesheuvel Cc: Vincent Whitchurch , Will Deacon , Linux ARM , kernel@axis.com, Arnd Bergmann Subject: Re: [PATCH] arm64: Make ARCH_DMA_MINALIGN configurable Message-ID: <20210517141514.GC1106@arm.com> References: <20210517074332.28280-1-vincent.whitchurch@axis.com> <20210517110406.GA1106@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210517_071519_041123_C0EE292A X-CRM114-Status: GOOD ( 29.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, May 17, 2021 at 02:01:54PM +0200, Ard Biesheuvel wrote: > On Mon, 17 May 2021 at 13:06, Catalin Marinas wrote: > > On Mon, May 17, 2021 at 09:43:32AM +0200, Vincent Whitchurch wrote: > > > ARCH_DMA_MINALIGN is hardcoded to 128, but this wastes memory if the > > > kernel is only intended to be run on platforms with cache line sizes of > > > 64 bytes. > > > > > > Make this configurable (hidden under CONFIG_EXPERT). Setting this to 64 > > > bytes reduces the slab memory usage of my Cortex-A53-based system by > > > ~6%, measured right after startup. > > > > I agree that we waste some memory since the kmalloc caches start from > > 128 but I don't think a config option is the right. > > > > An option would be to try not to rely on the hard-coded > > ARCH_DMA_MINALIGN when the slab caches are created but use > > cache_line_size(). It's a bit tricky as the cache_line_size() returned > > value may be tweaked by DT or PPTT after the boot caches have been > > created (see commit 7b8c87b297a7). > > > > Another option I recall discussing with Arnd about two years ago was to > > start with the default 128 at boot but add the smaller slab caches > > later, once we have more information. This can be just another 64 byte > > cache or even go all the way down to 8 byte if all the devices are > > cache coherent. > > ARCH_SLAB_MINALIGN is also used to statically align (members of) > struct types, so doing this at runtime is going to have limited > effect. You probably mean ARCH_KMALLOC_MINALIGN. We don't touch ARCH_SLAB_MINALIGN unless KASAN_{SW,HW}_TAGS is enabled and it is still maximum 16 (I wonder if it's safe to reduce this to 8 as it might be the case with KASAN_SW_TAGS). > If a) ThunderX is the only platform we care about (do we?) that has > 128 byte cachelines, and b) DMA is cache coherent on such platforms, > couldn't we separate ARCH_SLAB_MINALIGN from ARCH_DMA_MINALIGN? I.e., > set the first to 64 and keep the second at 128? ARCH_KMALLOC_MINALIGN is indeed the same as ARCH_DMA_MINALIGN. We can't do much about when the requirement is DMA. For example, struct devres has a data[] array aligned to ARCH_KMALLOC_MINALIGN with a comment that it may be needed for DMA. I can't tell how many SoCs out there have 128 byte cache lines (it can be in a system level cache that reacts to cache maintenance by VA) and are not fully coherent. I know there is one with 256 byte caches but luckily it's cache coherent. Even if we can't fix the build-time structure alignment, I think there's still sufficient benefit in allowing smaller slab caches. -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel