From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,LOTS_OF_MONEY,MAILING_LIST_MULTI,MONEY_NOHTML, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3B7CC07E95 for ; Wed, 7 Jul 2021 09:29:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A6C3C61CC1 for ; Wed, 7 Jul 2021 09:29:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231253AbhGGJcR (ORCPT ); Wed, 7 Jul 2021 05:32:17 -0400 Received: from mout.kundenserver.de ([212.227.126.133]:42409 "EHLO mout.kundenserver.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230429AbhGGJcR (ORCPT ); Wed, 7 Jul 2021 05:32:17 -0400 Received: from mail-wm1-f48.google.com ([209.85.128.48]) by mrelayeu.kundenserver.de (mreue009 [213.165.67.97]) with ESMTPSA (Nemesis) id 1N0G5n-1lDtIt1jfz-00xLlZ for ; Wed, 07 Jul 2021 11:29:36 +0200 Received: by mail-wm1-f48.google.com with SMTP id o22so1407694wms.0 for ; Wed, 07 Jul 2021 02:29:36 -0700 (PDT) X-Gm-Message-State: AOAM530BhtKenoPebIJwqT/10AS3bVrsrs2yC3ecNgfrw6ix3ci8yUjW 48MAuHU5DAVwy8NerCOQosSGnY3Z/+vKvh7YK40= X-Google-Smtp-Source: ABdhPJxy9VGMkJsQ2xLLoHD9KhuY49v75rKOXdVcERo4CB/SzAd5LgjZYkH+UrCI7NzD7/EpMdgZQ2cNpiSkwbT9ERU= X-Received: by 2002:a05:600c:3205:: with SMTP id r5mr5526288wmp.75.1625650176018; Wed, 07 Jul 2021 02:29:36 -0700 (PDT) MIME-Version: 1.0 References: <20210527124356.22367-1-will@kernel.org> <59800d6c-364a-f4be-e341-c5b531657ba3@arm.com> <20210706133314.GB20327@willie-the-truck> <87zguz7b6b.wl-maz@kernel.org> In-Reply-To: From: Arnd Bergmann Date: Wed, 7 Jul 2021 11:29:20 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH] arm64: cache: Lower ARCH_DMA_MINALIGN to 64 (L1_CACHE_BYTES) To: Yassine Oudjana Cc: Marc Zyngier , Will Deacon , Robin Murphy , Catalin Marinas , Ard Biesheuvel , Android Kernel Team , Linux ARM , Mark Rutland , Vincent Whitchurch , linux-arm-msm , Bjorn Andersson Content-Type: text/plain; charset="UTF-8" X-Provags-ID: V03:K1:Wzx7QP6bv5PCoUZoj1gFSeZAfY07iI6Arm3y7dQgYSxFOulLmhG Dubs5KpwkJOYowj30SOpVcaGvA0OQSm58OHduJwz1vX9bKDy7gsgkeFmRn+EsYcu7PHoyIY Vlvgc2S21xLfb/mI6R7B69wQHoKS8ltpX5oGpaMflc/p3Tc6iYdOJ04KDs0TDI4bbSGRVfF rU6mFvAB2+p5if9JdT6Gw== X-UI-Out-Filterresults: notjunk:1;V03:K0:f7pEn5TD5ks=:/PELfdULn2BQ9fKaAoZ0Rp pWyVyLSdErJdw6+mVlTeiXVY9quVPKqVhToIC/ggg82u2cGu01/ssJ8S4K+gsnqaS9fSvZU86 6NMXUXLg4tzdZoAQYwPmcS/EJNzHNaH10wXK1DHiGxvA/4rdc+k/Sm5zZQQ+ILeKhnIZBIDpz L0bLWoHfXs/Qgy1LqFcLIDr1ndxZAakHH008QkRhMPmsw+okIHg4LCUO3gJ9UUQXyr4UtgIZN 7x8vusOffxwGjlcnvYmllei8UUciHhZ73GULiuxB6fWR5c9kWKlW/lNKzJOZpAEEOuDnDLt8Y BtfpOm6SKZS0U9rN+XP4OxzYnCIa0uzHSWm98LRyeOHJFdOtqOpOhiFAUYy/5Zse2XgwbLFr3 yBpW7e/nBjLiM9L5uNjS2vyrp+y5TOwK79lCd88GKmUSrTiLUJ6NsEVcxHTB2fhYZ+XWg++2F xgcRWFCSOS4d2rZopxBws2Zh0g6JpBU70UjBniX5Vs37igovBN9f4sBGsS5EsGLMp8TYQsf1/ I6a7ZYPdf9cyq3yI2cW9/60amK9hMgZjJSqyHf7RhCq7Wf4PkQD4XFPMzXv/TxA/6gBzx5wU/ NKvWlOqAGXN46FOIs46FT001ITqA0uL97jXMOISpYH0ldgFcmO3YOQUfi0wi4H/T4eg0T0F1n xY+d50zIkDxihKuGudv2IX/ipS+2Z49m7RXQjy/DXfNBGah+Ug4hmxJp3cBZjjspbW+pK3WZY qgcikYKBK5GmC6vVbG3N9P2b2+quo3kwP9fESav46jjL0JYN/37KDLVZoyJw2Th0s7J61RMru ILJMWeBB8A6R0km+Bbvl2rG54brb/y7LP03W1nQbRhYs14JW2slmY4oAZ5mZVBkzN4OTBLS Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org On Tue, Jul 6, 2021 at 6:20 PM Will Deacon wrote: > > I think the million dollar question is whether the 128-byte cache-lines > live in a cache above the PoC or not. If it's just a system level cache > through which all DMA is "coherent", then it doesn't matter. On Wed, Jul 7, 2021 at 10:24 AM Yassine Oudjana wrote: > > On Wednesday, July 7th, 2021 at 12:33 AM, Arnd Bergmann wrote: > > On Tue, Jul 6, 2021 at 7:15 PM Yassine Oudjana y.oudjana@protonmail.com wrote: > > > > > > $ numactl -C 0 line -M 1M > > > 128 > > > $ numactl -C 3 line -M 1M > > > 128 > > > > Can you rerun the the 'line' test with '-M 128K' to see if that confirms the 64 > > byte L1 line size that the 'cache' test reported? > > $ numactl -C 0 line -M 128K > 64 > $ numactl -C 3 line -M 128K > 64 Ok, so this seems to confirm that the L1 uses 64 byte lines, while the L2 (or possibly L3) uses 128 byte lines. On Wed, Jul 7, 2021 at 12:27 AM Bjorn Andersson wrote: > > I can confirm that MSM8996, and a few derivatives, has 128 byte cache lines. Ok, thanks. Assuming this is an outer cache and the L1 indeed has a smaller line size, can you also confirm that this 128 byte lines are north of the point of coherency? In other words, does the CTR_EL0.DminLine field also show 128 bytes (in which case it seems we already lost)? And if not, does a CPU store to the second half of a 128 byte L2 line cause DMA data in the first half to be clobbered? Arnd From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,LOTS_OF_MONEY, MAILING_LIST_MULTI,MONEY_NOHTML,SPF_HELO_NONE autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD556C07E9E for ; Wed, 7 Jul 2021 10:16:49 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7FE5161C92 for ; Wed, 7 Jul 2021 10:16:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7FE5161C92 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arndb.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xCsnFlfWmCLMZlZbFAJkxi+Ldpk3FPYR+COeMLNbkz8=; b=a3qBvsxKweIdXY FPvjxbZsX1LezTVW9w7sFV3nmjwkPbUa0kHva+Q8sf0c0d4GoSSmu2IHaHwhAKxWu5BVntHlcS9Eo PwITbagOBO36YlMHqxnwAfoGNqKpeOMAxDCnAzb3ncmvhj7UXw97Hh49xkLK6dCEnbAna4LeolqWc IaGMNC1wOeC4wvWswGG3Ct1/GL28bPAZkHAVSDZ8ebDVsk1YgAExFahL31f1fUHs7Vctyg797uMFw WARWaRx6A768PUBpwNejkCGwfn1LutmnLj6gknvaP04+xr0yTDnvbCtdI52zBVib0UDtQxxedyqAB l4p+MSQzv9lmn10KJVAg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m14ZU-00EQux-NU; Wed, 07 Jul 2021 10:14:27 +0000 Received: from mout.kundenserver.de ([212.227.126.134]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m13sB-00EFWV-6D for linux-arm-kernel@lists.infradead.org; Wed, 07 Jul 2021 09:29:40 +0000 Received: from mail-wm1-f51.google.com ([209.85.128.51]) by mrelayeu.kundenserver.de (mreue012 [213.165.67.97]) with ESMTPSA (Nemesis) id 1MVJZv-1lZjb52SX9-00SKKV for ; Wed, 07 Jul 2021 11:29:36 +0200 Received: by mail-wm1-f51.google.com with SMTP id i2-20020a05600c3542b02902058529ea07so1212363wmq.3 for ; Wed, 07 Jul 2021 02:29:36 -0700 (PDT) X-Gm-Message-State: AOAM533+FmtzXv5dY1ihYA0Z5u0oo3BXtvWXpmjVJZp97LYA9hZ3khNF hq6VXjetNwYlDkwtf4uVkXEAtDYbBQK0sBA2Luk= X-Google-Smtp-Source: ABdhPJxy9VGMkJsQ2xLLoHD9KhuY49v75rKOXdVcERo4CB/SzAd5LgjZYkH+UrCI7NzD7/EpMdgZQ2cNpiSkwbT9ERU= X-Received: by 2002:a05:600c:3205:: with SMTP id r5mr5526288wmp.75.1625650176018; Wed, 07 Jul 2021 02:29:36 -0700 (PDT) MIME-Version: 1.0 References: <20210527124356.22367-1-will@kernel.org> <59800d6c-364a-f4be-e341-c5b531657ba3@arm.com> <20210706133314.GB20327@willie-the-truck> <87zguz7b6b.wl-maz@kernel.org> In-Reply-To: From: Arnd Bergmann Date: Wed, 7 Jul 2021 11:29:20 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH] arm64: cache: Lower ARCH_DMA_MINALIGN to 64 (L1_CACHE_BYTES) To: Yassine Oudjana Cc: Marc Zyngier , Will Deacon , Robin Murphy , Catalin Marinas , Ard Biesheuvel , Android Kernel Team , Linux ARM , Mark Rutland , Vincent Whitchurch , linux-arm-msm , Bjorn Andersson X-Provags-ID: V03:K1:/ViHe7MalN+a+p8gUpX2mZvFlmjPi+HzzVgh3BpsIW2eahlaYXL MCxH3dSPx9I1/yM5G2oxOQKyb4bpznlK6MrCTLT417WY+TbtF9GyuZPTZEOZZikxD5Vcb32 g1l3wtxIxvaggmTx71fgO4QStSxTtmmHsNxVdubCUMnPsMJIFo0M538oIrEHnobZgJmbpDQ blGhwZaDsu4wR5Wta3sPA== X-UI-Out-Filterresults: notjunk:1;V03:K0:By0awVVZ79o=:7zMlzuhE6yIq4iZp1SaCFM rns6s1c8IaGaQAS2VTwzqhbNVz4z4bXLFmAbBiuomC98UQGaCnAf1IsjX5wvHQ5n9DFBkCFXF mGolK6VC4zeCeT6r1DRjZL1/xNoE2/Xd+0KqJ7Cnna/kb2eWP6gMWZOMH5RqqfDxrRbMWrIPO Usbz8l48rP59/GJf0IN0AF0l3mO18d84WS1fQJ+OfKLoIGl4WaBIip5Tz44bRQdxKdWzMpIYC 4zflRM8rb9khg0ad9eSZI+Po6o/7ZxvvGJ1RVzoGDOoMdKX00gK+wuRHQsTq/omdc838mn5BS zQyKp7tAsmJrhG/gQXj2YSq44pEgZxsSkLLTOpnQqMl7NykHERAICXVRJlnMTpUQQf5VVjb+m hDhtVyZ856Do0yisOT0P2o/O/1x5LxIAU56jIyaNRkc1CyhFPEqR0aLDcnnAd8L5X8lPWBxPz Zs0Q3xthMNp1OckWCEYNKcLD2HZuHQfB95N2n6jtIaz5deMAQsVxM01ZHkUP0NUjASMTtET3D gwYWjCdFYM1B1qqdvn18iBmab6RZwiy7Q6ORVz4l1Y7xIUQ2zYoU8McIMTqx77tkmy8T0JHpw 9PE0vChWVF2gO5vyv+H8dmcPFqDldcELVKBorM6XlqxeexO1OvXnsDMQSuH5xCbhwLJT+Ek5P 6wOKNuw9UzCVX6iPN2dSL80QhrmQM2aHMd+VyysTVWQaxL3EewufVTcR8QVpTHuapFW5qGdLP c/6cEunxsoLfWDXn3Y2kHvSkujW3LjU9mTM1+lL6R0AbCuNAjBAMjY6gdUZKZkUzySkwdPiMr VnchBPZ+jnGg0B+TaxZvbNzlMkYyIHQdi2zi5rY9fUQd8JFKEeIUWisVGnxHlHfpmPgepLA X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210707_022939_601802_8B672F7D X-CRM114-Status: GOOD ( 18.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Jul 6, 2021 at 6:20 PM Will Deacon wrote: > > I think the million dollar question is whether the 128-byte cache-lines > live in a cache above the PoC or not. If it's just a system level cache > through which all DMA is "coherent", then it doesn't matter. On Wed, Jul 7, 2021 at 10:24 AM Yassine Oudjana wrote: > > On Wednesday, July 7th, 2021 at 12:33 AM, Arnd Bergmann wrote: > > On Tue, Jul 6, 2021 at 7:15 PM Yassine Oudjana y.oudjana@protonmail.com wrote: > > > > > > $ numactl -C 0 line -M 1M > > > 128 > > > $ numactl -C 3 line -M 1M > > > 128 > > > > Can you rerun the the 'line' test with '-M 128K' to see if that confirms the 64 > > byte L1 line size that the 'cache' test reported? > > $ numactl -C 0 line -M 128K > 64 > $ numactl -C 3 line -M 128K > 64 Ok, so this seems to confirm that the L1 uses 64 byte lines, while the L2 (or possibly L3) uses 128 byte lines. On Wed, Jul 7, 2021 at 12:27 AM Bjorn Andersson wrote: > > I can confirm that MSM8996, and a few derivatives, has 128 byte cache lines. Ok, thanks. Assuming this is an outer cache and the L1 indeed has a smaller line size, can you also confirm that this 128 byte lines are north of the point of coherency? In other words, does the CTR_EL0.DminLine field also show 128 bytes (in which case it seems we already lost)? And if not, does a CPU store to the second half of a 128 byte L2 line cause DMA data in the first half to be clobbered? Arnd _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel