From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,LOTS_OF_MONEY,MAILING_LIST_MULTI,MONEY_NOHTML, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63087C07E95 for ; Wed, 7 Jul 2021 14:41:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4475561CC5 for ; Wed, 7 Jul 2021 14:41:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232020AbhGGOoI (ORCPT ); Wed, 7 Jul 2021 10:44:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43260 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232056AbhGGOoI (ORCPT ); Wed, 7 Jul 2021 10:44:08 -0400 Received: from mail-il1-x134.google.com (mail-il1-x134.google.com [IPv6:2607:f8b0:4864:20::134]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E7F6C061574 for ; Wed, 7 Jul 2021 07:41:26 -0700 (PDT) Received: by mail-il1-x134.google.com with SMTP id g3so2997692ilj.7 for ; Wed, 07 Jul 2021 07:41:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=nIwflScCI6Icjj1cWTTKEy9uQqC1wMtDQ5W0AmkNq+g=; b=qSsUSkhUDDeNTKV9IlCEbm5Hj4XmuLwhiJaM0jY4sL1uhWUMhgpGglFL3S092uXiDj /XMSg5ef6TxisfLtL4rp/0CVFyFA3VMjGerBWdJFKyRc95JI/pwk4RNsms4ev8lTdzdT kmf/HHtLD6sZKM5ba4zPBnhrt5oBNwIzA0pOchC937ROImjrDpTL07S8cHX+Smh0XuLi Rz6rT31y3iaIrs2yvFzo21OEssbLwi6J8IMdWQ9fwWcRYahQqSl3jOXj9IRtNUDh0hev 2zfVoVZWn8ERMpvpvBudbyfGW86s/+treNOC7kHqDEWtwbMfrbkEZdsFHyesocfgw5Va /HOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=nIwflScCI6Icjj1cWTTKEy9uQqC1wMtDQ5W0AmkNq+g=; b=l7Sx7TIiCqi3qI5ISCRIYxYqoOLnceV7mckmlRAo6dq3EjGJeMpz2E8xBMHRzfuaGH snClzNWtn8RO4o+l9FhDAFIccuDGmnNKoW2m3lZwFhSDCQ9W0dHzOA1h10/2+7fPPEWr AVNE0Wah7EdpPhZ0rLT9NOgh0MJIWKjFCOQv8VCsCiYNv0mcHPqoT3wOVq5zi7tE78AZ v1NnXLQNXqH35RdH5xHkuidUiqMuD2qsvSSgj7hXktZT6Ognt+iDIHWk8NFDMJUqJT8t oZVgHXoqsBytBUx2MhzizFDzS7AWiy8rbeHmz+QX4io51E+6h8X6Brss4otrOMq1PWOf aD4w== X-Gm-Message-State: AOAM533JAI1KPc1j/6EJQ9yossICBzly0JntqzCBMVd4orns5Xa6VXdl 9ghqqQLtiXhTfMfn5koZ4V2NxsbQIODIDROWnO4= X-Google-Smtp-Source: ABdhPJwQQMe3S+AM4KYV3Xo0/ehcLXF/J7ADlwe9sG4QkGCGpF1PS07nGxT96XXQEqLf3PdAJZC9KF8DNC5jsbg0epk= X-Received: by 2002:a05:6e02:58f:: with SMTP id c15mr18436980ils.102.1625668886115; Wed, 07 Jul 2021 07:41:26 -0700 (PDT) MIME-Version: 1.0 References: <20210527124356.22367-1-will@kernel.org> <59800d6c-364a-f4be-e341-c5b531657ba3@arm.com> <20210706133314.GB20327@willie-the-truck> <87zguz7b6b.wl-maz@kernel.org> In-Reply-To: From: Jeffrey Hugo Date: Wed, 7 Jul 2021 08:41:15 -0600 Message-ID: Subject: Re: [PATCH] arm64: cache: Lower ARCH_DMA_MINALIGN to 64 (L1_CACHE_BYTES) To: Arnd Bergmann Cc: Yassine Oudjana , Marc Zyngier , Will Deacon , Robin Murphy , Catalin Marinas , Ard Biesheuvel , Android Kernel Team , Linux ARM , Mark Rutland , Vincent Whitchurch , linux-arm-msm , Bjorn Andersson Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org On Wed, Jul 7, 2021 at 3:30 AM Arnd Bergmann wrote: > > On Tue, Jul 6, 2021 at 6:20 PM Will Deacon wrote: > > > > I think the million dollar question is whether the 128-byte cache-lines > > live in a cache above the PoC or not. If it's just a system level cache > > through which all DMA is "coherent", then it doesn't matter. > > On Wed, Jul 7, 2021 at 10:24 AM Yassine Oudjana > wrote: > > > > On Wednesday, July 7th, 2021 at 12:33 AM, Arnd Bergmann wrote: > > > On Tue, Jul 6, 2021 at 7:15 PM Yassine Oudjana y.oudjana@protonmail.com wrote: > > > > > > > > $ numactl -C 0 line -M 1M > > > > 128 > > > > $ numactl -C 3 line -M 1M > > > > 128 > > > > > > Can you rerun the the 'line' test with '-M 128K' to see if that confirms the 64 > > > byte L1 line size that the 'cache' test reported? > > > > $ numactl -C 0 line -M 128K > > 64 > > $ numactl -C 3 line -M 128K > > 64 > > Ok, so this seems to confirm that the L1 uses 64 byte lines, while the L2 (or > possibly L3) uses 128 byte lines. > > On Wed, Jul 7, 2021 at 12:27 AM Bjorn Andersson > wrote: > > > > I can confirm that MSM8996, and a few derivatives, has 128 byte cache lines. > > Ok, thanks. Assuming this is an outer cache and the L1 indeed has a smaller line > size, can you also confirm that this 128 byte lines are north of the point of > coherency? Finding this old documentation has been painful :) L0 I 64 byte cacheline L1 I 64 L1 D 64 L2 unified 128 (shared between the CPUs of a duplex) I believe L2 is within the POC, but I'm trying to dig up the old documentation to confirm. > > In other words, does the CTR_EL0.DminLine field also show 128 bytes > (in which case > it seems we already lost)? And if not, does a CPU store to the second half of a > 128 byte L2 line cause DMA data in the first half to be clobbered? Per the documentation I'm seeing, CTR_EL0.DminLine should show 128 bytes. I don't have hardware handy to confirm. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=1.0 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,LOTS_OF_MONEY,MAILING_LIST_MULTI,MONEY_NOHTML, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2C88C07E95 for ; Wed, 7 Jul 2021 14:42:56 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7C30161C6C for ; Wed, 7 Jul 2021 14:42:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7C30161C6C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=RnXu49dnubBZimuP5rgwF5XvfRqKPf5xNI+umtmJs1A=; b=g7GVsrlajkbGBR T3nVEj7X3WxUA5vRvw6fKqryPpyyEsIdwdHD7ngFiUuv8wFP+BzMQ9Edn9XlIv0utAw25/+OU5edq QOYxX59FK7Ck5PLPWCwHsM+awyRDcjmLWQ/1Clt42NuGbNhVZhFF87bl2NAzrSshcBXy/ElP3WnLF zNtvkzRAvI1rjUZNRMMWXtb++SRM8dzBPbdL73Ftp1mmZdwzAO5+1+g41qtexzoNKELpd3kTnjZLB 7FXPW/+aCw6esGHqe3tfRioaENQezy3lIt/YfSsYFuEYYGyvb++UMbzxDDxjBnF3ffrZFfknmEFt7 UCZ9AwKyHzs+PR4f3Q6w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m18k0-00F708-FV; Wed, 07 Jul 2021 14:41:32 +0000 Received: from mail-il1-x133.google.com ([2607:f8b0:4864:20::133]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m18jw-00F6yb-5N for linux-arm-kernel@lists.infradead.org; Wed, 07 Jul 2021 14:41:29 +0000 Received: by mail-il1-x133.google.com with SMTP id f12so2956137ils.11 for ; Wed, 07 Jul 2021 07:41:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=nIwflScCI6Icjj1cWTTKEy9uQqC1wMtDQ5W0AmkNq+g=; b=qSsUSkhUDDeNTKV9IlCEbm5Hj4XmuLwhiJaM0jY4sL1uhWUMhgpGglFL3S092uXiDj /XMSg5ef6TxisfLtL4rp/0CVFyFA3VMjGerBWdJFKyRc95JI/pwk4RNsms4ev8lTdzdT kmf/HHtLD6sZKM5ba4zPBnhrt5oBNwIzA0pOchC937ROImjrDpTL07S8cHX+Smh0XuLi Rz6rT31y3iaIrs2yvFzo21OEssbLwi6J8IMdWQ9fwWcRYahQqSl3jOXj9IRtNUDh0hev 2zfVoVZWn8ERMpvpvBudbyfGW86s/+treNOC7kHqDEWtwbMfrbkEZdsFHyesocfgw5Va /HOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=nIwflScCI6Icjj1cWTTKEy9uQqC1wMtDQ5W0AmkNq+g=; b=CzMawex47Wz2WMF4nBASE1thuL7z2hn8mBdvS1NRUQCxjbcNCJQcKoCMSnOQbELwOS pTeDQRRRxb79SeV/Fql90xa1T4HlpwHtsU0UMDtwa9YKYp6gERFsaE/m24THLcX2W97b O3lPuFt4mNGbiH+AG7IlFUWIAKkrof4v8l72mG8DjNML6hMwdNnCC7QQnpbPdZS1b8y0 /C2ZVJ0U+FoL+rV9p9gk8LnyvtezZ1huUYiERs6n9SxId2/kGmLRQk5aEOGhDSa2cIkn PV8QTWwEuBdGxN2xaiyjapIlfZmySQsUrGwYkEpo0Torn91go5B+DGMFZKfS6iaiJQky +Uxg== X-Gm-Message-State: AOAM530O/ld8XCi4EPzEERC0U+sr3X0z8bWIJB9d2W+1DhGaAQSZUIYy 6kYsaeWC3JlBtzNgT8M8qnsz0hUhb0Xr2Yz+pGE= X-Google-Smtp-Source: ABdhPJwQQMe3S+AM4KYV3Xo0/ehcLXF/J7ADlwe9sG4QkGCGpF1PS07nGxT96XXQEqLf3PdAJZC9KF8DNC5jsbg0epk= X-Received: by 2002:a05:6e02:58f:: with SMTP id c15mr18436980ils.102.1625668886115; Wed, 07 Jul 2021 07:41:26 -0700 (PDT) MIME-Version: 1.0 References: <20210527124356.22367-1-will@kernel.org> <59800d6c-364a-f4be-e341-c5b531657ba3@arm.com> <20210706133314.GB20327@willie-the-truck> <87zguz7b6b.wl-maz@kernel.org> In-Reply-To: From: Jeffrey Hugo Date: Wed, 7 Jul 2021 08:41:15 -0600 Message-ID: Subject: Re: [PATCH] arm64: cache: Lower ARCH_DMA_MINALIGN to 64 (L1_CACHE_BYTES) To: Arnd Bergmann Cc: Yassine Oudjana , Marc Zyngier , Will Deacon , Robin Murphy , Catalin Marinas , Ard Biesheuvel , Android Kernel Team , Linux ARM , Mark Rutland , Vincent Whitchurch , linux-arm-msm , Bjorn Andersson X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210707_074128_277477_D1F7EB3F X-CRM114-Status: GOOD ( 22.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Jul 7, 2021 at 3:30 AM Arnd Bergmann wrote: > > On Tue, Jul 6, 2021 at 6:20 PM Will Deacon wrote: > > > > I think the million dollar question is whether the 128-byte cache-lines > > live in a cache above the PoC or not. If it's just a system level cache > > through which all DMA is "coherent", then it doesn't matter. > > On Wed, Jul 7, 2021 at 10:24 AM Yassine Oudjana > wrote: > > > > On Wednesday, July 7th, 2021 at 12:33 AM, Arnd Bergmann wrote: > > > On Tue, Jul 6, 2021 at 7:15 PM Yassine Oudjana y.oudjana@protonmail.com wrote: > > > > > > > > $ numactl -C 0 line -M 1M > > > > 128 > > > > $ numactl -C 3 line -M 1M > > > > 128 > > > > > > Can you rerun the the 'line' test with '-M 128K' to see if that confirms the 64 > > > byte L1 line size that the 'cache' test reported? > > > > $ numactl -C 0 line -M 128K > > 64 > > $ numactl -C 3 line -M 128K > > 64 > > Ok, so this seems to confirm that the L1 uses 64 byte lines, while the L2 (or > possibly L3) uses 128 byte lines. > > On Wed, Jul 7, 2021 at 12:27 AM Bjorn Andersson > wrote: > > > > I can confirm that MSM8996, and a few derivatives, has 128 byte cache lines. > > Ok, thanks. Assuming this is an outer cache and the L1 indeed has a smaller line > size, can you also confirm that this 128 byte lines are north of the point of > coherency? Finding this old documentation has been painful :) L0 I 64 byte cacheline L1 I 64 L1 D 64 L2 unified 128 (shared between the CPUs of a duplex) I believe L2 is within the POC, but I'm trying to dig up the old documentation to confirm. > > In other words, does the CTR_EL0.DminLine field also show 128 bytes > (in which case > it seems we already lost)? And if not, does a CPU store to the second half of a > 128 byte L2 line cause DMA data in the first half to be clobbered? Per the documentation I'm seeing, CTR_EL0.DminLine should show 128 bytes. I don't have hardware handy to confirm. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel