From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5335C43603 for ; Fri, 5 Mar 2021 15:20:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7326F65092 for ; Fri, 5 Mar 2021 15:20:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230270AbhCEPUK (ORCPT ); Fri, 5 Mar 2021 10:20:10 -0500 Received: from marcansoft.com ([212.63.210.85]:58814 "EHLO mail.marcansoft.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229650AbhCEPTp (ORCPT ); Fri, 5 Mar 2021 10:19:45 -0500 Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: marcan@marcan.st) by mail.marcansoft.com (Postfix) with ESMTPSA id C160E3FA28; Fri, 5 Mar 2021 15:19:36 +0000 (UTC) To: Andy Shevchenko Cc: linux-arm Mailing List , Marc Zyngier , Rob Herring , Arnd Bergmann , Olof Johansson , Krzysztof Kozlowski , Mark Kettenis , Tony Lindgren , Mohamed Mediouni , Stan Skowronek , Alexander Graf , Will Deacon , Linus Walleij , Mark Rutland , Greg Kroah-Hartman , Jonathan Corbet , Catalin Marinas , Christoph Hellwig , "David S. Miller" , devicetree , "open list:SERIAL DRIVERS" , Linux Documentation List , Linux Samsung SOC , Linux-Arch , Linux Kernel Mailing List References: <20210304213902.83903-1-marcan@marcan.st> <20210304213902.83903-9-marcan@marcan.st> From: Hector Martin Subject: Re: [RFT PATCH v3 08/27] asm-generic/io.h: Add a non-posted variant of ioremap() Message-ID: <03f75e1f-3b3a-95a7-0298-c616dfed54ec@marcan.st> Date: Sat, 6 Mar 2021 00:19:34 +0900 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: es-ES Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/03/2021 23.45, Andy Shevchenko wrote: > On Thu, Mar 4, 2021 at 11:40 PM Hector Martin wrote: >> >> ARM64 currently defaults to posted MMIO (nGnRnE), but some devices >> require the use of non-posted MMIO (nGnRE). Introduce a new ioremap() >> variant to handle this case. ioremap_np() is aliased to ioremap() by >> default on arches that do not implement this variant. > > Hmm... But isn't it basically a requirement to those device drivers to > use readX()/writeX() instead of readX_relaxed() / writeX_relaxed()? No, the write ops used do not matter. It's just that on these Apple SoCs the fabric requires the mappings to be nGnRnE, else it just throws SErrors on all writes and ignores them. The difference between _relaxed and not is barrier behavior with regards to DMA/memory accesses; this applies regardless of whether the writes are E or nE. You can have relaxed accesses with nGnRnE and then you would still have race conditions if you do not have a barrier between the MMIO and accessing DMA memory. What nGnRnE buys you (on platforms/buses where it works properly) is that you do need a dummy read after a write to ensure completion. All of this is to some extent moot on these SoCs; it's not that we need the drivers to use nGnRnE for some correctness reason, it's that the SoCs force us to use it or else everything breaks, which was the motivation for this change. But since on most other SoCs both are valid options, this does allow some other drivers/platforms to opt into nGnRnE if they have a good reason to do so. Though you just made me notice two mistakes in the commit description: first, it describes the old v2 version, for v3 I made ioremap_np() just return NULL on arches that don't implement it. Second, nGnRnE and nGnRE are backwards. Oops. I'll fix it for the next version. >> #define IORESOURCE_MEM_32BIT (3<<3) >> #define IORESOURCE_MEM_SHADOWABLE (1<<5) /* dup: IORESOURCE_SHADOWABLE */ >> #define IORESOURCE_MEM_EXPANSIONROM (1<<6) >> +#define IORESOURCE_MEM_NONPOSTED (1<<7) > > Not sure it's the right location (in a bit field) for this flag. Do you have a better suggestion? It seemed logical to put it here, as a flag on memory-type I/O resources. -- Hector Martin (marcan@marcan.st) Public Key: https://mrcn.st/pub From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6D62C433DB for ; Fri, 5 Mar 2021 15:24:25 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 35CEE64F73 for ; Fri, 5 Mar 2021 15:24:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 35CEE64F73 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=marcan.st Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:Subject: From:References:Cc:To:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=6sMa0S+GhGNVKTBel21lrnVssCz4OUXcZq3eYmaIHNg=; b=OVCpBKD7inyu9ZUkYkSQRxHs2 hBH8Tvlg3kBtgj7LUEuZ0uREmadEGyd7SD4W/4qTlmKEk96zvA+E7a2UtCtA4rlGn7x2gaUQSyYXG kyGpaZUo65IApeea1XiyQWjBdQY1GYprNvwhUgrpy8sIY5kC4bM/sqg3CH5MMnOZ8XwCrWRcMmnFg QPIHC6CPG3MpPjwf3McDY8AOirfYndLowumaFB6cD9FAA6mk9qY29doOsjK49Ml30OK2C4QT+Aqpi XI2gdxe/M5ZJeDAI7aO50MpbviWKTvzE3o03OS0yVYqthP7PDi4kRJJwyWPC5rV+JOQzA1hwxhGXt hGBBGq3CQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lICGi-00FQEK-9d; Fri, 05 Mar 2021 15:21:32 +0000 Received: from marcansoft.com ([2a01:298:fe:f::2] helo=mail.marcansoft.com) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lICEy-00FPzI-SR for linux-arm-kernel@lists.infradead.org; Fri, 05 Mar 2021 15:20:03 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: marcan@marcan.st) by mail.marcansoft.com (Postfix) with ESMTPSA id C160E3FA28; Fri, 5 Mar 2021 15:19:36 +0000 (UTC) To: Andy Shevchenko Cc: linux-arm Mailing List , Marc Zyngier , Rob Herring , Arnd Bergmann , Olof Johansson , Krzysztof Kozlowski , Mark Kettenis , Tony Lindgren , Mohamed Mediouni , Stan Skowronek , Alexander Graf , Will Deacon , Linus Walleij , Mark Rutland , Greg Kroah-Hartman , Jonathan Corbet , Catalin Marinas , Christoph Hellwig , "David S. Miller" , devicetree , "open list:SERIAL DRIVERS" , Linux Documentation List , Linux Samsung SOC , Linux-Arch , Linux Kernel Mailing List References: <20210304213902.83903-1-marcan@marcan.st> <20210304213902.83903-9-marcan@marcan.st> From: Hector Martin Subject: Re: [RFT PATCH v3 08/27] asm-generic/io.h: Add a non-posted variant of ioremap() Message-ID: <03f75e1f-3b3a-95a7-0298-c616dfed54ec@marcan.st> Date: Sat, 6 Mar 2021 00:19:34 +0900 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: Content-Language: es-ES X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210305_151951_063719_E7671158 X-CRM114-Status: GOOD ( 23.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 05/03/2021 23.45, Andy Shevchenko wrote: > On Thu, Mar 4, 2021 at 11:40 PM Hector Martin wrote: >> >> ARM64 currently defaults to posted MMIO (nGnRnE), but some devices >> require the use of non-posted MMIO (nGnRE). Introduce a new ioremap() >> variant to handle this case. ioremap_np() is aliased to ioremap() by >> default on arches that do not implement this variant. > > Hmm... But isn't it basically a requirement to those device drivers to > use readX()/writeX() instead of readX_relaxed() / writeX_relaxed()? No, the write ops used do not matter. It's just that on these Apple SoCs the fabric requires the mappings to be nGnRnE, else it just throws SErrors on all writes and ignores them. The difference between _relaxed and not is barrier behavior with regards to DMA/memory accesses; this applies regardless of whether the writes are E or nE. You can have relaxed accesses with nGnRnE and then you would still have race conditions if you do not have a barrier between the MMIO and accessing DMA memory. What nGnRnE buys you (on platforms/buses where it works properly) is that you do need a dummy read after a write to ensure completion. All of this is to some extent moot on these SoCs; it's not that we need the drivers to use nGnRnE for some correctness reason, it's that the SoCs force us to use it or else everything breaks, which was the motivation for this change. But since on most other SoCs both are valid options, this does allow some other drivers/platforms to opt into nGnRnE if they have a good reason to do so. Though you just made me notice two mistakes in the commit description: first, it describes the old v2 version, for v3 I made ioremap_np() just return NULL on arches that don't implement it. Second, nGnRnE and nGnRE are backwards. Oops. I'll fix it for the next version. >> #define IORESOURCE_MEM_32BIT (3<<3) >> #define IORESOURCE_MEM_SHADOWABLE (1<<5) /* dup: IORESOURCE_SHADOWABLE */ >> #define IORESOURCE_MEM_EXPANSIONROM (1<<6) >> +#define IORESOURCE_MEM_NONPOSTED (1<<7) > > Not sure it's the right location (in a bit field) for this flag. Do you have a better suggestion? It seemed logical to put it here, as a flag on memory-type I/O resources. -- Hector Martin (marcan@marcan.st) Public Key: https://mrcn.st/pub _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel