From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 929AEC433E6 for ; Thu, 11 Mar 2021 09:13:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6DD5B64FB9 for ; Thu, 11 Mar 2021 09:13:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231937AbhCKJMq (ORCPT ); Thu, 11 Mar 2021 04:12:46 -0500 Received: from mail.kernel.org ([198.145.29.99]:46640 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231960AbhCKJMV (ORCPT ); Thu, 11 Mar 2021 04:12:21 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 6E02B64FD3; Thu, 11 Mar 2021 09:12:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615453940; bh=UNAIXrkfTNShut/I3gLeR6MyhyTRWg+W8FSgOB8eYQ4=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=DI4pa8FmJBv1+nb41cqRzZo7Hptz7QQQzvlRjcJTBIw0e/bhAXWQ7W/gwVc+jd0k9 ZhqqzIBC7e8rnatmT33kA0ARL5vzlopH0zM+MiiCEFAq49oH9QUGlrZhDDZ9HowwEy lrFjsqsuQtlzhtCZWchRd9MeM8YeCZe0xYbzjGaWZf+kyrzeZ0vM7VHhwB5ReoXTvT e0QH/bJZxd66iwjZbtaqAuumCK8Xo5edq0Kim+f5qYxrZ+QJHZefURHKQ5iK9hMsKl YzNhCe3+7KZ+eKoEl/RBKaSAA+e1J4B+4QeR+cRu3cQSFA7S2n8+W3egWk1uEsn7G1 xl69qAw6Tm39g== Received: by mail-oo1-f52.google.com with SMTP id e19-20020a4a73530000b02901b62c0e1bb6so511976oof.11; Thu, 11 Mar 2021 01:12:20 -0800 (PST) X-Gm-Message-State: AOAM530a8gom26bMc/UCqdtbMuFQLu0HhU4qH5U5jPPwOAtgQHdFOqgY cssk5aZXoZppqGwhxvF5z2j/AxQJlgHyE2imuN4= X-Google-Smtp-Source: ABdhPJygUghNxY2XSM2ghWokFwLHqZ3nZ8W75yDA/iVYNJIzeGBm7t3OFNlQOYjAFsneiZxfCa4ihGnIYWF71hoNWi4= X-Received: by 2002:a4a:8ed2:: with SMTP id c18mr5911583ool.66.1615453939509; Thu, 11 Mar 2021 01:12:19 -0800 (PST) MIME-Version: 1.0 References: <20210304213902.83903-1-marcan@marcan.st> <20210304213902.83903-13-marcan@marcan.st> <6e4880b3-1fb6-0cbf-c1a5-7a46fd9ccf62@marcan.st> <20210308211306.GA2920998@robh.at.kernel.org> <332c0b9a-dcfd-4c3b-9038-47cbda90eb3f@marcan.st> In-Reply-To: From: Arnd Bergmann Date: Thu, 11 Mar 2021 10:12:02 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [RFT PATCH v3 12/27] of/address: Add infrastructure to declare MMIO as non-posted To: Rob Herring Cc: Hector Martin , linux-arm-kernel , Marc Zyngier , Olof Johansson , Krzysztof Kozlowski , Mark Kettenis , Tony Lindgren , Mohamed Mediouni , Stan Skowronek , Alexander Graf , Will Deacon , Linus Walleij , Mark Rutland , Andy Shevchenko , Greg Kroah-Hartman , Jonathan Corbet , Catalin Marinas , Christoph Hellwig , "David S. Miller" , DTML , "open list:SERIAL DRIVERS" , Linux Doc Mailing List , linux-samsung-soc , "open list:GENERIC INCLUDE/ASM HEADER FILES" , "linux-kernel@vger.kernel.org" Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Mar 10, 2021 at 6:01 PM Rob Herring wrote: > > On Wed, Mar 10, 2021 at 1:27 AM Hector Martin wrote: > > > > On 10/03/2021 07.06, Rob Herring wrote: > > >> My main concern here is that this creates an inconsistency in the device > > >> tree representation that only works because PCI drivers happen not to > > >> use these code paths. Logically, having "nonposted-mmio" above the PCI > > >> controller would imply that it applies to that bus too. Sure, it doesn't > > >> matter for Linux since it is ignored, but this creates an implicit > > >> exception that PCI buses always use posted modes. > > > > > > We could be stricter that "nonposted-mmio" must be in the immediate > > > parent. That's kind of in line with how addressing already works. > > > Every level has to have 'ranges' to be an MMIO address, and the > > > address cell size is set by the immediate parent. > > > > > >> Then if a device comes along that due to some twisted fabric logic needs > > >> nonposted nGnRnE mappings for PCIe (even though the actual PCIe ops will > > >> end up posted at the bus anyway)... how do we represent that? Declare > > >> that another "nonposted-mmio" on the PCIe bus means "no, really, use > > >> nonposted mmio for this"? > > > > > > If we're strict, yes. The PCI host bridge would have to have "nonposted-mmio". > > > > Works for me; then let's just make it non-recursive. > > > > Do you think we can get rid of the Apple-only optimization if we do > > this? It would mean only looking at the parent during address > > resolution, not recursing all the way to the top, so presumably the > > performance impact would be quite minimal. Works for me. > Yeah, that should be fine. I'd keep an IS_ENABLED() config check > though. Then I'll also know if anyone else needs this. Ok, makes sense. Conceptually, I'd like to then see a check that verifies that the property is only set for nodes whose parent also has it set, since that is how AXI defines it: A bus can wait for the ack from its child node, or it can acknowledge the write to its parent early. However, this breaks down as soon as a bus does the early ack: all its children by definition use posted writes (as seen by the CPU), even if they wait for stores that come from other masters. Does this make sense to you? Arnd From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3807C433DB for ; Thu, 11 Mar 2021 09:13:52 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 043BC64E5F for ; Thu, 11 Mar 2021 09:13:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 043BC64E5F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=5I11e3UZVoQrfmNoQ9UaTsJXev4sWlwqBM+MXAcg4GQ=; b=NvemiuKy/P12EoU54MJBqbRVn DzLiidYBIst7P6qi/wda+HpRsnQNJkT1Wf3+8kfwftFEgzprsRqQ3IhQyfir6Al6MyYSyfzfrtZV8 0lVmDSDnhJVcGABksjsJF4pP7jveg0kimzEZAJoUPCK5rdK7gQzjSP9DPKD2w5iQC8kP0bCze0g2m /u2K33JGwu6BxpXw2rMV//uklJHwpLkpjGxhHBVCu/h4Iw4wDNb56bnKqGWoB+DNm5ASYJqxZoQUn YQTyINLGLfCOBCEq3BaA26wHLwZ5z7+qBysXzxWDaeVk4Q4lc9J56pcehop1zv8QAR8DhsGQRv1E8 sVNWWiD5g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKHMp-008ivM-5T; Thu, 11 Mar 2021 09:12:27 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lKHMk-008iuZ-0j for linux-arm-kernel@lists.infradead.org; Thu, 11 Mar 2021 09:12:24 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 627B564FB0 for ; Thu, 11 Mar 2021 09:12:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615453940; bh=UNAIXrkfTNShut/I3gLeR6MyhyTRWg+W8FSgOB8eYQ4=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=DI4pa8FmJBv1+nb41cqRzZo7Hptz7QQQzvlRjcJTBIw0e/bhAXWQ7W/gwVc+jd0k9 ZhqqzIBC7e8rnatmT33kA0ARL5vzlopH0zM+MiiCEFAq49oH9QUGlrZhDDZ9HowwEy lrFjsqsuQtlzhtCZWchRd9MeM8YeCZe0xYbzjGaWZf+kyrzeZ0vM7VHhwB5ReoXTvT e0QH/bJZxd66iwjZbtaqAuumCK8Xo5edq0Kim+f5qYxrZ+QJHZefURHKQ5iK9hMsKl YzNhCe3+7KZ+eKoEl/RBKaSAA+e1J4B+4QeR+cRu3cQSFA7S2n8+W3egWk1uEsn7G1 xl69qAw6Tm39g== Received: by mail-oo1-f45.google.com with SMTP id c13-20020a4ab18d0000b02901b5b67cff5eso511911ooo.2 for ; Thu, 11 Mar 2021 01:12:20 -0800 (PST) X-Gm-Message-State: AOAM531BfjRrmYrMtUiigxCaMhfrlk8yXTnRHWOLDeobXM1wt8QHWG2G XOnSZQslgsqpDjNKuPWr59m71DWbTRlJW5peSFo= X-Google-Smtp-Source: ABdhPJygUghNxY2XSM2ghWokFwLHqZ3nZ8W75yDA/iVYNJIzeGBm7t3OFNlQOYjAFsneiZxfCa4ihGnIYWF71hoNWi4= X-Received: by 2002:a4a:8ed2:: with SMTP id c18mr5911583ool.66.1615453939509; Thu, 11 Mar 2021 01:12:19 -0800 (PST) MIME-Version: 1.0 References: <20210304213902.83903-1-marcan@marcan.st> <20210304213902.83903-13-marcan@marcan.st> <6e4880b3-1fb6-0cbf-c1a5-7a46fd9ccf62@marcan.st> <20210308211306.GA2920998@robh.at.kernel.org> <332c0b9a-dcfd-4c3b-9038-47cbda90eb3f@marcan.st> In-Reply-To: From: Arnd Bergmann Date: Thu, 11 Mar 2021 10:12:02 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [RFT PATCH v3 12/27] of/address: Add infrastructure to declare MMIO as non-posted To: Rob Herring Cc: Hector Martin , linux-arm-kernel , Marc Zyngier , Olof Johansson , Krzysztof Kozlowski , Mark Kettenis , Tony Lindgren , Mohamed Mediouni , Stan Skowronek , Alexander Graf , Will Deacon , Linus Walleij , Mark Rutland , Andy Shevchenko , Greg Kroah-Hartman , Jonathan Corbet , Catalin Marinas , Christoph Hellwig , "David S. Miller" , DTML , "open list:SERIAL DRIVERS" , Linux Doc Mailing List , linux-samsung-soc , "open list:GENERIC INCLUDE/ASM HEADER FILES" , "linux-kernel@vger.kernel.org" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210311_091222_538016_A71A76CB X-CRM114-Status: GOOD ( 32.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Mar 10, 2021 at 6:01 PM Rob Herring wrote: > > On Wed, Mar 10, 2021 at 1:27 AM Hector Martin wrote: > > > > On 10/03/2021 07.06, Rob Herring wrote: > > >> My main concern here is that this creates an inconsistency in the device > > >> tree representation that only works because PCI drivers happen not to > > >> use these code paths. Logically, having "nonposted-mmio" above the PCI > > >> controller would imply that it applies to that bus too. Sure, it doesn't > > >> matter for Linux since it is ignored, but this creates an implicit > > >> exception that PCI buses always use posted modes. > > > > > > We could be stricter that "nonposted-mmio" must be in the immediate > > > parent. That's kind of in line with how addressing already works. > > > Every level has to have 'ranges' to be an MMIO address, and the > > > address cell size is set by the immediate parent. > > > > > >> Then if a device comes along that due to some twisted fabric logic needs > > >> nonposted nGnRnE mappings for PCIe (even though the actual PCIe ops will > > >> end up posted at the bus anyway)... how do we represent that? Declare > > >> that another "nonposted-mmio" on the PCIe bus means "no, really, use > > >> nonposted mmio for this"? > > > > > > If we're strict, yes. The PCI host bridge would have to have "nonposted-mmio". > > > > Works for me; then let's just make it non-recursive. > > > > Do you think we can get rid of the Apple-only optimization if we do > > this? It would mean only looking at the parent during address > > resolution, not recursing all the way to the top, so presumably the > > performance impact would be quite minimal. Works for me. > Yeah, that should be fine. I'd keep an IS_ENABLED() config check > though. Then I'll also know if anyone else needs this. Ok, makes sense. Conceptually, I'd like to then see a check that verifies that the property is only set for nodes whose parent also has it set, since that is how AXI defines it: A bus can wait for the ack from its child node, or it can acknowledge the write to its parent early. However, this breaks down as soon as a bus does the early ack: all its children by definition use posted writes (as seen by the CPU), even if they wait for stores that come from other masters. Does this make sense to you? Arnd _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel