From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B6A49C636CC for ; Thu, 16 Feb 2023 21:53:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: In-Reply-To:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date: Reply-To:Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date :Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=tXh+5tlBKXvDIaW4v7Po3WPZOVRgdD3RonXr0l0eGZ8=; b=PS49UjDLSzms2xsNv0tfy+FRCs Irgv//+IEUOo4Hly/4SC2Ta2NB4HaxPxb4VCH1J9XiFM8ssqBvwMkcEkyNzKFL79uyczJC0nME4Bc LDk3QXVskM/FqXzwwBadvmcBmUkDqKdJP/0ULPDUplzzx8AZ1Dc6VSEj9bxsaLiY2e2Y3SynDcQA9 C1gAoozMhCRNQSScUUHnkKwE6B4eg06oQncDxQGNFH40pjYR/7SWSdmA6kitjmQoMUC/NzLojlXpd oiEsqxoz7CRTdthF0cvpMA4q6vP5UVNvV/8AB6ZB9QxHflhwYHLNVcH1f3xPKodHJK5ThoCDQ8iMg e2YmZGbw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pSmC4-00BxEL-E5; Thu, 16 Feb 2023 21:53:32 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pSmBr-00BxC3-O7; Thu, 16 Feb 2023 21:53:21 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4A8BB60C1D; Thu, 16 Feb 2023 21:53:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7EEA3C4339E; Thu, 16 Feb 2023 21:53:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676584390; bh=/89NA4NAoXcXzdPfjHUemplkBWp+8gbW+gPen4ZFpgo=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=Xx7q4GXXB4Ze6YTbHhjjCXHrGwNp71BIJV5GuJj4gBwBiwzTg6nkKStAHxNTmYAcy qbDmYTSljpicwjrAbhw5o/ypDDD0tgcN26EfyZPgeM1yI1U7lOOg7oHb35651sjpBe PZ3IeInK+WNQUl00mwuiCFvStJUKoaCwq6saPl6xj8Aa3Se3DH/+s8LiyHyAFtqu+H dsRq+jSUQU7Ns+eRE5UkrZotfNyo1vvj0nuOX633djBUYop/B4g5vkZjFroJsa5C8D ptZXCA31Et8eKsHz+ywE8GbM+0R51PxwoNZwLz68UUZrQNR0n+X07KI3BbaeeNo1gW mupdA/vmOkC8w== Date: Thu, 16 Feb 2023 21:53:02 +0000 From: Conor Dooley To: Cristian Ciocaltea , Emil Renner Berthing , daire.mcnamara@microchip.com Cc: Lee Jones , Rob Herring , Krzysztof Kozlowski , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Emil Renner Berthing , Palmer Dabbelt , Paul Walmsley , Albert Ou , Giuseppe Cavallaro , Alexandre Torgue , Jose Abreu , Maxime Coquelin , Richard Cochran , Sagar Kadam , Yanhong Wang , devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, linux-riscv@lists.infradead.org, linux-stm32@st-md-mailman.stormreply.com, linux-arm-kernel@lists.infradead.org, kernel@collabora.com Subject: Re: [PATCH 02/12] dt-bindings: riscv: sifive-ccache: Add 'uncached-offset' property Message-ID: References: <20230211031821.976408-1-cristian.ciocaltea@collabora.com> <20230211031821.976408-3-cristian.ciocaltea@collabora.com> MIME-Version: 1.0 In-Reply-To: <20230211031821.976408-3-cristian.ciocaltea@collabora.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230216_135319_909577_787EFB23 X-CRM114-Status: GOOD ( 29.68 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============4144466286773729091==" Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org --===============4144466286773729091== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="+OUnDp7k4pWMwsmV" Content-Disposition: inline --+OUnDp7k4pWMwsmV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hey all, On Sat, Feb 11, 2023 at 05:18:11AM +0200, Cristian Ciocaltea wrote: > Add the 'uncached-offset' property to be used for specifying the > uncached memory offset required for handling non-coherent DMA > transactions. >=20 > Signed-off-by: Cristian Ciocaltea > --- > Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml | 5 +++++ > 1 file changed, 5 insertions(+) >=20 > diff --git a/Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml = b/Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml > index 2b864b2f12c9..60cd87a2810a 100644 > --- a/Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml > +++ b/Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml > @@ -82,6 +82,11 @@ properties: > =20 > next-level-cache: true > =20 > + uncached-offset: > + $ref: /schemas/types.yaml#/definitions/uint64 > + description: | > + Uncached memory offset for handling non-coherent DMA transactions. Firstly, this pretty tied to the StarFive stuff, where there is only one "bank" of memory that neatly maps to one bank of non-cached memory. On PolarFire SoC, where we would also like to make use of non-coherent DMA for some transfers using the FPGA fabric, things are a bit more complex. Instead of a region & a non-cached alias, we have 2 regions and 2 non-cached regions. These regions lie at 0x8000_0000 & 0x10_0000_0000 and the non-cached regions are at 0xc000_0000 & 0x14_0000_0000. As you can tell, one fixed offset isn't going to work there! The other bit of a problem is that there is no fixed concept of aliases, as seems to be the case on the jh7100. Instead, where the regions "point" to in physical DDR is something that is configurable at runtime. Practically speaking, it is set by firmware very early on in boot & is fixed from there out, but will vary between boards and FPGA fabric configuration. Effectively that means that from the PoV of a Devicetree it is constant, but a good bit of flexibility is going to be needed. What we have been doing on PolarFire SoC (although mostly internally at this point) is, rather than creating a property like uncached-offset, we instead are using the dma-ranges properties to induce the same affect. In an example configuration with memory at: reg =3D <0x0 0x80000000 0x0 0x4000000>; reg =3D <0x0 0x8a000000 0x0 0x8000000>; reg =3D <0x0 0xc4000000 0x0 0x6000000>; reg =3D <0x10 0x22000000 0x0 0x5e000000>; reg =3D <0x14 0x12000000 0x0 0x10000000>; a reserved memory section then covering the non-cached region at 0x14_0000_0000: dma_non_cached_high: non-cached-high-buffer { compatible =3D "shared-dma-pool"; size =3D <0x0 0x10000000>; no-map; linux,dma-default; alloc-ranges =3D <0x14 0x12000000 0x0 0x10000000>; }; and dma-ranges: dma-ranges =3D <0x14 0x0 0x0 0x80000000 0x0 0x4000000>, <0x14 0x4000000 0x0 0xc4000000 0x0 0x6000000>, <0x14 0xa000000 0x0 0x8a000000 0x0 0x8000000>, In this configuration, 0x8000_0000, 0x10_0000_0000, 0xc000_0000 & 0x14_0000_0000 are all aliases of the same address. With this setup, we're able to do non-coherent DMA to the FPGA fabric, to the PCI for example. The DTS does grow a bit of complexity, with reserved memory regions and dma-ranges - but at least they're standard properties! Emil, if you want to take a look at that it is here: https://github.com/linux4microchip/linux linux-5.15-mchp I think I said to you before that it was based on one of Atish's early approaches, the one from the 5.15 development cycle IIRC since we're using that LTS. Obviously it'll need changes to be upstreamable so we're not wedded to this approach. For instance, it's being controlled by a compile time option at the moment, so that clearly needs to become runtime for upstream (and realistically needs to be one in our vendor tree too...) I'll try to hack that approach into the visionfive v1 soonTM and see how it goes, but it'll not be this side of March before I have time to do that. Cheers, Conor. --+OUnDp7k4pWMwsmV Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYIAB0WIQRh246EGq/8RLhDjO14tDGHoIJi0gUCY+6lvgAKCRB4tDGHoIJi 0ulqAQCULkPATgqWtudHs9arilghmWnIl+5HOdnui0TiEobVIwD9EJWZRVVAGTBQ VrKlrkCPLqc12a+YcCC0aBjkaR6SDwI= =ZTRK -----END PGP SIGNATURE----- --+OUnDp7k4pWMwsmV-- --===============4144466286773729091== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv --===============4144466286773729091==-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC072C61DA4 for ; Thu, 16 Feb 2023 21:54:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230225AbjBPVys (ORCPT ); Thu, 16 Feb 2023 16:54:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44160 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229947AbjBPVyp (ORCPT ); Thu, 16 Feb 2023 16:54:45 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55D119772; Thu, 16 Feb 2023 13:54:12 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id F07CBB829C0; Thu, 16 Feb 2023 21:53:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7EEA3C4339E; Thu, 16 Feb 2023 21:53:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676584390; bh=/89NA4NAoXcXzdPfjHUemplkBWp+8gbW+gPen4ZFpgo=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=Xx7q4GXXB4Ze6YTbHhjjCXHrGwNp71BIJV5GuJj4gBwBiwzTg6nkKStAHxNTmYAcy qbDmYTSljpicwjrAbhw5o/ypDDD0tgcN26EfyZPgeM1yI1U7lOOg7oHb35651sjpBe PZ3IeInK+WNQUl00mwuiCFvStJUKoaCwq6saPl6xj8Aa3Se3DH/+s8LiyHyAFtqu+H dsRq+jSUQU7Ns+eRE5UkrZotfNyo1vvj0nuOX633djBUYop/B4g5vkZjFroJsa5C8D ptZXCA31Et8eKsHz+ywE8GbM+0R51PxwoNZwLz68UUZrQNR0n+X07KI3BbaeeNo1gW mupdA/vmOkC8w== Date: Thu, 16 Feb 2023 21:53:02 +0000 From: Conor Dooley To: Cristian Ciocaltea , Emil Renner Berthing , daire.mcnamara@microchip.com Cc: Lee Jones , Rob Herring , Krzysztof Kozlowski , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Emil Renner Berthing , Palmer Dabbelt , Paul Walmsley , Albert Ou , Giuseppe Cavallaro , Alexandre Torgue , Jose Abreu , Maxime Coquelin , Richard Cochran , Sagar Kadam , Yanhong Wang , devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, linux-riscv@lists.infradead.org, linux-stm32@st-md-mailman.stormreply.com, linux-arm-kernel@lists.infradead.org, kernel@collabora.com Subject: Re: [PATCH 02/12] dt-bindings: riscv: sifive-ccache: Add 'uncached-offset' property Message-ID: References: <20230211031821.976408-1-cristian.ciocaltea@collabora.com> <20230211031821.976408-3-cristian.ciocaltea@collabora.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="+OUnDp7k4pWMwsmV" Content-Disposition: inline In-Reply-To: <20230211031821.976408-3-cristian.ciocaltea@collabora.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org --+OUnDp7k4pWMwsmV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hey all, On Sat, Feb 11, 2023 at 05:18:11AM +0200, Cristian Ciocaltea wrote: > Add the 'uncached-offset' property to be used for specifying the > uncached memory offset required for handling non-coherent DMA > transactions. >=20 > Signed-off-by: Cristian Ciocaltea > --- > Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml | 5 +++++ > 1 file changed, 5 insertions(+) >=20 > diff --git a/Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml = b/Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml > index 2b864b2f12c9..60cd87a2810a 100644 > --- a/Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml > +++ b/Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml > @@ -82,6 +82,11 @@ properties: > =20 > next-level-cache: true > =20 > + uncached-offset: > + $ref: /schemas/types.yaml#/definitions/uint64 > + description: | > + Uncached memory offset for handling non-coherent DMA transactions. Firstly, this pretty tied to the StarFive stuff, where there is only one "bank" of memory that neatly maps to one bank of non-cached memory. On PolarFire SoC, where we would also like to make use of non-coherent DMA for some transfers using the FPGA fabric, things are a bit more complex. Instead of a region & a non-cached alias, we have 2 regions and 2 non-cached regions. These regions lie at 0x8000_0000 & 0x10_0000_0000 and the non-cached regions are at 0xc000_0000 & 0x14_0000_0000. As you can tell, one fixed offset isn't going to work there! The other bit of a problem is that there is no fixed concept of aliases, as seems to be the case on the jh7100. Instead, where the regions "point" to in physical DDR is something that is configurable at runtime. Practically speaking, it is set by firmware very early on in boot & is fixed from there out, but will vary between boards and FPGA fabric configuration. Effectively that means that from the PoV of a Devicetree it is constant, but a good bit of flexibility is going to be needed. What we have been doing on PolarFire SoC (although mostly internally at this point) is, rather than creating a property like uncached-offset, we instead are using the dma-ranges properties to induce the same affect. In an example configuration with memory at: reg =3D <0x0 0x80000000 0x0 0x4000000>; reg =3D <0x0 0x8a000000 0x0 0x8000000>; reg =3D <0x0 0xc4000000 0x0 0x6000000>; reg =3D <0x10 0x22000000 0x0 0x5e000000>; reg =3D <0x14 0x12000000 0x0 0x10000000>; a reserved memory section then covering the non-cached region at 0x14_0000_0000: dma_non_cached_high: non-cached-high-buffer { compatible =3D "shared-dma-pool"; size =3D <0x0 0x10000000>; no-map; linux,dma-default; alloc-ranges =3D <0x14 0x12000000 0x0 0x10000000>; }; and dma-ranges: dma-ranges =3D <0x14 0x0 0x0 0x80000000 0x0 0x4000000>, <0x14 0x4000000 0x0 0xc4000000 0x0 0x6000000>, <0x14 0xa000000 0x0 0x8a000000 0x0 0x8000000>, In this configuration, 0x8000_0000, 0x10_0000_0000, 0xc000_0000 & 0x14_0000_0000 are all aliases of the same address. With this setup, we're able to do non-coherent DMA to the FPGA fabric, to the PCI for example. The DTS does grow a bit of complexity, with reserved memory regions and dma-ranges - but at least they're standard properties! Emil, if you want to take a look at that it is here: https://github.com/linux4microchip/linux linux-5.15-mchp I think I said to you before that it was based on one of Atish's early approaches, the one from the 5.15 development cycle IIRC since we're using that LTS. Obviously it'll need changes to be upstreamable so we're not wedded to this approach. For instance, it's being controlled by a compile time option at the moment, so that clearly needs to become runtime for upstream (and realistically needs to be one in our vendor tree too...) I'll try to hack that approach into the visionfive v1 soonTM and see how it goes, but it'll not be this side of March before I have time to do that. Cheers, Conor. --+OUnDp7k4pWMwsmV Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYIAB0WIQRh246EGq/8RLhDjO14tDGHoIJi0gUCY+6lvgAKCRB4tDGHoIJi 0ulqAQCULkPATgqWtudHs9arilghmWnIl+5HOdnui0TiEobVIwD9EJWZRVVAGTBQ VrKlrkCPLqc12a+YcCC0aBjkaR6SDwI= =ZTRK -----END PGP SIGNATURE----- --+OUnDp7k4pWMwsmV-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C03DCC636CC for ; Thu, 16 Feb 2023 21:54:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: In-Reply-To:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date: Reply-To:Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date :Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=HWkljLb/NxJa/7Wd96QJ0MgiAg6bJIgIN2Ca6Tx4o9Y=; b=hvCzlMfZSiZrMkADKsub5hpoZe 8xs7TS1/KLlEVddeJ3kZEGcsnX1a/8hTL1C5gV4fK352psseNHVQ69/N9B43lCsIz9NOh368fxB3Y gbdH3uggBDEWxVaj5GSWjbnK0YDkxAtu+TJLeNtunoac5TqZZh1CXwfqWOqQsxWQB8578NMIX2kIZ 6B/dcRWS7KdI5SGDSMNaS0zBao3zGNNCe1EUzJp/VDXZRBp/xgjeUElejErL33gNkGkwZcQtwnP3v LtfAi5pCufAtGO35iv7gUHiJwKXue4G6+eCRj1IA1BF2s5gP5XrGF87rv3lhd3bNmWYZzOc343uK8 DGe4s6kQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pSmBx-00BxDb-ET; Thu, 16 Feb 2023 21:53:25 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pSmBr-00BxC3-O7; Thu, 16 Feb 2023 21:53:21 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4A8BB60C1D; Thu, 16 Feb 2023 21:53:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7EEA3C4339E; Thu, 16 Feb 2023 21:53:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676584390; bh=/89NA4NAoXcXzdPfjHUemplkBWp+8gbW+gPen4ZFpgo=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=Xx7q4GXXB4Ze6YTbHhjjCXHrGwNp71BIJV5GuJj4gBwBiwzTg6nkKStAHxNTmYAcy qbDmYTSljpicwjrAbhw5o/ypDDD0tgcN26EfyZPgeM1yI1U7lOOg7oHb35651sjpBe PZ3IeInK+WNQUl00mwuiCFvStJUKoaCwq6saPl6xj8Aa3Se3DH/+s8LiyHyAFtqu+H dsRq+jSUQU7Ns+eRE5UkrZotfNyo1vvj0nuOX633djBUYop/B4g5vkZjFroJsa5C8D ptZXCA31Et8eKsHz+ywE8GbM+0R51PxwoNZwLz68UUZrQNR0n+X07KI3BbaeeNo1gW mupdA/vmOkC8w== Date: Thu, 16 Feb 2023 21:53:02 +0000 From: Conor Dooley To: Cristian Ciocaltea , Emil Renner Berthing , daire.mcnamara@microchip.com Cc: Lee Jones , Rob Herring , Krzysztof Kozlowski , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Emil Renner Berthing , Palmer Dabbelt , Paul Walmsley , Albert Ou , Giuseppe Cavallaro , Alexandre Torgue , Jose Abreu , Maxime Coquelin , Richard Cochran , Sagar Kadam , Yanhong Wang , devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, linux-riscv@lists.infradead.org, linux-stm32@st-md-mailman.stormreply.com, linux-arm-kernel@lists.infradead.org, kernel@collabora.com Subject: Re: [PATCH 02/12] dt-bindings: riscv: sifive-ccache: Add 'uncached-offset' property Message-ID: References: <20230211031821.976408-1-cristian.ciocaltea@collabora.com> <20230211031821.976408-3-cristian.ciocaltea@collabora.com> MIME-Version: 1.0 In-Reply-To: <20230211031821.976408-3-cristian.ciocaltea@collabora.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230216_135319_909577_787EFB23 X-CRM114-Status: GOOD ( 29.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============4748043798155223410==" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org --===============4748043798155223410== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="+OUnDp7k4pWMwsmV" Content-Disposition: inline --+OUnDp7k4pWMwsmV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hey all, On Sat, Feb 11, 2023 at 05:18:11AM +0200, Cristian Ciocaltea wrote: > Add the 'uncached-offset' property to be used for specifying the > uncached memory offset required for handling non-coherent DMA > transactions. >=20 > Signed-off-by: Cristian Ciocaltea > --- > Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml | 5 +++++ > 1 file changed, 5 insertions(+) >=20 > diff --git a/Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml = b/Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml > index 2b864b2f12c9..60cd87a2810a 100644 > --- a/Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml > +++ b/Documentation/devicetree/bindings/riscv/sifive,ccache0.yaml > @@ -82,6 +82,11 @@ properties: > =20 > next-level-cache: true > =20 > + uncached-offset: > + $ref: /schemas/types.yaml#/definitions/uint64 > + description: | > + Uncached memory offset for handling non-coherent DMA transactions. Firstly, this pretty tied to the StarFive stuff, where there is only one "bank" of memory that neatly maps to one bank of non-cached memory. On PolarFire SoC, where we would also like to make use of non-coherent DMA for some transfers using the FPGA fabric, things are a bit more complex. Instead of a region & a non-cached alias, we have 2 regions and 2 non-cached regions. These regions lie at 0x8000_0000 & 0x10_0000_0000 and the non-cached regions are at 0xc000_0000 & 0x14_0000_0000. As you can tell, one fixed offset isn't going to work there! The other bit of a problem is that there is no fixed concept of aliases, as seems to be the case on the jh7100. Instead, where the regions "point" to in physical DDR is something that is configurable at runtime. Practically speaking, it is set by firmware very early on in boot & is fixed from there out, but will vary between boards and FPGA fabric configuration. Effectively that means that from the PoV of a Devicetree it is constant, but a good bit of flexibility is going to be needed. What we have been doing on PolarFire SoC (although mostly internally at this point) is, rather than creating a property like uncached-offset, we instead are using the dma-ranges properties to induce the same affect. In an example configuration with memory at: reg =3D <0x0 0x80000000 0x0 0x4000000>; reg =3D <0x0 0x8a000000 0x0 0x8000000>; reg =3D <0x0 0xc4000000 0x0 0x6000000>; reg =3D <0x10 0x22000000 0x0 0x5e000000>; reg =3D <0x14 0x12000000 0x0 0x10000000>; a reserved memory section then covering the non-cached region at 0x14_0000_0000: dma_non_cached_high: non-cached-high-buffer { compatible =3D "shared-dma-pool"; size =3D <0x0 0x10000000>; no-map; linux,dma-default; alloc-ranges =3D <0x14 0x12000000 0x0 0x10000000>; }; and dma-ranges: dma-ranges =3D <0x14 0x0 0x0 0x80000000 0x0 0x4000000>, <0x14 0x4000000 0x0 0xc4000000 0x0 0x6000000>, <0x14 0xa000000 0x0 0x8a000000 0x0 0x8000000>, In this configuration, 0x8000_0000, 0x10_0000_0000, 0xc000_0000 & 0x14_0000_0000 are all aliases of the same address. With this setup, we're able to do non-coherent DMA to the FPGA fabric, to the PCI for example. The DTS does grow a bit of complexity, with reserved memory regions and dma-ranges - but at least they're standard properties! Emil, if you want to take a look at that it is here: https://github.com/linux4microchip/linux linux-5.15-mchp I think I said to you before that it was based on one of Atish's early approaches, the one from the 5.15 development cycle IIRC since we're using that LTS. Obviously it'll need changes to be upstreamable so we're not wedded to this approach. For instance, it's being controlled by a compile time option at the moment, so that clearly needs to become runtime for upstream (and realistically needs to be one in our vendor tree too...) I'll try to hack that approach into the visionfive v1 soonTM and see how it goes, but it'll not be this side of March before I have time to do that. Cheers, Conor. --+OUnDp7k4pWMwsmV Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYIAB0WIQRh246EGq/8RLhDjO14tDGHoIJi0gUCY+6lvgAKCRB4tDGHoIJi 0ulqAQCULkPATgqWtudHs9arilghmWnIl+5HOdnui0TiEobVIwD9EJWZRVVAGTBQ VrKlrkCPLqc12a+YcCC0aBjkaR6SDwI= =ZTRK -----END PGP SIGNATURE----- --+OUnDp7k4pWMwsmV-- --===============4748043798155223410== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel --===============4748043798155223410==--