From: Samuel Holland <samuel@sholland.org> To: Prabhakar <prabhakar.csengg@gmail.com> Cc: Jisheng Zhang <jszhang@kernel.org>, Atish Patra <atishp@rivosinc.com>, Anup Patel <apatel@ventanamicro.com>, Andrew Jones <ajones@ventanamicro.com>, Nathan Chancellor <nathan@kernel.org>, Philipp Tomsich <philipp.tomsich@vrull.eu>, devicetree@vger.kernel.org, linux-riscv@lists.infradead.org, linux-renesas-soc@vger.kernel.org, linux-kernel@vger.kernel.org, Biju Das <biju.das.jz@bp.renesas.com>, Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>, Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>, Geert Uytterhoeven <geert+renesas@glider.be>, Magnus Damm <magnus.damm@gmail.com>, Heiko Stuebner <heiko@sntech.de>, Conor Dooley <conor.dooley@microchip.com>, Guo Ren <guoren@kernel.org>, Rob Herring <robh+dt@kernel.org>, Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org> Subject: Re: [PATCH v5 6/6] soc: renesas: Add L2 cache management for RZ/Five SoC Date: Tue, 27 Dec 2022 21:16:17 -0600 [thread overview] Message-ID: <2df0317a-5696-8d3e-70a1-72730fbd6a78@sholland.org> (raw) In-Reply-To: <20221212115505.36770-7-prabhakar.mahadev-lad.rj@bp.renesas.com> On 12/12/22 05:55, Prabhakar wrote: > From: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> > > I/O Coherence Port (IOCP) provides an AXI interface for connecting > external non-caching masters, such as DMA controllers. The accesses > from IOCP are coherent with D-Caches and L2 Cache. > > IOCP is a specification option and is disabled on the Renesas RZ/Five > SoC due to this reason IP blocks using DMA will fail. > > The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA) > block that allows dynamic adjustment of memory attributes in the runtime. > It contains a configurable amount of PMA entries implemented as CSR > registers to control the attributes of memory locations in interest. > Below are the memory attributes supported: > * Device, Non-bufferable > * Device, bufferable > * Memory, Non-cacheable, Non-bufferable > * Memory, Non-cacheable, Bufferable > * Memory, Write-back, No-allocate > * Memory, Write-back, Read-allocate > * Memory, Write-back, Write-allocate > * Memory, Write-back, Read and Write-allocate > > More info about PMA (section 10.3): > Link: http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf > > As a workaround for SoCs with IOCP disabled CMO needs to be handled by > software. Firstly OpenSBI configures the memory region as > "Memory, Non-cacheable, Bufferable" and passes this region as a global > shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA > allocations happen from this region and synchronization callbacks are > implemented to synchronize when doing DMA transactions. > > Example PMA region passes as a DT node from OpenSBI: > reserved-memory { > #address-cells = <2>; > #size-cells = <2>; > ranges; > > pma_resv0@58000000 { > compatible = "shared-dma-pool"; > reg = <0x0 0x58000000 0x0 0x08000000>; > no-map; > linux,dma-default; > }; > }; > > Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> > --- > v4 -> v5 > * Dropped code for configuring L2 cache > * Dropped code for configuring PMA > * Updated commit message > * Added comments > * Changed static branch enable/disable order > > RFC v3 -> v4 > * Made use of runtime patching instead of compile time > * Now just exposing single function ax45mp_no_iocp_cmo() for CMO handling > * Added a check to make sure cache line size is always 64 bytes > * Renamed folder rzf -> rzfive > * Improved Kconfig description > * Dropped L2 cache configuration > * Dropped unnecessary casts > * Fixed comments pointed by Geert. > --- > arch/riscv/include/asm/cacheflush.h | 8 + > arch/riscv/include/asm/errata_list.h | 28 ++- > drivers/soc/renesas/Kconfig | 6 + > drivers/soc/renesas/Makefile | 2 + > drivers/soc/renesas/rzfive/Kconfig | 6 + > drivers/soc/renesas/rzfive/Makefile | 3 + > drivers/soc/renesas/rzfive/ax45mp_cache.c | 256 ++++++++++++++++++++++ > 7 files changed, 303 insertions(+), 6 deletions(-) > create mode 100644 drivers/soc/renesas/rzfive/Kconfig > create mode 100644 drivers/soc/renesas/rzfive/Makefile > create mode 100644 drivers/soc/renesas/rzfive/ax45mp_cache.c Thanks for the updates! This looks much cleaner and easier to understand now that the driver is only trying to do one thing. > diff --git a/drivers/soc/renesas/rzfive/Makefile b/drivers/soc/renesas/rzfive/Makefile > new file mode 100644 > index 000000000000..2012e7fb978d > --- /dev/null > +++ b/drivers/soc/renesas/rzfive/Makefile ... > +void ax45mp_no_iocp_cmo(unsigned int cache_size, void *vaddr, size_t size, int dir, int ops) > +{ > + if (ops == NON_COHERENT_DMA_PREP) > + return; > + > + if (!static_branch_unlikely(&ax45mp_l2c_configured)) > + return; > + > + if (ops == NON_COHERENT_SYNC_DMA_FOR_DEVICE) { > + switch (dir) { > + case DMA_FROM_DEVICE: > + ax45mp_cpu_dma_inval_range(vaddr, size); > + break; > + case DMA_TO_DEVICE: > + case DMA_BIDIRECTIONAL: > + ax45mp_cpu_dma_wb_range(vaddr, size); > + break; > + default: > + break; > + } > + return; > + } > + > + /* op == NON_COHERENT_SYNC_DMA_FOR_CPU */ > + if (dir == DMA_BIDIRECTIONAL || dir == DMA_FROM_DEVICE) > + ax45mp_cpu_dma_inval_range(vaddr, size); I think this at least deserves a comment explaining why it differs from the clean/flush/invalidate choices in arch/riscv/mm/dma-noncoherent.c. Regards, Samuel
WARNING: multiple messages have this Message-ID (diff)
From: Samuel Holland <samuel@sholland.org> To: Prabhakar <prabhakar.csengg@gmail.com> Cc: Jisheng Zhang <jszhang@kernel.org>, Atish Patra <atishp@rivosinc.com>, Anup Patel <apatel@ventanamicro.com>, Andrew Jones <ajones@ventanamicro.com>, Nathan Chancellor <nathan@kernel.org>, Philipp Tomsich <philipp.tomsich@vrull.eu>, devicetree@vger.kernel.org, linux-riscv@lists.infradead.org, linux-renesas-soc@vger.kernel.org, linux-kernel@vger.kernel.org, Biju Das <biju.das.jz@bp.renesas.com>, Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>, Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>, Geert Uytterhoeven <geert+renesas@glider.be>, Magnus Damm <magnus.damm@gmail.com>, Heiko Stuebner <heiko@sntech.de>, Conor Dooley <conor.dooley@microchip.com>, Guo Ren <guoren@kernel.org>, Rob Herring <robh+dt@kernel.org>, Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org> Subject: Re: [PATCH v5 6/6] soc: renesas: Add L2 cache management for RZ/Five SoC Date: Tue, 27 Dec 2022 21:16:17 -0600 [thread overview] Message-ID: <2df0317a-5696-8d3e-70a1-72730fbd6a78@sholland.org> (raw) In-Reply-To: <20221212115505.36770-7-prabhakar.mahadev-lad.rj@bp.renesas.com> On 12/12/22 05:55, Prabhakar wrote: > From: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> > > I/O Coherence Port (IOCP) provides an AXI interface for connecting > external non-caching masters, such as DMA controllers. The accesses > from IOCP are coherent with D-Caches and L2 Cache. > > IOCP is a specification option and is disabled on the Renesas RZ/Five > SoC due to this reason IP blocks using DMA will fail. > > The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA) > block that allows dynamic adjustment of memory attributes in the runtime. > It contains a configurable amount of PMA entries implemented as CSR > registers to control the attributes of memory locations in interest. > Below are the memory attributes supported: > * Device, Non-bufferable > * Device, bufferable > * Memory, Non-cacheable, Non-bufferable > * Memory, Non-cacheable, Bufferable > * Memory, Write-back, No-allocate > * Memory, Write-back, Read-allocate > * Memory, Write-back, Write-allocate > * Memory, Write-back, Read and Write-allocate > > More info about PMA (section 10.3): > Link: http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf > > As a workaround for SoCs with IOCP disabled CMO needs to be handled by > software. Firstly OpenSBI configures the memory region as > "Memory, Non-cacheable, Bufferable" and passes this region as a global > shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA > allocations happen from this region and synchronization callbacks are > implemented to synchronize when doing DMA transactions. > > Example PMA region passes as a DT node from OpenSBI: > reserved-memory { > #address-cells = <2>; > #size-cells = <2>; > ranges; > > pma_resv0@58000000 { > compatible = "shared-dma-pool"; > reg = <0x0 0x58000000 0x0 0x08000000>; > no-map; > linux,dma-default; > }; > }; > > Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> > --- > v4 -> v5 > * Dropped code for configuring L2 cache > * Dropped code for configuring PMA > * Updated commit message > * Added comments > * Changed static branch enable/disable order > > RFC v3 -> v4 > * Made use of runtime patching instead of compile time > * Now just exposing single function ax45mp_no_iocp_cmo() for CMO handling > * Added a check to make sure cache line size is always 64 bytes > * Renamed folder rzf -> rzfive > * Improved Kconfig description > * Dropped L2 cache configuration > * Dropped unnecessary casts > * Fixed comments pointed by Geert. > --- > arch/riscv/include/asm/cacheflush.h | 8 + > arch/riscv/include/asm/errata_list.h | 28 ++- > drivers/soc/renesas/Kconfig | 6 + > drivers/soc/renesas/Makefile | 2 + > drivers/soc/renesas/rzfive/Kconfig | 6 + > drivers/soc/renesas/rzfive/Makefile | 3 + > drivers/soc/renesas/rzfive/ax45mp_cache.c | 256 ++++++++++++++++++++++ > 7 files changed, 303 insertions(+), 6 deletions(-) > create mode 100644 drivers/soc/renesas/rzfive/Kconfig > create mode 100644 drivers/soc/renesas/rzfive/Makefile > create mode 100644 drivers/soc/renesas/rzfive/ax45mp_cache.c Thanks for the updates! This looks much cleaner and easier to understand now that the driver is only trying to do one thing. > diff --git a/drivers/soc/renesas/rzfive/Makefile b/drivers/soc/renesas/rzfive/Makefile > new file mode 100644 > index 000000000000..2012e7fb978d > --- /dev/null > +++ b/drivers/soc/renesas/rzfive/Makefile ... > +void ax45mp_no_iocp_cmo(unsigned int cache_size, void *vaddr, size_t size, int dir, int ops) > +{ > + if (ops == NON_COHERENT_DMA_PREP) > + return; > + > + if (!static_branch_unlikely(&ax45mp_l2c_configured)) > + return; > + > + if (ops == NON_COHERENT_SYNC_DMA_FOR_DEVICE) { > + switch (dir) { > + case DMA_FROM_DEVICE: > + ax45mp_cpu_dma_inval_range(vaddr, size); > + break; > + case DMA_TO_DEVICE: > + case DMA_BIDIRECTIONAL: > + ax45mp_cpu_dma_wb_range(vaddr, size); > + break; > + default: > + break; > + } > + return; > + } > + > + /* op == NON_COHERENT_SYNC_DMA_FOR_CPU */ > + if (dir == DMA_BIDIRECTIONAL || dir == DMA_FROM_DEVICE) > + ax45mp_cpu_dma_inval_range(vaddr, size); I think this at least deserves a comment explaining why it differs from the clean/flush/invalidate choices in arch/riscv/mm/dma-noncoherent.c. Regards, Samuel _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2022-12-28 3:16 UTC|newest] Thread overview: 132+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-12-12 11:54 [PATCH v5 0/6] AX45MP: Add support to non-coherent DMA Prabhakar 2022-12-12 11:54 ` Prabhakar 2022-12-12 11:55 ` [PATCH v5 1/6] riscv: asm: alternative-macros: Introduce ALTERNATIVE_3() macro Prabhakar 2022-12-12 11:55 ` Prabhakar 2022-12-12 12:32 ` Heiko Stuebner 2022-12-12 12:32 ` Heiko Stuebner 2022-12-13 17:21 ` Geert Uytterhoeven 2022-12-13 17:21 ` Geert Uytterhoeven 2022-12-13 17:49 ` Lad, Prabhakar 2022-12-13 17:49 ` Lad, Prabhakar 2022-12-14 14:34 ` Andrew Jones 2022-12-14 14:34 ` Andrew Jones 2022-12-17 21:41 ` Conor Dooley 2022-12-17 21:41 ` Conor Dooley 2022-12-19 11:15 ` Lad, Prabhakar 2022-12-19 11:15 ` Lad, Prabhakar 2022-12-19 16:22 ` Conor Dooley 2022-12-19 16:22 ` Conor Dooley 2022-12-19 16:24 ` Lad, Prabhakar 2022-12-19 16:24 ` Lad, Prabhakar 2022-12-12 11:55 ` [PATCH v5 2/6] riscv: asm: vendorid_list: Add Andes Technology to the vendors list Prabhakar 2022-12-12 11:55 ` Prabhakar 2022-12-12 11:55 ` [PATCH v5 3/6] riscv: errata: Add Andes alternative ports Prabhakar 2022-12-12 11:55 ` Prabhakar 2022-12-17 21:19 ` Conor Dooley 2022-12-17 21:19 ` Conor Dooley 2022-12-19 11:19 ` Lad, Prabhakar 2022-12-19 11:19 ` Lad, Prabhakar 2022-12-19 16:20 ` Conor Dooley 2022-12-19 16:20 ` Conor Dooley 2022-12-21 0:31 ` Lad, Prabhakar 2022-12-21 0:31 ` Lad, Prabhakar 2022-12-12 11:55 ` [PATCH v5 4/6] riscv: mm: dma-noncoherent: Pass direction and operation to ALT_CMO_OP() Prabhakar 2022-12-12 11:55 ` Prabhakar 2022-12-13 17:14 ` Geert Uytterhoeven 2022-12-13 17:14 ` Geert Uytterhoeven 2022-12-13 17:57 ` Lad, Prabhakar 2022-12-13 17:57 ` Lad, Prabhakar 2022-12-17 20:52 ` Conor Dooley 2022-12-17 20:52 ` Conor Dooley 2022-12-19 11:21 ` Lad, Prabhakar 2022-12-19 11:21 ` Lad, Prabhakar 2022-12-12 11:55 ` [PATCH v5 5/6] dt-bindings: cache: r9a07g043f-l2-cache: Add DT binding documentation for L2 cache controller Prabhakar 2022-12-12 11:55 ` Prabhakar 2022-12-12 17:28 ` Rob Herring 2022-12-12 17:28 ` Rob Herring 2022-12-12 11:55 ` [PATCH v5 6/6] soc: renesas: Add L2 cache management for RZ/Five SoC Prabhakar 2022-12-12 11:55 ` Prabhakar 2022-12-15 10:34 ` Geert Uytterhoeven 2022-12-15 10:34 ` Geert Uytterhoeven 2022-12-15 11:03 ` Lad, Prabhakar 2022-12-15 11:03 ` Lad, Prabhakar 2022-12-15 11:10 ` Geert Uytterhoeven 2022-12-15 11:10 ` Geert Uytterhoeven 2022-12-15 17:46 ` Lad, Prabhakar 2022-12-15 17:46 ` Lad, Prabhakar 2022-12-15 19:54 ` Conor Dooley 2022-12-15 19:54 ` Conor Dooley 2022-12-15 20:17 ` Geert Uytterhoeven 2022-12-15 20:17 ` Geert Uytterhoeven 2022-12-15 20:32 ` Conor Dooley 2022-12-15 20:32 ` Conor Dooley 2022-12-15 21:40 ` Palmer Dabbelt 2022-12-15 21:40 ` Palmer Dabbelt 2022-12-16 7:02 ` Christoph Hellwig 2022-12-16 7:02 ` Christoph Hellwig 2022-12-16 16:32 ` Palmer Dabbelt 2022-12-16 16:32 ` Palmer Dabbelt 2022-12-16 20:04 ` Arnd Bergmann 2022-12-16 20:04 ` Arnd Bergmann 2022-12-17 22:52 ` Conor Dooley 2022-12-17 22:52 ` Conor Dooley 2022-12-19 12:43 ` Lad, Prabhakar 2022-12-19 12:43 ` Lad, Prabhakar 2022-12-19 16:08 ` Conor Dooley 2022-12-19 16:08 ` Conor Dooley 2022-12-29 14:05 ` Arnd Bergmann 2022-12-29 14:05 ` Arnd Bergmann 2022-12-29 14:42 ` Conor Dooley 2022-12-29 14:42 ` Conor Dooley 2023-01-03 21:03 ` [RFC v5.1 0/9] Generic function based cache management operations (was Re: [PATCH v5 6/6] soc: renesas: Add L2 cache management for RZ/Five SoC) Conor Dooley 2023-01-03 21:03 ` Conor Dooley 2023-01-03 21:03 ` [RFC v5.1 1/9] riscv: asm: alternative-macros: Introduce ALTERNATIVE_3() macro Conor Dooley 2023-01-03 21:03 ` Conor Dooley 2023-01-03 21:03 ` [RFC v5.1 2/9] riscv: asm: vendorid_list: Add Andes Technology to the vendors list Conor Dooley 2023-01-03 21:03 ` Conor Dooley 2023-01-03 21:03 ` [RFC v5.1 3/9] riscv: errata: Add Andes alternative ports Conor Dooley 2023-01-03 21:03 ` Conor Dooley 2023-01-03 21:03 ` [RFC v5.1 4/9] riscv: mm: dma-noncoherent: Pass direction and operation to ALT_CMO_OP() Conor Dooley 2023-01-03 21:03 ` Conor Dooley 2023-01-03 21:03 ` [RFC v5.1 5/9] dt-bindings: cache: r9a07g043f-l2-cache: Add DT binding documentation for L2 cache controller Conor Dooley 2023-01-03 21:03 ` Conor Dooley 2023-01-03 21:03 ` [RFC v5.1 6/9] cache,soc: Move SiFive CCache driver & create drivers/cache Conor Dooley 2023-01-03 21:03 ` Conor Dooley 2023-01-04 9:50 ` Ben Dooks 2023-01-04 9:50 ` Ben Dooks 2023-01-04 10:18 ` Conor Dooley 2023-01-04 10:18 ` Conor Dooley 2023-01-03 21:03 ` [RFC v5.1 7/9] RISC-V: create a function based cache management interface Conor Dooley 2023-01-03 21:03 ` Conor Dooley 2023-01-03 21:04 ` [RFC v5.1 8/9] soc: renesas: Add L2 cache management for RZ/Five SoC Conor Dooley 2023-01-03 21:04 ` Conor Dooley 2023-01-03 21:04 ` [RFC v5.1 9/9] [DON'T APPLY] cache: sifive-ccache: add cache flushing capability Conor Dooley 2023-01-03 21:04 ` Conor Dooley 2023-01-03 21:25 ` Palmer Dabbelt 2023-01-03 21:25 ` Palmer Dabbelt 2023-01-03 21:28 ` Arnd Bergmann 2023-01-03 21:28 ` Arnd Bergmann 2023-01-04 0:00 ` Conor Dooley 2023-01-04 0:00 ` Conor Dooley 2023-01-04 8:17 ` Arnd Bergmann 2023-01-04 8:17 ` Arnd Bergmann 2023-01-04 9:23 ` Conor Dooley 2023-01-04 9:23 ` Conor Dooley 2023-01-04 10:19 ` Arnd Bergmann 2023-01-04 10:19 ` Arnd Bergmann 2023-01-04 11:56 ` Conor Dooley 2023-01-04 11:56 ` Conor Dooley 2023-01-04 12:18 ` Arnd Bergmann 2023-01-04 12:18 ` Arnd Bergmann 2023-01-04 13:20 ` Conor Dooley 2023-01-04 13:20 ` Conor Dooley 2023-01-04 14:15 ` Arnd Bergmann 2023-01-04 14:15 ` Arnd Bergmann 2023-01-04 9:45 ` Ben Dooks 2023-01-04 9:45 ` Ben Dooks 2023-01-04 9:57 ` Conor Dooley 2023-01-04 9:57 ` Conor Dooley 2022-12-17 21:35 ` [PATCH v5 6/6] soc: renesas: Add L2 cache management for RZ/Five SoC Conor Dooley 2022-12-17 21:35 ` Conor Dooley 2022-12-28 3:16 ` Samuel Holland [this message] 2022-12-28 3:16 ` Samuel Holland
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=2df0317a-5696-8d3e-70a1-72730fbd6a78@sholland.org \ --to=samuel@sholland.org \ --cc=ajones@ventanamicro.com \ --cc=aou@eecs.berkeley.edu \ --cc=apatel@ventanamicro.com \ --cc=atishp@rivosinc.com \ --cc=biju.das.jz@bp.renesas.com \ --cc=conor.dooley@microchip.com \ --cc=devicetree@vger.kernel.org \ --cc=geert+renesas@glider.be \ --cc=guoren@kernel.org \ --cc=heiko@sntech.de \ --cc=jszhang@kernel.org \ --cc=krzysztof.kozlowski+dt@linaro.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-renesas-soc@vger.kernel.org \ --cc=linux-riscv@lists.infradead.org \ --cc=magnus.damm@gmail.com \ --cc=nathan@kernel.org \ --cc=palmer@dabbelt.com \ --cc=paul.walmsley@sifive.com \ --cc=philipp.tomsich@vrull.eu \ --cc=prabhakar.csengg@gmail.com \ --cc=prabhakar.mahadev-lad.rj@bp.renesas.com \ --cc=robh+dt@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.