From: Conor Dooley <conor.dooley@microchip.com> To: Arnd Bergmann <arnd@arndb.de> Cc: Prabhakar <prabhakar.csengg@gmail.com>, "Geert Uytterhoeven" <geert+renesas@glider.be>, "Heiko Stübner" <heiko@sntech.de>, guoren <guoren@kernel.org>, "Andrew Jones" <ajones@ventanamicro.com>, "Paul Walmsley" <paul.walmsley@sifive.com>, "Palmer Dabbelt" <palmer@dabbelt.com>, "Albert Ou" <aou@eecs.berkeley.edu>, "open list:RISC-V ARCHITECTURE" <linux-riscv@lists.infradead.org>, "open list" <linux-kernel@vger.kernel.org>, devicetree@vger.kernel.org, Linux-Renesas <linux-renesas-soc@vger.kernel.org>, "Lad, Prabhakar" <prabhakar.mahadev-lad.rj@bp.renesas.com>, "Philipp Tomsich" <philipp.tomsich@vrull.eu>, "Nathan Chancellor" <nathan@kernel.org>, "Atish Patra" <atishp@rivosinc.com>, "Anup Patel" <apatel@ventanamicro.com>, "Tsukasa OI" <research_trasio@irq.a4lg.com>, "Jisheng Zhang" <jszhang@kernel.org>, "Mayuresh Chitale" <mchitale@ventanamicro.com>, "Christoph Hellwig" <hch@lst.de>, "Will Deacon" <will@kernel.org> Subject: Re: [RFC PATCH v6 1/6] riscv: mm: dma-noncoherent: Switch using function pointers for cache management Date: Mon, 9 Jan 2023 13:27:31 +0000 [thread overview] Message-ID: <Y7wWQ200sdN2rAmX@wendy> (raw) In-Reply-To: <45d6eb0c-cbe3-4a83-aa12-3483638473ae@app.fastmail.com> [-- Attachment #1: Type: text/plain, Size: 3851 bytes --] On Mon, Jan 09, 2023 at 01:59:12PM +0100, Arnd Bergmann wrote: > On Mon, Jan 9, 2023, at 13:03, Lad, Prabhakar wrote: > > On Sun, Jan 8, 2023 at 12:08 AM Arnd Bergmann <arnd@arndb.de> wrote: > >> >> > +struct riscv_cache_ops { > >> >> > + void (*clean_range)(unsigned long addr, unsigned long size); > >> >> > + void (*inv_range)(unsigned long addr, unsigned long size); > >> >> > + void (*flush_range)(unsigned long addr, unsigned long size); > >> >> > + void (*riscv_dma_noncoherent_cmo_ops)(void *vaddr, size_t size, > >> >> > + enum dma_data_direction dir, > >> >> > + enum dma_noncoherent_ops ops); > >> >> > +}; > >> >> > >> >> I don't quite see how the fourth operation is used here. > >> >> Are there cache controllers that need something beyond > >> >> clean/inv/flush? > >> >> > >> > This is for platforms that dont follow standard cache operations (like > >> > done in patch 5/6) and there drivers decide on the operations > >> > depending on the ops and dir. > >> > >> My feeling is that the set of operations that get called should > >> not depend on the cache controller but at best the CPU. I tried to > >> enumerate how zicbom and ax45 differ here, and how that compares > >> to other architectures: > >> > >> zicbom ax45,mips,arc arm arm64 > >> fromdevice clean/flush inval/inval inval/inval clean/inval > >> todevice clean/- clean/- clean/- clean/- > >> bidi flush/flush flush/inval clean/inval clean/inval I did a bit of digging on lore for context on why the ops are what they are.. In v3 of the Zicbom enablement patchset, things looked like: fromdevice inval/inval todevice clean/- bidi flush/inval v3: https://lore.kernel.org/linux-riscv/20220610004308.1903626-3-heiko@sntech.de/ Samuel had some comments about the invals: https://lore.kernel.org/linux-riscv/342e3c12-ebb0-badf-7d4c-c444a2b842b2@sholland.org/ In v4 it was changed to: fromdevice inval/flush todevice clean/- bidi flush/flush v4: https://lore.kernel.org/linux-riscv/20220619203212.3604485-4-heiko@sntech.de/ Christoph replied to that one, linking the thread belonging to the commit you pointed out earlier: https://lore.kernel.org/linux-riscv/20220620061607.GB10485@lst.de/ v5 produced what you have in your table above: https://lore.kernel.org/linux-riscv/20220629215944.397952-4-heiko@sntech.de/ > >> > >> So everyone does the same operation for DMA_TO_DEVICE, but > >> they differ in the DMA_FROM_DEVICE handling, for reasons I > >> don't quite see: > >> > >> Your ax45 code does the same as arc and mips. arm and > >> arm64 skip invalidating the cache before bidi mappings, > >> but arm has a FIXME comment about that. arm64 does a > >> 'clean' instead of 'inval' when mapping a fromdevice > >> page, which seems valid but slower than necessary. > >> > >> Could the zicbom operations be changed to do the same > >> things as the ax45/mips/arc ones, or are there specific > >> details in the zicbom spec that require this? > >> > > I'll let the RISC-V experts respond here. > > Adding Christoph Hellwig and Will Deacon to Cc as well. > > I had another look at the arm64 side, which (like the zicbom > variant) uses 'clean' on dma_sync_single_for_device(DMA_FROM_DEVICE), > as that has changed not that long ago, see > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c50f11c6196f45c92ca48b16a5071615d4ae0572 > > I'm still not sure what the correct set of operations has > to be, but nothing in that patch description sounds ISA > or even microarchitecture specific. Hope the lore archaeology helps jog people's memories... Conor [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 228 bytes --]
WARNING: multiple messages have this Message-ID (diff)
From: Conor Dooley <conor.dooley@microchip.com> To: Arnd Bergmann <arnd@arndb.de> Cc: Prabhakar <prabhakar.csengg@gmail.com>, "Geert Uytterhoeven" <geert+renesas@glider.be>, "Heiko Stübner" <heiko@sntech.de>, guoren <guoren@kernel.org>, "Andrew Jones" <ajones@ventanamicro.com>, "Paul Walmsley" <paul.walmsley@sifive.com>, "Palmer Dabbelt" <palmer@dabbelt.com>, "Albert Ou" <aou@eecs.berkeley.edu>, "open list:RISC-V ARCHITECTURE" <linux-riscv@lists.infradead.org>, "open list" <linux-kernel@vger.kernel.org>, devicetree@vger.kernel.org, Linux-Renesas <linux-renesas-soc@vger.kernel.org>, "Lad, Prabhakar" <prabhakar.mahadev-lad.rj@bp.renesas.com>, "Philipp Tomsich" <philipp.tomsich@vrull.eu>, "Nathan Chancellor" <nathan@kernel.org>, "Atish Patra" <atishp@rivosinc.com>, "Anup Patel" <apatel@ventanamicro.com>, "Tsukasa OI" <research_trasio@irq.a4lg.com>, "Jisheng Zhang" <jszhang@kernel.org>, "Mayuresh Chitale" <mchitale@ventanamicro.com>, "Christoph Hellwig" <hch@lst.de>, "Will Deacon" <will@kernel.org> Subject: Re: [RFC PATCH v6 1/6] riscv: mm: dma-noncoherent: Switch using function pointers for cache management Date: Mon, 9 Jan 2023 13:27:31 +0000 [thread overview] Message-ID: <Y7wWQ200sdN2rAmX@wendy> (raw) In-Reply-To: <45d6eb0c-cbe3-4a83-aa12-3483638473ae@app.fastmail.com> [-- Attachment #1.1: Type: text/plain, Size: 3851 bytes --] On Mon, Jan 09, 2023 at 01:59:12PM +0100, Arnd Bergmann wrote: > On Mon, Jan 9, 2023, at 13:03, Lad, Prabhakar wrote: > > On Sun, Jan 8, 2023 at 12:08 AM Arnd Bergmann <arnd@arndb.de> wrote: > >> >> > +struct riscv_cache_ops { > >> >> > + void (*clean_range)(unsigned long addr, unsigned long size); > >> >> > + void (*inv_range)(unsigned long addr, unsigned long size); > >> >> > + void (*flush_range)(unsigned long addr, unsigned long size); > >> >> > + void (*riscv_dma_noncoherent_cmo_ops)(void *vaddr, size_t size, > >> >> > + enum dma_data_direction dir, > >> >> > + enum dma_noncoherent_ops ops); > >> >> > +}; > >> >> > >> >> I don't quite see how the fourth operation is used here. > >> >> Are there cache controllers that need something beyond > >> >> clean/inv/flush? > >> >> > >> > This is for platforms that dont follow standard cache operations (like > >> > done in patch 5/6) and there drivers decide on the operations > >> > depending on the ops and dir. > >> > >> My feeling is that the set of operations that get called should > >> not depend on the cache controller but at best the CPU. I tried to > >> enumerate how zicbom and ax45 differ here, and how that compares > >> to other architectures: > >> > >> zicbom ax45,mips,arc arm arm64 > >> fromdevice clean/flush inval/inval inval/inval clean/inval > >> todevice clean/- clean/- clean/- clean/- > >> bidi flush/flush flush/inval clean/inval clean/inval I did a bit of digging on lore for context on why the ops are what they are.. In v3 of the Zicbom enablement patchset, things looked like: fromdevice inval/inval todevice clean/- bidi flush/inval v3: https://lore.kernel.org/linux-riscv/20220610004308.1903626-3-heiko@sntech.de/ Samuel had some comments about the invals: https://lore.kernel.org/linux-riscv/342e3c12-ebb0-badf-7d4c-c444a2b842b2@sholland.org/ In v4 it was changed to: fromdevice inval/flush todevice clean/- bidi flush/flush v4: https://lore.kernel.org/linux-riscv/20220619203212.3604485-4-heiko@sntech.de/ Christoph replied to that one, linking the thread belonging to the commit you pointed out earlier: https://lore.kernel.org/linux-riscv/20220620061607.GB10485@lst.de/ v5 produced what you have in your table above: https://lore.kernel.org/linux-riscv/20220629215944.397952-4-heiko@sntech.de/ > >> > >> So everyone does the same operation for DMA_TO_DEVICE, but > >> they differ in the DMA_FROM_DEVICE handling, for reasons I > >> don't quite see: > >> > >> Your ax45 code does the same as arc and mips. arm and > >> arm64 skip invalidating the cache before bidi mappings, > >> but arm has a FIXME comment about that. arm64 does a > >> 'clean' instead of 'inval' when mapping a fromdevice > >> page, which seems valid but slower than necessary. > >> > >> Could the zicbom operations be changed to do the same > >> things as the ax45/mips/arc ones, or are there specific > >> details in the zicbom spec that require this? > >> > > I'll let the RISC-V experts respond here. > > Adding Christoph Hellwig and Will Deacon to Cc as well. > > I had another look at the arm64 side, which (like the zicbom > variant) uses 'clean' on dma_sync_single_for_device(DMA_FROM_DEVICE), > as that has changed not that long ago, see > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c50f11c6196f45c92ca48b16a5071615d4ae0572 > > I'm still not sure what the correct set of operations has > to be, but nothing in that patch description sounds ISA > or even microarchitecture specific. Hope the lore archaeology helps jog people's memories... Conor [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 228 bytes --] [-- Attachment #2: Type: text/plain, Size: 161 bytes --] _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2023-01-09 13:28 UTC|newest] Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-01-06 18:55 [PATCH v6 0/6] RISC-V non-coherent function pointer based cache management operations + non-coherent DMA support for AX45MP Prabhakar 2023-01-06 18:55 ` Prabhakar 2023-01-06 18:55 ` [RFC PATCH v6 1/6] riscv: mm: dma-noncoherent: Switch using function pointers for cache management Prabhakar 2023-01-06 18:55 ` Prabhakar 2023-01-06 22:31 ` Arnd Bergmann 2023-01-06 23:29 ` Conor Dooley 2023-01-06 23:29 ` Conor Dooley 2023-01-07 21:52 ` Arnd Bergmann 2023-01-07 21:52 ` Arnd Bergmann 2023-01-07 22:21 ` Conor Dooley 2023-01-07 22:21 ` Conor Dooley 2023-01-08 16:37 ` Conor Dooley 2023-01-08 16:37 ` Conor Dooley 2023-01-07 22:10 ` Lad, Prabhakar 2023-01-07 22:10 ` Lad, Prabhakar 2023-01-08 0:07 ` Arnd Bergmann 2023-01-08 0:07 ` Arnd Bergmann 2023-01-09 12:03 ` Lad, Prabhakar 2023-01-09 12:03 ` Lad, Prabhakar 2023-01-09 12:59 ` Arnd Bergmann 2023-01-09 12:59 ` Arnd Bergmann 2023-01-09 13:27 ` Conor Dooley [this message] 2023-01-09 13:27 ` Conor Dooley 2023-01-10 7:01 ` Christoph Hellwig 2023-01-10 7:01 ` Christoph Hellwig 2023-01-10 15:03 ` Arnd Bergmann 2023-01-10 15:03 ` Arnd Bergmann 2023-01-10 15:11 ` Will Deacon 2023-01-10 15:11 ` Will Deacon 2023-01-13 5:48 ` Christoph Hellwig 2023-01-13 5:48 ` Christoph Hellwig 2023-01-20 17:04 ` Arnd Bergmann 2023-01-20 17:04 ` Arnd Bergmann 2023-01-21 14:37 ` Christoph Hellwig 2023-01-21 14:37 ` Christoph Hellwig 2023-01-21 19:30 ` Arnd Bergmann 2023-01-21 19:30 ` Arnd Bergmann 2023-01-22 7:27 ` Christoph Hellwig 2023-01-22 7:27 ` Christoph Hellwig 2023-01-22 11:04 ` Arnd Bergmann 2023-01-22 11:04 ` Arnd Bergmann 2023-01-23 14:46 ` Christoph Hellwig 2023-01-23 14:46 ` Christoph Hellwig 2023-01-06 23:47 ` Conor Dooley 2023-01-06 23:47 ` Conor Dooley 2023-01-07 22:36 ` Lad, Prabhakar 2023-01-07 22:36 ` Lad, Prabhakar 2023-01-06 18:55 ` [PATCH v6 2/6] riscv: asm: vendorid_list: Add Andes Technology to the vendors list Prabhakar 2023-01-06 18:55 ` Prabhakar 2023-01-06 18:55 ` [PATCH v6 3/6] riscv: errata: Add Andes alternative ports Prabhakar 2023-01-06 18:55 ` Prabhakar 2023-01-06 21:44 ` Conor Dooley 2023-01-06 21:44 ` Conor Dooley 2023-01-06 18:55 ` [PATCH v6 4/6] dt-bindings: cache: r9a07g043f-l2-cache: Add DT binding documentation for L2 cache controller Prabhakar 2023-01-06 18:55 ` Prabhakar 2023-01-06 21:53 ` Conor Dooley 2023-01-06 21:53 ` Conor Dooley 2023-01-07 20:43 ` Lad, Prabhakar 2023-01-07 20:43 ` Lad, Prabhakar 2023-01-09 12:15 ` Geert Uytterhoeven 2023-01-09 12:15 ` Geert Uytterhoeven 2023-01-09 13:14 ` Lad, Prabhakar 2023-01-09 13:14 ` Lad, Prabhakar 2023-01-06 18:55 ` [PATCH v6 5/6] cache: Add L2 cache management for Andes AX45MP RISC-V core Prabhakar 2023-01-06 18:55 ` Prabhakar 2023-01-07 0:09 ` Conor Dooley 2023-01-07 0:09 ` Conor Dooley 2023-01-07 20:49 ` Lad, Prabhakar 2023-01-07 20:49 ` Lad, Prabhakar 2023-01-06 18:55 ` [PATCH v6 6/6] soc: renesas: Kconfig: Select the required configs for RZ/Five SoC Prabhakar 2023-01-06 18:55 ` Prabhakar 2023-01-06 23:49 ` Conor Dooley 2023-01-06 23:49 ` Conor Dooley
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=Y7wWQ200sdN2rAmX@wendy \ --to=conor.dooley@microchip.com \ --cc=ajones@ventanamicro.com \ --cc=aou@eecs.berkeley.edu \ --cc=apatel@ventanamicro.com \ --cc=arnd@arndb.de \ --cc=atishp@rivosinc.com \ --cc=devicetree@vger.kernel.org \ --cc=geert+renesas@glider.be \ --cc=guoren@kernel.org \ --cc=hch@lst.de \ --cc=heiko@sntech.de \ --cc=jszhang@kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-renesas-soc@vger.kernel.org \ --cc=linux-riscv@lists.infradead.org \ --cc=mchitale@ventanamicro.com \ --cc=nathan@kernel.org \ --cc=palmer@dabbelt.com \ --cc=paul.walmsley@sifive.com \ --cc=philipp.tomsich@vrull.eu \ --cc=prabhakar.csengg@gmail.com \ --cc=prabhakar.mahadev-lad.rj@bp.renesas.com \ --cc=research_trasio@irq.a4lg.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.