From: Palmer Dabbelt <palmer@dabbelt.com> To: guoren@kernel.org Cc: Arnd Bergmann <arnd@arndb.de>, Anup Patel <Anup.Patel@wdc.com>, anup@brainfault.org, drew@beagleboard.org, Christoph Hellwig <hch@lst.de>, wefu@redhat.com, lazyparser@gmail.com, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-sunxi@lists.linux.dev, guoren@linux.alibaba.com, Paul Walmsley <paul.walmsley@sifive.com> Subject: Re: [PATCH RFC 0/3] riscv: Add DMA_COHERENT support Date: Fri, 04 Jun 2021 09:12:51 -0700 (PDT) [thread overview] Message-ID: <mhng-423aeaad-9339-4695-9a85-f947dd6135ac@palmerdabbelt-glaptop> (raw) In-Reply-To: <CAJF2gTTpurWpPUcA2JkF0rOFztKQgFBhOF9zQyuyi_-sxszhRQ@mail.gmail.com> On Fri, 04 Jun 2021 07:47:22 PDT (-0700), guoren@kernel.org wrote: > Hi Arnd & Palmer, > > Sorry for the delayed reply, I'm working on the next version of the patch. > > On Fri, Jun 4, 2021 at 5:56 PM Arnd Bergmann <arnd@arndb.de> wrote: >> >> On Thu, Jun 3, 2021 at 5:39 PM Palmer Dabbelt <palmer@dabbelt.com> wrote: >> > On Wed, 02 Jun 2021 23:00:29 PDT (-0700), Anup Patel wrote: >> > >> This implementation, which adds some Kconfig entries that control page table >> > >> bits, definately isn't suitable for upstream. Allowing users to set arbitrary >> > >> page table bits will eventually conflict with the standard, and is just going to >> > >> be a mess. It'll also lead to kernels that are only compatible with specific >> > >> designs, which we're trying very hard to avoid. At a bare minimum we'll need >> > >> some way to detect systems with these page table bits before setting them, >> > >> and some description of what the bits actually do so we can reason about >> > >> them. >> > > >> > > Yes, vendor specific Kconfig options are strict NO NO. We can't give-up the >> > > goal of unified kernel image for all platforms. > Okay, Agree. Please help review the next version of the patch. > >> > >> > I think this is just a phrasing issue, but just to be sure: >> > >> > IMO it's not that they're vendor-specific Kconfig options, it's that >> > turning them on will conflict with standard systems (and other vendors). >> > We've already got the ability to select sets of Kconfig settings that >> > will only boot on one vendor's system, which is fine, as long as there >> > remains a set of Kconfig settings that will boot on all systems. >> > >> > An example here would be the errata: every system has errata of some >> > sort, so if we start flipping off various vendor's errata Kconfigs >> > you'll end up with kernels that only function properly on some systems. >> > That's fine with me, as long as it's possible to turn on all vendor's >> > errata Kconfigs at the same time and the resulting kernel functions >> > correctly on all systems. >> >> Yes, this is generally the goal, and it would be great to have that >> working in a way where a 'defconfig' build just turns on all the options >> that are needed to use any SoC specific features and drivers while >> still working on all hardware. There are however limits you may run >> into at some point, and other architectures usually only manage to span >> some 10 to 15 years of hardware implementations with a single >> kernel before it get really hard. > I could follow the goal in the next version of the patchset. Please > help review, thx. IMO we're essentially here now with the RISC-V stuff: defconfig flips on everything necesasry to boot normal-smelling SOCs, with everything being detected as the system boots. We have some wacky configurations like !MMU and XIP that are coupled to the hardware, but (and sorry for crossing the other threads, I missed your pointer as it's early here) as I said in the other thread it might be time to make it explicit that those things are non-portable. The hope here has always been that we'd have enough in the standards that we could avoid a proliferation of vendor-specific code. We've always put a strong "things keep working forever" stake in the ground in RISC-V land, but that's largely been because we were counting on the standards existing that make support easy. In practice we don't have those standards so we're ending up with a fairly large software base that is required to support everything. We don't have all that much hardware right now so we'll have to see how it goes, but for now I'm in favor of keeping defconfig as a "boots on everything" sort of setup -- both because it makes life easier for users, and because it makes issues like the non-portable Kconfigs that showed up here quite explicit. If we get to 10/15 years of hardware then I'm sure we'll be removing old systems from defconfig (or maybe even the kernel entirely, a lot of this stuff isn't in production). I'm just hoping we make it that far ;) >> To give some common examples that make it break down: >> >> - 32-bit vs 64-bit already violates that rule on risc-v (as it does on >> most other architectures) Yes, and there's no way around that on RISC-V. They're different base ISAs therefor re-define the same instructions, so we're essentially at two kernel binaries by that point. The platform spec says rv64gc, so we can kind of punt on this one for now. If rv32 hardware shows up we'll probably want a standard system there too, which is why we've avoided coupling kernel portability to XLEN. >> - architectures that support both big-endian and little-endian kernels >> tend to have platforms that require one or the other (e.g. mips, >> though not arm). Not an issue for you. It is now! We've added big-endian to RISC-V. There's no hardware yet and very little software support. IMO the right answer is to ban that from the platform spec, but again it'll depnd on what vendors want to build (though anyone is listening, please don't make my life miserable ;)). >> - page table formats are the main cause of incompatibility: arm32 >> and x86-32 require three-level tables for certain features, but those >> are incompatible with older cores, arm64 supports three different >> page sizes, but none of them works on all cores (4KB almost works >> everywhere). We actually have some support on the works for multiple page table levels in a single binary, which should help with a lot of that incompatibility. I don't know of any plans to couple other page table features to the number of levels, though. >> - SMP-enabled ARMv7 kernels can be configured to run on either >> ARMv6 or ARMv8, but not both, in this case because of incompatible >> barrier instructions. Our barriers aren't quite split the same way, but we do have two memory models (RVWMO and TSO). IIUC we should be able to support both in the same kernels with some patching, but the resulting kernels would be biased towards one memory models over the other WRT performance. Again, we'll have to see what the vendors do and I'm hoping we don't end up with too many headaches. >> - 32-bit Arm has a couple more remaining features that require building >> a machine specific kernel if enabled because they hardcode physical >> addresses: early printk (debug_ll, not the normal earlycon), NOMMU, >> and XIP. We've got NOMMU and XIP as well, but we have some SBI support for early printk. IMO we're not really sure if we've decoupled all the PA layout dependencies yet from Linux, as we really only support one vendor's systems, but we've had a lot of work lately on beefing up our memory layout so with any luck we'll be able to quickly sort out anything that comes up.
WARNING: multiple messages have this Message-ID (diff)
From: Palmer Dabbelt <palmer@dabbelt.com> To: guoren@kernel.org Cc: Arnd Bergmann <arnd@arndb.de>, Anup Patel <Anup.Patel@wdc.com>, anup@brainfault.org, drew@beagleboard.org, Christoph Hellwig <hch@lst.de>, wefu@redhat.com, lazyparser@gmail.com, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-sunxi@lists.linux.dev, guoren@linux.alibaba.com, Paul Walmsley <paul.walmsley@sifive.com> Subject: Re: [PATCH RFC 0/3] riscv: Add DMA_COHERENT support Date: Fri, 04 Jun 2021 09:12:51 -0700 (PDT) [thread overview] Message-ID: <mhng-423aeaad-9339-4695-9a85-f947dd6135ac@palmerdabbelt-glaptop> (raw) In-Reply-To: <CAJF2gTTpurWpPUcA2JkF0rOFztKQgFBhOF9zQyuyi_-sxszhRQ@mail.gmail.com> On Fri, 04 Jun 2021 07:47:22 PDT (-0700), guoren@kernel.org wrote: > Hi Arnd & Palmer, > > Sorry for the delayed reply, I'm working on the next version of the patch. > > On Fri, Jun 4, 2021 at 5:56 PM Arnd Bergmann <arnd@arndb.de> wrote: >> >> On Thu, Jun 3, 2021 at 5:39 PM Palmer Dabbelt <palmer@dabbelt.com> wrote: >> > On Wed, 02 Jun 2021 23:00:29 PDT (-0700), Anup Patel wrote: >> > >> This implementation, which adds some Kconfig entries that control page table >> > >> bits, definately isn't suitable for upstream. Allowing users to set arbitrary >> > >> page table bits will eventually conflict with the standard, and is just going to >> > >> be a mess. It'll also lead to kernels that are only compatible with specific >> > >> designs, which we're trying very hard to avoid. At a bare minimum we'll need >> > >> some way to detect systems with these page table bits before setting them, >> > >> and some description of what the bits actually do so we can reason about >> > >> them. >> > > >> > > Yes, vendor specific Kconfig options are strict NO NO. We can't give-up the >> > > goal of unified kernel image for all platforms. > Okay, Agree. Please help review the next version of the patch. > >> > >> > I think this is just a phrasing issue, but just to be sure: >> > >> > IMO it's not that they're vendor-specific Kconfig options, it's that >> > turning them on will conflict with standard systems (and other vendors). >> > We've already got the ability to select sets of Kconfig settings that >> > will only boot on one vendor's system, which is fine, as long as there >> > remains a set of Kconfig settings that will boot on all systems. >> > >> > An example here would be the errata: every system has errata of some >> > sort, so if we start flipping off various vendor's errata Kconfigs >> > you'll end up with kernels that only function properly on some systems. >> > That's fine with me, as long as it's possible to turn on all vendor's >> > errata Kconfigs at the same time and the resulting kernel functions >> > correctly on all systems. >> >> Yes, this is generally the goal, and it would be great to have that >> working in a way where a 'defconfig' build just turns on all the options >> that are needed to use any SoC specific features and drivers while >> still working on all hardware. There are however limits you may run >> into at some point, and other architectures usually only manage to span >> some 10 to 15 years of hardware implementations with a single >> kernel before it get really hard. > I could follow the goal in the next version of the patchset. Please > help review, thx. IMO we're essentially here now with the RISC-V stuff: defconfig flips on everything necesasry to boot normal-smelling SOCs, with everything being detected as the system boots. We have some wacky configurations like !MMU and XIP that are coupled to the hardware, but (and sorry for crossing the other threads, I missed your pointer as it's early here) as I said in the other thread it might be time to make it explicit that those things are non-portable. The hope here has always been that we'd have enough in the standards that we could avoid a proliferation of vendor-specific code. We've always put a strong "things keep working forever" stake in the ground in RISC-V land, but that's largely been because we were counting on the standards existing that make support easy. In practice we don't have those standards so we're ending up with a fairly large software base that is required to support everything. We don't have all that much hardware right now so we'll have to see how it goes, but for now I'm in favor of keeping defconfig as a "boots on everything" sort of setup -- both because it makes life easier for users, and because it makes issues like the non-portable Kconfigs that showed up here quite explicit. If we get to 10/15 years of hardware then I'm sure we'll be removing old systems from defconfig (or maybe even the kernel entirely, a lot of this stuff isn't in production). I'm just hoping we make it that far ;) >> To give some common examples that make it break down: >> >> - 32-bit vs 64-bit already violates that rule on risc-v (as it does on >> most other architectures) Yes, and there's no way around that on RISC-V. They're different base ISAs therefor re-define the same instructions, so we're essentially at two kernel binaries by that point. The platform spec says rv64gc, so we can kind of punt on this one for now. If rv32 hardware shows up we'll probably want a standard system there too, which is why we've avoided coupling kernel portability to XLEN. >> - architectures that support both big-endian and little-endian kernels >> tend to have platforms that require one or the other (e.g. mips, >> though not arm). Not an issue for you. It is now! We've added big-endian to RISC-V. There's no hardware yet and very little software support. IMO the right answer is to ban that from the platform spec, but again it'll depnd on what vendors want to build (though anyone is listening, please don't make my life miserable ;)). >> - page table formats are the main cause of incompatibility: arm32 >> and x86-32 require three-level tables for certain features, but those >> are incompatible with older cores, arm64 supports three different >> page sizes, but none of them works on all cores (4KB almost works >> everywhere). We actually have some support on the works for multiple page table levels in a single binary, which should help with a lot of that incompatibility. I don't know of any plans to couple other page table features to the number of levels, though. >> - SMP-enabled ARMv7 kernels can be configured to run on either >> ARMv6 or ARMv8, but not both, in this case because of incompatible >> barrier instructions. Our barriers aren't quite split the same way, but we do have two memory models (RVWMO and TSO). IIUC we should be able to support both in the same kernels with some patching, but the resulting kernels would be biased towards one memory models over the other WRT performance. Again, we'll have to see what the vendors do and I'm hoping we don't end up with too many headaches. >> - 32-bit Arm has a couple more remaining features that require building >> a machine specific kernel if enabled because they hardcode physical >> addresses: early printk (debug_ll, not the normal earlycon), NOMMU, >> and XIP. We've got NOMMU and XIP as well, but we have some SBI support for early printk. IMO we're not really sure if we've decoupled all the PA layout dependencies yet from Linux, as we really only support one vendor's systems, but we've had a lot of work lately on beefing up our memory layout so with any luck we'll be able to quickly sort out anything that comes up. _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2021-06-04 16:12 UTC|newest] Thread overview: 124+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-05-19 5:04 [PATCH RFC 0/3] riscv: Add DMA_COHERENT support guoren 2021-05-19 5:04 ` guoren 2021-05-19 5:04 ` [PATCH RFC 1/3] riscv: pgtable.h: Fixup _PAGE_CHG_MASK usage guoren 2021-05-19 5:04 ` guoren 2021-05-19 5:04 ` [PATCH RFC 2/3] riscv: Add DMA_COHERENT for custom PTE attributes guoren 2021-05-19 5:04 ` guoren 2021-05-19 5:04 ` [PATCH RFC 3/3] riscv: Add SYNC_DMA_FOR_CPU/DEVICE for DMA_COHERENT guoren 2021-05-19 5:04 ` guoren 2021-05-19 6:32 ` Guo Ren 2021-05-19 6:32 ` Guo Ren 2021-05-19 6:32 ` Guo Ren 2021-05-19 5:20 ` [PATCH RFC 0/3] riscv: Add DMA_COHERENT support Christoph Hellwig 2021-05-19 5:20 ` Christoph Hellwig 2021-05-19 5:48 ` Guo Ren 2021-05-19 5:48 ` Guo Ren 2021-05-19 5:48 ` Guo Ren 2021-05-19 5:55 ` Christoph Hellwig 2021-05-19 5:55 ` Christoph Hellwig 2021-05-19 6:09 ` Guo Ren 2021-05-19 6:09 ` Guo Ren 2021-05-19 6:09 ` Guo Ren 2021-05-19 6:44 ` Drew Fustini 2021-05-19 6:44 ` Drew Fustini 2021-05-19 6:53 ` Christoph Hellwig 2021-05-19 6:53 ` Christoph Hellwig 2021-05-20 1:45 ` Guo Ren 2021-05-20 1:45 ` Guo Ren 2021-05-20 1:45 ` Guo Ren 2021-05-20 5:48 ` Christoph Hellwig 2021-05-20 5:48 ` Christoph Hellwig 2021-06-06 18:14 ` Nick Kossifidis 2021-06-06 18:14 ` Nick Kossifidis 2021-06-07 0:04 ` Guo Ren 2021-06-07 0:04 ` Guo Ren 2021-06-07 2:16 ` Nick Kossifidis 2021-06-07 2:16 ` Nick Kossifidis 2021-06-07 3:19 ` Guo Ren 2021-06-07 3:19 ` Guo Ren 2021-06-07 6:27 ` Christoph Hellwig 2021-06-07 6:27 ` Christoph Hellwig 2021-06-07 6:41 ` Guo Ren 2021-06-07 6:41 ` Guo Ren 2021-06-07 6:51 ` Christoph Hellwig 2021-06-07 6:51 ` Christoph Hellwig 2021-06-07 7:46 ` Guo Ren 2021-06-07 7:46 ` Guo Ren 2021-06-08 15:00 ` David Laight 2021-06-08 15:00 ` David Laight 2021-06-08 15:32 ` 'Christoph Hellwig' 2021-06-08 15:32 ` 'Christoph Hellwig' 2021-06-08 16:11 ` David Laight 2021-06-08 16:11 ` David Laight 2021-06-07 8:35 ` Nick Kossifidis 2021-06-07 8:35 ` Nick Kossifidis 2021-06-09 3:28 ` Guo Ren 2021-06-09 3:28 ` Guo Ren 2021-06-09 6:05 ` Jisheng Zhang 2021-06-09 6:05 ` Jisheng Zhang 2021-06-09 9:45 ` Nick Kossifidis 2021-06-09 9:45 ` Nick Kossifidis 2021-06-09 12:43 ` Guo Ren 2021-06-09 12:43 ` Guo Ren 2021-05-19 6:05 ` Guo Ren 2021-05-19 6:05 ` Guo Ren 2021-05-19 6:05 ` Guo Ren 2021-05-19 6:06 ` Christoph Hellwig 2021-05-19 6:06 ` Christoph Hellwig 2021-05-19 6:11 ` Guo Ren 2021-05-19 6:11 ` Guo Ren 2021-05-19 6:11 ` Guo Ren 2021-05-19 6:54 ` Drew Fustini 2021-05-19 6:54 ` Drew Fustini 2021-05-19 6:56 ` Christoph Hellwig 2021-05-19 6:56 ` Christoph Hellwig 2021-05-19 7:14 ` Anup Patel 2021-05-19 7:14 ` Anup Patel 2021-05-19 8:25 ` Damien Le Moal 2021-05-19 8:25 ` Damien Le Moal 2021-05-20 1:47 ` Guo Ren 2021-05-20 1:47 ` Guo Ren 2021-05-20 1:59 ` Guo Ren 2021-05-20 1:59 ` Guo Ren 2021-05-22 0:36 ` Guo Ren 2021-05-22 0:36 ` Guo Ren 2021-05-30 0:30 ` Palmer Dabbelt 2021-05-30 0:30 ` Palmer Dabbelt 2021-06-03 4:13 ` Palmer Dabbelt 2021-06-03 4:13 ` Palmer Dabbelt 2021-06-03 6:00 ` Anup Patel 2021-06-03 6:00 ` Anup Patel 2021-06-03 15:39 ` Palmer Dabbelt 2021-06-03 15:39 ` Palmer Dabbelt 2021-06-04 9:02 ` David Laight 2021-06-04 9:02 ` David Laight 2021-06-04 9:53 ` Arnd Bergmann 2021-06-04 9:53 ` Arnd Bergmann 2021-06-04 9:53 ` Arnd Bergmann 2021-06-04 14:47 ` Guo Ren 2021-06-04 14:47 ` Guo Ren 2021-06-04 16:12 ` Palmer Dabbelt [this message] 2021-06-04 16:12 ` Palmer Dabbelt 2021-06-04 21:26 ` Arnd Bergmann 2021-06-04 21:26 ` Arnd Bergmann 2021-06-04 21:26 ` Arnd Bergmann 2021-06-04 22:10 ` Palmer Dabbelt 2021-06-04 22:10 ` Palmer Dabbelt 2021-06-08 12:26 ` Guo Ren 2021-06-08 12:26 ` Guo Ren 2021-06-06 17:11 ` Guo Ren 2021-06-06 17:11 ` Guo Ren 2021-06-06 17:11 ` Guo Ren 2021-06-07 3:38 ` Anup Patel 2021-06-07 3:38 ` Anup Patel 2021-06-07 4:22 ` Guo Ren 2021-06-07 4:22 ` Guo Ren 2021-06-07 4:22 ` Guo Ren 2021-06-07 4:47 ` Anup Patel 2021-06-07 4:47 ` Anup Patel 2021-06-07 5:08 ` Guo Ren 2021-06-07 5:08 ` Guo Ren 2021-06-07 5:08 ` Guo Ren 2021-06-07 5:13 ` Guo Ren 2021-06-07 5:13 ` Guo Ren 2021-06-07 5:13 ` Guo Ren
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=mhng-423aeaad-9339-4695-9a85-f947dd6135ac@palmerdabbelt-glaptop \ --to=palmer@dabbelt.com \ --cc=Anup.Patel@wdc.com \ --cc=anup@brainfault.org \ --cc=arnd@arndb.de \ --cc=drew@beagleboard.org \ --cc=guoren@kernel.org \ --cc=guoren@linux.alibaba.com \ --cc=hch@lst.de \ --cc=lazyparser@gmail.com \ --cc=linux-arch@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-riscv@lists.infradead.org \ --cc=linux-sunxi@lists.linux.dev \ --cc=paul.walmsley@sifive.com \ --cc=wefu@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.