Re: [RFC 3/5] dma-mapping: Enable global non-coherent pool support for RISC-V

From: Atish Patra <atishp@atishpatra.org>
To: Christoph Hellwig <hch@lst.de>
Cc: Atish Patra <atish.patra@wdc.com>,
	devicetree <devicetree@vger.kernel.org>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Tobias Klauser <tklauser@distanz.ch>,
	Robin Murphy <robin.murphy@arm.com>,
	"linux-kernel@vger.kernel.org List"
	<linux-kernel@vger.kernel.org>, Rob Herring <robh+dt@kernel.org>,
	iommu@lists.linux-foundation.org,
	Guo Ren <guoren@linux.alibaba.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	linux-riscv <linux-riscv@lists.infradead.org>,
	Frank Rowand <frowand.list@gmail.com>,
	Dmitry Vyukov <dvyukov@google.com>
Subject: Re: [RFC 3/5] dma-mapping: Enable global non-coherent pool support for RISC-V
Date: Mon, 2 Aug 2021 11:22:42 -0700	[thread overview]
Message-ID: <CAOnJCUJgCHOR5nifNVjkQawKniGf81Zvsh226brdAPuLNkXAdQ@mail.gmail.com> (raw)
In-Reply-To: <20210727085244.GA20609@lst.de>

On Tue, Jul 27, 2021 at 1:52 AM Christoph Hellwig <hch@lst.de> wrote:
>
> On Mon, Jul 26, 2021 at 03:47:54PM -0700, Atish Patra wrote:
> > arch_dma_set_uncached works as well in this case. However, mips,
> > niops2 & xtensa uses a
> > fixed (via config) value for the offset. Similar approach can't be
> > used here because the platform specific
> > offset value has to be determined at runtime so that a single kernel
> > image can boot on all platforms.
>
> Nothing in the interface requires a fixed offset.  And using the offset
> has one enormous advantage in that there is no need to declare a
> statically sized pool - allocations are fully dynamic.  And any kind of
> fixed pool tends to cause huge problems.
>
> > 1. a new DT property so that arch specific code is aware of the
> > non-cacheable window offset.
>
> Yes.
>
> > individual device if a per-device non-cacheable
> >    window support is required in future. As of now, the beagleV memory
>
> If you require a per-device noncachable area you can use the per-device
> coherent pools.  But why would you want that?
>
> > region lies in 0x10_0000_00000 - x17_FFFF_FFFF
> >    which is mapped to start of DRAM 0x80000000. All of the
> > non-coherent devices can do 32bit DMA only.
>
> Adjust ZONE_DMA32 so that it takes the uncached offset into account.
>
> > > > -     mem = dma_init_coherent_memory(phys_addr, phys_addr, size, true);
> > > > +     if (phys_addr == device_addr)
> > > > +             mem = dma_init_coherent_memory(phys_addr, device_addr, size, true);
> > > > +     else
> > > > +             mem = dma_init_coherent_memory(phys_addr, device_addr, size, false);
> > >
> > > Nak.  The phys_addr != device_addr support is goign away.  This needs
> >
> > ok.
> >
> > > to be filled in using dma-ranges property hanging of the struct device.
> >
> > struct device is only accessible in rmem_dma_device_init. I couldn't
> > find a proper way to access it during
> > dma_reserved_default_memory setup under global pool.
> >
> > Does that mean we should use a per-device memory pool instead of a
> > global non-coherent pool ?
>
> Indeed, that would be a problem in this case.  But if we can just
> use the uncached offset directly I think everything will be much
> simpler.

Yes. I was planning to change this to use an uncached offset.
However, the planned mass production for beaglev starlight sbc is
cancelled now [1].
As there is no other board that requires an uncached offset, I don't
think there is no usecase
for adding uncached offset support for RISC-V right now. I will
revisit(hopefully we don't have to)
this in case any platform implements uncached window support in future.

[1] https://www.cnx-software.com/2021/07/31/beaglev-starlight-sbc-wont-be-mass-manufactured-redesigned-beaglev-risc-v-sbc-expected-in-q1-2022/
-- 
Regards,
Atish