Re: [PATCH v7 14/15] dt-bindings: of: Add restricted DMA pool

From: Will Deacon <will@kernel.org>
To: Claire Chang <tientzu@chromium.org>
Cc: Jim Quinlan <james.quinlan@broadcom.com>,
	heikki.krogerus@linux.intel.com,
	linux-devicetree <devicetree@vger.kernel.org>,
	peterz@infradead.org, dri-devel@lists.freedesktop.org,
	chris@chris-wilson.co.uk, grant.likely@arm.com, paulus@samba.org,
	Frank Rowand <frowand.list@gmail.com>,
	Christoph Hellwig <hch@lst.de>, Jianxiong Gao <jxgao@google.com>,
	sstabellini@kernel.org, Saravana Kannan <saravanak@google.com>,
	mpe@ellerman.id.au,
	"Rafael J . Wysocki" <rafael.j.wysocki@intel.com>,
	mingo@kernel.org, Bartosz Golaszewski <bgolaszewski@baylibre.com>,
	bskeggs@redhat.com, linux-pci@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	Thierry Reding <treding@nvidia.com>,
	matthew.auld@intel.com, Nicolas Boichat <drinkcat@chromium.org>,
	thomas.hellstrom@linux.intel.com, jgross@suse.com,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	intel-gfx@lists.freedesktop.org,
	Robin Murphy <robin.murphy@arm.com>,
	Rob Herring <robh+dt@kernel.org>,
	rodrigo.vivi@intel.com, Bjorn Helgaas <bhelgaas@google.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
	boris.ostrovsky@oracle.com, airlied@linux.ie,
	Greg KH <gregkh@linuxfoundation.org>,
	Randy Dunlap <rdunlap@infradead.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"list@263.net:IOMMU DRIVERS" <iommu@lists.linux-foundation.org>,
	xypron.glpk@gmx.de, linuxppc-dev@lists.ozlabs.org,
	bauerman@linux.ibm.com
Subject: Re: [PATCH v7 14/15] dt-bindings: of: Add restricted DMA pool
Date: Thu, 27 May 2021 12:34:57 +0100	[thread overview]
Message-ID: <20210527113456.GA22019@willie-the-truck> (raw)
In-Reply-To: <CALiNf2_sVXnb97++yWusB5PWz8Pzfn9bCKZc6z3tY4bx6-nW8w@mail.gmail.com>

On Thu, May 27, 2021 at 07:29:20PM +0800, Claire Chang wrote:
> On Wed, May 26, 2021 at 11:53 PM Will Deacon <will@kernel.org> wrote:
> >
> > On Wed, May 26, 2021 at 01:13:22PM +0100, Will Deacon wrote:
> > > On Tue, May 18, 2021 at 02:42:14PM +0800, Claire Chang wrote:
> > > > @@ -138,4 +160,9 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB).
> > > >             memory-region = <&multimedia_reserved>;
> > > >             /* ... */
> > > >     };
> > > > +
> > > > +   pcie_device: pcie_device@0,0 {
> > > > +           memory-region = <&restricted_dma_mem_reserved>;
> > > > +           /* ... */
> > > > +   };
> > >
> > > I still don't understand how this works for individual PCIe devices -- how
> > > is dev->of_node set to point at the node you have above?
> > >
> > > I tried adding the memory-region to the host controller instead, and then
> > > I see it crop up in dmesg:
> > >
> > >   | pci-host-generic 40000000.pci: assigned reserved memory node restricted_dma_mem_reserved
> > >
> > > but none of the actual PCI devices end up with 'dma_io_tlb_mem' set, and
> > > so the restricted DMA area is not used. In fact, swiotlb isn't used at all.
> > >
> > > What am I missing to make this work with PCIe devices?
> >
> > Aha, looks like we're just missing the logic to inherit the DMA
> > configuration. The diff below gets things working for me.
> 
> I guess what was missing is the reg property in the pcie_device node.
> Will update the example dts.

Thanks. I still think something like my diff makes sense, if you wouldn't mind including
it, as it allows restricted DMA to be used for situations where the PCIe
topology is not static.

Perhaps we should prefer dev->of_node if it exists, but then use the node
of the host bridge's parent node otherwise?

Will