From: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
To: Amit Pundir <amit.pundir@linaro.org>
Cc: jeremy.linton@arm.com, iommu@lists.linux-foundation.org,
linux-rpi-kernel@lists.infradead.org,
David Rientjes <rientjes@google.com>,
Robin Murphy <robin.murphy@arm.com>,
Christoph Hellwig <hch@lst.de>
Subject: Re: dma-pool fixes
Date: Fri, 31 Jul 2020 12:47:12 +0200 [thread overview]
Message-ID: <a0c8eb70cedb05e310c2957a1c0f7968c1e39d53.camel@suse.de> (raw)
In-Reply-To: <CAMi1Hd3QNtZVEFUgDprT==wcVHKv6TsvpT3RurVJTDb1op2LnA@mail.gmail.com>
Hi Amit,
On Wed, 2020-07-29 at 17:52 +0530, Amit Pundir wrote:
> On Wed, 29 Jul 2020 at 16:15, Nicolas Saenz Julienne
> <nsaenzjulienne@suse.de> wrote:
> > On Tue, 2020-07-28 at 17:30 +0200, Christoph Hellwig wrote:
> > > On Tue, Jul 28, 2020 at 06:18:41PM +0530, Amit Pundir wrote:
> > > > > Oh well, this leaves me confused again. It looks like your setup
> > > > > really needs a CMA in zone normal for the dma or dma32 pool.
> > > >
> > > > Anything I should look up in the downstream kernel/dts?
> > >
> > > I don't have a good idea right now. Nicolas, can you think of something
> > > else?
> >
> > To summarise, the device is:
> > - Using the dma-direct code path.
> > - Requesting ZONE_DMA memory to then fail when provided memory falls in
> > ZONE_DMA. Actually, the only acceptable memory comes from CMA, which is
> > located topmost of the 4GB boundary.
> >
> > My wild guess is that we may be abusing an iommu identity mapping setup by
> > firmware.
> >
> > That said, what would be helpful to me is to find out the troublesome device.
> > Amit, could you try adding this patch along with Christoph's modified series
> > (so the board boots). Ultimately DMA atomic allocations are not that common, so
> > we should get only a few hits:
>
> Hi, still not hitting dma_alloc_from_pool().
Sorry I insisted, but not hitting the atomic path makes the issue even harder
to understand.
> I hit the following direct alloc path only once, at starting:
>
> dma_alloc_coherent ()
> -> dma_alloc_attrs()
> -> dma_is_direct() -> dma_direct_alloc()
> -> dma_direct_alloc_pages()
> -> dma_should_alloc_from_pool() #returns FALSE from here
>
> After that I'm hitting following iommu dma alloc path all the time:
>
> dma_alloc_coherent()
> -> dma_alloc_attrs()
> -> (ops->alloc) -> iommu_dma_alloc()
> -> iommu_dma_alloc_remap() #always returns from here
>
> So dma_alloc_from_pool() is not getting called at all in either of the
> above cases.
Ok, so lets see who's doing what and with what constraints:
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 4959f5df21bd..d28b3e4b91d3 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -594,6 +594,9 @@ static void *iommu_dma_alloc_remap(struct device *dev, size_t size,
dma_addr_t iova;
void *vaddr;
+ dev_info(dev, "%s, bus_dma_limit %llx, dma_mask %llx, coherent_dma_mask %llx, in irq %lu, size %lu, gfp %x, attrs %lx\n",
+ __func__, dev->bus_dma_limit, *dev->dma_mask, dev->coherent_dma_mask, in_interrupt(), size, gfp, attrs);
+
*dma_handle = DMA_MAPPING_ERROR;
if (unlikely(iommu_dma_deferred_attach(dev, domain)))
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index bb0041e99659..e5474e709e7b 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -160,6 +160,9 @@ void *dma_direct_alloc_pages(struct device *dev, size_t size,
size = PAGE_ALIGN(size);
+ dev_info(dev, "%s, bus_dma_limit %llx, dma_mask %llx, coherent_dma_mask %llx, in irq %lu, size %lu, gfp %x, attrs %lx\n",
+ __func__, dev->bus_dma_limit, *dev->dma_mask, dev->coherent_dma_mask, in_interrupt(), size, gfp, attrs);
+
if (dma_should_alloc_from_pool(dev, gfp, attrs)) {
ret = dma_alloc_from_pool(dev, size, &page, gfp);
if (!ret)
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2020-07-31 10:47 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-28 10:47 dma-pool fixes Christoph Hellwig
2020-07-28 10:47 ` [PATCH 1/2] dma-pool: fix coherent pool allocations for IOMMU mappings Christoph Hellwig
2020-07-28 14:56 ` Nicolas Saenz Julienne
2020-07-28 15:28 ` Christoph Hellwig
2020-07-28 10:47 ` [PATCH 2/2] dma-pool: Only allocate from CMA when in same memory zone Christoph Hellwig
2020-07-28 12:02 ` dma-pool fixes Amit Pundir
2020-07-28 12:07 ` Christoph Hellwig
2020-07-28 12:25 ` Amit Pundir
2020-07-28 12:41 ` Christoph Hellwig
2020-07-28 12:48 ` Amit Pundir
2020-07-28 15:30 ` Christoph Hellwig
2020-07-29 10:45 ` Nicolas Saenz Julienne
2020-07-29 12:22 ` Amit Pundir
2020-07-31 7:46 ` Amit Pundir
2020-07-31 13:09 ` Christoph Hellwig
2020-07-31 14:15 ` Amit Pundir
2020-07-31 19:04 ` David Rientjes via iommu
2020-08-01 8:20 ` David Rientjes via iommu
2020-08-01 8:57 ` revert scope for 5.8, was " Christoph Hellwig
2020-08-01 11:57 ` Amit Pundir
2020-08-01 16:59 ` Nicolas Saenz Julienne
2020-08-01 17:39 ` Christoph Hellwig
2020-08-02 4:46 ` Amit Pundir
2020-08-02 15:04 ` Amit Pundir
2020-08-03 4:14 ` David Rientjes via iommu
2020-08-03 6:44 ` Christoph Hellwig
2020-08-03 18:30 ` David Rientjes via iommu
2020-08-01 18:28 ` Linus Torvalds
2020-08-02 4:35 ` Amit Pundir
2020-08-01 8:29 ` Christoph Hellwig
2020-07-31 10:47 ` Nicolas Saenz Julienne [this message]
2020-07-31 11:17 ` Amit Pundir
2020-07-31 14:15 ` Nicolas Saenz Julienne
2020-07-31 14:20 ` Amit Pundir
2020-08-01 7:34 ` Amit Pundir
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a0c8eb70cedb05e310c2957a1c0f7968c1e39d53.camel@suse.de \
--to=nsaenzjulienne@suse.de \
--cc=amit.pundir@linaro.org \
--cc=hch@lst.de \
--cc=iommu@lists.linux-foundation.org \
--cc=jeremy.linton@arm.com \
--cc=linux-rpi-kernel@lists.infradead.org \
--cc=rientjes@google.com \
--cc=robin.murphy@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).