From: Catalin Marinas <catalin.marinas@arm.com>
To: Arnd Bergmann <arnd@arndb.de>
Cc: "linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
Will Deacon <Will.Deacon@arm.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
Ding Tianhong <dingtianhong@huawei.com>
Subject: Re: For the problem when using swiotlb
Date: Fri, 21 Nov 2014 11:36:18 +0000 [thread overview]
Message-ID: <20141121113618.GD19783@e104818-lin.cambridge.arm.com> (raw)
In-Reply-To: <2310899.ejrSzj6BIr@wuerfel>
On Fri, Nov 21, 2014 at 11:26:45AM +0000, Arnd Bergmann wrote:
> On Friday 21 November 2014 11:06:10 Catalin Marinas wrote:
> > On Wed, Nov 19, 2014 at 03:56:42PM +0000, Arnd Bergmann wrote:
> > > On Wednesday 19 November 2014 15:46:35 Catalin Marinas wrote:
> > > > Going back to original topic, the dma_supported() function on arm64
> > > > calls swiotlb_dma_supported() which actually checks whether the swiotlb
> > > > bounce buffer is within the dma mask. This transparent bouncing (unlike
> > > > arm32 where it needs to be explicit) is not always optimal, though
> > > > required for 32-bit only devices on a 64-bit system. The problem is when
> > > > the driver is 64-bit capable but forgets to call
> > > > dma_set_mask_and_coherent() (that's not the only question I got about
> > > > running out of swiotlb buffers).
> > >
> > > I think it would be nice to warn once per device that starts using the
> > > swiotlb. Really all 32-bit DMA masters should have a proper IOMMU
> > > attached.
> >
> > It would be nice to have a dev_warn_once().
> >
> > I think it makes sense on arm64 to avoid swiotlb bounce buffers for
> > coherent allocations altogether. The __dma_alloc_coherent() function
> > already checks coherent_dma_mask and sets GFP_DMA accordingly. If we
> > have a device that cannot even cope with a 32-bit ZONE_DMA, we should
> > just not support DMA at all on it (without an IOMMU). The arm32
> > __dma_supported() has a similar check.
>
> If we ever encounter this case, we may have to add a smaller ZONE_DMA
> and use ZONE_DMA32 for the normal dma allocations.
Traditionally on x86 I think ZONE_DMA was for ISA and ZONE_DMA32 had to
cover the 32-bit physical address space. On arm64 we don't expect ISA,
so we only use ZONE_DMA (which is 4G, similar to IA-64, sparc). We had
ZONE_DMA32 originally but it broke swiotlb which assumes ZONE_DMA for
its bounce buffer.
> > Swiotlb is still required for the streaming DMA since we get bouncing
> > for pages allocated outside the driver control (e.g. VFS layer which
> > doesn't care about GFP_DMA), hoping a 16M bounce buffer would be enough.
> >
> > Ding seems to imply that CMA fixes the problem, which means that the
> > issue is indeed coherent allocations.
>
> I wonder what's going on here, since swiotlb_alloc_coherent() actually
> tries a regular __get_free_pages(flags, order) first, and when ZONE_DMA
> is set here, it just work without using the pool.
As long as coherent_dma_mask is sufficient for ZONE_DMA. I have no idea
what this mask is set to in Ding's case (but I've seen the problem
previously with an out of tree driver where coherent_dma_mask was some
random number; so better reporting here would help).
There could be another case where dma_pfn_offset is required but let's
wait for some more info from Ding (ZONE_DMA is 32-bit from the start of
RAM which could be 40-bit like on Seattle, so basically such devices
would need to set dma_pfn_offset).
--
Catalin
next prev parent reply other threads:[~2014-11-21 11:36 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-11-17 11:56 For the problem when using swiotlb Ding Tianhong
2014-11-17 12:18 ` Arnd Bergmann
2014-11-17 18:09 ` Catalin Marinas
2014-11-19 3:17 ` Ding Tianhong
2014-11-19 8:45 ` Arnd Bergmann
2014-11-19 11:29 ` Catalin Marinas
2014-11-19 12:48 ` Arnd Bergmann
2014-11-19 15:46 ` Catalin Marinas
2014-11-19 15:56 ` Arnd Bergmann
2014-11-21 11:06 ` Catalin Marinas
2014-11-21 11:26 ` Arnd Bergmann
2014-11-21 11:36 ` Catalin Marinas [this message]
2014-11-21 12:27 ` Arnd Bergmann
2014-11-20 2:57 ` Ding Tianhong
2014-11-20 7:40 ` Arnd Bergmann
2014-11-20 8:34 ` Ding Tianhong
2014-11-20 9:02 ` Arnd Bergmann
2014-11-20 9:21 ` Ding Tianhong
2014-11-21 9:35 ` Catalin Marinas
2014-11-21 10:32 ` Catalin Marinas
2014-11-21 12:48 ` Arnd Bergmann
2014-11-21 16:57 ` Catalin Marinas
2014-11-21 17:04 ` Arnd Bergmann
2014-11-21 17:51 ` Catalin Marinas
2014-11-21 18:09 ` Catalin Marinas
2014-11-24 20:12 ` Arnd Bergmann
2014-11-25 10:58 ` Catalin Marinas
2014-11-25 11:29 ` Russell King - ARM Linux
2014-11-25 12:23 ` Catalin Marinas
2014-11-27 2:36 ` Ding Tianhong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20141121113618.GD19783@e104818-lin.cambridge.arm.com \
--to=catalin.marinas@arm.com \
--cc=Will.Deacon@arm.com \
--cc=arnd@arndb.de \
--cc=dingtianhong@huawei.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).