Re: [PATCH 1/2] [RFC] ata: ahci: Respect bus DMA constraints

From: Geert Uytterhoeven <geert@linux-m68k.org>
To: Marek Vasut <marek.vasut@gmail.com>
Cc: Christoph Hellwig <hch@lst.de>,
	Robin Murphy <robin.murphy@arm.com>,
	linux-ide@vger.kernel.org, linux-nvme@lists.infradead.org,
	Marek Vasut <marek.vasut+renesas@gmail.com>,
	Geert Uytterhoeven <geert+renesas@glider.be>,
	Jens Axboe <axboe@fb.com>, Jens Axboe <axboe@kernel.dk>,
	Keith Busch <keith.busch@intel.com>,
	Sagi Grimberg <sagi@grimberg.me>,
	Wolfram Sang <wsa+renesas@sang-engineering.com>,
	Linux-Renesas <linux-renesas-soc@vger.kernel.org>
Subject: Re: [PATCH 1/2] [RFC] ata: ahci: Respect bus DMA constraints
Date: Sun, 17 Mar 2019 11:24:22 +0100	[thread overview]
Message-ID: <CAMuHMdU5S8UGctWR8+4sFqgQxrkoH71_ZzZ2ENFhrPc+svfGLA@mail.gmail.com> (raw)
In-Reply-To: <3b665597-a616-70fc-8cd0-dfde236fe669@gmail.com>

Hi Marek,

On Sat, Mar 16, 2019 at 10:26 PM Marek Vasut <marek.vasut@gmail.com> wrote:
> On 3/13/19 7:30 PM, Christoph Hellwig wrote:
> > On Sat, Mar 09, 2019 at 12:23:15AM +0100, Marek Vasut wrote:
> >> On 3/8/19 8:18 AM, Christoph Hellwig wrote:
> >>> On Thu, Mar 07, 2019 at 12:14:06PM +0100, Marek Vasut wrote:
> >>>>> Right, but whoever *interprets* the device masks after the driver has
> >>>>> overridden them should be taking the (smaller) bus mask into account as
> >>>>> well, so the question is where is *that* not being done correctly?
> >>>>
> >>>> Do you have a hint where I should look for that ?
> >>>
> >>> If this a 32-bit ARM platform it might the complete lack of support
> >>> for bus_dma_mask in arch/arm/mm/dma-mapping.c..
> >>
> >> It's an ARM 64bit platform, just the PCIe controller is limited to 32bit
> >> address range, so the devices on the PCIe bus cannot read the host's
> >> DRAM above the 32bit limit.
> >
> > arm64 should take the mask into account both for the swiotlb and
> > iommu case.  What are the exact symptoms you see?
>
> With the nvme, the device is recognized, but cannot be used.
> It boils down to PCI BAR access being possible, since that's all below
> the 32bit boundary, but when the device tries to do any sort of DMA,
> that transfer returns nonsense data.
>
> But when I call dma_set_mask_and_coherent(dev->dev, DMA_BIT_MASK(32) in
> the affected driver (thus far I tried this nvme, xhci-pci and ahci-pci
> drivers), it all starts to work fine.
>
> Could it be that the driver overwrites the (coherent_)dma_mask and
> that's why the swiotlb/iommu code cannot take this into account ?
>
> > Does it involve
> > swiotlb not kicking in, or iommu issues?
>
> How can I check ? I added printks into arch/arm64/mm/dma-mapping.c and
> drivers/iommu/dma-iommu.c , but I suspect I need to look elsewhere.

So far the IOMMU is disabled in upstream, as no devices are whitelisted
in drivers/iommu/ipmmu-vmsa.c.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds