[PATCH] usb: dwc3: support 64 bit DMA in platform driver

[PATCH] usb: dwc3: support 64 bit DMA in platform driver

[Sven Peter]

Currently, the dwc3 platform driver does not explicitly ask for
a DMA mask. This makes it fall back to the default 32-bit mask which
breaks the driver on systems that only have RAM starting above the
first 4G like the Apple M1 SoC.

Fix this by using the same logic already present in xhci-plat.c:
First, try to set a coherent dma mask for 64-bit, and then attempt
again with a 32-bit mask if this fails.

Signed-off-by: Sven Peter <sven@svenpeter.dev>
---

I have taken the code directly from the xhci-plat.c driver so
I think this change should be fairly low risk.
Unfortunately I only have the Apple M1 to test this on but here
the driver still works with the iommu enabled which limits the
address space to 32 bit. It also enables to use this with the iommu
in bypass mode which requires 64 bit addresses.

I believe this has been working fine so far since the dwc3 driver
only uses a few very small buffers in host mode which might still
fit within the first 4G of address space on many devices. The
majority of DMA buffers are allocated inside the xhci driver which
will already call dma_set_mask_and_coherent.

Best,

Sven

 drivers/usb/dwc3/core.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
index b6e53d8212cd..ef6bb6aaffd8 100644
--- a/drivers/usb/dwc3/core.c
+++ b/drivers/usb/dwc3/core.c
@@ -1545,6 +1545,21 @@ static int dwc3_probe(struct platform_device *pdev)
 
 	dwc3_get_properties(dwc);
 
+	/* Try to set 64-bit DMA first */
+	if (WARN_ON(!dwc->sysdev->dma_mask))
+		/* Platform did not initialize dma_mask */
+		ret = dma_coerce_mask_and_coherent(dwc->sysdev,
+						   DMA_BIT_MASK(64));
+	else
+		ret = dma_set_mask_and_coherent(dwc->sysdev, DMA_BIT_MASK(64));
+
+	/* If seting 64-bit DMA mask fails, fall back to 32-bit DMA mask */
+	if (ret) {
+		ret = dma_set_mask_and_coherent(dwc->sysdev, DMA_BIT_MASK(32));
+		if (ret)
+			return ret;
+	}
+
 	dwc->reset = devm_reset_control_array_get_optional_shared(dev);
 	if (IS_ERR(dwc->reset))
 		return PTR_ERR(dwc->reset);
-- 
2.25.1

Re: [PATCH] usb: dwc3: support 64 bit DMA in platform driver

[Greg Kroah-Hartman]

On Sun, Jun 06, 2021 at 11:36:29AM +0200, Sven Peter wrote:
> Currently, the dwc3 platform driver does not explicitly ask for
> a DMA mask. This makes it fall back to the default 32-bit mask which
> breaks the driver on systems that only have RAM starting above the
> first 4G like the Apple M1 SoC.
> 
> Fix this by using the same logic already present in xhci-plat.c:
> First, try to set a coherent dma mask for 64-bit, and then attempt
> again with a 32-bit mask if this fails.
> 
> Signed-off-by: Sven Peter <sven@svenpeter.dev>
> ---
> 
> I have taken the code directly from the xhci-plat.c driver so
> I think this change should be fairly low risk.
> Unfortunately I only have the Apple M1 to test this on but here
> the driver still works with the iommu enabled which limits the
> address space to 32 bit. It also enables to use this with the iommu
> in bypass mode which requires 64 bit addresses.
> 
> I believe this has been working fine so far since the dwc3 driver
> only uses a few very small buffers in host mode which might still
> fit within the first 4G of address space on many devices. The
> majority of DMA buffers are allocated inside the xhci driver which
> will already call dma_set_mask_and_coherent.
> 
> Best,
> 
> Sven
> 
>  drivers/usb/dwc3/core.c | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
> 
> diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
> index b6e53d8212cd..ef6bb6aaffd8 100644
> --- a/drivers/usb/dwc3/core.c
> +++ b/drivers/usb/dwc3/core.c
> @@ -1545,6 +1545,21 @@ static int dwc3_probe(struct platform_device *pdev)
>  
>  	dwc3_get_properties(dwc);
>  
> +	/* Try to set 64-bit DMA first */
> +	if (WARN_ON(!dwc->sysdev->dma_mask))

This will cause systems to reboot if they have panic-on-warn set.  Are
you sure you want that to happen?

I know you copied this from xhci-plat, but let's not duplicate bugs
please :)

thnaks,

greg k-h

Re: [PATCH] usb: dwc3: support 64 bit DMA in platform driver

[Sven Peter]

On Sun, Jun 6, 2021, at 11:52, Greg Kroah-Hartman wrote:
> On Sun, Jun 06, 2021 at 11:36:29AM +0200, Sven Peter wrote:
> > Currently, the dwc3 platform driver does not explicitly ask for
> > a DMA mask. This makes it fall back to the default 32-bit mask which
> > breaks the driver on systems that only have RAM starting above the
> > first 4G like the Apple M1 SoC.
> > 
> > Fix this by using the same logic already present in xhci-plat.c:
> > First, try to set a coherent dma mask for 64-bit, and then attempt
> > again with a 32-bit mask if this fails.
> > 
> > Signed-off-by: Sven Peter <sven@svenpeter.dev>
> > ---
> > 
> > I have taken the code directly from the xhci-plat.c driver so
> > I think this change should be fairly low risk.
> > Unfortunately I only have the Apple M1 to test this on but here
> > the driver still works with the iommu enabled which limits the
> > address space to 32 bit. It also enables to use this with the iommu
> > in bypass mode which requires 64 bit addresses.
> > 
> > I believe this has been working fine so far since the dwc3 driver
> > only uses a few very small buffers in host mode which might still
> > fit within the first 4G of address space on many devices. The
> > majority of DMA buffers are allocated inside the xhci driver which
> > will already call dma_set_mask_and_coherent.
> > 
> > Best,
> > 
> > Sven
> > 
> >  drivers/usb/dwc3/core.c | 15 +++++++++++++++
> >  1 file changed, 15 insertions(+)
> > 
> > diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
> > index b6e53d8212cd..ef6bb6aaffd8 100644
> > --- a/drivers/usb/dwc3/core.c
> > +++ b/drivers/usb/dwc3/core.c
> > @@ -1545,6 +1545,21 @@ static int dwc3_probe(struct platform_device *pdev)
> >  
> >  	dwc3_get_properties(dwc);
> >  
> > +	/* Try to set 64-bit DMA first */
> > +	if (WARN_ON(!dwc->sysdev->dma_mask))
> 
> This will cause systems to reboot if they have panic-on-warn set.  Are
> you sure you want that to happen?

Probably not :) I'll remove the WARN_ON.

thanks,

Sven

Re: [PATCH] usb: dwc3: support 64 bit DMA in platform driver

[Christoph Hellwig]

On Sun, Jun 06, 2021 at 11:36:29AM +0200, Sven Peter wrote:
> +	/* Try to set 64-bit DMA first */
> +	if (WARN_ON(!dwc->sysdev->dma_mask))
> +		/* Platform did not initialize dma_mask */
> +		ret = dma_coerce_mask_and_coherent(dwc->sysdev,
> +						   DMA_BIT_MASK(64));
> +	else
> +		ret = dma_set_mask_and_coherent(dwc->sysdev, DMA_BIT_MASK(64));

WARN_ON + fallback seems weird.  I'd suggest to just error out for the
warn case.

> +	/* If seting 64-bit DMA mask fails, fall back to 32-bit DMA mask */
> +	if (ret) {
> +		ret = dma_set_mask_and_coherent(dwc->sysdev, DMA_BIT_MASK(32));
> +		if (ret)
> +			return ret;
> +	}

Setting a 64-bit mask will not fail.  No need for the fallback.