Linux-Block Archive on lore.kernel.org
 help / color / Atom feed
From: Biju Das <biju.das@bp.renesas.com>
To: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>,
	Christoph Hellwig <hch@lst.de>
Cc: Linux-Renesas <linux-renesas-soc@vger.kernel.org>,
	"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>,
	"linux-usb@vger.kernel.org" <linux-usb@vger.kernel.org>
Subject: RE: How to resolve an issue in swiotlb environment?
Date: Mon, 10 Jun 2019 07:31:34 +0000
Message-ID: <OSBPR01MB21038C8C8186797D9EC18489B8130@OSBPR01MB2103.jpnprd01.prod.outlook.com> (raw)
In-Reply-To: <OSAPR01MB3089D50DBDAA6C7D427E72EED8100@OSAPR01MB3089.jpnprd01.prod.outlook.com>

Hi All,

Any update on the below issue. I am seeing similar issue on RZ/G2M board with Linux version 5.2.0-rc3.

root@hihope-rz-g2m:~# [   35.414177] usb 2-1: new SuperSpeed Gen 1 USB device number 2 using xhci-hcd
[   35.449402] usb-storage 2-1:1.0: USB Mass Storage device detected
[   35.455915] scsi host0: usb-storage 2-1:1.0
[   36.482585] scsi 0:0:0:0: Direct-Access     SanDisk  Extreme          0001 PQ: 0 ANSI: 6
[   36.491260] sd 0:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[   36.499823] sd 0:0:0:0: [sda] Write Protect is off
[   36.505474] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   36.518074]  sda: sda1
[   36.523163] sd 0:0:0:0: [sda] Attached SCSI disk

root@hihope-rz-g2m:~# mkdir -p /tmp/rmnt/sda1
root@hihope-rz-g2m:~# mount -t auto /dev/sda1 /tmp/rmnt/sda1
root@hihope-rz-g2m:~# dd if=/dev/urandom of=/tmp/sda1-random bs=1024 count=10240
10240+0 records in
10240+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0.187696 s, 55.9 MB/s
root@hihope-rz-g2m:~# cp /tmp/sda1-random /tmp/rmnt/sda1/sda1-random
root@hihope-rz-g2m:~# [  218.861212] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  218.871885] xhci-hcd ee000000.usb: overflow 0x000000067430b000+1003520 of DMA mask ffffffff bus mask 0
[  218.881233] WARNING: CPU: 0 PID: 258 at kernel/dma/direct.c:43 report_addr+0x38/0xa8
[  218.888974] Modules linked in: renesas_usb3 usb_dmac phy_rcar_gen3_usb3
[  218.895594] CPU: 0 PID: 258 Comm: usb-storage Not tainted 5.2.0-rc3-00017-gc80b083-dirty #5
[  218.903940] Hardware name: HopeRun HiHope RZ/G2M with sub board (DT)
[  218.910291] pstate: 40000005 (nZcv daif -PAN -UAO)
[  218.915078] pc : report_addr+0x38/0xa8
[  218.918821] lr : report_addr+0xa0/0xa8
[  218.922564] sp : ffff0000125fb970
[  218.925872] x29: ffff0000125fb970 x28: 0000000000000000
[  218.931180] x27: 0000000000000000 x26: 000000001f020280
[  218.936487] x25: ffff8006394a82a8 x24: 0000000000000000
[  218.941794] x23: 0000000000000001 x22: 0000000000000000
[  218.947101] x21: 00000000000f5000 x20: ffff000011309000
[  218.952408] x19: ffff80063a600010 x18: ffffffffffffffff
[  218.957714] x17: 0000000000000000 x16: 0000000000000000
[  218.963023] x15: ffff0000113096c8 x14: 4d4420666f203032
[  218.968331] x13: 35333030312b3030 x12: 3062303334373630
[  218.973638] x11: 3030303030307830 x10: ffff000011309f20
[  218.978945] x9 : ffff0000112e3018 x8 : 0000000000000123
[  218.984252] x7 : 0000000000000005 x6 : ffff80063b578180
[  218.989559] x5 : ffff80063b578180 x4 : 0000000000000000
[  218.994865] x3 : ffff80063b57ef10 x2 : eed25f279b69f300
[  219.000172] x1 : eed25f279b69f300 x0 : 0000000000000000
[  219.005481] Call trace:
[  219.007923]  report_addr+0x38/0xa8
[  219.011321]  dma_direct_map_page+0x148/0x158
[  219.015586]  dma_direct_map_sg+0x78/0xe0
[  219.019510]  usb_hcd_map_urb_for_dma+0x2fc/0x468
[  219.024124]  xhci_map_urb_for_dma+0x54/0x68
[  219.028303]  usb_hcd_submit_urb+0x88/0x968
[  219.032394]  usb_submit_urb+0x3b0/0x570
[  219.036226]  usb_sg_wait+0x98/0x158
[  219.039711]  usb_stor_bulk_transfer_sglist.part.3+0x94/0x128
[  219.045366]  usb_stor_bulk_srb+0x48/0x88
[  219.049283]  usb_stor_Bulk_transport+0x10c/0x390
[  219.053896]  usb_stor_invoke_transport+0x3c/0x500
[  219.058595]  usb_stor_transparent_scsi_command+0xc/0x18
[  219.063816]  usb_stor_control_thread+0x1c4/0x260
[  219.068431]  kthread+0x124/0x128
[  219.071660]  ret_from_fork+0x10/0x18
[  219.075229] ---[ end trace dd9ef2a6b7fef860 ]---
[  219.080087] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  219.090810] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  219.101510] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  219.112209] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  219.122901] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  219.133591] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  219.144283] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  219.154973] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  219.165674] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  223.861717] swiotlb_tbl_map_single: 67451 callbacks suppressed
[  223.861721] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  223.878249] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  223.888940] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  223.899630] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  223.910318] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  223.921005] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  223.931695] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  223.942387] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  223.953077] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  223.963765] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  228.865664] swiotlb_tbl_map_single: 70409 callbacks suppressed
[  228.865668] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  228.882188] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  228.892878] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  228.903567] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  228.914256] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  228.924944] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  228.935636] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  228.946326] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  228.957015] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)
[  228.967705] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 1003520 bytes), total 32768 (slots), used 1088 (slots)

Regards,
Biju

> Subject: RE: How to resolve an issue in swiotlb environment?
> 
> Hi Christoph,
> 
> I think we should continue to discuss on this email thread instead of the fixed
> DMA-API.txt patch [1]
> 
> [1]
> https://marc.info/?t=155989412200001&r=1&w=2
> 
> > From: Yoshihiro Shimoda, Sent: Monday, June 3, 2019 3:42 PM
> >
> > Hi linux-block and iommu mailing lists,
> >
> > I have an issue that a USB SSD with xHCI on R-Car H3 causes "swiotlb is full"
> like below.
> >
> >     [   36.745286] xhci-hcd ee000000.usb: swiotlb buffer is full (sz: 524288
> bytes), total 32768 (slots), used 1338 (slots)
> >
> > I have investigated this issue by using git bisect, and then I found the
> following commit:
> >
> > ---
> > commit 09324d32d2a0843e66652a087da6f77924358e62
> > Author: Christoph Hellwig <hch@lst.de>
> > Date:   Tue May 21 09:01:41 2019 +0200
> >
> >     block: force an unlimited segment size on queues with a virt
> > boundary
> > ---
> 
> Thank you for your comment on other email thread [2] like below:
> ---
> Turns out it isn't as simple as I thought, as there doesn't seem to be an easy
> way to get to the struct device used for DMA mapping from USB drivers.  I'll
> need to think a bit more how to handle that best.
> ---
> 
> [2]
> https://marc.info/?l=linux-doc&m=155989651620473&w=2
> 
> I'm not sure this is a correct way, but the issue disappears if I applied a patch
> below to USB storage driver. Especially, WARNING happened on
> blk_queue_max_segment_size().
> Maybe we need to expand the argument "struct device *" of
> blk_queue_virt_boundary() to call dma_max_mapping_size()?
> ---
> diff --git a/drivers/usb/storage/scsiglue.c b/drivers/usb/storage/scsiglue.c
> index 59190d8..fa37b39 100644
> --- a/drivers/usb/storage/scsiglue.c
> +++ b/drivers/usb/storage/scsiglue.c
> @@ -28,6 +28,7 @@
>   * status of a command.
>   */
> 
> +#include <linux/dma-mapping.h>
>  #include <linux/module.h>
>  #include <linux/mutex.h>
> 
> @@ -83,6 +84,15 @@ static int slave_alloc (struct scsi_device *sdev)
>  	maxp = usb_maxpacket(us->pusb_dev, us->recv_bulk_pipe, 0);
>  	blk_queue_virt_boundary(sdev->request_queue, maxp - 1);
> 
> +{
> +	struct device *dev = us->pusb_dev->bus->controller;
> +
> +	dev_info(dev, "%s: size = %zu\n", __func__,
> dma_max_mapping_size(dev));
> +	blk_queue_max_segment_size(sdev->request_queue,
> +				   dma_max_mapping_size(dev));
> +}
> +
> +
>  	/*
>  	 * Some host controllers may have alignment requirements.
>  	 * We'll play it safe by requiring 512-byte alignment always.
> ---
> 
> Best regards,
> Yoshihiro Shimoda


  reply index

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-06-03  6:42 Yoshihiro Shimoda
2019-06-07 12:00 ` Yoshihiro Shimoda
2019-06-10  7:31   ` Biju Das [this message]
2019-06-10 11:13   ` Yoshihiro Shimoda
2019-06-10 12:32     ` Christoph Hellwig
2019-06-10 18:46       ` Alan Stern
2019-06-11  6:41         ` Christoph Hellwig
2019-06-11 14:51           ` Alan Stern
2019-06-12  7:30             ` Christoph Hellwig
2019-06-12  8:52               ` Yoshihiro Shimoda
2019-06-12 11:31                 ` Christoph Hellwig
2019-06-13  4:52                   ` Yoshihiro Shimoda
2019-06-12 11:46               ` Oliver Neukum
2019-06-12 12:06                 ` Christoph Hellwig
2019-06-12 14:43                   ` Alan Stern
2019-06-13  7:39                     ` Christoph Hellwig
2019-06-13 16:57                       ` Martin K. Petersen
2019-06-13 17:16                       ` Alan Stern
2019-06-13 18:18                         ` Greg KH
2019-06-13 23:01                         ` shuah
2019-06-14 14:44                           ` Alan Stern
2019-06-18 15:28                             ` shuah
2019-06-19 20:23                               ` shuah
2019-06-19 21:05                                 ` Alan Stern
2019-06-21 17:43                                   ` Suwan Kim
2019-06-11  6:49         ` Yoshihiro Shimoda

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=OSBPR01MB21038C8C8186797D9EC18489B8130@OSBPR01MB2103.jpnprd01.prod.outlook.com \
    --to=biju.das@bp.renesas.com \
    --cc=hch@lst.de \
    --cc=iommu@lists.linux-foundation.org \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-renesas-soc@vger.kernel.org \
    --cc=linux-usb@vger.kernel.org \
    --cc=yoshihiro.shimoda.uh@renesas.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Block Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-block/0 linux-block/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-block linux-block/ https://lore.kernel.org/linux-block \
		linux-block@vger.kernel.org linux-block@archiver.kernel.org
	public-inbox-index linux-block


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-block


AGPL code for this site: git clone https://public-inbox.org/ public-inbox