linux-usb.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* reeze while write on external usb 3.0 hard disk [Bug 204095]
@ 2019-08-17  9:54 Piergiorgio Sartor
  2019-08-19 14:14 ` Alan Stern
  0 siblings, 1 reply; 22+ messages in thread
From: Piergiorgio Sartor @ 2019-08-17  9:54 UTC (permalink / raw)
  To: linux-usb

Hi all,

bug 204095 from bugzilla.kernel.org was closed,
but apparently something is left unfixed.

Ref.: https://bugzilla.kernel.org/show_bug.cgi?id=204095

Below the two last entries I did about the topic:

--- --- ---

I've Fedora 30 and there was an update, from kernel 5.1.20
to 5.2.5, and now I've the logs full of:

ehci-pci 0000:00:13.2: swiotlb buffer is full (sz: 327680 bytes), total 32768 (slots), used 97 (slots)

Or similarly:

ehci-pci 0000:00:13.2: swiotlb buffer is full (sz: 315392 bytes), total 32768 (slots), used 103 (slots)

This happens whenever I access an external USB3.0 drive
connected to a USB2.0 port, I'm not sure if this makes
any difference.

It is enough something like "hdparm -t /dev/sdX" to trigger
the above (and cause a lock-up of the command and 100% CPU
load for "usb-storage").
Nevertheless, it seems that the first access, likely few bytes
read, is successful since the device is recognize by the file
manager.
Anything else, (large) file copy causes the issue.

Could it be the same bug? That is the fix for 5.2.5 does not
fix it completely?

--- --- ---

Some more comments.

I tested a different PC, this does not show any problem, neither
with the "old" 5.1.20 nor with the new 5.2.5 kernel.

Second, the original PC still have problems, here below the log
output at the moment of the "crash" due to "hdparm -t /dev/sdX":

[   47.212609] xhci_hcd 0000:02:00.0: swiotlb buffer is full (sz: 421888 bytes), total 32768 (slots), used 1 (slots)
[   47.212620] xhci_hcd 0000:02:00.0: overflow 0x0000000383a19000+421888 of DMA mask ffffffff bus mask 0
[   47.212646] WARNING: CPU: 0 PID: 454 at kernel/dma/direct.c:43 report_addr+0x33/0x60
[   47.212649] Modules linked in: fuse cfg80211 rfkill nf_log_ipv4 nf_log_common ipt_REJECT nf_reject_ipv4 xt_state xt_conntrack xt_owner xt_LOG iptable_filter nf_conntrack_ftp xt_CT nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_raw hwmon_vid sunrpc dm_crypt dm_cache_smq nvidia_drm(POE) dm_cache nvidia_modeset(POE) dm_persistent_data dm_bio_prison libcrc32c nvidia_uvm(OE) amd64_edac_mod edac_mce_amd kvm_amd ccp kvm snd_hda_codec_hdmi irqbypass joydev crct10dif_pclmul nvidia(POE) crc32_pclmul ghash_clmulni_intel snd_hda_codec_via snd_hda_codec_generic ledtrig_audio snd_hda_intel drm_kms_helper wmi_bmof snd_hda_codec fam15h_power k10temp snd_hda_core sp5100_tco snd_hwdep i2c_piix4 snd_seq snd_seq_device drm snd_pcm snd_timer ipmi_devintf snd ipmi_msghandler asus_atk0110 soundcore pcc_cpufreq acpi_cpufreq vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) binfmt_misc ip_tables firewire_sbp2 raid1 raid10 crc32c_intel firewire_ohci firewire_core serio_raw crc_itu_t r8169 wmi uas
[   47.212719]  usb_storage hid_logitech ff_memless ecryptfs
[   47.212734] CPU: 0 PID: 454 Comm: usb-storage Tainted: P           OE     5.2.6-200.fc30.x86_64 #1
[   47.212738] Hardware name: System manufacturer System Product Name/M5A78L-M/USB3, BIOS 2101    12/02/2014
[   47.212744] RIP: 0010:report_addr+0x33/0x60
[   47.212750] Code: 48 8b 87 28 02 00 00 48 89 34 24 48 85 c0 74 2d 4c 8b 00 b8 fe ff ff ff 49 39 c0 76 14 80 3d a3 ae 41 01 00 0f 84 a9 06 00 00 <0f> 0b 48 83 c4 08 c3 48 83 bf 38 02 00 00 00 74 ef eb e0 80 3d 84
[   47.212754] RSP: 0018:ffffa85941bffbc8 EFLAGS: 00010246
[   47.212758] RAX: 0000000000000000 RBX: 0000000000000002 RCX: 0000000000000000
[   47.212762] RDX: 0000000000000007 RSI: 0000000000000092 RDI: ffff9a45af617900
[   47.212765] RBP: ffff9a45accc00b0 R08: ffffa85941bff92d R09: 00000000000003a3
[   47.212769] R10: ffffffffb6bece08 R11: ffffa85941bff92d R12: 0000000000067000
[   47.212772] R13: 0000000000000002 R14: 0000000000000000 R15: ffff9a45a40502b0
[   47.212777] FS:  0000000000000000(0000) GS:ffff9a45af600000(0000) knlGS:0000000000000000
[   47.212780] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   47.212784] CR2: 000055b215f34c50 CR3: 00000003d622e000 CR4: 00000000000406f0
[   47.212787] Call Trace:
[   47.212797]  dma_direct_map_page+0xdf/0xf0
[   47.212803]  dma_direct_map_sg+0x67/0xb0
[   47.212811]  usb_hcd_map_urb_for_dma+0x343/0x530
[   47.212817]  usb_hcd_submit_urb+0x9a/0xbb0
[   47.212824]  ? schedule_timeout+0x209/0x300
[   47.212829]  ? usb_hcd_submit_urb+0xbf/0xbb0
[   47.212835]  ? __switch_to_asm+0x40/0x70
[   47.212840]  ? __switch_to_asm+0x34/0x70
[   47.212845]  ? _cond_resched+0x15/0x30
[   47.212852]  ? __kmalloc+0x16c/0x210
[   47.212857]  ? _cond_resched+0x15/0x30
[   47.212863]  ? usb_alloc_urb+0x35/0x60
[   47.212867]  usb_sg_wait+0x7e/0x150
[   47.212879]  usb_stor_bulk_transfer_sglist.part.0+0x64/0xb0 [usb_storage]
[   47.212886]  usb_stor_bulk_srb+0x49/0x80 [usb_storage]
[   47.212893]  usb_stor_Bulk_transport+0x163/0x3e0 [usb_storage]
[   47.212898]  ? schedule+0x33/0x90
[   47.212905]  usb_stor_invoke_transport+0x3a/0x500 [usb_storage]
[   47.212910]  ? wait_for_completion_interruptible+0x156/0x1a0
[   47.212915]  ? wake_up_q+0x60/0x60
[   47.212922]  usb_stor_control_thread+0x183/0x280 [usb_storage]
[   47.212928]  kthread+0xfb/0x130
[   47.212934]  ? usb_stor_disconnect+0xb0/0xb0 [usb_storage]
[   47.212938]  ? kthread_park+0x80/0x80
[   47.212943]  ret_from_fork+0x22/0x40
[   47.212948] ---[ end trace 848c6eec31ed7f76 ]---

--- --- ---

More information.

With kernel 5.2.8 (still from Fedora) the problem appears,
but not any more with "hdparm -t /dev/sdX".
It is required or to copy large files, or something like
"dd if=/dev/sdX of=/dev/null bs=1M".

Sometime, with small blocks (bs=4k, for example), it seems
better, but I'll have to test more.

Finally, no difference between USB3.0 or USB2.0 in any
combination of devices.

On other PCs, all with Intel HW, I could not experience
the issue, yet.

If you need more information, please let me know, I'll
try to support as much a I can.

Thanks,

bye,

-- 

piergiorgio

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-08-17  9:54 reeze while write on external usb 3.0 hard disk [Bug 204095] Piergiorgio Sartor
@ 2019-08-19 14:14 ` Alan Stern
  2019-08-20  7:23   ` Christoph Hellwig
  0 siblings, 1 reply; 22+ messages in thread
From: Alan Stern @ 2019-08-19 14:14 UTC (permalink / raw)
  To: Piergiorgio Sartor, Jens Axboe, Christoph Hellwig
  Cc: USB list, linux-block, Kernel development list

Let's bring this to the attention of some more people.

It looks like the bug that was supposed to be fixed by commit
d74ffae8b8dd ("usb-storage: Add a limitation for
blk_queue_max_hw_sectors()"), which is part of 5.2.5, but apparently
the bug still occurs.

Alan Stern

On Sat, 17 Aug 2019, Piergiorgio Sartor wrote:

> Hi all,
> 
> bug 204095 from bugzilla.kernel.org was closed,
> but apparently something is left unfixed.
> 
> Ref.: https://bugzilla.kernel.org/show_bug.cgi?id=204095
> 
> Below the two last entries I did about the topic:
> 
> --- --- ---
> 
> I've Fedora 30 and there was an update, from kernel 5.1.20
> to 5.2.5, and now I've the logs full of:
> 
> ehci-pci 0000:00:13.2: swiotlb buffer is full (sz: 327680 bytes), total 32768 (slots), used 97 (slots)
> 
> Or similarly:
> 
> ehci-pci 0000:00:13.2: swiotlb buffer is full (sz: 315392 bytes), total 32768 (slots), used 103 (slots)
> 
> This happens whenever I access an external USB3.0 drive
> connected to a USB2.0 port, I'm not sure if this makes
> any difference.
> 
> It is enough something like "hdparm -t /dev/sdX" to trigger
> the above (and cause a lock-up of the command and 100% CPU
> load for "usb-storage").
> Nevertheless, it seems that the first access, likely few bytes
> read, is successful since the device is recognize by the file
> manager.
> Anything else, (large) file copy causes the issue.
> 
> Could it be the same bug? That is the fix for 5.2.5 does not
> fix it completely?
> 
> --- --- ---
> 
> Some more comments.
> 
> I tested a different PC, this does not show any problem, neither
> with the "old" 5.1.20 nor with the new 5.2.5 kernel.
> 
> Second, the original PC still have problems, here below the log
> output at the moment of the "crash" due to "hdparm -t /dev/sdX":
> 
> [   47.212609] xhci_hcd 0000:02:00.0: swiotlb buffer is full (sz: 421888 bytes), total 32768 (slots), used 1 (slots)
> [   47.212620] xhci_hcd 0000:02:00.0: overflow 0x0000000383a19000+421888 of DMA mask ffffffff bus mask 0
> [   47.212646] WARNING: CPU: 0 PID: 454 at kernel/dma/direct.c:43 report_addr+0x33/0x60
> [   47.212649] Modules linked in: fuse cfg80211 rfkill nf_log_ipv4 nf_log_common ipt_REJECT nf_reject_ipv4 xt_state xt_conntrack xt_owner xt_LOG iptable_filter nf_conntrack_ftp xt_CT nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_raw hwmon_vid sunrpc dm_crypt dm_cache_smq nvidia_drm(POE) dm_cache nvidia_modeset(POE) dm_persistent_data dm_bio_prison libcrc32c nvidia_uvm(OE) amd64_edac_mod edac_mce_amd kvm_amd ccp kvm snd_hda_codec_hdmi irqbypass joydev crct10dif_pclmul nvidia(POE) crc32_pclmul ghash_clmulni_intel snd_hda_codec_via snd_hda_codec_generic ledtrig_audio snd_hda_intel drm_kms_helper wmi_bmof snd_hda_codec fam15h_power k10temp snd_hda_core sp5100_tco snd_hwdep i2c_piix4 snd_seq snd_seq_device drm snd_pcm snd_timer ipmi_devintf snd ipmi_msghandler asus_atk0110 soundcore pcc_cpufreq acpi_cpufreq vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) binfmt_misc ip_tables firewire_sbp2 raid1 raid10 crc32c_intel firewire_ohci firewire_core serio_raw crc_itu_t r8169 wmi uas
> [   47.212719]  usb_storage hid_logitech ff_memless ecryptfs
> [   47.212734] CPU: 0 PID: 454 Comm: usb-storage Tainted: P           OE     5.2.6-200.fc30.x86_64 #1
> [   47.212738] Hardware name: System manufacturer System Product Name/M5A78L-M/USB3, BIOS 2101    12/02/2014
> [   47.212744] RIP: 0010:report_addr+0x33/0x60
> [   47.212750] Code: 48 8b 87 28 02 00 00 48 89 34 24 48 85 c0 74 2d 4c 8b 00 b8 fe ff ff ff 49 39 c0 76 14 80 3d a3 ae 41 01 00 0f 84 a9 06 00 00 <0f> 0b 48 83 c4 08 c3 48 83 bf 38 02 00 00 00 74 ef eb e0 80 3d 84
> [   47.212754] RSP: 0018:ffffa85941bffbc8 EFLAGS: 00010246
> [   47.212758] RAX: 0000000000000000 RBX: 0000000000000002 RCX: 0000000000000000
> [   47.212762] RDX: 0000000000000007 RSI: 0000000000000092 RDI: ffff9a45af617900
> [   47.212765] RBP: ffff9a45accc00b0 R08: ffffa85941bff92d R09: 00000000000003a3
> [   47.212769] R10: ffffffffb6bece08 R11: ffffa85941bff92d R12: 0000000000067000
> [   47.212772] R13: 0000000000000002 R14: 0000000000000000 R15: ffff9a45a40502b0
> [   47.212777] FS:  0000000000000000(0000) GS:ffff9a45af600000(0000) knlGS:0000000000000000
> [   47.212780] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   47.212784] CR2: 000055b215f34c50 CR3: 00000003d622e000 CR4: 00000000000406f0
> [   47.212787] Call Trace:
> [   47.212797]  dma_direct_map_page+0xdf/0xf0
> [   47.212803]  dma_direct_map_sg+0x67/0xb0
> [   47.212811]  usb_hcd_map_urb_for_dma+0x343/0x530
> [   47.212817]  usb_hcd_submit_urb+0x9a/0xbb0
> [   47.212824]  ? schedule_timeout+0x209/0x300
> [   47.212829]  ? usb_hcd_submit_urb+0xbf/0xbb0
> [   47.212835]  ? __switch_to_asm+0x40/0x70
> [   47.212840]  ? __switch_to_asm+0x34/0x70
> [   47.212845]  ? _cond_resched+0x15/0x30
> [   47.212852]  ? __kmalloc+0x16c/0x210
> [   47.212857]  ? _cond_resched+0x15/0x30
> [   47.212863]  ? usb_alloc_urb+0x35/0x60
> [   47.212867]  usb_sg_wait+0x7e/0x150
> [   47.212879]  usb_stor_bulk_transfer_sglist.part.0+0x64/0xb0 [usb_storage]
> [   47.212886]  usb_stor_bulk_srb+0x49/0x80 [usb_storage]
> [   47.212893]  usb_stor_Bulk_transport+0x163/0x3e0 [usb_storage]
> [   47.212898]  ? schedule+0x33/0x90
> [   47.212905]  usb_stor_invoke_transport+0x3a/0x500 [usb_storage]
> [   47.212910]  ? wait_for_completion_interruptible+0x156/0x1a0
> [   47.212915]  ? wake_up_q+0x60/0x60
> [   47.212922]  usb_stor_control_thread+0x183/0x280 [usb_storage]
> [   47.212928]  kthread+0xfb/0x130
> [   47.212934]  ? usb_stor_disconnect+0xb0/0xb0 [usb_storage]
> [   47.212938]  ? kthread_park+0x80/0x80
> [   47.212943]  ret_from_fork+0x22/0x40
> [   47.212948] ---[ end trace 848c6eec31ed7f76 ]---
> 
> --- --- ---
> 
> More information.
> 
> With kernel 5.2.8 (still from Fedora) the problem appears,
> but not any more with "hdparm -t /dev/sdX".
> It is required or to copy large files, or something like
> "dd if=/dev/sdX of=/dev/null bs=1M".
> 
> Sometime, with small blocks (bs=4k, for example), it seems
> better, but I'll have to test more.
> 
> Finally, no difference between USB3.0 or USB2.0 in any
> combination of devices.
> 
> On other PCs, all with Intel HW, I could not experience
> the issue, yet.
> 
> If you need more information, please let me know, I'll
> try to support as much a I can.
> 
> Thanks,
> 
> bye,


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-08-19 14:14 ` Alan Stern
@ 2019-08-20  7:23   ` Christoph Hellwig
  2019-08-20 16:37     ` Piergiorgio Sartor
  0 siblings, 1 reply; 22+ messages in thread
From: Christoph Hellwig @ 2019-08-20  7:23 UTC (permalink / raw)
  To: Alan Stern
  Cc: Piergiorgio Sartor, Jens Axboe, Christoph Hellwig, USB list,
	linux-block, Kernel development list

On Mon, Aug 19, 2019 at 10:14:25AM -0400, Alan Stern wrote:
> Let's bring this to the attention of some more people.
> 
> It looks like the bug that was supposed to be fixed by commit
> d74ffae8b8dd ("usb-storage: Add a limitation for
> blk_queue_max_hw_sectors()"), which is part of 5.2.5, but apparently
> the bug still occurs.

Piergiorgio,

can you dump the content of max_hw_sectors_kb file for your USB storage
device and send that to this thread?

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-08-20  7:23   ` Christoph Hellwig
@ 2019-08-20 16:37     ` Piergiorgio Sartor
  2019-08-26 17:38       ` Piergiorgio Sartor
  0 siblings, 1 reply; 22+ messages in thread
From: Piergiorgio Sartor @ 2019-08-20 16:37 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Alan Stern, Piergiorgio Sartor, Jens Axboe, USB list,
	linux-block, Kernel development list

On Tue, Aug 20, 2019 at 09:23:26AM +0200, Christoph Hellwig wrote:
> On Mon, Aug 19, 2019 at 10:14:25AM -0400, Alan Stern wrote:
> > Let's bring this to the attention of some more people.
> > 
> > It looks like the bug that was supposed to be fixed by commit
> > d74ffae8b8dd ("usb-storage: Add a limitation for
> > blk_queue_max_hw_sectors()"), which is part of 5.2.5, but apparently
> > the bug still occurs.
> 
> Piergiorgio,
> 
> can you dump the content of max_hw_sectors_kb file for your USB storage
> device and send that to this thread?

Hi all,

for both kernels, 5.1.20 (working) and 5.2.8 (not working),
the content of /sys/dev/x:y/queue/max_hw_sectors_kb is 512
for USB storage devices (2.0 and 3.0).

This is for the PC showing the issue.

In an other PC, which does not show the issus at the moment,
the values are 120, for USB2.0, and 256, for USB3.0.

bye,

-- 

piergiorgio

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-08-20 16:37     ` Piergiorgio Sartor
@ 2019-08-26 17:38       ` Piergiorgio Sartor
  2019-08-29 14:08         ` Christoph Hellwig
  2019-09-25 17:07         ` Piergiorgio Sartor
  0 siblings, 2 replies; 22+ messages in thread
From: Piergiorgio Sartor @ 2019-08-26 17:38 UTC (permalink / raw)
  To: Piergiorgio Sartor
  Cc: Christoph Hellwig, Alan Stern, Jens Axboe, USB list, linux-block,
	Kernel development list

On Tue, Aug 20, 2019 at 06:37:22PM +0200, Piergiorgio Sartor wrote:
> On Tue, Aug 20, 2019 at 09:23:26AM +0200, Christoph Hellwig wrote:
> > On Mon, Aug 19, 2019 at 10:14:25AM -0400, Alan Stern wrote:
> > > Let's bring this to the attention of some more people.
> > > 
> > > It looks like the bug that was supposed to be fixed by commit
> > > d74ffae8b8dd ("usb-storage: Add a limitation for
> > > blk_queue_max_hw_sectors()"), which is part of 5.2.5, but apparently
> > > the bug still occurs.
> > 
> > Piergiorgio,
> > 
> > can you dump the content of max_hw_sectors_kb file for your USB storage
> > device and send that to this thread?
> 
> Hi all,
> 
> for both kernels, 5.1.20 (working) and 5.2.8 (not working),
> the content of /sys/dev/x:y/queue/max_hw_sectors_kb is 512
> for USB storage devices (2.0 and 3.0).
> 
> This is for the PC showing the issue.
> 
> In an other PC, which does not show the issus at the moment,
> the values are 120, for USB2.0, and 256, for USB3.0.

Hi again,

any news on this?

Is there anything I can do to help?

Should I report this somewhere else too?

Currently this is quite a huge problem for me,
since the only working external storage is an
old 1394 HDD...

Thanks,

bye,

-- 

piergiorgio

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-08-26 17:38       ` Piergiorgio Sartor
@ 2019-08-29 14:08         ` Christoph Hellwig
  2019-09-25 17:07         ` Piergiorgio Sartor
  1 sibling, 0 replies; 22+ messages in thread
From: Christoph Hellwig @ 2019-08-29 14:08 UTC (permalink / raw)
  To: Piergiorgio Sartor
  Cc: Christoph Hellwig, Alan Stern, Jens Axboe, USB list, linux-block,
	Kernel development list

On Mon, Aug 26, 2019 at 07:38:33PM +0200, Piergiorgio Sartor wrote:
> any news on this?

Sorry, I've been dropping the ball here, and I'm a little puzzled
on what is going on.  Including on why we are bounce buffering for
xhci at all.  What kind of system (CPU / mainboard) is this?

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-08-26 17:38       ` Piergiorgio Sartor
  2019-08-29 14:08         ` Christoph Hellwig
@ 2019-09-25 17:07         ` Piergiorgio Sartor
  2019-09-25 18:31           ` Alan Stern
  1 sibling, 1 reply; 22+ messages in thread
From: Piergiorgio Sartor @ 2019-09-25 17:07 UTC (permalink / raw)
  To: Piergiorgio Sartor
  Cc: Christoph Hellwig, Alan Stern, Jens Axboe, USB list, linux-block,
	Kernel development list

On Mon, Aug 26, 2019 at 07:38:33PM +0200, Piergiorgio Sartor wrote:
> On Tue, Aug 20, 2019 at 06:37:22PM +0200, Piergiorgio Sartor wrote:
> > On Tue, Aug 20, 2019 at 09:23:26AM +0200, Christoph Hellwig wrote:
> > > On Mon, Aug 19, 2019 at 10:14:25AM -0400, Alan Stern wrote:
> > > > Let's bring this to the attention of some more people.
> > > > 
> > > > It looks like the bug that was supposed to be fixed by commit
> > > > d74ffae8b8dd ("usb-storage: Add a limitation for
> > > > blk_queue_max_hw_sectors()"), which is part of 5.2.5, but apparently
> > > > the bug still occurs.
> > > 
> > > Piergiorgio,
> > > 
> > > can you dump the content of max_hw_sectors_kb file for your USB storage
> > > device and send that to this thread?
> > 
> > Hi all,
> > 
> > for both kernels, 5.1.20 (working) and 5.2.8 (not working),
> > the content of /sys/dev/x:y/queue/max_hw_sectors_kb is 512
> > for USB storage devices (2.0 and 3.0).
> > 
> > This is for the PC showing the issue.
> > 
> > In an other PC, which does not show the issus at the moment,
> > the values are 120, for USB2.0, and 256, for USB3.0.
> 
> Hi again,
> 
> any news on this?
> 
> Is there anything I can do to help?
> 
> Should I report this somewhere else too?
> 
> Currently this is quite a huge problem for me,
> since the only working external storage is an
> old 1394 HDD...

Hi all,

I'm now on kernel 5.2.16, from Fedora, and still I
see the same issue.

I guess it is not a chipset quirk, since there
are two involved here.
For the USB 2.0 I've (with "lspci"):

USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller (prog-if 20 [EHCI])

For USB 3.0 I've:

USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller (prog-if 30 [XHCI])

Any idea on how to proceed?

Thanks a lot.

bye,

-- 

piergiorgio

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-09-25 17:07         ` Piergiorgio Sartor
@ 2019-09-25 18:31           ` Alan Stern
  2019-09-27  9:04             ` Peter Chen
  2019-09-29 20:13             ` Piergiorgio Sartor
  0 siblings, 2 replies; 22+ messages in thread
From: Alan Stern @ 2019-09-25 18:31 UTC (permalink / raw)
  To: Piergiorgio Sartor
  Cc: Christoph Hellwig, Jens Axboe, USB list, linux-block,
	Kernel development list

On Wed, 25 Sep 2019, Piergiorgio Sartor wrote:

> On Mon, Aug 26, 2019 at 07:38:33PM +0200, Piergiorgio Sartor wrote:
> > On Tue, Aug 20, 2019 at 06:37:22PM +0200, Piergiorgio Sartor wrote:
> > > On Tue, Aug 20, 2019 at 09:23:26AM +0200, Christoph Hellwig wrote:
> > > > On Mon, Aug 19, 2019 at 10:14:25AM -0400, Alan Stern wrote:
> > > > > Let's bring this to the attention of some more people.
> > > > > 
> > > > > It looks like the bug that was supposed to be fixed by commit
> > > > > d74ffae8b8dd ("usb-storage: Add a limitation for
> > > > > blk_queue_max_hw_sectors()"), which is part of 5.2.5, but apparently
> > > > > the bug still occurs.
> > > > 
> > > > Piergiorgio,
> > > > 
> > > > can you dump the content of max_hw_sectors_kb file for your USB storage
> > > > device and send that to this thread?
> > > 
> > > Hi all,
> > > 
> > > for both kernels, 5.1.20 (working) and 5.2.8 (not working),
> > > the content of /sys/dev/x:y/queue/max_hw_sectors_kb is 512
> > > for USB storage devices (2.0 and 3.0).
> > > 
> > > This is for the PC showing the issue.
> > > 
> > > In an other PC, which does not show the issus at the moment,
> > > the values are 120, for USB2.0, and 256, for USB3.0.
> > 
> > Hi again,
> > 
> > any news on this?
> > 
> > Is there anything I can do to help?
> > 
> > Should I report this somewhere else too?
> > 
> > Currently this is quite a huge problem for me,
> > since the only working external storage is an
> > old 1394 HDD...
> 
> Hi all,
> 
> I'm now on kernel 5.2.16, from Fedora, and still I
> see the same issue.
> 
> I guess it is not a chipset quirk, since there
> are two involved here.
> For the USB 2.0 I've (with "lspci"):
> 
> USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller (prog-if 20 [EHCI])
> 
> For USB 3.0 I've:
> 
> USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller (prog-if 30 [XHCI])
> 
> Any idea on how to proceed?
> 
> Thanks a lot.

One thing you can try is git bisect from 5.1.20 (or maybe just 5.1.0)  
to 5.2.8.  If you can identify a particular commit which caused the
problem to start, that would help.

Alan Stern


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-09-25 18:31           ` Alan Stern
@ 2019-09-27  9:04             ` Peter Chen
  2019-09-29 20:13             ` Piergiorgio Sartor
  1 sibling, 0 replies; 22+ messages in thread
From: Peter Chen @ 2019-09-27  9:04 UTC (permalink / raw)
  To: Alan Stern
  Cc: Piergiorgio Sartor, Christoph Hellwig, Jens Axboe, USB list,
	linux-block, Kernel development list

On 19-09-25 14:31:58, Alan Stern wrote:
> On Wed, 25 Sep 2019, Piergiorgio Sartor wrote:
> 
> > On Mon, Aug 26, 2019 at 07:38:33PM +0200, Piergiorgio Sartor wrote:
> > > On Tue, Aug 20, 2019 at 06:37:22PM +0200, Piergiorgio Sartor wrote:
> > > > On Tue, Aug 20, 2019 at 09:23:26AM +0200, Christoph Hellwig wrote:
> > > > > On Mon, Aug 19, 2019 at 10:14:25AM -0400, Alan Stern wrote:
> > > > > > Let's bring this to the attention of some more people.
> > > > > > 
> > > > > > It looks like the bug that was supposed to be fixed by commit
> > > > > > d74ffae8b8dd ("usb-storage: Add a limitation for
> > > > > > blk_queue_max_hw_sectors()"), which is part of 5.2.5, but apparently
> > > > > > the bug still occurs.
> > > > > 
> > > > > Piergiorgio,
> > > > > 
> > > > > can you dump the content of max_hw_sectors_kb file for your USB storage
> > > > > device and send that to this thread?
> > > > 
> > > > Hi all,
> > > > 
> > > > for both kernels, 5.1.20 (working) and 5.2.8 (not working),
> > > > the content of /sys/dev/x:y/queue/max_hw_sectors_kb is 512
> > > > for USB storage devices (2.0 and 3.0).
> > > > 
> > > > This is for the PC showing the issue.
> > > > 
> > > > In an other PC, which does not show the issus at the moment,
> > > > the values are 120, for USB2.0, and 256, for USB3.0.
> > > 
> > > Hi again,
> > > 
> > > any news on this?
> > > 
> > > Is there anything I can do to help?
> > > 
> > > Should I report this somewhere else too?
> > > 
> > > Currently this is quite a huge problem for me,
> > > since the only working external storage is an
> > > old 1394 HDD...
> > 
> > Hi all,
> > 
> > I'm now on kernel 5.2.16, from Fedora, and still I
> > see the same issue.
> > 
> > I guess it is not a chipset quirk, since there
> > are two involved here.
> > For the USB 2.0 I've (with "lspci"):
> > 
> > USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller (prog-if 20 [EHCI])
> > 
> > For USB 3.0 I've:
> > 
> > USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller (prog-if 30 [XHCI])
> > 
> > Any idea on how to proceed?
> > 
> > Thanks a lot.
> 
> One thing you can try is git bisect from 5.1.20 (or maybe just 5.1.0)  
> to 5.2.8.  If you can identify a particular commit which caused the
> problem to start, that would help.
> 
> Alan Stern

Hi Piergiorgio,

Would you please check if below patch helps?

commit 449fa54d6815be8c2c1f68fa9dbbae9384a7c03e
Author: Fugang Duan <fugang.duan@nxp.com>
Date:   Fri Jul 19 17:26:48 2019 +0800

    dma-direct: correct the physical addr in dma_direct_sync_sg_for_cpu/device
        

-- 

Thanks,
Peter Chen

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-09-25 18:31           ` Alan Stern
  2019-09-27  9:04             ` Peter Chen
@ 2019-09-29 20:13             ` Piergiorgio Sartor
  2019-09-30  1:01               ` Alan Stern
  1 sibling, 1 reply; 22+ messages in thread
From: Piergiorgio Sartor @ 2019-09-29 20:13 UTC (permalink / raw)
  To: Alan Stern
  Cc: Piergiorgio Sartor, Christoph Hellwig, Jens Axboe, USB list,
	linux-block, Kernel development list

On Wed, Sep 25, 2019 at 02:31:58PM -0400, Alan Stern wrote:
> On Wed, 25 Sep 2019, Piergiorgio Sartor wrote:
> 
> > On Mon, Aug 26, 2019 at 07:38:33PM +0200, Piergiorgio Sartor wrote:
> > > On Tue, Aug 20, 2019 at 06:37:22PM +0200, Piergiorgio Sartor wrote:
> > > > On Tue, Aug 20, 2019 at 09:23:26AM +0200, Christoph Hellwig wrote:
> > > > > On Mon, Aug 19, 2019 at 10:14:25AM -0400, Alan Stern wrote:
> > > > > > Let's bring this to the attention of some more people.
> > > > > > 
> > > > > > It looks like the bug that was supposed to be fixed by commit
> > > > > > d74ffae8b8dd ("usb-storage: Add a limitation for
> > > > > > blk_queue_max_hw_sectors()"), which is part of 5.2.5, but apparently
> > > > > > the bug still occurs.
> > > > > 
> > > > > Piergiorgio,
> > > > > 
> > > > > can you dump the content of max_hw_sectors_kb file for your USB storage
> > > > > device and send that to this thread?
> > > > 
> > > > Hi all,
> > > > 
> > > > for both kernels, 5.1.20 (working) and 5.2.8 (not working),
> > > > the content of /sys/dev/x:y/queue/max_hw_sectors_kb is 512
> > > > for USB storage devices (2.0 and 3.0).
> > > > 
> > > > This is for the PC showing the issue.
> > > > 
> > > > In an other PC, which does not show the issus at the moment,
> > > > the values are 120, for USB2.0, and 256, for USB3.0.
> > > 
> > > Hi again,
> > > 
> > > any news on this?
> > > 
> > > Is there anything I can do to help?
> > > 
> > > Should I report this somewhere else too?
> > > 
> > > Currently this is quite a huge problem for me,
> > > since the only working external storage is an
> > > old 1394 HDD...
> > 
> > Hi all,
> > 
> > I'm now on kernel 5.2.16, from Fedora, and still I
> > see the same issue.
> > 
> > I guess it is not a chipset quirk, since there
> > are two involved here.
> > For the USB 2.0 I've (with "lspci"):
> > 
> > USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller (prog-if 20 [EHCI])
> > 
> > For USB 3.0 I've:
> > 
> > USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller (prog-if 30 [XHCI])
> > 
> > Any idea on how to proceed?
> > 
> > Thanks a lot.
> 
> One thing you can try is git bisect from 5.1.20 (or maybe just 5.1.0)  
> to 5.2.8.  If you can identify a particular commit which caused the
> problem to start, that would help.

OK, I tried a bisect (2 days compilations...).
Assuming I've done everything correctly (how to
test this? How to remove the guilty patch?), this
was the result:

09324d32d2a0843e66652a087da6f77924358e62 is the first bad commit
commit 09324d32d2a0843e66652a087da6f77924358e62
Author: Christoph Hellwig <hch@lst.de>
Date:   Tue May 21 09:01:41 2019 +0200

    block: force an unlimited segment size on queues with a virt boundary

    We currently fail to update the front/back segment size in the bio when
    deciding to allow an otherwise gappy segement to a device with a
    virt boundary.  The reason why this did not cause problems is that
    devices with a virt boundary fundamentally don't use segments as we
    know it and thus don't care.  Make that assumption formal by forcing
    an unlimited segement size in this case.

    Fixes: f6970f83ef79 ("block: don't check if adjacent bvecs in one bio can be mergeable")
    Signed-off-by: Christoph Hellwig <hch@lst.de>
    Reviewed-by: Ming Lei <ming.lei@redhat.com>
    Reviewed-by: Hannes Reinecke <hare@suse.com>
    Signed-off-by: Jens Axboe <axboe@kernel.dk>

:040000 040000 57ba04a02f948022c0f6ba24bfa36f3b565b2440 8c925f71ce75042529c001bf244b30565d19ebf3 M      block

What to do now?

Thanks,

bye,

-- 

piergiorgio

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-09-29 20:13             ` Piergiorgio Sartor
@ 2019-09-30  1:01               ` Alan Stern
  2019-09-30 18:25                 ` Piergiorgio Sartor
  0 siblings, 1 reply; 22+ messages in thread
From: Alan Stern @ 2019-09-30  1:01 UTC (permalink / raw)
  To: Piergiorgio Sartor
  Cc: Christoph Hellwig, Jens Axboe, USB list, linux-block,
	Kernel development list

On Sun, 29 Sep 2019, Piergiorgio Sartor wrote:

> On Wed, Sep 25, 2019 at 02:31:58PM -0400, Alan Stern wrote:
> > On Wed, 25 Sep 2019, Piergiorgio Sartor wrote:
> > 
> > > On Mon, Aug 26, 2019 at 07:38:33PM +0200, Piergiorgio Sartor wrote:
> > > > On Tue, Aug 20, 2019 at 06:37:22PM +0200, Piergiorgio Sartor wrote:
> > > > > On Tue, Aug 20, 2019 at 09:23:26AM +0200, Christoph Hellwig wrote:
> > > > > > On Mon, Aug 19, 2019 at 10:14:25AM -0400, Alan Stern wrote:
> > > > > > > Let's bring this to the attention of some more people.
> > > > > > > 
> > > > > > > It looks like the bug that was supposed to be fixed by commit
> > > > > > > d74ffae8b8dd ("usb-storage: Add a limitation for
> > > > > > > blk_queue_max_hw_sectors()"), which is part of 5.2.5, but apparently
> > > > > > > the bug still occurs.
> > > > > > 
> > > > > > Piergiorgio,
> > > > > > 
> > > > > > can you dump the content of max_hw_sectors_kb file for your USB storage
> > > > > > device and send that to this thread?
> > > > > 
> > > > > Hi all,
> > > > > 
> > > > > for both kernels, 5.1.20 (working) and 5.2.8 (not working),
> > > > > the content of /sys/dev/x:y/queue/max_hw_sectors_kb is 512
> > > > > for USB storage devices (2.0 and 3.0).
> > > > > 
> > > > > This is for the PC showing the issue.
> > > > > 
> > > > > In an other PC, which does not show the issus at the moment,
> > > > > the values are 120, for USB2.0, and 256, for USB3.0.

> > One thing you can try is git bisect from 5.1.20 (or maybe just 5.1.0)  
> > to 5.2.8.  If you can identify a particular commit which caused the
> > problem to start, that would help.
> 
> OK, I tried a bisect (2 days compilations...).
> Assuming I've done everything correctly (how to
> test this? How to remove the guilty patch?), this
> was the result:
> 
> 09324d32d2a0843e66652a087da6f77924358e62 is the first bad commit
> commit 09324d32d2a0843e66652a087da6f77924358e62
> Author: Christoph Hellwig <hch@lst.de>
> Date:   Tue May 21 09:01:41 2019 +0200
> 
>     block: force an unlimited segment size on queues with a virt boundary
> 
>     We currently fail to update the front/back segment size in the bio when
>     deciding to allow an otherwise gappy segement to a device with a
>     virt boundary.  The reason why this did not cause problems is that
>     devices with a virt boundary fundamentally don't use segments as we
>     know it and thus don't care.  Make that assumption formal by forcing
>     an unlimited segement size in this case.
> 
>     Fixes: f6970f83ef79 ("block: don't check if adjacent bvecs in one bio can be mergeable")
>     Signed-off-by: Christoph Hellwig <hch@lst.de>
>     Reviewed-by: Ming Lei <ming.lei@redhat.com>
>     Reviewed-by: Hannes Reinecke <hare@suse.com>
>     Signed-off-by: Jens Axboe <axboe@kernel.dk>
> 
> :040000 040000 57ba04a02f948022c0f6ba24bfa36f3b565b2440 8c925f71ce75042529c001bf244b30565d19ebf3 M      block
> 
> What to do now?

Here's how to verify that the bisection got a correct result.  First, 
do a git checkout of commit 09324d32d2a0, build the kernel, and make 
sure that it exhibits the problem.

Next, have git write out the contents of that commit in the form of a
patch (git show commit-id >patchfile), and revert it (git apply -R
patchfile).  Build the kernel from that tree, and make sure that it
does not exhibit the problem.  If it doesn't, you have definitely shown
that this commit is the cause (or at least, is _one_ of the causes).

Alan Stern


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-09-30  1:01               ` Alan Stern
@ 2019-09-30 18:25                 ` Piergiorgio Sartor
  2019-10-13 18:11                   ` Piergiorgio Sartor
  0 siblings, 1 reply; 22+ messages in thread
From: Piergiorgio Sartor @ 2019-09-30 18:25 UTC (permalink / raw)
  To: Alan Stern
  Cc: Piergiorgio Sartor, Christoph Hellwig, Jens Axboe, USB list,
	linux-block, Kernel development list

On Sun, Sep 29, 2019 at 09:01:48PM -0400, Alan Stern wrote:
> On Sun, 29 Sep 2019, Piergiorgio Sartor wrote:
> 
> > On Wed, Sep 25, 2019 at 02:31:58PM -0400, Alan Stern wrote:
> > > On Wed, 25 Sep 2019, Piergiorgio Sartor wrote:
> > > 
> > > > On Mon, Aug 26, 2019 at 07:38:33PM +0200, Piergiorgio Sartor wrote:
> > > > > On Tue, Aug 20, 2019 at 06:37:22PM +0200, Piergiorgio Sartor wrote:
> > > > > > On Tue, Aug 20, 2019 at 09:23:26AM +0200, Christoph Hellwig wrote:
> > > > > > > On Mon, Aug 19, 2019 at 10:14:25AM -0400, Alan Stern wrote:
> > > > > > > > Let's bring this to the attention of some more people.
> > > > > > > > 
> > > > > > > > It looks like the bug that was supposed to be fixed by commit
> > > > > > > > d74ffae8b8dd ("usb-storage: Add a limitation for
> > > > > > > > blk_queue_max_hw_sectors()"), which is part of 5.2.5, but apparently
> > > > > > > > the bug still occurs.
> > > > > > > 
> > > > > > > Piergiorgio,
> > > > > > > 
> > > > > > > can you dump the content of max_hw_sectors_kb file for your USB storage
> > > > > > > device and send that to this thread?
> > > > > > 
> > > > > > Hi all,
> > > > > > 
> > > > > > for both kernels, 5.1.20 (working) and 5.2.8 (not working),
> > > > > > the content of /sys/dev/x:y/queue/max_hw_sectors_kb is 512
> > > > > > for USB storage devices (2.0 and 3.0).
> > > > > > 
> > > > > > This is for the PC showing the issue.
> > > > > > 
> > > > > > In an other PC, which does not show the issus at the moment,
> > > > > > the values are 120, for USB2.0, and 256, for USB3.0.
> 
> > > One thing you can try is git bisect from 5.1.20 (or maybe just 5.1.0)  
> > > to 5.2.8.  If you can identify a particular commit which caused the
> > > problem to start, that would help.
> > 
> > OK, I tried a bisect (2 days compilations...).
> > Assuming I've done everything correctly (how to
> > test this? How to remove the guilty patch?), this
> > was the result:
> > 
> > 09324d32d2a0843e66652a087da6f77924358e62 is the first bad commit
> > commit 09324d32d2a0843e66652a087da6f77924358e62
> > Author: Christoph Hellwig <hch@lst.de>
> > Date:   Tue May 21 09:01:41 2019 +0200
> > 
> >     block: force an unlimited segment size on queues with a virt boundary
> > 
> >     We currently fail to update the front/back segment size in the bio when
> >     deciding to allow an otherwise gappy segement to a device with a
> >     virt boundary.  The reason why this did not cause problems is that
> >     devices with a virt boundary fundamentally don't use segments as we
> >     know it and thus don't care.  Make that assumption formal by forcing
> >     an unlimited segement size in this case.
> > 
> >     Fixes: f6970f83ef79 ("block: don't check if adjacent bvecs in one bio can be mergeable")
> >     Signed-off-by: Christoph Hellwig <hch@lst.de>
> >     Reviewed-by: Ming Lei <ming.lei@redhat.com>
> >     Reviewed-by: Hannes Reinecke <hare@suse.com>
> >     Signed-off-by: Jens Axboe <axboe@kernel.dk>
> > 
> > :040000 040000 57ba04a02f948022c0f6ba24bfa36f3b565b2440 8c925f71ce75042529c001bf244b30565d19ebf3 M      block
> > 
> > What to do now?
> 
> Here's how to verify that the bisection got a correct result.  First, 
> do a git checkout of commit 09324d32d2a0, build the kernel, and make 
> sure that it exhibits the problem.
> 
> Next, have git write out the contents of that commit in the form of a
> patch (git show commit-id >patchfile), and revert it (git apply -R
> patchfile).  Build the kernel from that tree, and make sure that it
> does not exhibit the problem.  If it doesn't, you have definitely shown
> that this commit is the cause (or at least, is _one_ of the causes).

I tried as suggested, i.e. jumping to commit
09324d32d2a0843e66652a087da6f77924358e62, testing,
removing the patch, testing.
The result was as expected.
I was able to reproduce the issue with the commit,
I was not able to reproduce it without.
It seems this patch / commit is causing the problem.
Directly or indirectly.

What are the next steps?

Thanks!

bye,

-- 

piergiorgio

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-09-30 18:25                 ` Piergiorgio Sartor
@ 2019-10-13 18:11                   ` Piergiorgio Sartor
  2019-10-16 17:01                     ` Alan Stern
  0 siblings, 1 reply; 22+ messages in thread
From: Piergiorgio Sartor @ 2019-10-13 18:11 UTC (permalink / raw)
  To: Piergiorgio Sartor
  Cc: Alan Stern, Christoph Hellwig, Jens Axboe, USB list, linux-block,
	Kernel development list

On Mon, Sep 30, 2019 at 08:25:01PM +0200, Piergiorgio Sartor wrote:
> On Sun, Sep 29, 2019 at 09:01:48PM -0400, Alan Stern wrote:
> > On Sun, 29 Sep 2019, Piergiorgio Sartor wrote:
> > 
> > > On Wed, Sep 25, 2019 at 02:31:58PM -0400, Alan Stern wrote:
> > > > On Wed, 25 Sep 2019, Piergiorgio Sartor wrote:
> > > > 
> > > > > On Mon, Aug 26, 2019 at 07:38:33PM +0200, Piergiorgio Sartor wrote:
> > > > > > On Tue, Aug 20, 2019 at 06:37:22PM +0200, Piergiorgio Sartor wrote:
> > > > > > > On Tue, Aug 20, 2019 at 09:23:26AM +0200, Christoph Hellwig wrote:
> > > > > > > > On Mon, Aug 19, 2019 at 10:14:25AM -0400, Alan Stern wrote:
> > > > > > > > > Let's bring this to the attention of some more people.
> > > > > > > > > 
> > > > > > > > > It looks like the bug that was supposed to be fixed by commit
> > > > > > > > > d74ffae8b8dd ("usb-storage: Add a limitation for
> > > > > > > > > blk_queue_max_hw_sectors()"), which is part of 5.2.5, but apparently
> > > > > > > > > the bug still occurs.
> > > > > > > > 
> > > > > > > > Piergiorgio,
> > > > > > > > 
> > > > > > > > can you dump the content of max_hw_sectors_kb file for your USB storage
> > > > > > > > device and send that to this thread?
> > > > > > > 
> > > > > > > Hi all,
> > > > > > > 
> > > > > > > for both kernels, 5.1.20 (working) and 5.2.8 (not working),
> > > > > > > the content of /sys/dev/x:y/queue/max_hw_sectors_kb is 512
> > > > > > > for USB storage devices (2.0 and 3.0).
> > > > > > > 
> > > > > > > This is for the PC showing the issue.
> > > > > > > 
> > > > > > > In an other PC, which does not show the issus at the moment,
> > > > > > > the values are 120, for USB2.0, and 256, for USB3.0.
> > 
> > > > One thing you can try is git bisect from 5.1.20 (or maybe just 5.1.0)  
> > > > to 5.2.8.  If you can identify a particular commit which caused the
> > > > problem to start, that would help.
> > > 
> > > OK, I tried a bisect (2 days compilations...).
> > > Assuming I've done everything correctly (how to
> > > test this? How to remove the guilty patch?), this
> > > was the result:
> > > 
> > > 09324d32d2a0843e66652a087da6f77924358e62 is the first bad commit
> > > commit 09324d32d2a0843e66652a087da6f77924358e62
> > > Author: Christoph Hellwig <hch@lst.de>
> > > Date:   Tue May 21 09:01:41 2019 +0200
> > > 
> > >     block: force an unlimited segment size on queues with a virt boundary
> > > 
> > >     We currently fail to update the front/back segment size in the bio when
> > >     deciding to allow an otherwise gappy segement to a device with a
> > >     virt boundary.  The reason why this did not cause problems is that
> > >     devices with a virt boundary fundamentally don't use segments as we
> > >     know it and thus don't care.  Make that assumption formal by forcing
> > >     an unlimited segement size in this case.
> > > 
> > >     Fixes: f6970f83ef79 ("block: don't check if adjacent bvecs in one bio can be mergeable")
> > >     Signed-off-by: Christoph Hellwig <hch@lst.de>
> > >     Reviewed-by: Ming Lei <ming.lei@redhat.com>
> > >     Reviewed-by: Hannes Reinecke <hare@suse.com>
> > >     Signed-off-by: Jens Axboe <axboe@kernel.dk>
> > > 
> > > :040000 040000 57ba04a02f948022c0f6ba24bfa36f3b565b2440 8c925f71ce75042529c001bf244b30565d19ebf3 M      block
> > > 
> > > What to do now?
> > 
> > Here's how to verify that the bisection got a correct result.  First, 
> > do a git checkout of commit 09324d32d2a0, build the kernel, and make 
> > sure that it exhibits the problem.
> > 
> > Next, have git write out the contents of that commit in the form of a
> > patch (git show commit-id >patchfile), and revert it (git apply -R
> > patchfile).  Build the kernel from that tree, and make sure that it
> > does not exhibit the problem.  If it doesn't, you have definitely shown
> > that this commit is the cause (or at least, is _one_ of the causes).
> 
> I tried as suggested, i.e. jumping to commit
> 09324d32d2a0843e66652a087da6f77924358e62, testing,
> removing the patch, testing.
> The result was as expected.
> I was able to reproduce the issue with the commit,
> I was not able to reproduce it without.
> It seems this patch / commit is causing the problem.
> Directly or indirectly.
> 
> What are the next steps?

Hi all,

I tested kernel 5.3.5 (Fedora kernel-5.3.5-200.fc30.x86_64),
with same problematic results.

Again, what should be done now?
Could you please revert the patch?

Or is there something else to check?

Thanks,

bye,

-- 

piergiorgio

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-10-13 18:11                   ` Piergiorgio Sartor
@ 2019-10-16 17:01                     ` Alan Stern
  2019-10-17 17:53                       ` Piergiorgio Sartor
  0 siblings, 1 reply; 22+ messages in thread
From: Alan Stern @ 2019-10-16 17:01 UTC (permalink / raw)
  To: Piergiorgio Sartor
  Cc: Christoph Hellwig, Jens Axboe, USB list, linux-block,
	Kernel development list

On Sun, 13 Oct 2019, Piergiorgio Sartor wrote:

> On Mon, Sep 30, 2019 at 08:25:01PM +0200, Piergiorgio Sartor wrote:
> > On Sun, Sep 29, 2019 at 09:01:48PM -0400, Alan Stern wrote:
> > > On Sun, 29 Sep 2019, Piergiorgio Sartor wrote:
> > > 
> > > > On Wed, Sep 25, 2019 at 02:31:58PM -0400, Alan Stern wrote:
> > > > > On Wed, 25 Sep 2019, Piergiorgio Sartor wrote:
> > > > > 
> > > > > > On Mon, Aug 26, 2019 at 07:38:33PM +0200, Piergiorgio Sartor wrote:
> > > > > > > On Tue, Aug 20, 2019 at 06:37:22PM +0200, Piergiorgio Sartor wrote:
> > > > > > > > On Tue, Aug 20, 2019 at 09:23:26AM +0200, Christoph Hellwig wrote:
> > > > > > > > > On Mon, Aug 19, 2019 at 10:14:25AM -0400, Alan Stern wrote:
> > > > > > > > > > Let's bring this to the attention of some more people.
> > > > > > > > > > 
> > > > > > > > > > It looks like the bug that was supposed to be fixed by commit
> > > > > > > > > > d74ffae8b8dd ("usb-storage: Add a limitation for
> > > > > > > > > > blk_queue_max_hw_sectors()"), which is part of 5.2.5, but apparently
> > > > > > > > > > the bug still occurs.
> > > > > > > > > 
> > > > > > > > > Piergiorgio,
> > > > > > > > > 
> > > > > > > > > can you dump the content of max_hw_sectors_kb file for your USB storage
> > > > > > > > > device and send that to this thread?
> > > > > > > > 
> > > > > > > > Hi all,
> > > > > > > > 
> > > > > > > > for both kernels, 5.1.20 (working) and 5.2.8 (not working),
> > > > > > > > the content of /sys/dev/x:y/queue/max_hw_sectors_kb is 512
> > > > > > > > for USB storage devices (2.0 and 3.0).
> > > > > > > > 
> > > > > > > > This is for the PC showing the issue.
> > > > > > > > 
> > > > > > > > In an other PC, which does not show the issus at the moment,
> > > > > > > > the values are 120, for USB2.0, and 256, for USB3.0.
> > > 
> > > > > One thing you can try is git bisect from 5.1.20 (or maybe just 5.1.0)  
> > > > > to 5.2.8.  If you can identify a particular commit which caused the
> > > > > problem to start, that would help.
> > > > 
> > > > OK, I tried a bisect (2 days compilations...).
> > > > Assuming I've done everything correctly (how to
> > > > test this? How to remove the guilty patch?), this
> > > > was the result:
> > > > 
> > > > 09324d32d2a0843e66652a087da6f77924358e62 is the first bad commit
> > > > commit 09324d32d2a0843e66652a087da6f77924358e62
> > > > Author: Christoph Hellwig <hch@lst.de>
> > > > Date:   Tue May 21 09:01:41 2019 +0200
> > > > 
> > > >     block: force an unlimited segment size on queues with a virt boundary
> > > > 
> > > >     We currently fail to update the front/back segment size in the bio when
> > > >     deciding to allow an otherwise gappy segement to a device with a
> > > >     virt boundary.  The reason why this did not cause problems is that
> > > >     devices with a virt boundary fundamentally don't use segments as we
> > > >     know it and thus don't care.  Make that assumption formal by forcing
> > > >     an unlimited segement size in this case.
> > > > 
> > > >     Fixes: f6970f83ef79 ("block: don't check if adjacent bvecs in one bio can be mergeable")
> > > >     Signed-off-by: Christoph Hellwig <hch@lst.de>
> > > >     Reviewed-by: Ming Lei <ming.lei@redhat.com>
> > > >     Reviewed-by: Hannes Reinecke <hare@suse.com>
> > > >     Signed-off-by: Jens Axboe <axboe@kernel.dk>
> > > > 
> > > > :040000 040000 57ba04a02f948022c0f6ba24bfa36f3b565b2440 8c925f71ce75042529c001bf244b30565d19ebf3 M      block
> > > > 
> > > > What to do now?
> > > 
> > > Here's how to verify that the bisection got a correct result.  First, 
> > > do a git checkout of commit 09324d32d2a0, build the kernel, and make 
> > > sure that it exhibits the problem.
> > > 
> > > Next, have git write out the contents of that commit in the form of a
> > > patch (git show commit-id >patchfile), and revert it (git apply -R
> > > patchfile).  Build the kernel from that tree, and make sure that it
> > > does not exhibit the problem.  If it doesn't, you have definitely shown
> > > that this commit is the cause (or at least, is _one_ of the causes).
> > 
> > I tried as suggested, i.e. jumping to commit
> > 09324d32d2a0843e66652a087da6f77924358e62, testing,
> > removing the patch, testing.
> > The result was as expected.
> > I was able to reproduce the issue with the commit,
> > I was not able to reproduce it without.
> > It seems this patch / commit is causing the problem.
> > Directly or indirectly.
> > 
> > What are the next steps?
> 
> Hi all,
> 
> I tested kernel 5.3.5 (Fedora kernel-5.3.5-200.fc30.x86_64),
> with same problematic results.
> 
> Again, what should be done now?
> Could you please revert the patch?
> 
> Or is there something else to check?

Here is one more thing you can try.  I have no idea whether it will 
make any difference, but the Changelog entry for the patch you 
identified suggests that it might help.

Alan Stern



Index: usb-devel/drivers/usb/storage/scsiglue.c
===================================================================
--- usb-devel.orig/drivers/usb/storage/scsiglue.c
+++ usb-devel/drivers/usb/storage/scsiglue.c
@@ -68,7 +68,6 @@ static const char* host_info(struct Scsi
 static int slave_alloc (struct scsi_device *sdev)
 {
 	struct us_data *us = host_to_us(sdev->host);
-	int maxp;
 
 	/*
 	 * Set the INQUIRY transfer length to 36.  We don't use any of
@@ -78,15 +77,6 @@ static int slave_alloc (struct scsi_devi
 	sdev->inquiry_len = 36;
 
 	/*
-	 * USB has unusual scatter-gather requirements: the length of each
-	 * scatterlist element except the last must be divisible by the
-	 * Bulk maxpacket value.  Fortunately this value is always a
-	 * power of 2.  Inform the block layer about this requirement.
-	 */
-	maxp = usb_maxpacket(us->pusb_dev, us->recv_bulk_pipe, 0);
-	blk_queue_virt_boundary(sdev->request_queue, maxp - 1);
-
-	/*
 	 * Some host controllers may have alignment requirements.
 	 * We'll play it safe by requiring 512-byte alignment always.
 	 */


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-10-16 17:01                     ` Alan Stern
@ 2019-10-17 17:53                       ` Piergiorgio Sartor
  2019-10-17 19:23                         ` Alan Stern
  2019-10-21 15:48                         ` [PATCH] usb-storage: Revert commit 747668dbc061 ("usb-storage: Set virt_boundary_mask to avoid SG overflows") Alan Stern
  0 siblings, 2 replies; 22+ messages in thread
From: Piergiorgio Sartor @ 2019-10-17 17:53 UTC (permalink / raw)
  To: Alan Stern
  Cc: Piergiorgio Sartor, Christoph Hellwig, Jens Axboe, USB list,
	linux-block, Kernel development list

On Wed, Oct 16, 2019 at 01:01:20PM -0400, Alan Stern wrote:
> On Sun, 13 Oct 2019, Piergiorgio Sartor wrote:
> 
> > On Mon, Sep 30, 2019 at 08:25:01PM +0200, Piergiorgio Sartor wrote:
> > > On Sun, Sep 29, 2019 at 09:01:48PM -0400, Alan Stern wrote:
> > > > On Sun, 29 Sep 2019, Piergiorgio Sartor wrote:
> > > > 
> > > > > On Wed, Sep 25, 2019 at 02:31:58PM -0400, Alan Stern wrote:
> > > > > > On Wed, 25 Sep 2019, Piergiorgio Sartor wrote:
> > > > > > 
> > > > > > > On Mon, Aug 26, 2019 at 07:38:33PM +0200, Piergiorgio Sartor wrote:
> > > > > > > > On Tue, Aug 20, 2019 at 06:37:22PM +0200, Piergiorgio Sartor wrote:
> > > > > > > > > On Tue, Aug 20, 2019 at 09:23:26AM +0200, Christoph Hellwig wrote:
> > > > > > > > > > On Mon, Aug 19, 2019 at 10:14:25AM -0400, Alan Stern wrote:
> > > > > > > > > > > Let's bring this to the attention of some more people.
> > > > > > > > > > > 
> > > > > > > > > > > It looks like the bug that was supposed to be fixed by commit
> > > > > > > > > > > d74ffae8b8dd ("usb-storage: Add a limitation for
> > > > > > > > > > > blk_queue_max_hw_sectors()"), which is part of 5.2.5, but apparently
> > > > > > > > > > > the bug still occurs.
> > > > > > > > > > 
> > > > > > > > > > Piergiorgio,
> > > > > > > > > > 
> > > > > > > > > > can you dump the content of max_hw_sectors_kb file for your USB storage
> > > > > > > > > > device and send that to this thread?
> > > > > > > > > 
> > > > > > > > > Hi all,
> > > > > > > > > 
> > > > > > > > > for both kernels, 5.1.20 (working) and 5.2.8 (not working),
> > > > > > > > > the content of /sys/dev/x:y/queue/max_hw_sectors_kb is 512
> > > > > > > > > for USB storage devices (2.0 and 3.0).
> > > > > > > > > 
> > > > > > > > > This is for the PC showing the issue.
> > > > > > > > > 
> > > > > > > > > In an other PC, which does not show the issus at the moment,
> > > > > > > > > the values are 120, for USB2.0, and 256, for USB3.0.
> > > > 
> > > > > > One thing you can try is git bisect from 5.1.20 (or maybe just 5.1.0)  
> > > > > > to 5.2.8.  If you can identify a particular commit which caused the
> > > > > > problem to start, that would help.
> > > > > 
> > > > > OK, I tried a bisect (2 days compilations...).
> > > > > Assuming I've done everything correctly (how to
> > > > > test this? How to remove the guilty patch?), this
> > > > > was the result:
> > > > > 
> > > > > 09324d32d2a0843e66652a087da6f77924358e62 is the first bad commit
> > > > > commit 09324d32d2a0843e66652a087da6f77924358e62
> > > > > Author: Christoph Hellwig <hch@lst.de>
> > > > > Date:   Tue May 21 09:01:41 2019 +0200
> > > > > 
> > > > >     block: force an unlimited segment size on queues with a virt boundary
> > > > > 
> > > > >     We currently fail to update the front/back segment size in the bio when
> > > > >     deciding to allow an otherwise gappy segement to a device with a
> > > > >     virt boundary.  The reason why this did not cause problems is that
> > > > >     devices with a virt boundary fundamentally don't use segments as we
> > > > >     know it and thus don't care.  Make that assumption formal by forcing
> > > > >     an unlimited segement size in this case.
> > > > > 
> > > > >     Fixes: f6970f83ef79 ("block: don't check if adjacent bvecs in one bio can be mergeable")
> > > > >     Signed-off-by: Christoph Hellwig <hch@lst.de>
> > > > >     Reviewed-by: Ming Lei <ming.lei@redhat.com>
> > > > >     Reviewed-by: Hannes Reinecke <hare@suse.com>
> > > > >     Signed-off-by: Jens Axboe <axboe@kernel.dk>
> > > > > 
> > > > > :040000 040000 57ba04a02f948022c0f6ba24bfa36f3b565b2440 8c925f71ce75042529c001bf244b30565d19ebf3 M      block
> > > > > 
> > > > > What to do now?
> > > > 
> > > > Here's how to verify that the bisection got a correct result.  First, 
> > > > do a git checkout of commit 09324d32d2a0, build the kernel, and make 
> > > > sure that it exhibits the problem.
> > > > 
> > > > Next, have git write out the contents of that commit in the form of a
> > > > patch (git show commit-id >patchfile), and revert it (git apply -R
> > > > patchfile).  Build the kernel from that tree, and make sure that it
> > > > does not exhibit the problem.  If it doesn't, you have definitely shown
> > > > that this commit is the cause (or at least, is _one_ of the causes).
> > > 
> > > I tried as suggested, i.e. jumping to commit
> > > 09324d32d2a0843e66652a087da6f77924358e62, testing,
> > > removing the patch, testing.
> > > The result was as expected.
> > > I was able to reproduce the issue with the commit,
> > > I was not able to reproduce it without.
> > > It seems this patch / commit is causing the problem.
> > > Directly or indirectly.
> > > 
> > > What are the next steps?
> > 
> > Hi all,
> > 
> > I tested kernel 5.3.5 (Fedora kernel-5.3.5-200.fc30.x86_64),
> > with same problematic results.
> > 
> > Again, what should be done now?
> > Could you please revert the patch?
> > 
> > Or is there something else to check?
> 
> Here is one more thing you can try.  I have no idea whether it will 
> make any difference, but the Changelog entry for the patch you 
> identified suggests that it might help.
> 
> Alan Stern
> 
> 
> 
> Index: usb-devel/drivers/usb/storage/scsiglue.c
> ===================================================================
> --- usb-devel.orig/drivers/usb/storage/scsiglue.c
> +++ usb-devel/drivers/usb/storage/scsiglue.c
> @@ -68,7 +68,6 @@ static const char* host_info(struct Scsi
>  static int slave_alloc (struct scsi_device *sdev)
>  {
>  	struct us_data *us = host_to_us(sdev->host);
> -	int maxp;
>  
>  	/*
>  	 * Set the INQUIRY transfer length to 36.  We don't use any of
> @@ -78,15 +77,6 @@ static int slave_alloc (struct scsi_devi
>  	sdev->inquiry_len = 36;
>  
>  	/*
> -	 * USB has unusual scatter-gather requirements: the length of each
> -	 * scatterlist element except the last must be divisible by the
> -	 * Bulk maxpacket value.  Fortunately this value is always a
> -	 * power of 2.  Inform the block layer about this requirement.
> -	 */
> -	maxp = usb_maxpacket(us->pusb_dev, us->recv_bulk_pipe, 0);
> -	blk_queue_virt_boundary(sdev->request_queue, maxp - 1);
> -
> -	/*
>  	 * Some host controllers may have alignment requirements.
>  	 * We'll play it safe by requiring 512-byte alignment always.
>  	 */

Hi,

I tested the patch.

Assumming I did everything properly, add patch,
test, issue not showing up, remove patch, re-test,
issue present.

It seems this patch you provide solves the issue.

Thanks a lot for the support and the solution.

I guess now this patch will be integrated into
mainline sometimes.
Please let me know, in this thread or directly, in
which kernel it will be available.

Thanks again, it would be for me impossible to
solve without your support,

bye,

-- 

piergiorgio

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-10-17 17:53                       ` Piergiorgio Sartor
@ 2019-10-17 19:23                         ` Alan Stern
  2019-11-08 23:05                           ` Piergiorgio Sartor
  2019-10-21 15:48                         ` [PATCH] usb-storage: Revert commit 747668dbc061 ("usb-storage: Set virt_boundary_mask to avoid SG overflows") Alan Stern
  1 sibling, 1 reply; 22+ messages in thread
From: Alan Stern @ 2019-10-17 19:23 UTC (permalink / raw)
  To: Piergiorgio Sartor
  Cc: Christoph Hellwig, Jens Axboe, USB list, linux-block,
	Kernel development list

On Thu, 17 Oct 2019, Piergiorgio Sartor wrote:

> > Here is one more thing you can try.  I have no idea whether it will 
> > make any difference, but the Changelog entry for the patch you 
> > identified suggests that it might help.
> > 
> > Alan Stern
> > 
> > 
> > 
> > Index: usb-devel/drivers/usb/storage/scsiglue.c
> > ===================================================================
> > --- usb-devel.orig/drivers/usb/storage/scsiglue.c
> > +++ usb-devel/drivers/usb/storage/scsiglue.c
> > @@ -68,7 +68,6 @@ static const char* host_info(struct Scsi
> >  static int slave_alloc (struct scsi_device *sdev)
> >  {
> >  	struct us_data *us = host_to_us(sdev->host);
> > -	int maxp;
> >  
> >  	/*
> >  	 * Set the INQUIRY transfer length to 36.  We don't use any of
> > @@ -78,15 +77,6 @@ static int slave_alloc (struct scsi_devi
> >  	sdev->inquiry_len = 36;
> >  
> >  	/*
> > -	 * USB has unusual scatter-gather requirements: the length of each
> > -	 * scatterlist element except the last must be divisible by the
> > -	 * Bulk maxpacket value.  Fortunately this value is always a
> > -	 * power of 2.  Inform the block layer about this requirement.
> > -	 */
> > -	maxp = usb_maxpacket(us->pusb_dev, us->recv_bulk_pipe, 0);
> > -	blk_queue_virt_boundary(sdev->request_queue, maxp - 1);
> > -
> > -	/*
> >  	 * Some host controllers may have alignment requirements.
> >  	 * We'll play it safe by requiring 512-byte alignment always.
> >  	 */
> 
> Hi,
> 
> I tested the patch.
> 
> Assumming I did everything properly, add patch,
> test, issue not showing up, remove patch, re-test,
> issue present.
> 
> It seems this patch you provide solves the issue.
> 
> Thanks a lot for the support and the solution.
> 
> I guess now this patch will be integrated into
> mainline sometimes.
> Please let me know, in this thread or directly, in
> which kernel it will be available.

I'm busy for the next few days, but I will submit the patch next week.

> Thanks again, it would be for me impossible to
> solve without your support,

You're welcome.

Alan Stern


^ permalink raw reply	[flat|nested] 22+ messages in thread

* [PATCH] usb-storage: Revert commit 747668dbc061 ("usb-storage: Set virt_boundary_mask to avoid SG overflows")
  2019-10-17 17:53                       ` Piergiorgio Sartor
  2019-10-17 19:23                         ` Alan Stern
@ 2019-10-21 15:48                         ` Alan Stern
  2019-10-23  1:53                           ` Christoph Hellwig
  1 sibling, 1 reply; 22+ messages in thread
From: Alan Stern @ 2019-10-21 15:48 UTC (permalink / raw)
  To: Greg KH
  Cc: Piergiorgio Sartor, Seth Bollinger, Christoph Hellwig,
	Jens Axboe, USB list

Commit 747668dbc061 ("usb-storage: Set virt_boundary_mask to avoid SG
overflows") attempted to solve a problem involving scatter-gather I/O
and USB/IP by setting the virt_boundary_mask for mass-storage devices.

However, it now turns out that this interacts badly with commit
09324d32d2a0 ("block: force an unlimited segment size on queues with a
virt boundary"), which was added later.  A typical error message is:

	ehci-pci 0000:00:13.2: swiotlb buffer is full (sz: 327680 bytes),
	total 32768 (slots), used 97 (slots)

There is no longer any reason to keep the virt_boundary_mask setting
for usb-storage.  It was needed in the first place only for handling
devices with a block size smaller than the maxpacket size and where
the host controller was not capable of fully general scatter-gather
operation (that is, able to merge two SG segments into a single USB
packet).  But:

	High-speed or slower connections never use a bulk maxpacket
	value larger than 512;

	The SCSI layer does not handle block devices with a block size
	smaller than 512 bytes;

	All the host controllers capable of SuperSpeed operation can
	handle fully general SG;

	Since commit ea44d190764b ("usbip: Implement SG support to
	vhci-hcd and stub driver") was merged, the USB/IP driver can
	also handle SG.

Therefore all supported device/controller combinations should be okay
with no need for any special virt_boundary_mask.  So in order to fix
the swiotlb problem, this patch reverts commit 747668dbc061.

Reported-and-tested-by: Piergiorgio Sartor <piergiorgio.sartor@nexgo.de>
Link: https://marc.info/?l=linux-usb&m=157134199501202&w=2
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
CC: Seth Bollinger <Seth.Bollinger@digi.com>
CC: <stable@vger.kernel.org>

---


[as1922]


 drivers/usb/storage/scsiglue.c |   10 ----------
 1 file changed, 10 deletions(-)

Index: usb-devel/drivers/usb/storage/scsiglue.c
===================================================================
--- usb-devel.orig/drivers/usb/storage/scsiglue.c
+++ usb-devel/drivers/usb/storage/scsiglue.c
@@ -68,7 +68,6 @@ static const char* host_info(struct Scsi
 static int slave_alloc (struct scsi_device *sdev)
 {
 	struct us_data *us = host_to_us(sdev->host);
-	int maxp;
 
 	/*
 	 * Set the INQUIRY transfer length to 36.  We don't use any of
@@ -78,15 +77,6 @@ static int slave_alloc (struct scsi_devi
 	sdev->inquiry_len = 36;
 
 	/*
-	 * USB has unusual scatter-gather requirements: the length of each
-	 * scatterlist element except the last must be divisible by the
-	 * Bulk maxpacket value.  Fortunately this value is always a
-	 * power of 2.  Inform the block layer about this requirement.
-	 */
-	maxp = usb_maxpacket(us->pusb_dev, us->recv_bulk_pipe, 0);
-	blk_queue_virt_boundary(sdev->request_queue, maxp - 1);
-
-	/*
 	 * Some host controllers may have alignment requirements.
 	 * We'll play it safe by requiring 512-byte alignment always.
 	 */


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH] usb-storage: Revert commit 747668dbc061 ("usb-storage: Set virt_boundary_mask to avoid SG overflows")
  2019-10-21 15:48                         ` [PATCH] usb-storage: Revert commit 747668dbc061 ("usb-storage: Set virt_boundary_mask to avoid SG overflows") Alan Stern
@ 2019-10-23  1:53                           ` Christoph Hellwig
  2019-10-23 14:12                             ` Alan Stern
  2019-10-23 15:34                             ` [PATCH] UAS: Revert commit 3ae62a42090f ("UAS: fix alignment of scatter/gather segments") Alan Stern
  0 siblings, 2 replies; 22+ messages in thread
From: Christoph Hellwig @ 2019-10-23  1:53 UTC (permalink / raw)
  To: Alan Stern
  Cc: Greg KH, Piergiorgio Sartor, Seth Bollinger, Christoph Hellwig,
	Jens Axboe, USB list

On Mon, Oct 21, 2019 at 11:48:06AM -0400, Alan Stern wrote:
> There is no longer any reason to keep the virt_boundary_mask setting
> for usb-storage.  It was needed in the first place only for handling
> devices with a block size smaller than the maxpacket size and where
> the host controller was not capable of fully general scatter-gather
> operation (that is, able to merge two SG segments into a single USB
> packet).  But:
> 
> 	High-speed or slower connections never use a bulk maxpacket
> 	value larger than 512;
> 
> 	The SCSI layer does not handle block devices with a block size
> 	smaller than 512 bytes;
> 
> 	All the host controllers capable of SuperSpeed operation can
> 	handle fully general SG;
> 
> 	Since commit ea44d190764b ("usbip: Implement SG support to
> 	vhci-hcd and stub driver") was merged, the USB/IP driver can
> 	also handle SG.
> 
> Therefore all supported device/controller combinations should be okay
> with no need for any special virt_boundary_mask.  So in order to fix
> the swiotlb problem, this patch reverts commit 747668dbc061.

That's great to know.  The same should also apply to uas, shouldn't
it?

Otherwise:

Acked-by: Christoph Hellwig <hch@lst.de>

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH] usb-storage: Revert commit 747668dbc061 ("usb-storage: Set virt_boundary_mask to avoid SG overflows")
  2019-10-23  1:53                           ` Christoph Hellwig
@ 2019-10-23 14:12                             ` Alan Stern
  2019-10-23 15:34                             ` [PATCH] UAS: Revert commit 3ae62a42090f ("UAS: fix alignment of scatter/gather segments") Alan Stern
  1 sibling, 0 replies; 22+ messages in thread
From: Alan Stern @ 2019-10-23 14:12 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Greg KH, Piergiorgio Sartor, Seth Bollinger, Jens Axboe, USB list

On Wed, 23 Oct 2019, Christoph Hellwig wrote:

> On Mon, Oct 21, 2019 at 11:48:06AM -0400, Alan Stern wrote:
> > There is no longer any reason to keep the virt_boundary_mask setting
> > for usb-storage.  It was needed in the first place only for handling
> > devices with a block size smaller than the maxpacket size and where
> > the host controller was not capable of fully general scatter-gather
> > operation (that is, able to merge two SG segments into a single USB
> > packet).  But:
> > 
> > 	High-speed or slower connections never use a bulk maxpacket
> > 	value larger than 512;
> > 
> > 	The SCSI layer does not handle block devices with a block size
> > 	smaller than 512 bytes;
> > 
> > 	All the host controllers capable of SuperSpeed operation can
> > 	handle fully general SG;
> > 
> > 	Since commit ea44d190764b ("usbip: Implement SG support to
> > 	vhci-hcd and stub driver") was merged, the USB/IP driver can
> > 	also handle SG.
> > 
> > Therefore all supported device/controller combinations should be okay
> > with no need for any special virt_boundary_mask.  So in order to fix
> > the swiotlb problem, this patch reverts commit 747668dbc061.
> 
> That's great to know.  The same should also apply to uas, shouldn't
> it?

Ah, yes, excellent point -- I had forgotten about it.  Additional patch 
coming up soon...

Alan Stern

> Otherwise:
> 
> Acked-by: Christoph Hellwig <hch@lst.de>


^ permalink raw reply	[flat|nested] 22+ messages in thread

* [PATCH] UAS: Revert commit 3ae62a42090f ("UAS: fix alignment of scatter/gather segments")
  2019-10-23  1:53                           ` Christoph Hellwig
  2019-10-23 14:12                             ` Alan Stern
@ 2019-10-23 15:34                             ` Alan Stern
  2019-10-24  1:10                               ` Christoph Hellwig
  1 sibling, 1 reply; 22+ messages in thread
From: Alan Stern @ 2019-10-23 15:34 UTC (permalink / raw)
  To: Greg KH, Oliver Neukum
  Cc: Piergiorgio Sartor, Seth Bollinger, Christoph Hellwig,
	Jens Axboe, USB list

Commit 3ae62a42090f ("UAS: fix alignment of scatter/gather segments"),
copying a similar commit for usb-storage, attempted to solve a problem
involving scatter-gather I/O and USB/IP by setting the
virt_boundary_mask for mass-storage devices.

However, it now turns out that the analogous change in usb-storage
interacted badly with commit 09324d32d2a0 ("block: force an unlimited
segment size on queues with a virt boundary"), which was added later.
A typical error message is:

	ehci-pci 0000:00:13.2: swiotlb buffer is full (sz: 327680 bytes),
	total 32768 (slots), used 97 (slots)

There is no longer any reason to keep the virt_boundary_mask setting
in the uas driver.  It was needed in the first place only for
handling devices with a block size smaller than the maxpacket size and
where the host controller was not capable of fully general
scatter-gather operation (that is, able to merge two SG segments into
a single USB packet).  But:

	High-speed or slower connections never use a bulk maxpacket
	value larger than 512;

	The SCSI layer does not handle block devices with a block size
	smaller than 512 bytes;

	All the host controllers capable of SuperSpeed operation can
	handle fully general SG;

	Since commit ea44d190764b ("usbip: Implement SG support to
	vhci-hcd and stub driver") was merged, the USB/IP driver can
	also handle SG.

Therefore all supported device/controller combinations should be okay
with no need for any special virt_boundary_mask.  So in order to head
off potential problems similar to those affecting usb-storage, this
patch reverts commit 3ae62a42090f.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
CC: Oliver Neukum <oneukum@suse.com>
CC: <stable@vger.kernel.org>

---


[as1923]


 drivers/usb/storage/uas.c |   20 --------------------
 1 file changed, 20 deletions(-)

Index: usb-devel/drivers/usb/storage/uas.c
===================================================================
--- usb-devel.orig/drivers/usb/storage/uas.c
+++ usb-devel/drivers/usb/storage/uas.c
@@ -789,30 +789,10 @@ static int uas_slave_alloc(struct scsi_d
 {
 	struct uas_dev_info *devinfo =
 		(struct uas_dev_info *)sdev->host->hostdata;
-	int maxp;
 
 	sdev->hostdata = devinfo;
 
 	/*
-	 * We have two requirements here. We must satisfy the requirements
-	 * of the physical HC and the demands of the protocol, as we
-	 * definitely want no additional memory allocation in this path
-	 * ruling out using bounce buffers.
-	 *
-	 * For a transmission on USB to continue we must never send
-	 * a package that is smaller than maxpacket. Hence the length of each
-         * scatterlist element except the last must be divisible by the
-         * Bulk maxpacket value.
-	 * If the HC does not ensure that through SG,
-	 * the upper layer must do that. We must assume nothing
-	 * about the capabilities off the HC, so we use the most
-	 * pessimistic requirement.
-	 */
-
-	maxp = usb_maxpacket(devinfo->udev, devinfo->data_in_pipe, 0);
-	blk_queue_virt_boundary(sdev->request_queue, maxp - 1);
-
-	/*
 	 * The protocol has no requirements on alignment in the strict sense.
 	 * Controllers may or may not have alignment restrictions.
 	 * As this is not exported, we use an extremely conservative guess.


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH] UAS: Revert commit 3ae62a42090f ("UAS: fix alignment of scatter/gather segments")
  2019-10-23 15:34                             ` [PATCH] UAS: Revert commit 3ae62a42090f ("UAS: fix alignment of scatter/gather segments") Alan Stern
@ 2019-10-24  1:10                               ` Christoph Hellwig
  0 siblings, 0 replies; 22+ messages in thread
From: Christoph Hellwig @ 2019-10-24  1:10 UTC (permalink / raw)
  To: Alan Stern
  Cc: Greg KH, Oliver Neukum, Piergiorgio Sartor, Seth Bollinger,
	Christoph Hellwig, Jens Axboe, USB list

On Wed, Oct 23, 2019 at 11:34:33AM -0400, Alan Stern wrote:
> Commit 3ae62a42090f ("UAS: fix alignment of scatter/gather segments"),
> copying a similar commit for usb-storage, attempted to solve a problem
> involving scatter-gather I/O and USB/IP by setting the
> virt_boundary_mask for mass-storage devices.
> 
> However, it now turns out that the analogous change in usb-storage
> interacted badly with commit 09324d32d2a0 ("block: force an unlimited
> segment size on queues with a virt boundary"), which was added later.
> A typical error message is:
> 
> 	ehci-pci 0000:00:13.2: swiotlb buffer is full (sz: 327680 bytes),
> 	total 32768 (slots), used 97 (slots)
> 
> There is no longer any reason to keep the virt_boundary_mask setting
> in the uas driver.  It was needed in the first place only for
> handling devices with a block size smaller than the maxpacket size and
> where the host controller was not capable of fully general
> scatter-gather operation (that is, able to merge two SG segments into
> a single USB packet).  But:
> 
> 	High-speed or slower connections never use a bulk maxpacket
> 	value larger than 512;
> 
> 	The SCSI layer does not handle block devices with a block size
> 	smaller than 512 bytes;
> 
> 	All the host controllers capable of SuperSpeed operation can
> 	handle fully general SG;
> 
> 	Since commit ea44d190764b ("usbip: Implement SG support to
> 	vhci-hcd and stub driver") was merged, the USB/IP driver can
> 	also handle SG.
> 
> Therefore all supported device/controller combinations should be okay
> with no need for any special virt_boundary_mask.  So in order to head
> off potential problems similar to those affecting usb-storage, this
> patch reverts commit 3ae62a42090f.
> 
> Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
> CC: Oliver Neukum <oneukum@suse.com>
> CC: <stable@vger.kernel.org>

Looks good,

Acked-by: Christoph Hellwig <hch@lst.de>

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: reeze while write on external usb 3.0 hard disk [Bug 204095]
  2019-10-17 19:23                         ` Alan Stern
@ 2019-11-08 23:05                           ` Piergiorgio Sartor
  0 siblings, 0 replies; 22+ messages in thread
From: Piergiorgio Sartor @ 2019-11-08 23:05 UTC (permalink / raw)
  To: Alan Stern
  Cc: Piergiorgio Sartor, Christoph Hellwig, Jens Axboe, USB list,
	linux-block, Kernel development list

On Thu, Oct 17, 2019 at 03:23:34PM -0400, Alan Stern wrote:
> On Thu, 17 Oct 2019, Piergiorgio Sartor wrote:
> 
> > > Here is one more thing you can try.  I have no idea whether it will 
> > > make any difference, but the Changelog entry for the patch you 
> > > identified suggests that it might help.
> > > 
> > > Alan Stern
> > > 
> > > 
> > > 
> > > Index: usb-devel/drivers/usb/storage/scsiglue.c
> > > ===================================================================
> > > --- usb-devel.orig/drivers/usb/storage/scsiglue.c
> > > +++ usb-devel/drivers/usb/storage/scsiglue.c
> > > @@ -68,7 +68,6 @@ static const char* host_info(struct Scsi
> > >  static int slave_alloc (struct scsi_device *sdev)
> > >  {
> > >  	struct us_data *us = host_to_us(sdev->host);
> > > -	int maxp;
> > >  
> > >  	/*
> > >  	 * Set the INQUIRY transfer length to 36.  We don't use any of
> > > @@ -78,15 +77,6 @@ static int slave_alloc (struct scsi_devi
> > >  	sdev->inquiry_len = 36;
> > >  
> > >  	/*
> > > -	 * USB has unusual scatter-gather requirements: the length of each
> > > -	 * scatterlist element except the last must be divisible by the
> > > -	 * Bulk maxpacket value.  Fortunately this value is always a
> > > -	 * power of 2.  Inform the block layer about this requirement.
> > > -	 */
> > > -	maxp = usb_maxpacket(us->pusb_dev, us->recv_bulk_pipe, 0);
> > > -	blk_queue_virt_boundary(sdev->request_queue, maxp - 1);
> > > -
> > > -	/*
> > >  	 * Some host controllers may have alignment requirements.
> > >  	 * We'll play it safe by requiring 512-byte alignment always.
> > >  	 */
> > 
> > Hi,
> > 
> > I tested the patch.
> > 
> > Assumming I did everything properly, add patch,
> > test, issue not showing up, remove patch, re-test,
> > issue present.
> > 
> > It seems this patch you provide solves the issue.
> > 
> > Thanks a lot for the support and the solution.
> > 
> > I guess now this patch will be integrated into
> > mainline sometimes.
> > Please let me know, in this thread or directly, in
> > which kernel it will be available.
> 
> I'm busy for the next few days, but I will submit the patch next week.

Hi again,

this message to let you know I tested
kernel 5.3.9 (always from Fedora), to
which Greg Kroah-Hartman added your
patch, and everything seems to work
fine, no problems detected so far.

Thanks,

bye,

-- 

piergiorgio

^ permalink raw reply	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2019-11-08 23:05 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-17  9:54 reeze while write on external usb 3.0 hard disk [Bug 204095] Piergiorgio Sartor
2019-08-19 14:14 ` Alan Stern
2019-08-20  7:23   ` Christoph Hellwig
2019-08-20 16:37     ` Piergiorgio Sartor
2019-08-26 17:38       ` Piergiorgio Sartor
2019-08-29 14:08         ` Christoph Hellwig
2019-09-25 17:07         ` Piergiorgio Sartor
2019-09-25 18:31           ` Alan Stern
2019-09-27  9:04             ` Peter Chen
2019-09-29 20:13             ` Piergiorgio Sartor
2019-09-30  1:01               ` Alan Stern
2019-09-30 18:25                 ` Piergiorgio Sartor
2019-10-13 18:11                   ` Piergiorgio Sartor
2019-10-16 17:01                     ` Alan Stern
2019-10-17 17:53                       ` Piergiorgio Sartor
2019-10-17 19:23                         ` Alan Stern
2019-11-08 23:05                           ` Piergiorgio Sartor
2019-10-21 15:48                         ` [PATCH] usb-storage: Revert commit 747668dbc061 ("usb-storage: Set virt_boundary_mask to avoid SG overflows") Alan Stern
2019-10-23  1:53                           ` Christoph Hellwig
2019-10-23 14:12                             ` Alan Stern
2019-10-23 15:34                             ` [PATCH] UAS: Revert commit 3ae62a42090f ("UAS: fix alignment of scatter/gather segments") Alan Stern
2019-10-24  1:10                               ` Christoph Hellwig

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).