Linux-USB Archive on lore.kernel.org
 help / color / Atom feed
From: Alan Stern <stern@rowland.harvard.edu>
To: Andrea Vai <andrea.vai@unipv.it>
Cc: Johannes Thumshirn <jthumshirn@suse.de>,
	Jens Axboe <axboe@kernel.dk>, <linux-usb@vger.kernel.org>,
	<linux-scsi@vger.kernel.org>,
	Himanshu Madhani <himanshu.madhani@cavium.com>,
	Hannes Reinecke <hare@suse.com>, Ming Lei <ming.lei@redhat.com>,
	Omar Sandoval <osandov@fb.com>,
	"Martin K. Petersen" <martin.petersen@oracle.com>,
	Greg KH <gregkh@linuxfoundation.org>
Subject: Re: Slow I/O on USB media after commit f664a3cc17b7d0a2bc3b3ab96181e1029b0ec0e6
Date: Tue, 13 Aug 2019 15:52:07 -0400 (EDT)
Message-ID: <Pine.LNX.4.44L0.1908131541170.1941-100000@iolanthe.rowland.org> (raw)
In-Reply-To: <e3f87757f7a0fdf551e911ad32fc8122eebe04c7.camel@unipv.it>

On Fri, 9 Aug 2019, Andrea Vai wrote:

> Il giorno lun, 08/07/2019 alle 11.38 -0400, Alan Stern ha scritto:
> > 
> > [...]
> >  Andrea, another thing you could try is to collect a usbmon trace
> > under 
> > > > one of the "slow" kernels.  Follow the instructions in 
> > > > Documentation/usb/usbmon.txt.  I think you could kill the file-
> > copy 
> > > > operation after just a couple of seconds; that should provide
> > enough 
> > > > trace information to help see what causes the slowdown.
> > > > 
> > > > (If you want, do the same test with a "fast" kernel and then
> > we'll 
> > > > compare the results.)
> > > > 
> > > > Alan Stern
> > > > 
> > > 
> > > Thanks Alan,
> > >   so I attach two ouputs, one for a "good" and one for a "bad"
> > kernel.
> > 
> > [...]
> > I don't know what the results mean, but I _can_ tell you what's
> > happening.  Every so often (at intervals of about a second) the pen
> > drive completely stops communicating with the "bad" kernel for about
> > one second and then starts up again.
> > 
> > Here's a short example from the "bad" trace:
> > 
> > ffff9169f0d399c0 513072808 S Bo:6:008:2 -115 122880 = 00000000
> > 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> > ffff9169f0d399c0 514262176 C Bo:6:008:2 0 122880 >
> > 
> > The second column is a timestamp (in microseconds).  This little
> > extract shows a 120-KB write starting at time 513.072 and ending at
> > 514.262, more than a second later.  Normally such a write would
> > complete in about 0.06 s.
> > 
> > The cumulative effect of all these delays is to slow the transfer 
> > drastically.  The "good" kernel trace shows a few delays like this,
> > but 
> > only one or two.
> > 
> > I have no idea how commit f664a3cc17b7, getting rid of the legacy
> > IO 
> > path, could have caused these delays.  It seems more likely that
> > the 
> > pen drive itself is the cause, perhaps because it is flushing
> > buffers 
> > more often under the "bad" kernel.
> > 
> > I'd like you to try doing another pair of usbmon tests.  This time,
> > start collecting the usbmon trace _before_ you plug in the pen
> > drive,
> > and stop the trace shortly after the pen drive has been mounted.  
> > Don't try to transfer any data.  Perhaps the two kernels are
> > initializing the pen drive with different settings and that accounts
> > for the different behaviors.
> 
> Thanks Alan, sorry for the delay (I haven't had physical access to the
> device for a while). I attach the two new usbmon logs.
> 
> Meanwhile, I can try with other pendrives, although I already tried
> two more and both behave "bad" (but one is identical to the first, and
> the other one is similar).

I looked through your two traces.  Unfortunately they don't help --
they are essentially identical.  Exactly the same sequence of commands
was sent to the device in both traces, except for a couple of
inconsequential TEST UNIT READY commands.

There are some timing differences, but they occur in between commands,
not during a command.  They probably are the times you spent typing
stuff into the terminal.

Can you run another test?  Similar to the last one, but this time start
collecting the usbmon trace at least 10 seconds after the drive is
plugged in.  Then mount the drive, copy a small file (say less than 500
bytes) to it, unmount it, and then stop the trace.

Alan Stern


       reply index

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <e3f87757f7a0fdf551e911ad32fc8122eebe04c7.camel@unipv.it>
2019-08-13 19:52 ` Alan Stern [this message]
     [not found] <307581a490b610c3025ee80f79a465a89d68ed19.camel@unipv.it>
2019-08-20 17:13 ` Alan Stern
2019-08-23 10:39   ` Andrea Vai
2019-08-23 20:42     ` Alan Stern
2019-08-26  6:09       ` Andrea Vai
2019-08-26 16:33         ` Alan Stern
2019-09-18 15:25           ` Andrea Vai
2019-09-18 16:30             ` Alan Stern
2019-09-19  7:33               ` Andrea Vai
2019-09-19 17:54                 ` Alan Stern
2019-09-20  7:25                   ` Andrea Vai
2019-09-20  7:44                     ` Greg KH
2019-09-19  8:26               ` Damien Le Moal
2019-09-19  8:55                 ` Ming Lei
2019-09-19  9:09                   ` Damien Le Moal
2019-09-19  9:21                     ` Ming Lei
2019-09-19 14:01                 ` Alan Stern
2019-09-19 14:14                   ` Damien Le Moal
2019-09-20  7:03                     ` Andrea Vai
2019-09-25 19:30                       ` Alan Stern
2019-09-25 19:36                         ` Jens Axboe
2019-09-27 15:47                           ` Andrea Vai
2019-11-04 16:00                             ` Andrea Vai
2019-11-04 18:20                               ` Alan Stern
2019-11-05 11:48                                 ` Andrea Vai
2019-11-05 18:31                                   ` Alan Stern
2019-11-05 23:29                                     ` Jens Axboe
2019-11-06 16:03                                       ` Alan Stern
2019-11-06 22:13                                         ` Damien Le Moal
2019-11-07  7:04                                           ` Andrea Vai
2019-11-07  7:54                                             ` Damien Le Moal
2019-11-07 18:59                                               ` Andrea Vai
2019-11-08  8:42                                                 ` Damien Le Moal
2019-11-08 14:33                                                   ` Jens Axboe
2019-11-11 10:46                                                     ` Andrea Vai
2019-11-09 10:09                                                   ` Ming Lei
2019-11-09 22:28                                                 ` Ming Lei
2019-11-11 10:50                                                   ` Andrea Vai
2019-11-11 11:05                                                     ` Ming Lei
2019-11-11 11:13                                                       ` Andrea Vai
2019-07-02 10:46 Andrea Vai
2019-07-02 11:51 ` Johannes Thumshirn
2019-07-02 22:36   ` Andrea Vai
2019-07-03  7:29     ` Johannes Thumshirn
2019-07-03 14:23       ` Alan Stern
2019-07-06 22:06         ` Andrea Vai
2019-07-08 15:38           ` Alan Stern
2019-07-02 12:01 ` Ming Lei
2019-07-02 22:39   ` Andrea Vai
2019-07-03  2:01     ` Ming Lei
2019-07-03  5:11       ` Andrea Vai
2019-07-03  6:36         ` Ming Lei
2019-07-03 15:27           ` Chris Murphy
2019-07-06  9:33           ` Andrea Vai
2019-07-08  1:01             ` Ming Lei
2019-07-09 21:18               ` Andrea Vai
2019-07-10  2:44                 ` Ming Lei

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Pine.LNX.4.44L0.1908131541170.1941-100000@iolanthe.rowland.org \
    --to=stern@rowland.harvard.edu \
    --cc=andrea.vai@unipv.it \
    --cc=axboe@kernel.dk \
    --cc=gregkh@linuxfoundation.org \
    --cc=hare@suse.com \
    --cc=himanshu.madhani@cavium.com \
    --cc=jthumshirn@suse.de \
    --cc=linux-scsi@vger.kernel.org \
    --cc=linux-usb@vger.kernel.org \
    --cc=martin.petersen@oracle.com \
    --cc=ming.lei@redhat.com \
    --cc=osandov@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-USB Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-usb/0 linux-usb/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-usb linux-usb/ https://lore.kernel.org/linux-usb \
		linux-usb@vger.kernel.org
	public-inbox-index linux-usb

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-usb


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git