From: Andrea Vai <andrea.vai@unipv.it>
To: Ming Lei <ming.lei@redhat.com>, "Theodore Y. Ts'o" <tytso@mit.edu>
Cc: "Schmid, Carsten" <Carsten_Schmid@mentor.com>,
Finn Thain <fthain@telegraphics.com.au>,
Damien Le Moal <Damien.LeMoal@wdc.com>,
Alan Stern <stern@rowland.harvard.edu>,
Jens Axboe <axboe@kernel.dk>,
Johannes Thumshirn <jthumshirn@suse.de>,
USB list <linux-usb@vger.kernel.org>,
SCSI development list <linux-scsi@vger.kernel.org>,
Himanshu Madhani <himanshu.madhani@cavium.com>,
Hannes Reinecke <hare@suse.com>, Omar Sandoval <osandov@fb.com>,
"Martin K. Petersen" <martin.petersen@oracle.com>,
Greg KH <gregkh@linuxfoundation.org>,
Hans Holmberg <Hans.Holmberg@wdc.com>,
Kernel development list <linux-kernel@vger.kernel.org>,
linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org
Subject: Re: AW: Slow I/O on USB media after commit f664a3cc17b7d0a2bc3b3ab96181e1029b0ec0e6
Date: Tue, 07 Jan 2020 08:51:41 +0100 [thread overview]
Message-ID: <5bd51904b1b6511748c5454bce437bdc038eeb1f.camel@unipv.it> (raw)
In-Reply-To: <20191226083706.GA17974@ming.t460p>
Il giorno gio, 26/12/2019 alle 16.37 +0800, Ming Lei ha scritto:
> On Wed, Dec 25, 2019 at 10:30:57PM -0500, Theodore Y. Ts'o wrote:
> > On Thu, Dec 26, 2019 at 10:27:02AM +0800, Ming Lei wrote:
> > > Maybe we need to be careful for HDD., since the request count in
> scheduler
> > > queue is double of in-flight request count, and in theory NCQ
> should only
> > > cover all in-flight 32 requests. I will find a sata HDD., and
> see if
> > > performance drop can be observed in the similar 'cp' test.
> >
> > Please try to measure it, but I'd be really surprised if it's
> > significant with with modern HDD's.
>
> Just find one machine with AHCI SATA, and run the following xfs
> overwrite test:
>
> #!/bin/bash
> DIR=$1
> echo 3 > /proc/sys/vm/drop_caches
> fio --readwrite=write --filesize=5g --overwrite=1 --
> filename=$DIR/fiofile \
> --runtime=60s --time_based --ioengine=psync --direct=0 --
> bs=4k
> --iodepth=128 --numjobs=2 --group_reporting=1 --
> name=overwrite
>
> FS is xfs, and disk is LVM over AHCI SATA with NCQ(depth 32),
> because the
> machine is picked up from RH beaker, and it is the only disk in the
> box.
>
> #lsblk
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> sda 8:0 0 931.5G 0 disk
> ├─sda1 8:1 0 1G 0 part /boot
> └─sda2 8:2 0 930.5G 0 part
> ├─rhel_hpe--ml10gen9--01-root 253:0 0 50G 0 lvm /
> ├─rhel_hpe--ml10gen9--01-swap 253:1 0 3.9G 0 lvm [SWAP]
> └─rhel_hpe--ml10gen9--01-home 253:2 0 876.6G 0 lvm /home
>
>
> kernel: 3a7ea2c483a53fc("scsi: provide mq_ops->busy() hook") which
> is
> the previous commit of f664a3cc17b7 ("scsi: kill off the legacy IO
> path").
>
> |scsi_mod.use_blk_mq=N |scsi_mod.use_blk_mq=Y |
> -----------------------------------------------------------
> throughput: |244MB/s |169MB/s |
> -----------------------------------------------------------
>
> Similar result can be observed on v5.4 kernel(184MB/s) with same
> test
> steps.
>
>
> > That because they typically have
> > a queue depth of 16, and a max_sectors_kb of 32767 (e.g., just
> under
> > 32 MiB). Sort seeks are typically 1-2 ms, with full stroke seeks
> > 8-10ms. Typical sequential write speeds on a 7200 RPM drive is
> > 125-150 MiB/s. So suppose every other request sent to the HDD is
> from
> > the other request stream. The disk will chose the 8 requests from
> its
> > queue that are contiguous, and so it will be writing around 256
> MiB,
> > which will take 2-3 seconds. If it then needs to spend between 1
> and
> > 10 ms seeking to another location of the disk, before it writes
> the
> > next 256 MiB, the worst case overhead of that seek is 10ms / 2s,
> or
> > 0.5%. That may very well be within your measurements' error bars.
>
> Looks you assume that disk seeking just happens once when writing
> around
> 256MB. This assumption may not be true, given all data can be in
> page
> cache before writing. So when two tasks are submitting IOs
> concurrently,
> IOs from each single task is sequential, and NCQ may order the
> current batch
> submitted from the two streams. However disk seeking may still be
> needed
> for the next batch handled by NCQ.
>
> > And of course, note that in real life, we are very *often* writing
> to
> > multiple files in parallel, for example, during a "make -j16"
> while
> > building the kernel. Writing a single large file is certainly
> > something people do (but even there people who are burning a 4G
> DVD
> > rip are often browsing the web while they are waiting for it to
> > complete, and the browser will be writing cache files, etc.). So
> > whether or not this is something where we should be stressing over
> > this specific workload is going to be quite debateable.
>
Hi,
is there any update on this? Sorry if I am making noise, but I would
like to help to improve the kernel (or fix it) if I can help.
Otherwise, please let me know how to consider this case,
Thanks, and bye
Andrea
next prev parent reply other threads:[~2020-01-07 7:51 UTC|newest]
Thread overview: 102+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <307581a490b610c3025ee80f79a465a89d68ed19.camel@unipv.it>
2019-08-20 17:13 ` Slow I/O on USB media after commit f664a3cc17b7d0a2bc3b3ab96181e1029b0ec0e6 Alan Stern
2019-08-23 10:39 ` Andrea Vai
2019-08-23 20:42 ` Alan Stern
2019-08-26 6:09 ` Andrea Vai
2019-08-26 16:33 ` Alan Stern
2019-09-18 15:25 ` Andrea Vai
2019-09-18 16:30 ` Alan Stern
2019-09-19 7:33 ` Andrea Vai
2019-09-19 17:54 ` Alan Stern
2019-09-20 7:25 ` Andrea Vai
2019-09-20 7:44 ` Greg KH
2019-09-19 8:26 ` Damien Le Moal
2019-09-19 8:55 ` Ming Lei
2019-09-19 9:09 ` Damien Le Moal
2019-09-19 9:21 ` Ming Lei
2019-09-19 14:01 ` Alan Stern
2019-09-19 14:14 ` Damien Le Moal
2019-09-20 7:03 ` Andrea Vai
2019-09-25 19:30 ` Alan Stern
2019-09-25 19:36 ` Jens Axboe
2019-09-27 15:47 ` Andrea Vai
2019-11-04 16:00 ` Andrea Vai
2019-11-04 18:20 ` Alan Stern
2019-11-05 11:48 ` Andrea Vai
2019-11-05 18:31 ` Alan Stern
2019-11-05 23:29 ` Jens Axboe
2019-11-06 16:03 ` Alan Stern
2019-11-06 22:13 ` Damien Le Moal
2019-11-07 7:04 ` Andrea Vai
2019-11-07 7:54 ` Damien Le Moal
2019-11-07 18:59 ` Andrea Vai
2019-11-08 8:42 ` Damien Le Moal
2019-11-08 14:33 ` Jens Axboe
2019-11-11 10:46 ` Andrea Vai
2019-11-09 10:09 ` Ming Lei
2019-11-09 22:28 ` Ming Lei
2019-11-11 10:50 ` Andrea Vai
2019-11-11 11:05 ` Ming Lei
2019-11-11 11:13 ` Andrea Vai
2019-11-22 19:16 ` Andrea Vai
2019-11-23 7:28 ` Ming Lei
2019-11-23 15:44 ` Andrea Vai
2019-11-25 3:54 ` Ming Lei
2019-11-25 10:11 ` Andrea Vai
2019-11-25 10:29 ` Ming Lei
2019-11-25 14:58 ` Andrea Vai
2019-11-25 15:15 ` Ming Lei
2019-11-25 18:51 ` Andrea Vai
2019-11-26 2:32 ` Ming Lei
2019-11-26 7:46 ` Andrea Vai
2019-11-26 9:15 ` Ming Lei
2019-11-26 10:24 ` Ming Lei
2019-11-26 11:14 ` Andrea Vai
2019-11-27 2:05 ` Ming Lei
2019-11-27 9:39 ` Andrea Vai
2019-11-27 13:08 ` Ming Lei
2019-11-27 15:01 ` Andrea Vai
2019-11-27 0:21 ` Finn Thain
2019-11-27 8:14 ` AW: " Schmid, Carsten
2019-11-27 21:49 ` Finn Thain
2019-11-28 7:46 ` Andrea Vai
2019-11-28 8:12 ` AW: " Schmid, Carsten
2019-11-28 11:40 ` Andrea Vai
2019-11-28 17:39 ` Alan Stern
2019-11-28 9:17 ` Ming Lei
2019-11-28 17:34 ` Andrea Vai
2019-11-29 0:57 ` Ming Lei
2019-11-29 2:35 ` Ming Lei
2019-11-29 14:41 ` Andrea Vai
2019-12-03 2:23 ` Ming Lei
2019-12-10 7:35 ` Andrea Vai
2019-12-10 8:05 ` Ming Lei
2019-12-11 2:41 ` Theodore Y. Ts'o
2019-12-11 4:00 ` Ming Lei
2019-12-11 16:07 ` Theodore Y. Ts'o
2019-12-11 21:33 ` Ming Lei
2019-12-12 7:34 ` Andrea Vai
2019-12-18 8:25 ` Andrea Vai
2019-12-18 9:48 ` Ming Lei
[not found] ` <b1b6a0e9d690ecd9432025acd2db4ac09f834040.camel@unipv.it>
2019-12-23 13:08 ` Ming Lei
2019-12-23 14:02 ` Andrea Vai
2019-12-24 1:32 ` Ming Lei
2019-12-24 8:04 ` Andrea Vai
2019-12-24 8:47 ` Ming Lei
2019-12-23 16:26 ` Theodore Y. Ts'o
2019-12-23 16:29 ` Andrea Vai
2019-12-23 17:22 ` Theodore Y. Ts'o
2019-12-23 18:45 ` Andrea Vai
2019-12-23 19:53 ` Theodore Y. Ts'o
2019-12-24 1:27 ` Ming Lei
2019-12-24 6:49 ` Andrea Vai
2019-12-24 8:51 ` Andrea Vai
2019-12-24 9:35 ` Ming Lei
2019-12-25 5:17 ` Theodore Y. Ts'o
2019-12-26 2:27 ` Ming Lei
2019-12-26 3:30 ` Theodore Y. Ts'o
2019-12-26 8:37 ` Ming Lei
2020-01-07 7:51 ` Andrea Vai [this message]
[not found] ` <20200101074310.10904-1-hdanton@sina.com>
2020-01-01 13:53 ` slow IO on USB media Ming Lei
2019-11-29 11:44 ` AW: Slow I/O on USB media after commit f664a3cc17b7d0a2bc3b3ab96181e1029b0ec0e6 Bernd Schubert
2019-12-02 7:01 ` Andrea Vai
2019-11-28 17:10 ` Andrea Vai
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5bd51904b1b6511748c5454bce437bdc038eeb1f.camel@unipv.it \
--to=andrea.vai@unipv.it \
--cc=Carsten_Schmid@mentor.com \
--cc=Damien.LeMoal@wdc.com \
--cc=Hans.Holmberg@wdc.com \
--cc=axboe@kernel.dk \
--cc=fthain@telegraphics.com.au \
--cc=gregkh@linuxfoundation.org \
--cc=hare@suse.com \
--cc=himanshu.madhani@cavium.com \
--cc=jthumshirn@suse.de \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=linux-usb@vger.kernel.org \
--cc=martin.petersen@oracle.com \
--cc=ming.lei@redhat.com \
--cc=osandov@fb.com \
--cc=stern@rowland.harvard.edu \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).