From: Ming Lei <ming.lei@redhat.com>
To: "Theodore Y. Ts'o" <tytso@mit.edu>
Cc: Andrea Vai <andrea.vai@unipv.it>,
"Schmid, Carsten" <Carsten_Schmid@mentor.com>,
Finn Thain <fthain@telegraphics.com.au>,
Damien Le Moal <Damien.LeMoal@wdc.com>,
Alan Stern <stern@rowland.harvard.edu>,
Jens Axboe <axboe@kernel.dk>,
Johannes Thumshirn <jthumshirn@suse.de>,
USB list <linux-usb@vger.kernel.org>,
SCSI development list <linux-scsi@vger.kernel.org>,
Himanshu Madhani <himanshu.madhani@cavium.com>,
Hannes Reinecke <hare@suse.com>, Omar Sandoval <osandov@fb.com>,
"Martin K. Petersen" <martin.petersen@oracle.com>,
Greg KH <gregkh@linuxfoundation.org>,
Hans Holmberg <Hans.Holmberg@wdc.com>,
Kernel development list <linux-kernel@vger.kernel.org>,
linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org
Subject: Re: AW: Slow I/O on USB media after commit f664a3cc17b7d0a2bc3b3ab96181e1029b0ec0e6
Date: Wed, 11 Dec 2019 12:00:58 +0800 [thread overview]
Message-ID: <20191211040058.GC6864@ming.t460p> (raw)
In-Reply-To: <20191211024137.GB61323@mit.edu>
On Tue, Dec 10, 2019 at 09:41:37PM -0500, Theodore Y. Ts'o wrote:
> On Tue, Dec 10, 2019 at 04:05:50PM +0800, Ming Lei wrote:
> > > > The path[2] is expected behaviour. Not sure path [1] is correct,
> > > > given
> > > > ext4_release_file() is supposed to be called when this inode is
> > > > released. That means the file is closed 4358 times during 1GB file
> > > > copying to usb storage.
> > > >
> > > > [1] insert requests when returning to user mode from syscall
> > > >
> > > > b'blk_mq_sched_request_inserted'
> > > > b'blk_mq_sched_request_inserted'
> > > > b'dd_insert_requests'
> > > > b'blk_mq_sched_insert_requests'
> > > > b'blk_mq_flush_plug_list'
> > > > b'blk_flush_plug_list'
> > > > b'io_schedule_prepare'
> > > > b'io_schedule'
> > > > b'rq_qos_wait'
> > > > b'wbt_wait'
> > > > b'__rq_qos_throttle'
> > > > b'blk_mq_make_request'
> > > > b'generic_make_request'
> > > > b'submit_bio'
> > > > b'ext4_io_submit'
> > > > b'ext4_writepages'
> > > > b'do_writepages'
> > > > b'__filemap_fdatawrite_range'
> > > > b'ext4_release_file'
> > > > b'__fput'
> > > > b'task_work_run'
> > > > b'exit_to_usermode_loop'
> > > > b'do_syscall_64'
> > > > b'entry_SYSCALL_64_after_hwframe'
> > > > 4358
>
> I'm guessing that your workload is repeatedly truncating a file (or
> calling open with O_TRUNC) and then writing data to it. When you do
> this, then when the file is closed, we assume that since you were
> replacing the previous contents of a file with new contents, that you
> would be unhappy if the file contents was replaced by a zero length
> file after a crash. That's because ten years, ago there were a *huge*
> number of crappy applications that would replace a file by reading it
> into memory, truncating it, and then write out the new contents of the
> file. This could be a high score file for a game, or a KDE or GNOME
> state file, etc.
>
> So if someone does open, truncate, write, close, we still immediately
> writing out the data on the close, assuming that the programmer really
> wanted open, truncate, write, fsync, close, but was too careless to
> actually do the right thing.
>
> Some workaround[1] like this is done by all of the major file systems,
> and was fallout the agreement from the "O_PONIES"[2] controversy.
> This was discussed and agreed to at the 2009 LSF/MM workshop. (See
> the "rename, fsync, and ponies" section.)
>
> [1] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/317781/comments/45
> [2] https://blahg.josefsipek.net/?p=364
> [3] https://lwn.net/Articles/327601/
>
> So if you're seeing a call to filemap_fdatawrite_range as the result
> of a fput, that's why.
>
> In any case, this behavior has been around for a decade, and it
> appears to be incidental to your performance difficulties with your
> USB thumbdrive and block-mq.
I didn't reproduce the issue in my test environment, and follows
Andrea's test commands[1]:
mount UUID=$uuid /mnt/pendrive 2>&1 |tee -a $logfile
SECONDS=0
cp $testfile /mnt/pendrive 2>&1 |tee -a $logfile
umount /mnt/pendrive 2>&1 |tee -a $logfile
The 'cp' command supposes to open/close the file just once, however
ext4_release_file() & write pages is observed to run for 4358 times
when executing the above 'cp' test.
[1] https://marc.info/?l=linux-kernel&m=157486689806734&w=2
Thanks,
Ming
next prev parent reply other threads:[~2019-12-11 4:01 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <0cd6ac36b7ab644576fc0f3f5bd4a880c33855d1.camel@unipv.it>
2019-11-05 18:31 ` Slow I/O on USB media after commit f664a3cc17b7d0a2bc3b3ab96181e1029b0ec0e6 Alan Stern
2019-11-05 23:29 ` Jens Axboe
2019-11-06 16:03 ` Alan Stern
2019-11-06 22:13 ` Damien Le Moal
2019-11-07 7:04 ` Andrea Vai
2019-11-07 7:54 ` Damien Le Moal
2019-11-07 18:59 ` Andrea Vai
2019-11-08 8:42 ` Damien Le Moal
2019-11-08 14:33 ` Jens Axboe
2019-11-11 10:46 ` Andrea Vai
2019-11-09 10:09 ` Ming Lei
2019-11-09 22:28 ` Ming Lei
2019-11-11 10:50 ` Andrea Vai
2019-11-11 11:05 ` Ming Lei
2019-11-11 11:13 ` Andrea Vai
2019-11-22 19:16 ` Andrea Vai
2019-11-23 7:28 ` Ming Lei
2019-11-23 15:44 ` Andrea Vai
2019-11-25 3:54 ` Ming Lei
2019-11-25 10:11 ` Andrea Vai
2019-11-25 10:29 ` Ming Lei
2019-11-25 14:58 ` Andrea Vai
2019-11-25 15:15 ` Ming Lei
2019-11-25 18:51 ` Andrea Vai
2019-11-26 2:32 ` Ming Lei
2019-11-26 7:46 ` Andrea Vai
2019-11-26 9:15 ` Ming Lei
2019-11-26 10:24 ` Ming Lei
2019-11-26 11:14 ` Andrea Vai
2019-11-27 2:05 ` Ming Lei
2019-11-27 9:39 ` Andrea Vai
2019-11-27 13:08 ` Ming Lei
2019-11-27 15:01 ` Andrea Vai
2019-11-27 0:21 ` Finn Thain
2019-11-27 8:14 ` AW: " Schmid, Carsten
2019-11-27 21:49 ` Finn Thain
2019-11-28 7:46 ` Andrea Vai
2019-11-28 8:12 ` AW: " Schmid, Carsten
2019-11-28 11:40 ` Andrea Vai
2019-11-28 17:39 ` Alan Stern
2019-11-28 9:17 ` Ming Lei
2019-11-28 17:34 ` Andrea Vai
2019-11-29 0:57 ` Ming Lei
2019-11-29 2:35 ` Ming Lei
2019-11-29 14:41 ` Andrea Vai
2019-12-03 2:23 ` Ming Lei
2019-12-10 7:35 ` Andrea Vai
2019-12-10 8:05 ` Ming Lei
2019-12-11 2:41 ` Theodore Y. Ts'o
2019-12-11 4:00 ` Ming Lei [this message]
2019-12-11 16:07 ` Theodore Y. Ts'o
2019-12-11 21:33 ` Ming Lei
2019-12-12 7:34 ` Andrea Vai
2019-12-18 8:25 ` Andrea Vai
2019-12-18 9:48 ` Ming Lei
[not found] ` <b1b6a0e9d690ecd9432025acd2db4ac09f834040.camel@unipv.it>
2019-12-23 13:08 ` Ming Lei
2019-12-23 14:02 ` Andrea Vai
2019-12-24 1:32 ` Ming Lei
2019-12-24 8:04 ` Andrea Vai
2019-12-24 8:47 ` Ming Lei
2019-12-23 16:26 ` Theodore Y. Ts'o
2019-12-23 16:29 ` Andrea Vai
2019-12-23 17:22 ` Theodore Y. Ts'o
2019-12-23 18:45 ` Andrea Vai
2019-12-23 19:53 ` Theodore Y. Ts'o
2019-12-24 1:27 ` Ming Lei
2019-12-24 6:49 ` Andrea Vai
2019-12-24 8:51 ` Andrea Vai
2019-12-24 9:35 ` Ming Lei
2019-12-25 5:17 ` Theodore Y. Ts'o
2019-12-26 2:27 ` Ming Lei
2019-12-26 3:30 ` Theodore Y. Ts'o
2019-12-26 8:37 ` Ming Lei
2020-01-07 7:51 ` Andrea Vai
[not found] ` <20200101074310.10904-1-hdanton@sina.com>
2020-01-01 13:53 ` slow IO on USB media Ming Lei
2019-11-29 11:44 ` AW: Slow I/O on USB media after commit f664a3cc17b7d0a2bc3b3ab96181e1029b0ec0e6 Bernd Schubert
2019-12-02 7:01 ` Andrea Vai
2019-11-28 17:10 ` Andrea Vai
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191211040058.GC6864@ming.t460p \
--to=ming.lei@redhat.com \
--cc=Carsten_Schmid@mentor.com \
--cc=Damien.LeMoal@wdc.com \
--cc=Hans.Holmberg@wdc.com \
--cc=andrea.vai@unipv.it \
--cc=axboe@kernel.dk \
--cc=fthain@telegraphics.com.au \
--cc=gregkh@linuxfoundation.org \
--cc=hare@suse.com \
--cc=himanshu.madhani@cavium.com \
--cc=jthumshirn@suse.de \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=linux-usb@vger.kernel.org \
--cc=martin.petersen@oracle.com \
--cc=osandov@fb.com \
--cc=stern@rowland.harvard.edu \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).