From: Ming Lei <ming.lei@redhat.com>
To: Andrea Vai <andrea.vai@unipv.it>
Cc: "Schmid, Carsten" <Carsten_Schmid@mentor.com>,
Finn Thain <fthain@telegraphics.com.au>,
Damien Le Moal <Damien.LeMoal@wdc.com>,
Alan Stern <stern@rowland.harvard.edu>,
Jens Axboe <axboe@kernel.dk>,
Johannes Thumshirn <jthumshirn@suse.de>,
USB list <linux-usb@vger.kernel.org>,
SCSI development list <linux-scsi@vger.kernel.org>,
Himanshu Madhani <himanshu.madhani@cavium.com>,
Hannes Reinecke <hare@suse.com>, Omar Sandoval <osandov@fb.com>,
"Martin K. Petersen" <martin.petersen@oracle.com>,
Greg KH <gregkh@linuxfoundation.org>,
Hans Holmberg <Hans.Holmberg@wdc.com>,
Kernel development list <linux-kernel@vger.kernel.org>,
linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org,
Theodore Ts'o <tytso@mit.edu>
Subject: Re: AW: Slow I/O on USB media after commit f664a3cc17b7d0a2bc3b3ab96181e1029b0ec0e6
Date: Tue, 3 Dec 2019 10:23:37 +0800 [thread overview]
Message-ID: <20191203022337.GE25002@ming.t460p> (raw)
In-Reply-To: <320b315b9c87543d4fb919ecbdf841596c8fbcea.camel@unipv.it>
On Fri, Nov 29, 2019 at 03:41:01PM +0100, Andrea Vai wrote:
> Il giorno ven, 29/11/2019 alle 10.35 +0800, Ming Lei ha scritto:
> > On Fri, Nov 29, 2019 at 08:57:34AM +0800, Ming Lei wrote:
> >
> > > [...]
> >
> > > Andrea, can you collect the following log when running the test
> > > on current new(bad) kernel?
> > >
> > > /usr/share/bcc/tools/stackcount -K blk_mq_make_request
> >
> > Instead, please run the following trace, given insert may be
> > called from other paths, such as flush plug:
> >
> > /usr/share/bcc/tools/stackcount -K t:block:block_rq_insert
>
> Attached, for new (patched) bad kernel.
>
> Produced by: start the trace script (with the pendrive already
> plugged), wait some seconds, run the test (1 trial, 1 GB), wait for
> the test to finish, stop the trace.
>
> The copy took ~1700 seconds.
See the two path[1][2] of inserting request, and path[1] is triggered
4358 times, and the path[2] is triggered 5763 times.
The path[2] is expected behaviour. Not sure path [1] is correct, given
ext4_release_file() is supposed to be called when this inode is
released. That means the file is closed 4358 times during 1GB file
copying to usb storage.
Cc filesystem list.
[1] insert requests when returning to user mode from syscall
b'blk_mq_sched_request_inserted'
b'blk_mq_sched_request_inserted'
b'dd_insert_requests'
b'blk_mq_sched_insert_requests'
b'blk_mq_flush_plug_list'
b'blk_flush_plug_list'
b'io_schedule_prepare'
b'io_schedule'
b'rq_qos_wait'
b'wbt_wait'
b'__rq_qos_throttle'
b'blk_mq_make_request'
b'generic_make_request'
b'submit_bio'
b'ext4_io_submit'
b'ext4_writepages'
b'do_writepages'
b'__filemap_fdatawrite_range'
b'ext4_release_file'
b'__fput'
b'task_work_run'
b'exit_to_usermode_loop'
b'do_syscall_64'
b'entry_SYSCALL_64_after_hwframe'
4358
[2] insert requests from writeback wq context
b'blk_mq_sched_request_inserted'
b'blk_mq_sched_request_inserted'
b'dd_insert_requests'
b'blk_mq_sched_insert_requests'
b'blk_mq_flush_plug_list'
b'blk_flush_plug_list'
b'io_schedule_prepare'
b'io_schedule'
b'rq_qos_wait'
b'wbt_wait'
b'__rq_qos_throttle'
b'blk_mq_make_request'
b'generic_make_request'
b'submit_bio'
b'ext4_io_submit'
b'ext4_bio_write_page'
b'mpage_submit_page'
b'mpage_process_page_bufs'
b'mpage_prepare_extent_to_map'
b'ext4_writepages'
b'do_writepages'
b'__writeback_single_inode'
b'writeback_sb_inodes'
b'__writeback_inodes_wb'
b'wb_writeback'
b'wb_workfn'
b'process_one_work'
b'worker_thread'
b'kthread'
b'ret_from_fork'
5763
Thanks,
Ming
next parent reply other threads:[~2019-12-03 2:24 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20191126023253.GA24501@ming.t460p>
[not found] ` <0598fe2754bf0717d81f7e72d3e9b3230c608cc6.camel@unipv.it>
[not found] ` <alpine.LNX.2.21.1.1911271055200.8@nippy.intranet>
[not found] ` <cb6e84781c4542229a3f31572cef19ab@SVR-IES-MBX-03.mgc.mentorg.com>
[not found] ` <c1358b840b3a4971aa35a25d8495c2c8953403ea.camel@unipv.it>
[not found] ` <20191128091712.GD15549@ming.t460p>
[not found] ` <f82fd5129e3dcacae703a689be60b20a7fedadf6.camel@unipv.it>
[not found] ` <20191129005734.GB1829@ming.t460p>
[not found] ` <20191129023555.GA8620@ming.t460p>
[not found] ` <320b315b9c87543d4fb919ecbdf841596c8fbcea.camel@unipv.it>
2019-12-03 2:23 ` Ming Lei [this message]
2019-12-10 7:35 ` AW: Slow I/O on USB media after commit f664a3cc17b7d0a2bc3b3ab96181e1029b0ec0e6 Andrea Vai
2019-12-10 8:05 ` Ming Lei
2019-12-11 2:41 ` Theodore Y. Ts'o
2019-12-11 4:00 ` Ming Lei
2019-12-11 16:07 ` Theodore Y. Ts'o
2019-12-11 21:33 ` Ming Lei
2019-12-12 7:34 ` Andrea Vai
2019-12-18 8:25 ` Andrea Vai
2019-12-18 9:48 ` Ming Lei
[not found] ` <b1b6a0e9d690ecd9432025acd2db4ac09f834040.camel@unipv.it>
2019-12-23 13:08 ` Ming Lei
2019-12-23 16:26 ` Theodore Y. Ts'o
2019-12-23 16:29 ` Andrea Vai
2019-12-23 17:22 ` Theodore Y. Ts'o
2019-12-23 18:45 ` Andrea Vai
2019-12-23 19:53 ` Theodore Y. Ts'o
2019-12-24 1:27 ` Ming Lei
2019-12-24 6:49 ` Andrea Vai
2019-12-24 8:51 ` Andrea Vai
2019-12-24 9:35 ` Ming Lei
2019-12-25 5:17 ` Theodore Y. Ts'o
2019-12-26 2:27 ` Ming Lei
2019-12-26 3:30 ` Theodore Y. Ts'o
2019-12-26 8:37 ` Ming Lei
2020-01-07 7:51 ` Andrea Vai
[not found] ` <20200101074310.10904-1-hdanton@sina.com>
2020-01-01 13:53 ` slow IO on USB media Ming Lei
[not found] ` <fc6f73fc5f57ade8890b472d63c7f0bd559de538.camel@unipv.it>
2019-12-24 1:32 ` AW: Slow I/O on USB media after commit f664a3cc17b7d0a2bc3b3ab96181e1029b0ec0e6 Ming Lei
2019-12-24 8:04 ` Andrea Vai
2019-12-24 8:47 ` Ming Lei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191203022337.GE25002@ming.t460p \
--to=ming.lei@redhat.com \
--cc=Carsten_Schmid@mentor.com \
--cc=Damien.LeMoal@wdc.com \
--cc=Hans.Holmberg@wdc.com \
--cc=andrea.vai@unipv.it \
--cc=axboe@kernel.dk \
--cc=fthain@telegraphics.com.au \
--cc=gregkh@linuxfoundation.org \
--cc=hare@suse.com \
--cc=himanshu.madhani@cavium.com \
--cc=jthumshirn@suse.de \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=linux-usb@vger.kernel.org \
--cc=martin.petersen@oracle.com \
--cc=osandov@fb.com \
--cc=stern@rowland.harvard.edu \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).