linux-block.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Marc Gonzalez <marc.w.gonzalez@free.fr>
To: fsdevel <linux-fsdevel@vger.kernel.org>,
	linux-block <linux-block@vger.kernel.org>
Cc: SCSI <linux-scsi@vger.kernel.org>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	Jan Kara <jack@suse.com>, Christoph Hellwig <hch@infradead.org>,
	Joao Pinto <jpinto@synopsys.com>, Jens Axboe <axboe@kernel.dk>,
	Fujita Tomonori <fujita.tomonori@lab.ntt.co.jp>,
	Paolo Valente <paolo.valente@linaro.org>
Subject: dd hangs when reading large partitions
Date: Fri, 18 Jan 2019 13:10:49 +0100	[thread overview]
Message-ID: <f792574c-e083-b218-13b4-c89be6566015@free.fr> (raw)

Hello,

I'm running into an issue which I don't know how to debug.
So I'm open to ideas and suggestions :-)

On my arm64 board, I have enabled Universal Flash Storage support.

I wanted to benchmark read performance, and noticed that the system
locks up when I read partitions larger than 3.5 GB, unless I tell
dd to use direct IO:

*** WITH O_DIRECT ***
# dd if=/dev/sda of=/dev/null bs=1M iflag=direct status=progress
57892929536 bytes (58 GB, 54 GiB) copied, 697.006 s, 83.1 MB/s 
55256+0 records in
55256+0 records out
57940115456 bytes (58 GB, 54 GiB) copied, 697.575 s, 83.1 MB/s

*** WITHOUT O_DIRECT ***
# dd if=/dev/sda of=/dev/null bs=1M status=progress
3853516800 bytes (3.9 GB, 3.6 GiB) copied, 49.0002 s, 78.6 MB/s


rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu:    1-...0: (8242 ticks this GP) idle=106/1/0x4000000000000000 softirq=168/171 fqs=2626 
rcu:    6-...0: (99 GPs behind) idle=ec2/1/0x4000000000000000 softirq=71/71 fqs=2626 
rcu:    (detected by 7, t=5254 jiffies, g=-275, q=2)
Task dump for CPU 1:
kworker/1:1H    R  running task        0   675      2 0x0000002a
Workqueue: kblockd blk_mq_run_work_fn
Call trace:
 __switch_to+0x168/0x1d0
 0xffffffc0f6efbbc8
 blk_mq_run_work_fn+0x28/0x40
 process_one_work+0x208/0x470
 worker_thread+0x48/0x460
 kthread+0x128/0x130
 ret_from_fork+0x10/0x1c
Task dump for CPU 6:
kthreadd        R  running task        0     2      0 0x0000002a
Call trace:
 __switch_to+0x168/0x1d0
 0x5b36396f4e7d4000


rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu:    1-...0: (8242 ticks this GP) idle=106/1/0x4000000000000000 softirq=168/171 fqs=10500 
rcu:    6-...0: (99 GPs behind) idle=ec2/1/0x4000000000000000 softirq=71/71 fqs=10500 
rcu:    (detected by 7, t=21009 jiffies, g=-275, q=2)
Task dump for CPU 1:
kworker/1:1H    R  running task        0   675      2 0x0000002a
Workqueue: kblockd blk_mq_run_work_fn
Call trace:
 __switch_to+0x168/0x1d0
 0xffffffc0f6efbbc8
 blk_mq_run_work_fn+0x28/0x40
 process_one_work+0x208/0x470
 worker_thread+0x48/0x460
 kthread+0x128/0x130
 ret_from_fork+0x10/0x1c
Task dump for CPU 6:
kthreadd        R  running task        0     2      0 0x0000002a
Call trace:
 __switch_to+0x168/0x1d0
 0x5b36396f4e7d4000


The system always hangs around the 3.6 GiB mark, wherever I start from.
How can I debug this issue?

Regards.

             reply	other threads:[~2019-01-18 12:11 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-01-18 12:10 Marc Gonzalez [this message]
2019-01-18 13:39 ` dd hangs when reading large partitions Ming Lei
2019-01-18 14:54   ` Marc Gonzalez
2019-01-18 15:18 ` jianchao.wang
2019-01-18 17:38   ` Marc Gonzalez
2019-01-18 17:48   ` Jens Axboe
2019-01-18 17:51     ` Bart Van Assche
2019-01-18 19:00       ` Jens Axboe
2019-01-19  9:56     ` Christoph Hellwig
2019-01-19 14:37       ` Jens Axboe
2019-01-19 16:09       ` Bart Van Assche
2019-01-21  8:33         ` Christoph Hellwig
2019-01-19 19:47       ` Marc Gonzalez
2019-01-19 20:45         ` Marc Gonzalez
2019-01-21  8:33         ` Christoph Hellwig
2019-01-21 15:22         ` Marc Gonzalez
2019-01-22  3:12           ` jianchao.wang
2019-01-22 10:59             ` Marc Gonzalez
2019-01-22 12:49               ` Marc Gonzalez
2019-01-22 16:17               ` Marc Gonzalez
2019-01-22 16:22                 ` Greg Kroah-Hartman
2019-01-22 19:07                   ` Evan Green
2019-01-23  3:10               ` jianchao.wang
2019-02-06 16:16                 ` Marc Gonzalez
2019-02-06 17:05                   ` Marc Gonzalez
2019-02-07 10:44                     ` Marc Gonzalez
2019-02-07 16:56                       ` Marc Gonzalez
2019-02-08 15:33                         ` Marc Gonzalez
2019-02-08 15:49                           ` Bart Van Assche
2019-02-09 11:57                             ` Marc Gonzalez
2019-02-11 16:36                             ` Marc Gonzalez
2019-02-11 17:27                               ` Marc Gonzalez
2019-02-12 15:26                                 ` [SOLVED] " Marc Gonzalez
2019-01-18 19:27 ` Douglas Gilbert

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f792574c-e083-b218-13b4-c89be6566015@free.fr \
    --to=marc.w.gonzalez@free.fr \
    --cc=axboe@kernel.dk \
    --cc=fujita.tomonori@lab.ntt.co.jp \
    --cc=hch@infradead.org \
    --cc=jack@suse.com \
    --cc=jpinto@synopsys.com \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=paolo.valente@linaro.org \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).