From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>, Tejun Heo <tj@kernel.org>
Cc: linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org
Subject: boot stall regression due to blk-mq: use percpu_ref for mq usage count
Date: Fri, 19 Sep 2014 13:38:15 +0200 [thread overview]
Message-ID: <20140919113815.GA10791@lst.de> (raw)
Hi Jens, hi Tejun,
I've seen multi-second boot stalls in one of my KVM setups during
the initial scsi scan:
[ 0.949892] scsi host0: Virtio SCSI HBA
[ 1.007864] scsi 0:0:0:0: Direct-Access QEMU QEMU HARDDISK 1.1. PQ: 0 ANSI: 5
[ 1.021299] scsi 0:0:1:0: Direct-Access QEMU QEMU HARDDISK 1.1. PQ: 0 ANSI: 5
[ 1.520356] tsc: Refined TSC clocksource calibration: 2491.910 MHz
<stall>
[ 16.186549] sd 0:0:0:0: Attached scsi generic sg0 type 0
[ 16.190478] sd 0:0:1:0: Attached scsi generic sg1 type 0
[ 16.194099] osd: LOADED open-osd 0.2.1
[ 16.203202] sd 0:0:0:0: [sda] 31457280 512-byte logical blocks: (16.1 GB/15.0 GiB)
[ 16.208478] sd 0:0:0:0: [sda] Write Protect is off
[ 16.211439] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 16.218771] sd 0:0:1:0: [sdb] 31457280 512-byte logical blocks: (16.1 GB/15.0 GiB)
[ 16.223264] sd 0:0:1:0: [sdb] Write Protect is off
[ 16.225682] sd 0:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
I've tracked this down to "blk-mq: use percpu_ref for mq usage count" in
a rather painful way as that one introduced enough other regressions
to mess up bisect.
If I revert the following commits:
dd840087086f3b93ac20f7472b4fca59aff7b79f
cddd5d17642cc6881352732693c2ae6930e9ce65
add703fda981b9719d37f371498b9f129acbd997
which are the above mentioned commit and two fixes to it the problem goes
away.
My qemu command line is below:
kvm \
-m 2048 \
-smp 1 \
-kernel arch/x86/boot/bzImage \
-append "root=/dev/vda console=tty0 console=ttyS0,115200n8 scsi_mod.use_blk_mq=Y" \
-nographic \
-drive if=virtio,file=/work/images/debian.qcow2,cache=none,serial="test1234" \
-drive if=none,id=test,file=/work/images/test.img,cache=none,aio=native \
-drive if=none,id=scratch,file=/work/images/scratch.img,cache=none,aio=native \
-device virtio-scsi-pci,id=scsi \
-device scsi-hd,drive=test \
-device scsi-hd,drive=scratch \
-drive file=/work/images/debian-7.3.0-amd64-netinst.iso,index=2,media=cdrom
next reply other threads:[~2014-09-19 11:38 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-19 11:38 Christoph Hellwig [this message]
2014-09-19 19:13 ` boot stall regression due to blk-mq: use percpu_ref for mq usage count Jens Axboe
2014-09-23 5:55 ` Christoph Hellwig
2014-09-23 5:56 ` Tejun Heo
2014-09-23 5:57 ` Tejun Heo
2014-09-23 5:59 ` Christoph Hellwig
2014-09-23 6:01 ` Tejun Heo
2014-09-23 6:03 ` Tejun Heo
2014-09-23 6:09 ` Christoph Hellwig
2014-09-23 6:11 ` Tejun Heo
2014-09-23 14:57 ` Elliott, Robert (Server Storage)
2014-09-23 15:49 ` Jens Axboe
2014-09-23 19:24 ` [PATCH block/for-3.17-fixes/core] blk-mq, percpu_ref: implement a kludge for SCSI blk-mq stall during probe Tejun Heo
2014-09-23 19:29 ` Jens Axboe
2014-09-23 19:48 ` Tejun Heo
2014-09-24 8:23 ` Christoph Hellwig
2014-09-24 14:30 ` Jens Axboe
2014-09-24 14:33 ` Tejun Heo
2014-09-24 14:33 ` Jens Axboe
2014-09-24 17:20 ` [PATCH percpu/for-3.18] Revert "blk-mq, percpu_ref: implement a kludge for SCSI blk-mq stall during probe" Tejun Heo
2014-09-23 6:08 ` [PATCH block/for-3.18/core] blk-mq: start q->mq_usage_counter in atomic mode Tejun Heo
2014-09-24 17:33 ` Tejun Heo
2014-09-24 17:43 ` [PATCH percpu/for-3.18] blk-mq, percpu_ref: " Tejun Heo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140919113815.GA10791@lst.de \
--to=hch@lst.de \
--cc=axboe@kernel.dk \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).