From: Christoph Hellwig <hch@infradead.org>
To: Jens Axboe <jens.axboe@oracle.com>
Cc: Tejun Heo <htejun@gmail.com>, David Greaves <david@dgreaves.com>,
"Rafael J. Wysocki" <rjw@sisk.pl>,
Linus Torvalds <torvalds@linux-foundation.org>,
David Chinner <dgc@sgi.com>,
xfs@oss.sgi.com,
"'linux-kernel@vger.kernel.org'" <linux-kernel@vger.kernel.org>,
linux-pm <linux-pm@lists.osdl.org>, Neil Brown <neilb@suse.de>,
Jeff Garzik <jgarzik@pobox.com>
Subject: Re: [PATCH] block: always requeue !fs requests at the front
Date: Sat, 16 Jun 2007 20:54:01 +0100 [thread overview]
Message-ID: <20070616195401.GA6929@infradead.org> (raw)
In-Reply-To: <20070615110544.GR6149@kernel.dk>
On Fri, Jun 15, 2007 at 01:05:44PM +0200, Jens Axboe wrote:
> On Fri, Jun 15 2007, Tejun Heo wrote:
> > SCSI marks internal commands with REQ_PREEMPT and push it at the front
> > of the request queue using blk_execute_rq(). When entering suspended
> > or frozen state, SCSI devices are quiesced using
> > scsi_device_quiesce(). In quiesced state, only REQ_PREEMPT requests
> > are processed. This is how SCSI blocks other requests out while
> > suspending and resuming. As all internal commands are pushed at the
> > front of the queue, this usually works.
> >
> > Unfortunately, this interacts badly with ordered requeueing. To
> > preserve request order on requeueing (due to busy device, active EH or
> > other failures), requests are sorted according to ordered sequence on
> > requeue if IO barrier is in progress.
> >
> > The following sequence deadlocks.
> >
> > 1. IO barrier sequence issues.
> >
> > 2. Suspend requested. Queue is quiesced with part of all of IO
> > barrier sequence at the front.
> >
> > 3. During suspending or resuming, SCSI issues internal command which
> > gets deferred and requeued for some reason. As the command is
> > issued after the IO barrier in #1, ordered requeueing code puts the
> > request after IO barrier sequence.
> >
> > 4. The device is ready to process requests again but still is in
> > quiesced state and the first request of the queue isn't
> > REQ_PREEMPT, so command processing is deadlocked -
> > suspending/resuming waits for the issued request to complete while
> > the request can't be processed till device is put back into
> > running state by resuming.
> >
> > This can be fixed by always putting !fs requests at the front when
> > requeueing.
> >
> > The following thread reports this deadlock.
> >
> > http://thread.gmane.org/gmane.linux.kernel/537473
> >
> > Signed-off-by: Tejun Heo <htejun@gmail.com>
> > Cc: Jenn Axboe <jens.axboe@oracle.com>
> > Cc: David Greaves <david@dgreaves.com>
> > ---
> > Okay, it took a lot of hours of debugging but boiled down to two liner
> > fix. I feel so empty. :-) RAID6 triggers this reliably because it
> > uses BIO_BARRIER heavily to update its superblock. The recent ATA
> > suspend/resume rewrite is hit by this because it uses SCSI internal
> > commands to spin down and up the drives for suspending and resuming.
> >
> > David, please test this. Jens, does it look okay?
>
> Yep looks good, except for the bad multi-line comment style, but that's
> minor stuff ;-)
>
> Acked-by: Jens Axboe <jens.axboe@oracle.com>
I'd much much prefer having a description of the problem in the actual
comment then a hyperlink. There's just too much chance of the latter
breaking over time, and it's impossible to update it when things change
that should be reflected in the comment.
next prev parent reply other threads:[~2007-06-16 19:54 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-06-01 21:23 2.6.22-rc3 hibernate(?) disables skge wol David Greaves
2007-06-01 21:42 ` Rafael J. Wysocki
2007-06-01 22:37 ` 2.6.22-rc3 hibernate(?) fails totally - regression David Greaves
2007-06-01 23:22 ` Rafael J. Wysocki
2007-06-02 22:31 ` David Greaves
2007-06-02 22:46 ` Linus Torvalds
2007-06-03 15:03 ` David Greaves
2007-06-06 8:33 ` Tejun Heo
2007-06-06 10:18 ` [PATCH] sata_promise: use TF interface for polling NODATA commands Tejun Heo
2007-06-06 10:19 ` Tejun Heo
2007-06-06 10:39 ` 2.6.22-rc3 hibernate(?) fails totally - regression (xfs on raid6) David Greaves
2007-06-07 5:53 ` Tejun Heo
2007-06-07 10:30 ` David Greaves
2007-06-07 11:07 ` David Chinner
2007-06-07 13:59 ` David Greaves
2007-06-07 22:28 ` David Chinner
2007-06-08 19:09 ` David Greaves
2007-06-12 18:43 ` Linus Torvalds
2007-06-13 11:16 ` David Greaves
2007-06-13 21:04 ` Linus Torvalds
2007-06-13 21:22 ` Jeff Garzik
2007-06-13 22:02 ` David Greaves
2007-06-13 22:12 ` Linus Torvalds
2007-06-13 23:15 ` Rafael J. Wysocki
2007-06-14 14:21 ` Tejun Heo
2007-06-14 15:10 ` Tejun Heo
2007-06-15 9:42 ` [PATCH] block: always requeue !fs requests at the front Tejun Heo
2007-06-15 11:05 ` Jens Axboe
2007-06-15 11:17 ` Tejun Heo
2007-06-15 11:21 ` Jens Axboe
2007-06-15 15:08 ` Jeff Garzik
2007-06-16 19:54 ` Christoph Hellwig [this message]
2007-06-17 7:29 ` Jens Axboe
2007-06-17 8:03 ` Tejun Heo
2007-06-15 13:58 ` David Greaves
2007-06-14 15:19 ` 2.6.22-rc3 hibernate(?) fails totally - regression (xfs on raid6) David Greaves
2007-06-14 0:28 ` David Chinner
2007-06-12 12:31 ` David Greaves
2007-06-10 18:43 ` Pavel Machek
2007-06-12 18:00 ` David Greaves
2007-06-12 21:31 ` Pavel Machek
2007-06-07 13:45 ` Duane Griffin
2007-06-07 14:00 ` David Greaves
2007-06-07 14:05 ` Tejun Heo
2007-06-07 14:36 ` Mark Lord
2007-06-07 15:20 ` David Greaves
2007-06-07 16:58 ` Rafael J. Wysocki
2007-06-07 20:12 ` Pavel Machek
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20070616195401.GA6929@infradead.org \
--to=hch@infradead.org \
--cc=david@dgreaves.com \
--cc=dgc@sgi.com \
--cc=htejun@gmail.com \
--cc=jens.axboe@oracle.com \
--cc=jgarzik@pobox.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pm@lists.osdl.org \
--cc=neilb@suse.de \
--cc=rjw@sisk.pl \
--cc=torvalds@linux-foundation.org \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).