From: Peter Zijlstra <peterz@infradead.org>
To: Tejun Heo <tj@kernel.org>
Cc: axboe@kernel.dk, linux-kernel@vger.kernel.org, oleg@redhat.com,
kernel-team@fb.com, osandov@fb.com
Subject: Re: [PATCHSET] blk-mq: reimplement timeout handling
Date: Mon, 11 Dec 2017 10:27:55 +0100 [thread overview]
Message-ID: <20171211092755.uptoa23wlukuryie@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <20171209192525.982030-1-tj@kernel.org>
On Sat, Dec 09, 2017 at 11:25:19AM -0800, Tejun Heo wrote:
> Currently, blk-mq timeout path synchronizes against the usual
> issue/completion path using a complex scheme involving atomic
> bitflags, REQ_ATOM_*, memory barriers and subtle memory coherence
> rules. Unfortunatley, it contains quite a few holes.
>
> It's pretty easy to make blk_mq_check_expired() terminate a later
> instance of a request. If we induce 5 sec delay before
> time_after_eq() test in blk_mq_check_expired(), shorten the timeout to
> 2s, and issue back-to-back large IOs, blk-mq starts timing out
> requests spuriously pretty quickly. Nothing actually timed out. It
> just made the call on a recycle instance of a request and then
> terminated a later instance long after the original instance finished.
> The scenario isn't theoretical either.
>
> This patchset replaces the broken synchronization mechanism with a RCU
> and generation number based one. Please read the patch description of
> the second path for more details.
>
> Oleg, Peter, I'd really appreciate if you guys can go over the
> reported breakages and the new implementation.
Great, yes that code seemed very suspicious when I looked at it; thanks
for making it go away. I'll try and find a spot to stare at the patches.
next prev parent reply other threads:[~2017-12-11 9:28 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-12-09 19:25 [PATCHSET] blk-mq: reimplement timeout handling Tejun Heo
2017-12-09 19:25 ` [PATCH 1/6] blk-mq: protect completion path with RCU Tejun Heo
2017-12-13 3:10 ` jianchao.wang
2017-12-09 19:25 ` [PATCH 2/6] blk-mq: replace timeout synchronization with a RCU and generation based scheme Tejun Heo
2017-12-12 10:09 ` Peter Zijlstra
2017-12-12 18:02 ` Tejun Heo
2017-12-12 10:10 ` Peter Zijlstra
2017-12-12 18:03 ` Tejun Heo
2017-12-12 11:56 ` Peter Zijlstra
2017-12-12 18:04 ` Tejun Heo
2017-12-09 19:25 ` [PATCH 3/6] blk-mq: use blk_mq_rq_state() instead of testing REQ_ATOM_COMPLETE Tejun Heo
2017-12-09 19:25 ` [PATCH 4/6] blk-mq: make blk_abort_request() trigger timeout path Tejun Heo
2017-12-09 19:25 ` [PATCH 5/6] blk-mq: remove REQ_ATOM_COMPLETE usages from blk-mq Tejun Heo
2017-12-09 19:25 ` [PATCH 6/6] blk-mq: remove REQ_ATOM_STARTED Tejun Heo
2017-12-12 10:09 ` jianchao.wang
2017-12-12 17:01 ` Tejun Heo
2017-12-12 17:26 ` Tejun Heo
2017-12-13 3:05 ` jianchao.wang
2017-12-13 16:09 ` Tejun Heo
2017-12-14 2:14 ` jianchao.wang
2017-12-12 11:17 ` Nikolay Borisov
2017-12-12 17:29 ` Tejun Heo
2017-12-11 9:27 ` Peter Zijlstra [this message]
2017-12-12 9:21 ` [PATCHSET] blk-mq: reimplement timeout handling Christoph Hellwig
2017-12-12 16:39 ` Tejun Heo
2017-12-12 16:08 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171211092755.uptoa23wlukuryie@hirez.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=axboe@kernel.dk \
--cc=kernel-team@fb.com \
--cc=linux-kernel@vger.kernel.org \
--cc=oleg@redhat.com \
--cc=osandov@fb.com \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).