From: Mike Snitzer <snitzer@redhat.com>
To: Ming Lei <tom.leiming@gmail.com>
Cc: Mikulas Patocka <mpatocka@redhat.com>,
Jens Axboe <axboe@kernel.dk>,
Kent Overstreet <kent.overstreet@gmail.com>,
dm-devel@redhat.com,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
"Alasdair G. Kergon" <agk@redhat.com>
Subject: Re: [PATCH v2] block: flush queued bios when the process blocks
Date: Thu, 15 Oct 2015 04:06:10 -0400 [thread overview]
Message-ID: <20151015080610.GA25536@redhat.com> (raw)
In-Reply-To: <CACVXFVPgvKEJo1dPigk303=GGVc3fJnuEk+QP+VEf=xa_Lz+vA@mail.gmail.com>
On Wed, Oct 14 2015 at 11:27pm -0400,
Ming Lei <tom.leiming@gmail.com> wrote:
> On Sat, Oct 10, 2015 at 3:52 AM, Mike Snitzer <snitzer@redhat.com> wrote:
> >
> > Turns out that this change:
> > http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=wip&id=2639638c77768a86216be456c2764e32a2bcd841
> >
> > needed to be reverted with:
> > http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=wip&id=ad3ccd760da7c05b90775372f9b39dc2964086fe
> >
> > Because nested plugs caused generic_make_request()'s onstack bio_list to
> > go out of scope (blk_finish_plug() wouldn't actually flush the list
> > within generic_make_request because XFS already added an outermost
> > plug).
>
> Looks you should have defined bio_list in plug as
>
> 'struct bio_list bio_list'
>
> instead of one pointer.
I realized that and fixed it (see commit ad3ccd760da7c05b90 referenced
above that does exactly that). That wasn't the problem.
> >
> > But even after fixing that I then hit issues with these changes now
> > resulting in imperfect 'in_generic_make_request' accounting that happens
> > lazily once the outermost plug completes blk_finish_plug. manifested as
> > dm-bufio.c:dm_bufio_prefetch's BUG_ON(dm_bufio_in_request()); hitting.
>
> Looks this problem should be related with above 'bio_list' definition too.
No, as I explained it was due to the nested plug:
> >
> > Basically using the blk-core's onstack plugging isn't workable for
> > fixing this deadlock and we're back to having to seriously consider
> > this (with its additional hook in the scheduler)
To elaborate, for the code in DM (and other subsystems like bcache) that
rely on accurate accounting of whether we're actively _in_
generic_make_request: using plug to store/manage the bio_list isn't
workable because nested plugs change the lifetime of when the bio_list
is processed (as I implemented it -- which was to respect nested plugs).
I could've forced the issue by making the bio_list get processed
regardless of nesting but that would've made the onstack plugging much
more convoluted (duality between nested vs not just for bio_list's
benefit and for what gain? Simply to avoid an extra conditional
immediately in the scheduler? That conditional was still added anyway
but just as part of blk_needs_flush_plug so in the end there wasn't any
benefit!).
Hopefully my middle-of-the-night reply is coherent and helped to clarify
my position that (ab)using blk_plug for the bio_list management is
_really_ awkward. ;)
Thanks,
Mike
next prev parent reply other threads:[~2015-10-15 8:06 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-05-27 15:03 [PATCH] block: flush queued bios when the process blocks Mikulas Patocka
2014-05-27 15:08 ` Jens Axboe
2014-05-27 15:23 ` Mikulas Patocka
2014-05-27 15:42 ` Jens Axboe
2014-05-27 16:26 ` Mikulas Patocka
2014-05-27 17:33 ` Mike Snitzer
2014-05-27 19:56 ` Kent Overstreet
2015-10-05 19:50 ` Mike Snitzer
2014-05-27 17:42 ` [PATCH] " Jens Axboe
2014-05-27 18:14 ` [dm-devel] " Christoph Hellwig
2014-05-27 19:59 ` Kent Overstreet
2014-05-27 19:56 ` Mikulas Patocka
2014-05-27 20:06 ` Kent Overstreet
2014-05-29 23:52 ` Mikulas Patocka
2015-10-05 20:59 ` Mike Snitzer
2015-10-06 13:28 ` Mikulas Patocka
2015-10-06 13:47 ` Mike Snitzer
2015-10-06 14:10 ` Mikulas Patocka
2015-10-06 14:26 ` Mikulas Patocka
2015-10-06 18:17 ` [dm-devel] " Mikulas Patocka
2015-10-06 18:50 ` Mike Snitzer
2015-10-06 20:16 ` [PATCH v2] " Mike Snitzer
2015-10-06 20:26 ` Mike Snitzer
2015-10-08 15:04 ` Mikulas Patocka
2015-10-08 15:08 ` Mike Snitzer
2015-10-09 19:52 ` Mike Snitzer
2015-10-09 19:59 ` Mike Snitzer
2015-10-14 20:47 ` [PATCH v3 for-4.4] block: flush queued bios when process blocks to avoid deadlock Mike Snitzer
2015-10-14 21:44 ` Jeff Moyer
2015-10-17 16:04 ` Ming Lei
2015-10-20 19:57 ` Mike Snitzer
2015-10-20 20:03 ` Mikulas Patocka
2015-10-21 16:38 ` Ming Lei
2015-10-21 21:49 ` Mikulas Patocka
2015-10-22 1:53 ` Ming Lei
2015-10-15 3:27 ` [PATCH v2] block: flush queued bios when the process blocks Ming Lei
2015-10-15 8:06 ` Mike Snitzer [this message]
2015-10-16 3:08 ` Ming Lei
2015-10-16 15:29 ` Mike Snitzer
2015-10-17 15:54 ` Ming Lei
2015-10-09 11:58 ` kbuild test robot
2014-05-27 17:59 ` [PATCH] " Kent Overstreet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20151015080610.GA25536@redhat.com \
--to=snitzer@redhat.com \
--cc=agk@redhat.com \
--cc=axboe@kernel.dk \
--cc=dm-devel@redhat.com \
--cc=kent.overstreet@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mpatocka@redhat.com \
--cc=tom.leiming@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).