From: Dave Chinner <david@fromorbit.com>
To: Damien Le Moal <Damien.LeMoal@wdc.com>
Cc: Jens Axboe <axboe@kernel.dk>,
"Darrick J. Wong" <darrick.wong@oracle.com>,
"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
xfs <linux-xfs@vger.kernel.org>
Subject: Re: Block device direct read EIO handling broken?
Date: Tue, 6 Aug 2019 10:23:53 +1000 [thread overview]
Message-ID: <20190806002353.GC7777@dread.disaster.area> (raw)
In-Reply-To: <BYAPR04MB5816D1AB6B586FAD664F8D79E7D50@BYAPR04MB5816.namprd04.prod.outlook.com>
On Tue, Aug 06, 2019 at 12:05:51AM +0000, Damien Le Moal wrote:
> On 2019/08/06 7:05, Damien Le Moal wrote:
> > On 2019/08/06 6:59, Damien Le Moal wrote:
> >> On 2019/08/06 6:28, Jens Axboe wrote:
> >>> On 8/5/19 2:27 PM, Damien Le Moal wrote:
> >>>> On 2019/08/06 6:26, Jens Axboe wrote:
> >>>>>> In any case, looking again at this code, it looks like there is a
> >>>>>> problem with dio->size being incremented early, even for fragments
> >>>>>> that get BLK_QC_T_EAGAIN, because dio->size is being used in
> >>>>>> blkdev_bio_end_io(). So an incorrect size can be reported to user
> >>>>>> space in that case on completion (e.g. large asynchronous no-wait dio
> >>>>>> that cannot be issued in one go).
> >>>>>>
> >>>>>> So maybe something like this ? (completely untested)
> >>>>>
> >>>>> I think that looks pretty good, I like not double accounting with
> >>>>> this_size and dio->size, and we retain the old style ordering for the
> >>>>> ret value.
> >>>>
> >>>> Do you want a proper patch with real testing backup ? I can send that
> >>>> later today.
> >>>
> >>> Yeah that'd be great, I like your approach better.
> >>>
> >>
> >> Looking again, I think this is not it yet: dio->size is being referenced after
> >> submit_bio(), so blkdev_bio_end_io() may see the old value if the bio completes
> >> before dio->size increment. So the use-after-free is still there. And since
> >> blkdev_bio_end_io() processes completion to user space only when dio->ref
> >> becomes 0, adding an atomic_inc/dec(&dio->ref) over the loop would not help and
> >> does not cover the single BIO case. Any idea how to address this one ?
> >>
> >
> > May be add a bio_get/put() over the 2 places that do submit_bio() would work,
> > for all cases (single/multi BIO, sync & async). E.g.:
> >
> > + bio_get(bio);
> > qc = submit_bio(bio);
> > if (qc == BLK_QC_T_EAGAIN) {
> > if (!dio->size)
> > ret = -EAGAIN;
> > + bio_put(bio);
> > goto error;
> > }
> > dio->size += bio_size;
> > + bio_put(bio);
> >
> > Thoughts ?
> >
>
> That does not work since the reference to dio->size in blkdev_bio_end_io()
> depends on atomic_dec_and_test(&dio->ref) which counts the BIO fragments for the
> dio (+1 for async multi-bio case). So completion of the last bio can still
> reference the old value of dio->size.
Didn't we fix this same use-after-free in iomap_dio_rw() in commit
4ea899ead278 ("iomap: fix a use after free in iomap_dio_rw")?
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2019-08-06 0:25 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-05 18:15 Block device direct read EIO handling broken? Darrick J. Wong
2019-08-05 18:31 ` Jens Axboe
2019-08-05 20:31 ` Jens Axboe
2019-08-05 20:54 ` Jens Axboe
2019-08-05 21:08 ` Damien Le Moal
2019-08-05 21:25 ` Jens Axboe
2019-08-05 21:27 ` Damien Le Moal
2019-08-05 21:28 ` Jens Axboe
2019-08-05 21:59 ` Damien Le Moal
2019-08-05 22:05 ` Damien Le Moal
2019-08-06 0:05 ` Damien Le Moal
2019-08-06 0:23 ` Dave Chinner [this message]
2019-08-06 11:32 ` Damien Le Moal
2019-08-06 4:09 ` Jens Axboe
2019-08-06 7:05 ` Damien Le Moal
2019-08-06 13:34 ` Jens Axboe
2019-08-07 9:42 ` Damien Le Moal
2019-08-06 13:23 ` Damien Le Moal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190806002353.GC7777@dread.disaster.area \
--to=david@fromorbit.com \
--cc=Damien.LeMoal@wdc.com \
--cc=axboe@kernel.dk \
--cc=darrick.wong@oracle.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).