From: Dave Chinner <david@fromorbit.com>
To: Jeff Moyer <jmoyer@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>,
bugzilla-daemon@bugzilla.kernel.org, linux-xfs@vger.kernel.org
Subject: Re: [Bug 208827] [fio io_uring] io_uring write data crc32c verify failed
Date: Wed, 12 Aug 2020 08:09:29 +1000 [thread overview]
Message-ID: <20200811220929.GQ2114@dread.disaster.area> (raw)
In-Reply-To: <x49zh70zyt6.fsf@segfault.boston.devel.redhat.com>
On Tue, Aug 11, 2020 at 04:56:37PM -0400, Jeff Moyer wrote:
> Jens Axboe <axboe@kernel.dk> writes:
>
> > So it seems to me like the file state is consistent, at least after the
> > run, and that this seems more likely to be a fio issue with short
> > read handling.
>
> Any idea why there was a short read, though?
Yes. See:
https://lore.kernel.org/linux-xfs/20200807024211.GG2114@dread.disaster.area/T/#maf3bd9325fb3ac0773089ca58609a2cea0386ddf
It's a race between the readahead io completion marking pages
uptodate and unlocking them, and the io_uring worker function
getting woken on the first page being unlocked and running the
buffered read before the entire readahead IO completion has unlocked
all the pages in the IO.
Basically, io_uring is re-running the IOCB_NOWAIT|IOCB_WAITQ IO when
there are still pages locked under IO. This will happen much more
frequently the larger the buffered read (these are only 64kB) and
the readahead windows are opened.
Essentially, the io_uring buffered read needs to wait until _all_
pages in the IO are marked up to date and unlocked, not just the
first one. And not just the last one, either - readahead can be
broken into multiple bios (because it spans extents) and there is no
guarantee of order of completion of the readahead bios given by the
readahead code....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2020-08-11 22:09 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-06 4:57 [Bug 208827] New: [fio io_uring] io_uring write data crc32c verify failed bugzilla-daemon
2020-08-07 2:42 ` Dave Chinner
2020-08-07 3:12 ` [Bug 208827] " bugzilla-daemon
2020-08-10 0:09 ` Dave Chinner
2020-08-10 3:56 ` Dave Chinner
2020-08-10 7:08 ` Dave Chinner
2020-08-10 9:08 ` Dave Chinner
2020-08-11 1:15 ` Jens Axboe
2020-08-11 1:50 ` Jens Axboe
2020-08-11 2:01 ` Jens Axboe
2020-08-11 3:01 ` Jens Axboe
2020-08-11 20:56 ` Jeff Moyer
2020-08-11 22:09 ` Dave Chinner [this message]
2020-08-12 15:13 ` Jens Axboe
2020-08-12 15:24 ` Jeff Moyer
2020-08-12 15:26 ` Jens Axboe
2020-08-11 2:00 ` Dave Chinner
2020-08-11 2:19 ` Jens Axboe
2020-08-11 5:53 ` Dave Chinner
2020-08-11 7:05 ` Dave Chinner
2020-08-11 13:10 ` Jens Axboe
2020-08-11 21:59 ` Dave Chinner
2020-08-11 23:00 ` Dave Chinner
2020-08-12 15:19 ` Jens Axboe
2020-08-11 1:07 ` Jens Axboe
2020-08-10 0:09 ` bugzilla-daemon
2020-08-10 3:56 ` bugzilla-daemon
2020-08-10 7:08 ` bugzilla-daemon
2020-08-10 9:09 ` bugzilla-daemon
2020-08-11 1:07 ` bugzilla-daemon
2020-08-11 1:15 ` bugzilla-daemon
2020-08-11 1:50 ` bugzilla-daemon
2020-08-11 2:00 ` bugzilla-daemon
2020-08-11 2:01 ` bugzilla-daemon
2020-08-11 2:20 ` bugzilla-daemon
2020-08-11 3:01 ` bugzilla-daemon
2020-08-11 5:53 ` bugzilla-daemon
2020-08-11 7:05 ` bugzilla-daemon
2020-08-11 13:10 ` bugzilla-daemon
2020-08-11 16:16 ` bugzilla-daemon
2020-08-11 20:56 ` bugzilla-daemon
2020-08-11 21:59 ` bugzilla-daemon
2020-08-11 22:09 ` bugzilla-daemon
2020-08-11 23:00 ` bugzilla-daemon
2020-08-12 3:15 ` bugzilla-daemon
2020-08-12 15:14 ` bugzilla-daemon
2020-08-12 15:19 ` bugzilla-daemon
2020-08-12 15:24 ` bugzilla-daemon
2020-08-12 15:26 ` bugzilla-daemon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200811220929.GQ2114@dread.disaster.area \
--to=david@fromorbit.com \
--cc=axboe@kernel.dk \
--cc=bugzilla-daemon@bugzilla.kernel.org \
--cc=jmoyer@redhat.com \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.