Re: loop: it looks like REQ_OP_FLUSH could return before IO completion.

From: Jens Axboe <axboe@kernel.dk>
To: Eric Wheeler <linux-block@lists.ewheeler.net>,
	Christoph Hellwig <hch@infradead.org>
Cc: Ming Lei <ming.lei@redhat.com>,
	linux-block@vger.kernel.org, linux-ext4@vger.kernel.org
Subject: Re: loop: it looks like REQ_OP_FLUSH could return before IO completion.
Date: Sat, 16 Apr 2022 18:59:34 -0600	[thread overview]
Message-ID: <7ae1f26a-cd09-85ff-2f4c-9e80af41ce66@kernel.dk> (raw)
In-Reply-To: <2815ce9-85f-7b56-be3f-7835eb9bb2c6@ewheeler.net>

On 4/16/22 2:05 PM, Eric Wheeler wrote:
> On Fri, 15 Apr 2022, Christoph Hellwig wrote:
>> On Fri, Apr 15, 2022 at 10:29:34PM +0800, Ming Lei wrote:
>>> If ext4 expects the following order, it is ext4's responsibility to
>>> maintain the order, and block layer may re-order all these IOs at will,
>>> so do not expect IOs are issued to device in submission order
>>
>> Yes, and it has been so since REQ_FLUSH (which later became
>> REQ_OP_FLUSH) replaced REQ_BARRIER 12 years ago:
>>
>> commit 28e7d1845216538303bb95d679d8fd4de50e2f1a
>> Author: Tejun Heo <tj@kernel.org>
>> Date:   Fri Sep 3 11:56:16 2010 +0200
>>
>> block: drop barrier ordering by queue draining
>>     
>>     Filesystems will take all the responsibilities for ordering requests
>>     around commit writes and will only indicate how the commit writes
>>     themselves should be handled by block layers.  This patch drops
>>     barrier ordering by queue draining from block layer.
> 
> Thanks Christoph. I think this answers my original question, too.
> 
> You may have already answered this implicitly above.  If you would be so 
> kind as to confirm my or correct my understanding with a few more 
> questions:
> 
> 1. Is the only way for a filesystem to know if one IO completed before a 
>    second IO to track the first IO's completion and submit the second IO 
>    when the first IO's completes (eg a journal commit followed by the 
>    subsequent metadata update)?  If not, then what block-layer mechanism 
>    should be used?

You either need to have a callback or wait on the IO, there's no other
way.

> 2. Are there any IO ordering flags or mechanisms in the block layer at 
>    this point---or---is it simply that all IOs entering the block layer 
>    can always be re-ordered before reaching the media?

No, no ordering flags are provided for this kind of use case. Any IO can
be reordered, hence the only reliable solution is to ensure the previous
have completed.

-- 
Jens Axboe