Re: [PATCH 2/2] block: Split and submit bios in LBA order

From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
To: Bart Van Assche <bvanassche@acm.org>, Christoph Hellwig <hch@lst.de>
Cc: Ming Lei <ming.lei@redhat.com>, Jens Axboe <axboe@kernel.dk>,
	linux-block@vger.kernel.org, Jaegeuk Kim <jaegeuk@kernel.org>,
	Jan Kara <jack@suse.cz>,
	Johannes Thumshirn <johannes.thumshirn@wdc.com>
Subject: Re: [PATCH 2/2] block: Split and submit bios in LBA order
Date: Sat, 25 Mar 2023 11:00:08 +0900	[thread overview]
Message-ID: <e83d1111-1841-7b2a-973b-16bea2e789c6@opensource.wdc.com> (raw)
In-Reply-To: <1e65e542-e8e9-bd3f-6ff1-1bbd4716a8c3@acm.org>

On 3/25/23 01:55, Bart Van Assche wrote:
> On 3/23/23 15:53, Damien Le Moal wrote:
>> On 3/24/23 01:27, Bart Van Assche wrote:
>>> On 3/23/23 03:28, Damien Le Moal wrote:
>>>> For the zone append emulation, the write locking is done by sd.c and the upper
>>>> layer does not restrict to one append per zone. So we actually could envision a
>>>> UFS version of the sd write locking calls that is optimized for the device
>>>> capabilities and we can keep a common upper layer (which is preferable in my
>>>> opinion).
>>>
>>> I see a blk_req_zone_write_trylock() call in
>>> sd_zbc_prepare_zone_append() and a blk_req_zone_write_unlock() call in
>>> sd_zbc_complete() for REQ_OP_ZONE_APPEND operations. Does this mean that
>>> the sd_zbc.c code restricts the queue depth to one per zone for
>>> REQ_OP_ZONE_APPEND operations?
>>
>> Yes, since the append becomes a regular write and HBAs are often happy to
>> reorder these commands, even for SMR, we need the locking.
>>
>> But if I understand your use case correctly, given that UFS gives guarantees on
>> the command dispatching order, you could probably relax this locking for zone
>> append requests. But you cannot for regular writes as the locking is up in the
>> block layer and needed to avoid block layer level reordering.
> 
> Hi Damien,
> 
> I don't think that we can achieve QD > 1 even if we would switch to
> REQ_OP_ZONE_APPEND. A SCSI LLD is allowed to respond with 
> SCSI_MLQUEUE_HOST_BUSY if the SCSI core asks it to queue a command. This 
> may lead to reordering and hence may cause UNALIGNED WRITE COMMAND 
> errors. We want to avoid these errors.

The trick here could be to have the UFS LLD to unlock the target zone of a write
when the command is sent to the device, instead of when the command completes.
This way, the zone is still locked when there is a requeue and there is no
reordering. That could allow for write qd > 1 in the case of UFS. And this
method could actually work for regular writes too.

-- 
Damien Le Moal
Western Digital Research