Re: [PATCH RFC 1/3] block: add flag 'nowait_requests' into queue limits

[Konstantin Khlebnikov <khlebnikov@yandex-team.ru>]

On 03/06/2020 07.58, Christoph Hellwig wrote:
> On Mon, Jun 01, 2020 at 03:37:09PM +0300, Konstantin Khlebnikov wrote:
>> Add flag for marking bio-based queues which support REQ_NOWAIT.
>> Set for all request based (mq) devices.
>>
>> Stacking device should set it after blk_set_stacking_limits() if method
>> make_request() itself doesn't delay requests or handles REQ_NOWAIT.
> 
> I don't think this belongs into the queue limits.  For example a
> stacking driver that always defers requests to a workqueue can support
> REQ_NOWAIT entirely independent of the underlying devices.  I think
> this just needs to be a simple queue flag.
> 

For O_DIRECT I/O REQ_NOWAIT not just about non-blocking submition.
It also provides instant feedback about contention. Like ECN from network.
This feedback is useful for rate-control and balancing load between replicas.

If upper layer simply remaps and forwards requests below then to forward
contention all layers of stacked device should support this feature.
That's why I've put it as flag into limits - to reuse limits stacking.

If any layer defers request, then it should somehow limit size of backlog
at the same time to provide sane behaviour for REQ_NOWAIT regardless of
behaviour lower devices. So, then it could simply set that flag in limits.

Also I want to add handing into blk-qos/throttler - never delay REQ_NOWAIT.