* [PATCH RFC 0/3] block: allow REQ_NOWAIT to some bio-based/stacked devices
@ 2020-06-01 12:37 Konstantin Khlebnikov
2020-06-01 12:37 ` [PATCH RFC 1/3] block: add flag 'nowait_requests' into queue limits Konstantin Khlebnikov
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Konstantin Khlebnikov @ 2020-06-01 12:37 UTC (permalink / raw)
To: linux-kernel, linux-block, dm-devel, linux-raid, Jens Axboe
Cc: Christoph Hellwig
Here is pretty straight forward attempt of handling REQ_NOWAIT for
bio-based and stacked devices.
They are marked with flag queue->limits.nowait_requests which tells that
queue method make_request() handles REQ_NOWAIT or doesn't delay requests,
and all backend devices do the same.
As a example second/third patches add support into md-raid0 and dm-linear.
---
Konstantin Khlebnikov (3):
block: add flag 'nowait_requests' into queue limits
md/raid0: enable REQ_NOWAIT
dm: add support for REQ_NOWAIT and enable for target dm-linear
drivers/md/dm-linear.c | 5 +++--
drivers/md/dm-table.c | 3 +++
drivers/md/dm.c | 4 +++-
drivers/md/raid0.c | 3 +++
include/linux/device-mapper.h | 6 ++++++
5 files changed, 18 insertions(+), 3 deletions(-)
--
Signature
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH RFC 1/3] block: add flag 'nowait_requests' into queue limits
2020-06-01 12:37 [PATCH RFC 0/3] block: allow REQ_NOWAIT to some bio-based/stacked devices Konstantin Khlebnikov
@ 2020-06-01 12:37 ` Konstantin Khlebnikov
2020-06-03 4:58 ` Christoph Hellwig
2020-06-01 12:37 ` [PATCH RFC 2/3] md/raid0: enable REQ_NOWAIT Konstantin Khlebnikov
2020-06-01 12:37 ` [PATCH RFC 3/3] dm: add support for REQ_NOWAIT and enable for target dm-linear Konstantin Khlebnikov
2 siblings, 1 reply; 6+ messages in thread
From: Konstantin Khlebnikov @ 2020-06-01 12:37 UTC (permalink / raw)
To: linux-kernel, linux-block, dm-devel, linux-raid, Jens Axboe
Cc: Christoph Hellwig
Add flag for marking bio-based queues which support REQ_NOWAIT.
Set for all request based (mq) devices.
Stacking device should set it after blk_set_stacking_limits() if method
make_request() itself doesn't delay requests or handles REQ_NOWAIT.
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
---
block/blk-core.c | 4 ++--
block/blk-mq.c | 3 +++
block/blk-settings.c | 3 +++
include/linux/blkdev.h | 1 +
4 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/block/blk-core.c b/block/blk-core.c
index c4b015004796..9139a316e6d4 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -892,9 +892,9 @@ generic_make_request_checks(struct bio *bio)
/*
* For a REQ_NOWAIT based request, return -EOPNOTSUPP
- * if queue is not a request based queue.
+ * if queue does not support this flag.
*/
- if ((bio->bi_opf & REQ_NOWAIT) && !queue_is_mq(q))
+ if ((bio->bi_opf & REQ_NOWAIT) && !q->limits.nowait_requests)
goto not_supported;
if (should_fail_bio(bio))
diff --git a/block/blk-mq.c b/block/blk-mq.c
index a7785df2c944..0c3daa0cda87 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2952,6 +2952,9 @@ struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set,
*/
q->poll_nsec = BLK_MQ_POLL_CLASSIC;
+ /* Request based queue always supports REQ_NOWAIT */
+ q->limits.nowait_requests = 1;
+
blk_mq_init_cpu_queues(q, set->nr_hw_queues);
blk_mq_add_queue_tag_set(set, q);
blk_mq_map_swqueue(q);
diff --git a/block/blk-settings.c b/block/blk-settings.c
index 14397b4c4b53..8f96c7324497 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -59,6 +59,7 @@ void blk_set_default_limits(struct queue_limits *lim)
lim->io_opt = 0;
lim->misaligned = 0;
lim->zoned = BLK_ZONED_NONE;
+ lim->nowait_requests = 0;
}
EXPORT_SYMBOL(blk_set_default_limits);
@@ -486,6 +487,8 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b,
t->max_segment_size = min_not_zero(t->max_segment_size,
b->max_segment_size);
+ t->nowait_requests &= b->nowait_requests;
+
t->misaligned |= b->misaligned;
alignment = queue_limit_alignment_offset(b, start);
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 32868fbedc9e..5f612dda34c2 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -346,6 +346,7 @@ struct queue_limits {
unsigned char misaligned;
unsigned char discard_misaligned;
unsigned char raid_partial_stripes_expensive;
+ unsigned char nowait_requests;
enum blk_zoned_model zoned;
};
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH RFC 2/3] md/raid0: enable REQ_NOWAIT
2020-06-01 12:37 [PATCH RFC 0/3] block: allow REQ_NOWAIT to some bio-based/stacked devices Konstantin Khlebnikov
2020-06-01 12:37 ` [PATCH RFC 1/3] block: add flag 'nowait_requests' into queue limits Konstantin Khlebnikov
@ 2020-06-01 12:37 ` Konstantin Khlebnikov
2020-06-01 12:37 ` [PATCH RFC 3/3] dm: add support for REQ_NOWAIT and enable for target dm-linear Konstantin Khlebnikov
2 siblings, 0 replies; 6+ messages in thread
From: Konstantin Khlebnikov @ 2020-06-01 12:37 UTC (permalink / raw)
To: linux-kernel, linux-block, dm-devel, linux-raid, Jens Axboe
Cc: Christoph Hellwig
Set limits.nowait_requests = 1 before stacking limits.
Raid itself does not delay bio in raid0_make_request().
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
---
drivers/md/raid0.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/md/raid0.c b/drivers/md/raid0.c
index 322386ff5d22..e34292b05488 100644
--- a/drivers/md/raid0.c
+++ b/drivers/md/raid0.c
@@ -406,6 +406,9 @@ static int raid0_run(struct mddev *mddev)
blk_queue_io_opt(mddev->queue,
(mddev->chunk_sectors << 9) * mddev->raid_disks);
+ /* raid0_make_request() does not delay requests */
+ mddev->queue->limits.nowait_requests = 1;
+
rdev_for_each(rdev, mddev) {
disk_stack_limits(mddev->gendisk, rdev->bdev,
rdev->data_offset << 9);
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH RFC 3/3] dm: add support for REQ_NOWAIT and enable for target dm-linear
2020-06-01 12:37 [PATCH RFC 0/3] block: allow REQ_NOWAIT to some bio-based/stacked devices Konstantin Khlebnikov
2020-06-01 12:37 ` [PATCH RFC 1/3] block: add flag 'nowait_requests' into queue limits Konstantin Khlebnikov
2020-06-01 12:37 ` [PATCH RFC 2/3] md/raid0: enable REQ_NOWAIT Konstantin Khlebnikov
@ 2020-06-01 12:37 ` Konstantin Khlebnikov
2 siblings, 0 replies; 6+ messages in thread
From: Konstantin Khlebnikov @ 2020-06-01 12:37 UTC (permalink / raw)
To: linux-kernel, linux-block, dm-devel, linux-raid, Jens Axboe
Cc: Christoph Hellwig
Add dm target feature flag DM_TARGET_NOWAIT which tells that target
has no problem with REQ_NOWAIT.
Set limits.nowait_requests if all targets and backends handle REQ_NOWAIT.
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
---
drivers/md/dm-linear.c | 5 +++--
drivers/md/dm-table.c | 3 +++
drivers/md/dm.c | 4 +++-
include/linux/device-mapper.h | 6 ++++++
4 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/drivers/md/dm-linear.c b/drivers/md/dm-linear.c
index e1db43446327..00774b5d7668 100644
--- a/drivers/md/dm-linear.c
+++ b/drivers/md/dm-linear.c
@@ -228,10 +228,11 @@ static struct target_type linear_target = {
.name = "linear",
.version = {1, 4, 0},
#ifdef CONFIG_BLK_DEV_ZONED
- .features = DM_TARGET_PASSES_INTEGRITY | DM_TARGET_ZONED_HM,
+ .features = DM_TARGET_PASSES_INTEGRITY | DM_TARGET_NOWAIT |
+ DM_TARGET_ZONED_HM,
.report_zones = linear_report_zones,
#else
- .features = DM_TARGET_PASSES_INTEGRITY,
+ .features = DM_TARGET_PASSES_INTEGRITY | DM_TARGET_NOWAIT,
#endif
.module = THIS_MODULE,
.ctr = linear_ctr,
diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c
index 0a2cc197f62b..f4610f79ebd6 100644
--- a/drivers/md/dm-table.c
+++ b/drivers/md/dm-table.c
@@ -1500,12 +1500,15 @@ int dm_calculate_queue_limits(struct dm_table *table,
unsigned int zone_sectors = 0;
blk_set_stacking_limits(limits);
+ limits->nowait_requests = 1;
for (i = 0; i < dm_table_get_num_targets(table); i++) {
blk_set_stacking_limits(&ti_limits);
ti = dm_table_get_target(table, i);
+ ti_limits.nowait_requests = dm_target_supports_nowait(ti->type);
+
if (!ti->type->iterate_devices)
goto combine_limits;
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index db9e46114653..767cd4d70341 100644
--- a/drivers/md/dm.c
+++ b/drivers/md/dm.c
@@ -1794,7 +1794,9 @@ static blk_qc_t dm_make_request(struct request_queue *q, struct bio *bio)
if (unlikely(test_bit(DMF_BLOCK_IO_FOR_SUSPEND, &md->flags))) {
dm_put_live_table(md, srcu_idx);
- if (!(bio->bi_opf & REQ_RAHEAD))
+ if (bio->bi_opf & REQ_NOWAIT)
+ bio_wouldblock_error(bio);
+ else if (!(bio->bi_opf & REQ_RAHEAD))
queue_io(md, bio);
else
bio_io_error(bio);
diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h
index af48d9da3916..4d4af1eeeba4 100644
--- a/include/linux/device-mapper.h
+++ b/include/linux/device-mapper.h
@@ -252,6 +252,12 @@ struct target_type {
#define DM_TARGET_ZONED_HM 0x00000040
#define dm_target_supports_zoned_hm(type) ((type)->features & DM_TARGET_ZONED_HM)
+/*
+ * A target handles REQ_NOWAIT
+ */
+#define DM_TARGET_NOWAIT 0x00000080
+#define dm_target_supports_nowait(type) (!!((type)->features & DM_TARGET_NOWAIT))
+
struct dm_target {
struct dm_table *table;
struct target_type *type;
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH RFC 1/3] block: add flag 'nowait_requests' into queue limits
2020-06-01 12:37 ` [PATCH RFC 1/3] block: add flag 'nowait_requests' into queue limits Konstantin Khlebnikov
@ 2020-06-03 4:58 ` Christoph Hellwig
2020-06-03 8:24 ` Konstantin Khlebnikov
0 siblings, 1 reply; 6+ messages in thread
From: Christoph Hellwig @ 2020-06-03 4:58 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: linux-kernel, linux-block, dm-devel, linux-raid, Jens Axboe,
Christoph Hellwig
On Mon, Jun 01, 2020 at 03:37:09PM +0300, Konstantin Khlebnikov wrote:
> Add flag for marking bio-based queues which support REQ_NOWAIT.
> Set for all request based (mq) devices.
>
> Stacking device should set it after blk_set_stacking_limits() if method
> make_request() itself doesn't delay requests or handles REQ_NOWAIT.
I don't think this belongs into the queue limits. For example a
stacking driver that always defers requests to a workqueue can support
REQ_NOWAIT entirely independent of the underlying devices. I think
this just needs to be a simple queue flag.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH RFC 1/3] block: add flag 'nowait_requests' into queue limits
2020-06-03 4:58 ` Christoph Hellwig
@ 2020-06-03 8:24 ` Konstantin Khlebnikov
0 siblings, 0 replies; 6+ messages in thread
From: Konstantin Khlebnikov @ 2020-06-03 8:24 UTC (permalink / raw)
To: Christoph Hellwig
Cc: linux-kernel, linux-block, dm-devel, linux-raid, Jens Axboe
On 03/06/2020 07.58, Christoph Hellwig wrote:
> On Mon, Jun 01, 2020 at 03:37:09PM +0300, Konstantin Khlebnikov wrote:
>> Add flag for marking bio-based queues which support REQ_NOWAIT.
>> Set for all request based (mq) devices.
>>
>> Stacking device should set it after blk_set_stacking_limits() if method
>> make_request() itself doesn't delay requests or handles REQ_NOWAIT.
>
> I don't think this belongs into the queue limits. For example a
> stacking driver that always defers requests to a workqueue can support
> REQ_NOWAIT entirely independent of the underlying devices. I think
> this just needs to be a simple queue flag.
>
For O_DIRECT I/O REQ_NOWAIT not just about non-blocking submition.
It also provides instant feedback about contention. Like ECN from network.
This feedback is useful for rate-control and balancing load between replicas.
If upper layer simply remaps and forwards requests below then to forward
contention all layers of stacked device should support this feature.
That's why I've put it as flag into limits - to reuse limits stacking.
If any layer defers request, then it should somehow limit size of backlog
at the same time to provide sane behaviour for REQ_NOWAIT regardless of
behaviour lower devices. So, then it could simply set that flag in limits.
Also I want to add handing into blk-qos/throttler - never delay REQ_NOWAIT.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-06-03 8:24 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-01 12:37 [PATCH RFC 0/3] block: allow REQ_NOWAIT to some bio-based/stacked devices Konstantin Khlebnikov
2020-06-01 12:37 ` [PATCH RFC 1/3] block: add flag 'nowait_requests' into queue limits Konstantin Khlebnikov
2020-06-03 4:58 ` Christoph Hellwig
2020-06-03 8:24 ` Konstantin Khlebnikov
2020-06-01 12:37 ` [PATCH RFC 2/3] md/raid0: enable REQ_NOWAIT Konstantin Khlebnikov
2020-06-01 12:37 ` [PATCH RFC 3/3] dm: add support for REQ_NOWAIT and enable for target dm-linear Konstantin Khlebnikov
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.