Re: [PATCH] blk-mq: plug request for shared sbitmap

[John Garry <john.garry@huawei.com>]

On 18/05/2021 12:54, Hannes Reinecke wrote:
>> In yanhui's virt workload(qemu, libaio, dio, high queue depth, single
>> job), the patch can improve throughput much(>50%) when running
>> sequential write(dio, libaio, 16 jobs) to XFS. And it is observed that
>> IO merge is recovered to level of disabling host tags.
>>
> I've found a good testcase for this by using null_blk:
> 
> modprobe null_blk submit_queues=32 queue_mode=2 irqmode=0 bs=4096
> hw_queue_depth=2048 completion_nsec=1 nr_devices=4 shared_tags=1
> shared_tag_bitmap=1
> 
> and using a simple fio job with libaio and rw=read and numjobs=32 will
> do the trick:
> 
> [nullb]
> rw=read
> size=16g
> ioengine=libaio
> direct=1
> buffered=0
> group_reporting=1
> bs=4096
> numjobs=32
> 
> (of course, the 'numjobs' and 'submit_queues' parameter would need to be
> adjusted to your machine).
> Omitting the 'shared_tag_bitmap' module parameter would yield around 12M
> iops; adding it would see a rather dramatic drop to 172k iops.
> With this patchset it's back to the expected iops value.

I will consider running this test myself, but do you mean that you get 
~12M with shared_tag_bitmap=1 and this patch?

I would always expect shared_tag_bitmap=1 to give a drop for null_blk, 
as we move from per-submit queue driver tag to all submit queues sharing 
the same driver tagset.

And I am not sure if you are fixing the IO sched from default.

Thanks,
John