Re: About sbitmap_bitmap_show() and cleared bits (was Re: [PATCH RFC v7 07/12] blk-mq: Add support in hctx_tags_bitmap_show() for a shared sbitmap)

From: John Garry <john.garry@huawei.com>
To: Hannes Reinecke <hare@suse.de>, <axboe@kernel.dk>,
	<linux-block@vger.kernel.org>
Cc: <jejb@linux.ibm.com>, <martin.petersen@oracle.com>,
	<don.brace@microsemi.com>, <kashyap.desai@broadcom.com>,
	<sumit.saxena@broadcom.com>, <ming.lei@redhat.com>,
	<bvanassche@acm.org>, <hare@suse.com>, <hch@lst.de>,
	<shivasharan.srikanteshwara@broadcom.com>,
	<linux-scsi@vger.kernel.org>, <esc.storagedev@microsemi.com>,
	<chenxiang66@hisilicon.com>, <megaraidlinux.pdl@broadcom.com>
Subject: Re: About sbitmap_bitmap_show() and cleared bits (was Re: [PATCH RFC v7 07/12] blk-mq: Add support in hctx_tags_bitmap_show() for a shared sbitmap)
Date: Tue, 30 Jun 2020 12:36:44 +0100	[thread overview]
Message-ID: <c95f6566-0f17-9d11-a52a-ac7433c8a2f0@huawei.com> (raw)
In-Reply-To: <a30ff47d-a06a-13d6-ef5d-8c90ba3261eb@huawei.com>

On 30/06/2020 08:30, John Garry wrote:
> On 30/06/2020 07:33, Hannes Reinecke wrote:
>> On 6/29/20 5:32 PM, John Garry wrote:
>>> Hi all,
>>>
>>> I noticed that sbitmap_bitmap_show() only shows set bits and does not 
>>> consider cleared bits. Is that the proper thing to do?
>>>
>>> I ask, as from trying to support sbitmap_bitmap_show() for hostwide 
>>> shared tags feature, we currently use blk_mq_queue_tag_busy_iter() to 
>>> find active requests (and associated tags/bits) for a particular 
>>> hctx. So, AFAICT, would give a change in behavior for 
>>> sbitmap_bitmap_show(), in that it would effectively show set and not 
>>> cleared bits.
>>>
>> Why would you need to do this?
>> Where would be the point traversing cleared bits?
> 
> I'm not talking about traversing cleared bits specifically. I just think 
> that today sbitmap_bitmap_show() only showing the bits in 
> sbitmap_word.word may not be useful or even misleading, as in reality 
> the "set" bits are sbitmap_word.word & ~sbitmap_word.cleared.
> 
> And for hostwide shared tags feature, iterating the busy rqs to find the 
> per-hctx tags/bits would effectively give us the "set" bits, above, so 
> there would be a difference.
> 

As an example, here's a sample tags_bitmap output:

00000000: 00f0 0fff 03c0 0000 0000 0000 efff fdff
00000010: 0000 c0f7 7fff ffff 0000 00e0 fef7 ffff
00000020: 0000 0000 f0ff ffff 0000 ffff 01d0 ffff
00000030: 0f80

And here's what we would have taking cleared bits into account:

00000000: 00f0 0fff 03c0 0000 0000 0000 0000 0000 (20 bits set)
00000010: 0000 0000 0000 0000 0000 0000 0000 0000
00000020: 0000 0000 0000 0000 0000 f8ff 0110 8000 (16 bits set)
00000030: 0f00					  (1 bit set)

Here's tags file output:

nr_tags=400
nr_reserved_tags=0
active_queues=2

bitmap_tags:
depth=400
busy=40
cleared=182
bits_per_word=64
map_nr=7
alloc_hint={22, 0, 0, 0, 103, 389, 231, 57, 377, 167, 0, 0, 69, 24, 44, 
50, 54,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0
, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}
wake_batch=8
wake_index=0

[snip]

20+16+1=39 more closely matches with busy=40.

So it seems sensible to go this way for whether hostwide tags are used 
or not.

thanks,
John