From: Jason Wang <jasowang@redhat.com>
To: JeffleXu <jefflexu@linux.alibaba.com>,
Joseph Qi <joseph.qi@linux.alibaba.com>,
"Michael S. Tsirkin" <mst@redhat.com>
Cc: linux-block@vger.kernel.org, virtualization@lists.linux-foundation.org
Subject: Re: [PATCH RFC] virtio-blk: support per-device queue depth
Date: Wed, 20 Jan 2021 10:47:40 +0800 [thread overview]
Message-ID: <b782bcb7-9b45-a122-0f80-8cfcd89b5f70@redhat.com> (raw)
In-Reply-To: <25d8430c-bf18-38ca-cf0c-c054507e1466@linux.alibaba.com>
On 2021/1/20 上午9:51, JeffleXu wrote:
>
> On 1/19/21 12:06 PM, Jason Wang wrote:
>> On 2021/1/19 上午9:33, JeffleXu wrote:
>>> On 1/18/21 1:25 PM, Jason Wang wrote:
>>>> On 2021/1/18 上午11:58, Joseph Qi wrote:
>>>>> module parameter 'virtblk_queue_depth' was firstly introduced for
>>>>> testing/benchmarking purposes described in commit fc4324b4597c
>>>>> ("virtio-blk: base queue-depth on virtqueue ringsize or module param").
>>>>> Since we have different virtio-blk devices which have different
>>>>> capabilities, it requires that we support per-device queue depth
>>>>> instead
>>>>> of per-module. So defaultly use vq free elements if module parameter
>>>>> 'virtblk_queue_depth' is not set.
>>>> I wonder if it's better to use sysfs instead (or whether it has already
>>>> had something like this in the blocker layer).
>>>>
>>> "/sys/block/<dev>/queue/nr_requests" indeed works, but isn't better to
>>> set queue_depth according to the hardware capability at the very first?
>>> AFAIK, nvme just set per-device queue_depth at initializing phase.
>>
>> I agree, the problem is that the current code may modify module parameter.
> The module parameter 'virtblk_queue_depth' is actually remained untainted.
>
> Actually it is the original code before this patch that changes the
> module parameter.
Yes, that's what I meant.
> When the module parameter is not set by boot cmdline
> (i.e., default to 0), it will be initialized to the queue_depth of the
> vring of the first probed virtio-blk device, and will be revealed to
> user space through '/sys/module/virtio_blk/parameters/queue_depth'. I'm
> not sure if this behavior is reasonable or not.
Right, it means the virtio-blk devices that is probed after the first
one can only use the queue_depth that is set according to the capability
of the first virtio-blk device.
>
> The only side effect of this patch is that, now
> '/sys/module/virtio_blk/parameters/queue_depth' will be kept as '0' when
> the module parameter is not set manually.
I think it's not an issue, the nr_request should be the correct way to
get per device queue depth.
Thanks
>
>
> Thanks,
> Jeffle
>
>
>>>>> Signed-off-by: Joseph Qi <joseph.qi@linux.alibaba.com>
>>>>> ---
>>>>> drivers/block/virtio_blk.c | 12 +++++++-----
>>>>> 1 file changed, 7 insertions(+), 5 deletions(-)
>>>>>
>>>>> diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
>>>>> index 145606d..f83a417 100644
>>>>> --- a/drivers/block/virtio_blk.c
>>>>> +++ b/drivers/block/virtio_blk.c
>>>>> @@ -705,6 +705,7 @@ static int virtblk_probe(struct virtio_device
>>>>> *vdev)
>>>>> u32 v, blk_size, max_size, sg_elems, opt_io_size;
>>>>> u16 min_io_size;
>>>>> u8 physical_block_exp, alignment_offset;
>>>>> + unsigned int queue_depth;
>>>>> if (!vdev->config->get) {
>>>>> dev_err(&vdev->dev, "%s failure: config access disabled\n",
>>>>> @@ -755,17 +756,18 @@ static int virtblk_probe(struct virtio_device
>>>>> *vdev)
>>>>> goto out_free_vq;
>>>>> }
>>>>> - /* Default queue sizing is to fill the ring. */
>>>>> - if (!virtblk_queue_depth) {
>>>>> - virtblk_queue_depth = vblk->vqs[0].vq->num_free;
>>>>> + if (likely(!virtblk_queue_depth)) {
>>>>> + queue_depth = vblk->vqs[0].vq->num_free;
>>>>> /* ... but without indirect descs, we use 2 descs per req */
>>>>> if (!virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC))
>>>>> - virtblk_queue_depth /= 2;
>>>>> + queue_depth /= 2;
>>>>> + } else {
>>>>> + queue_depth = virtblk_queue_depth;
>>>>> }
>>>>> memset(&vblk->tag_set, 0, sizeof(vblk->tag_set));
>>>>> vblk->tag_set.ops = &virtio_mq_ops;
>>>>> - vblk->tag_set.queue_depth = virtblk_queue_depth;
>>>>> + vblk->tag_set.queue_depth = queue_depth;
>>>>> vblk->tag_set.numa_node = NUMA_NO_NODE;
>>>>> vblk->tag_set.flags = BLK_MQ_F_SHOULD_MERGE;
>>>>> vblk->tag_set.cmd_size =
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
next prev parent reply other threads:[~2021-01-20 2:48 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <1610942338-78252-1-git-send-email-joseph.qi@linux.alibaba.com>
2021-01-18 5:25 ` [PATCH RFC] virtio-blk: support per-device queue depth Jason Wang
2021-01-19 1:33 ` JeffleXu
2021-01-19 4:06 ` Jason Wang
2021-01-20 1:51 ` JeffleXu
2021-01-20 2:47 ` Jason Wang [this message]
[not found] ` <9a736867-d420-26eb-3ee2-42869a069640@linux.alibaba.com>
2021-01-19 4:04 ` Jason Wang
2021-01-19 4:14 ` Jason Wang
[not found] ` <ce313c74-645f-3a55-44ac-4e757497c778@linux.alibaba.com>
2021-01-22 8:34 ` Michael S. Tsirkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b782bcb7-9b45-a122-0f80-8cfcd89b5f70@redhat.com \
--to=jasowang@redhat.com \
--cc=jefflexu@linux.alibaba.com \
--cc=joseph.qi@linux.alibaba.com \
--cc=linux-block@vger.kernel.org \
--cc=mst@redhat.com \
--cc=virtualization@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).