linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Dongli Zhang <dongli.zhang@oracle.com>
To: <virtualization@lists.linux-foundation.org>,
	<linux-block@vger.kernel.org>
Cc: <mst@redhat.com>, <axboe@kernel.dk>, <jasowang@redhat.com>,
	<linux-kernel@vger.kernel.org>
Subject: virtio-blk: should num_vqs be limited by num_possible_cpus()?
Date: Tue, 12 Mar 2019 10:22:46 -0700 (PDT)	[thread overview]
Message-ID: <e4afe4c5-0262-4500-aeec-60f30734b4fc@default> (raw)

I observed that there is one msix vector for config and one shared vector
for all queues in below qemu cmdline, when the num-queues for virtio-blk
is more than the number of possible cpus:

qemu: "-smp 4" while "-device virtio-blk-pci,drive=drive-0,id=virtblk0,num-queues=6"

# cat /proc/interrupts 
           CPU0       CPU1       CPU2       CPU3
... ...
 24:          0          0          0          0   PCI-MSI 65536-edge      virtio0-config
 25:          0          0          0         59   PCI-MSI 65537-edge      virtio0-virtqueues
... ...


However, when num-queues is the same as number of possible cpus:

qemu: "-smp 4" while "-device virtio-blk-pci,drive=drive-0,id=virtblk0,num-queues=4"

# cat /proc/interrupts 
           CPU0       CPU1       CPU2       CPU3
... ... 
 24:          0          0          0          0   PCI-MSI 65536-edge      virtio0-config
 25:          2          0          0          0   PCI-MSI 65537-edge      virtio0-req.0
 26:          0         35          0          0   PCI-MSI 65538-edge      virtio0-req.1
 27:          0          0         32          0   PCI-MSI 65539-edge      virtio0-req.2
 28:          0          0          0          0   PCI-MSI 65540-edge      virtio0-req.3
... ...

In above case, there is one msix vector per queue.


This is because the max number of queues is not limited by the number of
possible cpus.

By default, nvme (regardless about write_queues and poll_queues) and
xen-blkfront limit the number of queues with num_possible_cpus().


Is this by design on purpose, or can we fix with below?


diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
index 4bc083b..df95ce3 100644
--- a/drivers/block/virtio_blk.c
+++ b/drivers/block/virtio_blk.c
@@ -513,6 +513,8 @@ static int init_vq(struct virtio_blk *vblk)
 	if (err)
 		num_vqs = 1;
 
+	num_vqs = min(num_possible_cpus(), num_vqs);
+
 	vblk->vqs = kmalloc_array(num_vqs, sizeof(*vblk->vqs), GFP_KERNEL);
 	if (!vblk->vqs)
 		return -ENOMEM;
--


PS: The same issue is applicable to virtio-scsi as well.

Thank you very much!

Dongli Zhang

             reply	other threads:[~2019-03-12 17:22 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-03-12 17:22 Dongli Zhang [this message]
2019-03-12 17:33 ` Cornelia Huck
2019-03-13  3:26   ` Dongli Zhang
2019-03-13  9:39     ` Cornelia Huck
2019-03-14  6:12       ` Dongli Zhang
2019-03-14 12:13         ` Cornelia Huck
2019-03-14 16:08           ` Dongli Zhang
2019-03-15  4:50         ` Jason Wang
2019-03-15 12:41           ` Cornelia Huck
2019-03-18  7:47             ` Jason Wang
2019-03-19  2:22               ` Dongli Zhang
2019-03-20 12:53                 ` Jason Wang
2019-03-21  2:14                   ` Dongli Zhang
2019-03-21 15:57                   ` Stefan Hajnoczi
2019-03-14 12:32 ` Michael S. Tsirkin
2019-03-14 15:36   ` Dongli Zhang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=e4afe4c5-0262-4500-aeec-60f30734b4fc@default \
    --to=dongli.zhang@oracle.com \
    --cc=axboe@kernel.dk \
    --cc=jasowang@redhat.com \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mst@redhat.com \
    --cc=virtualization@lists.linux-foundation.org \
    --subject='Re: virtio-blk: should num_vqs be limited by num_possible_cpus()?' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).