Re: [Qemu-devel] [PATCH V3 00/14] Support more virtio queues

["Michael S. Tsirkin" <mst@redhat.com>]

On Thu, Mar 05, 2015 at 01:48:37PM +0800, Jason Wang wrote:
> We current limit the max virtio queues to 64. This is not sufficient
> to support multiqueue deivces (e.g recent Linux support up to 256
> tap queues). So this series try to let virtio to support more queues.

For some reason I was Cc'd only on some patches.
If you want me to apply series, pls Cc on all of them.

> No much works need to be done except:
> 
> - Patch 1 add a check to validate the queues supported by backend
>   against the virtio queue limitation
> - Patch 2 fixes a bug in virtio_del_queue()
> - Patch 3 introdues a bus specific queue limitation
> - Patch 4 - Patch 8 let each transport to use bus specific queue
>   limitation instead of VIRTIO_PCI_QUEUE_MAX
> - Patch 9 introduces a map from vector to queues that use this vector
> - Patch 10 introduces a helpr for get the queue index
> - Patch 11 speedups the MSI-X masking and unmasking through the vector
>   to queues mapping that introduced by Patch 9
> - Patch 12 increase the maximum number of queues supported by
>   virtio-pci from 64 to 513, and deal with the migration compatibility
>   with the changes.
> - Patch 13 tries to remove the hard coded MSI-X bar size (4096) and
>   allow up to 2048 MSI-X entries.
> - Patch 14 add a boolean property to let the virtio-net can calculate
>   the MSI-X bar size based on the number of MSI-X vectors and keep
>   migration compatibility with legacy version whose bar size is 4096.
> 
> With this patch, we can support up to 256 queues. Since x86 can only
> allow about 240 interrupt vectors for MSI-X, current Linux driver can
> only have about 80 queue pairs has their private MSI-X interrupt
> vectors. With sharing IRQ with queue pairs (RFC posted in
> https://lkml.org/lkml/2014/12/25/169), Linux driver can have up
> to about 186 queue pairs has their private MSI-X interrupt vectors.
> 
> Stress/migration test on virtio-pci, compile test/make check on
> s390x-softmmu.
> 
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Richard Henderson <rth@twiddle.net>
> Cc: Anthony Liguori <aliguori@amazon.com>
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Cc: Alexander Graf <agraf@suse.de>
> Cc: Keith Busch <keith.busch@intel.com>
> Cc: Kevin Wolf <kwolf@redhat.com>
> Cc: Stefan Hajnoczi <stefanha@redhat.com>
> Cc: Anthony Liguori <aliguori@amazon.com>
> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
> Cc: Cornelia Huck <cornelia.huck@de.ibm.com>
> Cc: Amit Shah <amit.shah@redhat.com>
> 
> Please review
> 
> Thanks
> 
> Changes from V2:
> - Move transport specific limitation to their implementation. (The
>   left is VIRTIO_CCW_QUEUE_MAX since I fail to find a common header
>   files other than virtio.h)
> - Use virtio_get_queue_max() if possbile in virtio.c
> - AdapterRoutes should use ccw limit
> - Introduce a vector to queue mapping for virito devices and speedup
>   the MSI-X masking and unmasking through this.
> 
> Changes from V1:
> - add a validation against the bus limitation
> - switch to use a bus specific queue limit instead of a global one,
>   this will allow us to just increase the limit of one transport
>   without disturbing others.
> - only increase the queue limit of virtio-pci
> - limit the maximum number of virtio queues to 64 for legacy machine
>   types
> 
> Jason Wang (14):
>   virtio-net: validate backend queue numbers against bus limitation
>   virtio-net: fix the upper bound when trying to delete queues
>   virito: introduce bus specific queue limit
>   virtio-ccw: introduce ccw specific queue limit
>   virtio-s390: switch to bus specific queue limit
>   virtio-serial-bus: switch to bus specific queue limit
>   virtio-mmio: switch to bus specific queue limit
>   virtio-pci: switch to use bus specific queue limit
>   virtio: introduce vector to virtqueues mapping
>   virtio: introduce virtio_queue_get_index()
>   virtio-pci: speedup MSI-X masking and unmasking
>   virtio-pci: increase the maximum number of virtqueues to 513
>   pci: remove hard-coded bar size in msix_init_exclusive_bar()
>   virtio-pci: introduce auto_msix_bar_size property
> 
>  hw/block/nvme.c                |  2 +-
>  hw/char/virtio-serial-bus.c    |  4 ++-
>  hw/i386/pc_piix.c              | 10 ++++++
>  hw/i386/pc_q35.c               |  9 +++++
>  hw/misc/ivshmem.c              |  2 +-
>  hw/net/virtio-net.c            |  9 ++++-
>  hw/pci/msix.c                  | 18 ++++------
>  hw/ppc/spapr.c                 |  5 +++
>  hw/s390x/s390-virtio-bus.c     |  7 ++--
>  hw/s390x/s390-virtio-ccw.c     |  7 ++--
>  hw/s390x/virtio-ccw.c          | 12 ++++---
>  hw/scsi/virtio-scsi.c          |  4 +--
>  hw/virtio/virtio-mmio.c        |  7 ++--
>  hw/virtio/virtio-pci.c         | 78 ++++++++++++++++++++++++++++--------------
>  hw/virtio/virtio-pci.h         |  3 ++
>  hw/virtio/virtio.c             | 74 ++++++++++++++++++++++++++++++---------
>  include/hw/compat.h            |  8 +++++
>  include/hw/pci/msix.h          |  2 +-
>  include/hw/s390x/s390_flic.h   |  2 +-
>  include/hw/virtio/virtio-bus.h |  2 ++
>  include/hw/virtio/virtio.h     |  7 +++-
>  21 files changed, 199 insertions(+), 73 deletions(-)
> 
> -- 
> 2.1.0