[PATCH v1 0/4] Increase default virtqueue size to improve performance

[PATCH v1 0/4] Increase default virtqueue size to improve performance

[Denis Plotnikov]

The goal is to increase the performance of the block layer on
1M reads/writes up to 4% by reducing the amount of requests issued by a guest
using virtio-scsi or virtio-blk devices.

Original problem description:
https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html


Denis Plotnikov (4):
  virtio: introduce VIRTQUEUE_DEFUALT_SIZE instead of hardcoded
    constants
  virtio: increase virtuqueue size for virtio-scsi and virtio-blk
  tests: add virtuqueue size checking to virtio_seg_max_adjust test
  tests: rename virtio_seg_max_adjust to virtio_check_params

 hw/block/virtio-blk.c                         |  6 ++--
 hw/core/machine.c                             |  3 ++
 hw/scsi/virtio-scsi.c                         |  5 +--
 include/hw/virtio/virtio.h                    |  1 +
 ...g_max_adjust.py => virtio_check_params.py} | 33 ++++++++++++-------
 5 files changed, 32 insertions(+), 16 deletions(-)
 rename tests/acceptance/{virtio_seg_max_adjust.py => virtio_check_params.py} (79%)

-- 
2.17.0

[PATCH v1 1/4] virtio: introduce VIRTQUEUE_DEFUALT_SIZE instead of hardcoded constants

[Denis Plotnikov]

Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
---
 hw/block/virtio-blk.c      | 6 ++++--
 hw/scsi/virtio-scsi.c      | 5 +++--
 include/hw/virtio/virtio.h | 1 +
 3 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 09f46ed85f..72f935033f 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -914,7 +914,8 @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
     memset(&blkcfg, 0, sizeof(blkcfg));
     virtio_stq_p(vdev, &blkcfg.capacity, capacity);
     virtio_stl_p(vdev, &blkcfg.seg_max,
-                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 : 128 - 2);
+                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 :
+                                          VIRTQUEUE_DEFAULT_SIZE - 2);
     virtio_stw_p(vdev, &blkcfg.geometry.cylinders, conf->cyls);
     virtio_stl_p(vdev, &blkcfg.blk_size, blk_size);
     virtio_stw_p(vdev, &blkcfg.min_io_size, conf->min_io_size / blk_size);
@@ -1272,7 +1273,8 @@ static Property virtio_blk_properties[] = {
     DEFINE_PROP_BIT("request-merging", VirtIOBlock, conf.request_merging, 0,
                     true),
     DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, 1),
-    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size, 128),
+    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size,
+                       VIRTQUEUE_DEFAULT_SIZE),
     DEFINE_PROP_BOOL("seg-max-adjust", VirtIOBlock, conf.seg_max_adjust, true),
     DEFINE_PROP_LINK("iothread", VirtIOBlock, conf.iothread, TYPE_IOTHREAD,
                      IOThread *),
diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index 3b61563609..36f66046ae 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -660,7 +660,8 @@ static void virtio_scsi_get_config(VirtIODevice *vdev,
 
     virtio_stl_p(vdev, &scsiconf->num_queues, s->conf.num_queues);
     virtio_stl_p(vdev, &scsiconf->seg_max,
-                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 : 128 - 2);
+                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 :
+                                          VIRTQUEUE_DEFAULT_SIZE - 2);
     virtio_stl_p(vdev, &scsiconf->max_sectors, s->conf.max_sectors);
     virtio_stl_p(vdev, &scsiconf->cmd_per_lun, s->conf.cmd_per_lun);
     virtio_stl_p(vdev, &scsiconf->event_info_size, sizeof(VirtIOSCSIEvent));
@@ -965,7 +966,7 @@ static void virtio_scsi_device_unrealize(DeviceState *dev, Error **errp)
 static Property virtio_scsi_properties[] = {
     DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, 1),
     DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSI,
-                                         parent_obj.conf.virtqueue_size, 128),
+                       parent_obj.conf.virtqueue_size, VIRTQUEUE_DEFAULT_SIZE),
     DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSI,
                       parent_obj.conf.seg_max_adjust, true),
     DEFINE_PROP_UINT32("max_sectors", VirtIOSCSI, parent_obj.conf.max_sectors,
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index b69d517496..a66ea2368b 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -48,6 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features,
 typedef struct VirtQueue VirtQueue;
 
 #define VIRTQUEUE_MAX_SIZE 1024
+#define VIRTQUEUE_DEFAULT_SIZE 128
 
 typedef struct VirtQueueElement
 {
-- 
2.17.0

[PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Denis Plotnikov]

The goal is to reduce the amount of requests issued by a guest on
1M reads/writes. This rises the performance up to 4% on that kind of
disk access pattern.

The maximum chunk size to be used for the guest disk accessing is
limited with seg_max parameter, which represents the max amount of
pices in the scatter-geather list in one guest disk request.

Since seg_max is virqueue_size dependent, increasing the virtqueue
size increases seg_max, which, in turn, increases the maximum size
of data to be read/write from guest disk.

More details in the original problem statment:
https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html

Suggested-by: Denis V. Lunev <den@openvz.org>
Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
---
 hw/core/machine.c          | 3 +++
 include/hw/virtio/virtio.h | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index 3e288bfceb..8bc401d8b7 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -28,6 +28,9 @@
 #include "hw/mem/nvdimm.h"
 
 GlobalProperty hw_compat_4_2[] = {
+    { "virtio-blk-device", "queue-size", "128"},
+    { "virtio-scsi-device", "virtqueue_size", "128"},
+    { "vhost-blk-device", "virtqueue_size", "128"},
     { "virtio-blk-device", "x-enable-wce-if-config-wce", "off" },
     { "virtio-blk-device", "seg-max-adjust", "off"},
     { "virtio-scsi-device", "seg_max_adjust", "off"},
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index a66ea2368b..16d540e390 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -48,7 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features,
 typedef struct VirtQueue VirtQueue;
 
 #define VIRTQUEUE_MAX_SIZE 1024
-#define VIRTQUEUE_DEFAULT_SIZE 128
+#define VIRTQUEUE_DEFAULT_SIZE 256
 
 typedef struct VirtQueueElement
 {
-- 
2.17.0

[PATCH v1 3/4] tests: add virtuqueue size checking to virtio_seg_max_adjust test

[Denis Plotnikov]

This is due to the change in the default virtqueue_size in the
latest machine type to improve guest disks performance.

Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
---
 tests/acceptance/virtio_seg_max_adjust.py | 33 ++++++++++++++---------
 1 file changed, 21 insertions(+), 12 deletions(-)

diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
index 5458573138..645d69b313 100755
--- a/tests/acceptance/virtio_seg_max_adjust.py
+++ b/tests/acceptance/virtio_seg_max_adjust.py
@@ -27,8 +27,10 @@ from qemu.machine import QEMUMachine
 from avocado_qemu import Test
 
 #list of machine types and virtqueue properties to test
-VIRTIO_SCSI_PROPS = {'seg_max_adjust': 'seg_max_adjust'}
-VIRTIO_BLK_PROPS = {'seg_max_adjust': 'seg-max-adjust'}
+VIRTIO_SCSI_PROPS = {'seg_max_adjust': 'seg_max_adjust',
+                     'queue_size': 'virtqueue_size'}
+VIRTIO_BLK_PROPS = {'seg_max_adjust': 'seg-max-adjust',
+                    'queue_size': 'queue-size'}
 
 DEV_TYPES = {'virtio-scsi-pci': VIRTIO_SCSI_PROPS,
              'virtio-blk-pci': VIRTIO_BLK_PROPS}
@@ -40,7 +42,7 @@ VM_DEV_PARAMS = {'virtio-scsi-pci': ['-device', 'virtio-scsi-pci,id=scsi0'],
                                     'driver=null-co,id=drive0,if=none']}
 
 
-class VirtioMaxSegSettingsCheck(Test):
+class VirtioPramsCheck(Test):
     @staticmethod
     def make_pattern(props):
         pattern_items = ['{0} = \w+'.format(prop) for prop in props]
@@ -72,20 +74,24 @@ class VirtioMaxSegSettingsCheck(Test):
                 props[p[0]] = p[1]
         return query_ok, props, error
 
-    def check_mt(self, mt, dev_type_name):
+    def check_mt(self, mt, expected_props, dev_type_name):
         with QEMUMachine(self.qemu_bin) as vm:
-            vm.set_machine(mt["name"])
+            vm.set_machine(mt)
             for s in VM_DEV_PARAMS[dev_type_name]:
                 vm.add_args(s)
             vm.launch()
             query_ok, props, error = self.query_virtqueue(vm, dev_type_name)
 
         if not query_ok:
-            self.fail('machine type {0}: {1}'.format(mt['name'], error))
+            self.fail('machine type {0}: {1}'.format(mt, error))
 
         for prop_name, prop_val in props.items():
-            expected_val = mt[prop_name]
-            self.assertEqual(expected_val, prop_val)
+            expected_val = expected_props[prop_name]
+            msg = 'Property value mismatch for (MT: {0}, '\
+                  'property name: {1}): expected value: "{2}" '\
+                  'actual value: "{3}"'\
+                  .format(mt, prop_name, expected_val, prop_val)
+            self.assertEqual(expected_val, prop_val, msg)
 
     @staticmethod
     def seg_max_adjust_enabled(mt):
@@ -120,15 +126,18 @@ class VirtioMaxSegSettingsCheck(Test):
 
         for dev_type in DEV_TYPES:
             # create the list of machine types and their parameters.
-            mtypes = list()
+            mtypes = dict()
             for m in machines:
                 if self.seg_max_adjust_enabled(m):
                     enabled = 'true'
+                    queue_size = '256'
                 else:
                     enabled = 'false'
-                mtypes.append({'name': m,
-                               DEV_TYPES[dev_type]['seg_max_adjust']: enabled})
+                    queue_size = '128'
+                mtypes[m] = {
+                    DEV_TYPES[dev_type]['seg_max_adjust']: enabled,
+                    DEV_TYPES[dev_type]['queue_size']: queue_size }
 
             # test each machine type for a device type
             for mt in mtypes:
-                self.check_mt(mt, dev_type)
+                self.check_mt(mt, mtypes[mt], dev_type)
-- 
2.17.0

[PATCH v1 4/4] tests: rename virtio_seg_max_adjust to virtio_check_params

[Denis Plotnikov]

Since, virtio_seg_max_adjust checks not only seg_max, but also
virtqueue_size parameter, let's make the test more general and
add new parameters to be checked there in the future.

Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
---
 .../{virtio_seg_max_adjust.py => virtio_check_params.py}          | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 rename tests/acceptance/{virtio_seg_max_adjust.py => virtio_check_params.py} (100%)

diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_check_params.py
similarity index 100%
rename from tests/acceptance/virtio_seg_max_adjust.py
rename to tests/acceptance/virtio_check_params.py
-- 
2.17.0

Re: [PATCH v1 1/4] virtio: introduce VIRTQUEUE_DEFUALT_SIZE instead of hardcoded constants

[Philippe Mathieu-Daudé]

On 1/29/20 3:06 PM, Denis Plotnikov wrote:
> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>

typo VIRTQUEUE_DEFUALT_SIZE -> VIRTQUEUE_DEFAULT_SIZE in subject

With subject fixed:
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

> ---
>   hw/block/virtio-blk.c      | 6 ++++--
>   hw/scsi/virtio-scsi.c      | 5 +++--
>   include/hw/virtio/virtio.h | 1 +
>   3 files changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
> index 09f46ed85f..72f935033f 100644
> --- a/hw/block/virtio-blk.c
> +++ b/hw/block/virtio-blk.c
> @@ -914,7 +914,8 @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
>       memset(&blkcfg, 0, sizeof(blkcfg));
>       virtio_stq_p(vdev, &blkcfg.capacity, capacity);
>       virtio_stl_p(vdev, &blkcfg.seg_max,
> -                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 : 128 - 2);
> +                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 :
> +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
>       virtio_stw_p(vdev, &blkcfg.geometry.cylinders, conf->cyls);
>       virtio_stl_p(vdev, &blkcfg.blk_size, blk_size);
>       virtio_stw_p(vdev, &blkcfg.min_io_size, conf->min_io_size / blk_size);
> @@ -1272,7 +1273,8 @@ static Property virtio_blk_properties[] = {
>       DEFINE_PROP_BIT("request-merging", VirtIOBlock, conf.request_merging, 0,
>                       true),
>       DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, 1),
> -    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size, 128),
> +    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size,
> +                       VIRTQUEUE_DEFAULT_SIZE),
>       DEFINE_PROP_BOOL("seg-max-adjust", VirtIOBlock, conf.seg_max_adjust, true),
>       DEFINE_PROP_LINK("iothread", VirtIOBlock, conf.iothread, TYPE_IOTHREAD,
>                        IOThread *),
> diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
> index 3b61563609..36f66046ae 100644
> --- a/hw/scsi/virtio-scsi.c
> +++ b/hw/scsi/virtio-scsi.c
> @@ -660,7 +660,8 @@ static void virtio_scsi_get_config(VirtIODevice *vdev,
>   
>       virtio_stl_p(vdev, &scsiconf->num_queues, s->conf.num_queues);
>       virtio_stl_p(vdev, &scsiconf->seg_max,
> -                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 : 128 - 2);
> +                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 :
> +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
>       virtio_stl_p(vdev, &scsiconf->max_sectors, s->conf.max_sectors);
>       virtio_stl_p(vdev, &scsiconf->cmd_per_lun, s->conf.cmd_per_lun);
>       virtio_stl_p(vdev, &scsiconf->event_info_size, sizeof(VirtIOSCSIEvent));
> @@ -965,7 +966,7 @@ static void virtio_scsi_device_unrealize(DeviceState *dev, Error **errp)
>   static Property virtio_scsi_properties[] = {
>       DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, 1),
>       DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSI,
> -                                         parent_obj.conf.virtqueue_size, 128),
> +                       parent_obj.conf.virtqueue_size, VIRTQUEUE_DEFAULT_SIZE),
>       DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSI,
>                         parent_obj.conf.seg_max_adjust, true),
>       DEFINE_PROP_UINT32("max_sectors", VirtIOSCSI, parent_obj.conf.max_sectors,
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index b69d517496..a66ea2368b 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -48,6 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features,
>   typedef struct VirtQueue VirtQueue;
>   
>   #define VIRTQUEUE_MAX_SIZE 1024
> +#define VIRTQUEUE_DEFAULT_SIZE 128
>   
>   typedef struct VirtQueueElement
>   {
> 

Re: [PATCH v1 1/4] virtio: introduce VIRTQUEUE_DEFUALT_SIZE instead of hardcoded constants

[Cornelia Huck]

On Wed, 29 Jan 2020 17:06:59 +0300
Denis Plotnikov <dplotnikov@virtuozzo.com> wrote:

> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
> ---
>  hw/block/virtio-blk.c      | 6 ++++--
>  hw/scsi/virtio-scsi.c      | 5 +++--
>  include/hw/virtio/virtio.h | 1 +
>  3 files changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
> index 09f46ed85f..72f935033f 100644
> --- a/hw/block/virtio-blk.c
> +++ b/hw/block/virtio-blk.c
> @@ -914,7 +914,8 @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
>      memset(&blkcfg, 0, sizeof(blkcfg));
>      virtio_stq_p(vdev, &blkcfg.capacity, capacity);
>      virtio_stl_p(vdev, &blkcfg.seg_max,
> -                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 : 128 - 2);
> +                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 :
> +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
>      virtio_stw_p(vdev, &blkcfg.geometry.cylinders, conf->cyls);
>      virtio_stl_p(vdev, &blkcfg.blk_size, blk_size);
>      virtio_stw_p(vdev, &blkcfg.min_io_size, conf->min_io_size / blk_size);
> @@ -1272,7 +1273,8 @@ static Property virtio_blk_properties[] = {
>      DEFINE_PROP_BIT("request-merging", VirtIOBlock, conf.request_merging, 0,
>                      true),
>      DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, 1),
> -    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size, 128),
> +    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size,
> +                       VIRTQUEUE_DEFAULT_SIZE),
>      DEFINE_PROP_BOOL("seg-max-adjust", VirtIOBlock, conf.seg_max_adjust, true),
>      DEFINE_PROP_LINK("iothread", VirtIOBlock, conf.iothread, TYPE_IOTHREAD,
>                       IOThread *),
> diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
> index 3b61563609..36f66046ae 100644
> --- a/hw/scsi/virtio-scsi.c
> +++ b/hw/scsi/virtio-scsi.c
> @@ -660,7 +660,8 @@ static void virtio_scsi_get_config(VirtIODevice *vdev,
>  
>      virtio_stl_p(vdev, &scsiconf->num_queues, s->conf.num_queues);
>      virtio_stl_p(vdev, &scsiconf->seg_max,
> -                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 : 128 - 2);
> +                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 :
> +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
>      virtio_stl_p(vdev, &scsiconf->max_sectors, s->conf.max_sectors);
>      virtio_stl_p(vdev, &scsiconf->cmd_per_lun, s->conf.cmd_per_lun);
>      virtio_stl_p(vdev, &scsiconf->event_info_size, sizeof(VirtIOSCSIEvent));
> @@ -965,7 +966,7 @@ static void virtio_scsi_device_unrealize(DeviceState *dev, Error **errp)
>  static Property virtio_scsi_properties[] = {
>      DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, 1),
>      DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSI,
> -                                         parent_obj.conf.virtqueue_size, 128),
> +                       parent_obj.conf.virtqueue_size, VIRTQUEUE_DEFAULT_SIZE),
>      DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSI,
>                        parent_obj.conf.seg_max_adjust, true),
>      DEFINE_PROP_UINT32("max_sectors", VirtIOSCSI, parent_obj.conf.max_sectors,
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index b69d517496..a66ea2368b 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -48,6 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features,
>  typedef struct VirtQueue VirtQueue;
>  
>  #define VIRTQUEUE_MAX_SIZE 1024
> +#define VIRTQUEUE_DEFAULT_SIZE 128

Going from the header only, this looks like a value that is supposed to
be used for every virtqueue... but from the users, this is only for blk
and scsi.

I don't think adding a default for everything makes sense, even if the
same value makes sense for blk and scsi.

>  
>  typedef struct VirtQueueElement
>  {

Re: [PATCH v1 1/4] virtio: introduce VIRTQUEUE_DEFUALT_SIZE instead of hardcoded constants

[Michael S. Tsirkin]

On Wed, Jan 29, 2020 at 05:06:59PM +0300, Denis Plotnikov wrote:
> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>


I'm not sure what the point is. It's more or less an accident that
these two devices share the queue size, this constance
makes no sense to me.

> ---
>  hw/block/virtio-blk.c      | 6 ++++--
>  hw/scsi/virtio-scsi.c      | 5 +++--
>  include/hw/virtio/virtio.h | 1 +
>  3 files changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
> index 09f46ed85f..72f935033f 100644
> --- a/hw/block/virtio-blk.c
> +++ b/hw/block/virtio-blk.c
> @@ -914,7 +914,8 @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
>      memset(&blkcfg, 0, sizeof(blkcfg));
>      virtio_stq_p(vdev, &blkcfg.capacity, capacity);
>      virtio_stl_p(vdev, &blkcfg.seg_max,
> -                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 : 128 - 2);
> +                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 :
> +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
>      virtio_stw_p(vdev, &blkcfg.geometry.cylinders, conf->cyls);
>      virtio_stl_p(vdev, &blkcfg.blk_size, blk_size);
>      virtio_stw_p(vdev, &blkcfg.min_io_size, conf->min_io_size / blk_size);
> @@ -1272,7 +1273,8 @@ static Property virtio_blk_properties[] = {
>      DEFINE_PROP_BIT("request-merging", VirtIOBlock, conf.request_merging, 0,
>                      true),
>      DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, 1),
> -    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size, 128),
> +    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size,
> +                       VIRTQUEUE_DEFAULT_SIZE),
>      DEFINE_PROP_BOOL("seg-max-adjust", VirtIOBlock, conf.seg_max_adjust, true),
>      DEFINE_PROP_LINK("iothread", VirtIOBlock, conf.iothread, TYPE_IOTHREAD,
>                       IOThread *),
> diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
> index 3b61563609..36f66046ae 100644
> --- a/hw/scsi/virtio-scsi.c
> +++ b/hw/scsi/virtio-scsi.c
> @@ -660,7 +660,8 @@ static void virtio_scsi_get_config(VirtIODevice *vdev,
>  
>      virtio_stl_p(vdev, &scsiconf->num_queues, s->conf.num_queues);
>      virtio_stl_p(vdev, &scsiconf->seg_max,
> -                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 : 128 - 2);
> +                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 :
> +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
>      virtio_stl_p(vdev, &scsiconf->max_sectors, s->conf.max_sectors);
>      virtio_stl_p(vdev, &scsiconf->cmd_per_lun, s->conf.cmd_per_lun);
>      virtio_stl_p(vdev, &scsiconf->event_info_size, sizeof(VirtIOSCSIEvent));
> @@ -965,7 +966,7 @@ static void virtio_scsi_device_unrealize(DeviceState *dev, Error **errp)
>  static Property virtio_scsi_properties[] = {
>      DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, 1),
>      DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSI,
> -                                         parent_obj.conf.virtqueue_size, 128),
> +                       parent_obj.conf.virtqueue_size, VIRTQUEUE_DEFAULT_SIZE),
>      DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSI,
>                        parent_obj.conf.seg_max_adjust, true),
>      DEFINE_PROP_UINT32("max_sectors", VirtIOSCSI, parent_obj.conf.max_sectors,
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index b69d517496..a66ea2368b 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -48,6 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features,
>  typedef struct VirtQueue VirtQueue;
>  
>  #define VIRTQUEUE_MAX_SIZE 1024
> +#define VIRTQUEUE_DEFAULT_SIZE 128
>  
>  typedef struct VirtQueueElement
>  {
> -- 
> 2.17.0

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Michael S. Tsirkin]

On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
> The goal is to reduce the amount of requests issued by a guest on
> 1M reads/writes. This rises the performance up to 4% on that kind of
> disk access pattern.
> 
> The maximum chunk size to be used for the guest disk accessing is
> limited with seg_max parameter, which represents the max amount of
> pices in the scatter-geather list in one guest disk request.
> 
> Since seg_max is virqueue_size dependent, increasing the virtqueue
> size increases seg_max, which, in turn, increases the maximum size
> of data to be read/write from guest disk.
> 
> More details in the original problem statment:
> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
> 
> Suggested-by: Denis V. Lunev <den@openvz.org>
> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>


looks good but let's just patch each device separately.

> ---
>  hw/core/machine.c          | 3 +++
>  include/hw/virtio/virtio.h | 2 +-
>  2 files changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index 3e288bfceb..8bc401d8b7 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -28,6 +28,9 @@
>  #include "hw/mem/nvdimm.h"
>  
>  GlobalProperty hw_compat_4_2[] = {
> +    { "virtio-blk-device", "queue-size", "128"},
> +    { "virtio-scsi-device", "virtqueue_size", "128"},
> +    { "vhost-blk-device", "virtqueue_size", "128"},
>      { "virtio-blk-device", "x-enable-wce-if-config-wce", "off" },
>      { "virtio-blk-device", "seg-max-adjust", "off"},
>      { "virtio-scsi-device", "seg_max_adjust", "off"},
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index a66ea2368b..16d540e390 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -48,7 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features,
>  typedef struct VirtQueue VirtQueue;
>  
>  #define VIRTQUEUE_MAX_SIZE 1024
> -#define VIRTQUEUE_DEFAULT_SIZE 128
> +#define VIRTQUEUE_DEFAULT_SIZE 256
>  
>  typedef struct VirtQueueElement
>  {
> -- 
> 2.17.0

Re: [PATCH v1 3/4] tests: add virtuqueue size checking to virtio_seg_max_adjust test

[Michael S. Tsirkin]

On Wed, Jan 29, 2020 at 05:07:01PM +0300, Denis Plotnikov wrote:
> This is due to the change in the default virtqueue_size in the
> latest machine type to improve guest disks performance.
>

Sorry what is due to the change?
 
> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
> ---
>  tests/acceptance/virtio_seg_max_adjust.py | 33 ++++++++++++++---------
>  1 file changed, 21 insertions(+), 12 deletions(-)
> 
> diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
> index 5458573138..645d69b313 100755
> --- a/tests/acceptance/virtio_seg_max_adjust.py
> +++ b/tests/acceptance/virtio_seg_max_adjust.py
> @@ -27,8 +27,10 @@ from qemu.machine import QEMUMachine
>  from avocado_qemu import Test
>  
>  #list of machine types and virtqueue properties to test
> -VIRTIO_SCSI_PROPS = {'seg_max_adjust': 'seg_max_adjust'}
> -VIRTIO_BLK_PROPS = {'seg_max_adjust': 'seg-max-adjust'}
> +VIRTIO_SCSI_PROPS = {'seg_max_adjust': 'seg_max_adjust',
> +                     'queue_size': 'virtqueue_size'}
> +VIRTIO_BLK_PROPS = {'seg_max_adjust': 'seg-max-adjust',
> +                    'queue_size': 'queue-size'}
>  
>  DEV_TYPES = {'virtio-scsi-pci': VIRTIO_SCSI_PROPS,
>               'virtio-blk-pci': VIRTIO_BLK_PROPS}
> @@ -40,7 +42,7 @@ VM_DEV_PARAMS = {'virtio-scsi-pci': ['-device', 'virtio-scsi-pci,id=scsi0'],
>                                      'driver=null-co,id=drive0,if=none']}
>  
>  
> -class VirtioMaxSegSettingsCheck(Test):
> +class VirtioPramsCheck(Test):
>      @staticmethod
>      def make_pattern(props):
>          pattern_items = ['{0} = \w+'.format(prop) for prop in props]
> @@ -72,20 +74,24 @@ class VirtioMaxSegSettingsCheck(Test):
>                  props[p[0]] = p[1]
>          return query_ok, props, error
>  
> -    def check_mt(self, mt, dev_type_name):
> +    def check_mt(self, mt, expected_props, dev_type_name):
>          with QEMUMachine(self.qemu_bin) as vm:
> -            vm.set_machine(mt["name"])
> +            vm.set_machine(mt)
>              for s in VM_DEV_PARAMS[dev_type_name]:
>                  vm.add_args(s)
>              vm.launch()
>              query_ok, props, error = self.query_virtqueue(vm, dev_type_name)
>  
>          if not query_ok:
> -            self.fail('machine type {0}: {1}'.format(mt['name'], error))
> +            self.fail('machine type {0}: {1}'.format(mt, error))
>  
>          for prop_name, prop_val in props.items():
> -            expected_val = mt[prop_name]
> -            self.assertEqual(expected_val, prop_val)
> +            expected_val = expected_props[prop_name]
> +            msg = 'Property value mismatch for (MT: {0}, '\
> +                  'property name: {1}): expected value: "{2}" '\
> +                  'actual value: "{3}"'\
> +                  .format(mt, prop_name, expected_val, prop_val)
> +            self.assertEqual(expected_val, prop_val, msg)


Looks like an unrelated change, no?

>      @staticmethod
>      def seg_max_adjust_enabled(mt):
> @@ -120,15 +126,18 @@ class VirtioMaxSegSettingsCheck(Test):
>  
>          for dev_type in DEV_TYPES:
>              # create the list of machine types and their parameters.
> -            mtypes = list()
> +            mtypes = dict()
>              for m in machines:
>                  if self.seg_max_adjust_enabled(m):
>                      enabled = 'true'
> +                    queue_size = '256'
>                  else:
>                      enabled = 'false'
> -                mtypes.append({'name': m,
> -                               DEV_TYPES[dev_type]['seg_max_adjust']: enabled})
> +                    queue_size = '128'
> +                mtypes[m] = {
> +                    DEV_TYPES[dev_type]['seg_max_adjust']: enabled,
> +                    DEV_TYPES[dev_type]['queue_size']: queue_size }
>  
>              # test each machine type for a device type
>              for mt in mtypes:
> -                self.check_mt(mt, dev_type)
> +                self.check_mt(mt, mtypes[mt], dev_type)
> -- 
> 2.17.0

Re: [PATCH v1 1/4] virtio: introduce VIRTQUEUE_DEFUALT_SIZE instead of hardcoded constants

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 4314 bytes --]

On Wed, Jan 29, 2020 at 06:55:18PM +0100, Cornelia Huck wrote:
> On Wed, 29 Jan 2020 17:06:59 +0300
> Denis Plotnikov <dplotnikov@virtuozzo.com> wrote:
> 
> > Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
> > ---
> >  hw/block/virtio-blk.c      | 6 ++++--
> >  hw/scsi/virtio-scsi.c      | 5 +++--
> >  include/hw/virtio/virtio.h | 1 +
> >  3 files changed, 8 insertions(+), 4 deletions(-)
> > 
> > diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
> > index 09f46ed85f..72f935033f 100644
> > --- a/hw/block/virtio-blk.c
> > +++ b/hw/block/virtio-blk.c
> > @@ -914,7 +914,8 @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
> >      memset(&blkcfg, 0, sizeof(blkcfg));
> >      virtio_stq_p(vdev, &blkcfg.capacity, capacity);
> >      virtio_stl_p(vdev, &blkcfg.seg_max,
> > -                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 : 128 - 2);
> > +                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 :
> > +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
> >      virtio_stw_p(vdev, &blkcfg.geometry.cylinders, conf->cyls);
> >      virtio_stl_p(vdev, &blkcfg.blk_size, blk_size);
> >      virtio_stw_p(vdev, &blkcfg.min_io_size, conf->min_io_size / blk_size);
> > @@ -1272,7 +1273,8 @@ static Property virtio_blk_properties[] = {
> >      DEFINE_PROP_BIT("request-merging", VirtIOBlock, conf.request_merging, 0,
> >                      true),
> >      DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, 1),
> > -    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size, 128),
> > +    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size,
> > +                       VIRTQUEUE_DEFAULT_SIZE),
> >      DEFINE_PROP_BOOL("seg-max-adjust", VirtIOBlock, conf.seg_max_adjust, true),
> >      DEFINE_PROP_LINK("iothread", VirtIOBlock, conf.iothread, TYPE_IOTHREAD,
> >                       IOThread *),
> > diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
> > index 3b61563609..36f66046ae 100644
> > --- a/hw/scsi/virtio-scsi.c
> > +++ b/hw/scsi/virtio-scsi.c
> > @@ -660,7 +660,8 @@ static void virtio_scsi_get_config(VirtIODevice *vdev,
> >  
> >      virtio_stl_p(vdev, &scsiconf->num_queues, s->conf.num_queues);
> >      virtio_stl_p(vdev, &scsiconf->seg_max,
> > -                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 : 128 - 2);
> > +                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 :
> > +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
> >      virtio_stl_p(vdev, &scsiconf->max_sectors, s->conf.max_sectors);
> >      virtio_stl_p(vdev, &scsiconf->cmd_per_lun, s->conf.cmd_per_lun);
> >      virtio_stl_p(vdev, &scsiconf->event_info_size, sizeof(VirtIOSCSIEvent));
> > @@ -965,7 +966,7 @@ static void virtio_scsi_device_unrealize(DeviceState *dev, Error **errp)
> >  static Property virtio_scsi_properties[] = {
> >      DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, 1),
> >      DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSI,
> > -                                         parent_obj.conf.virtqueue_size, 128),
> > +                       parent_obj.conf.virtqueue_size, VIRTQUEUE_DEFAULT_SIZE),
> >      DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSI,
> >                        parent_obj.conf.seg_max_adjust, true),
> >      DEFINE_PROP_UINT32("max_sectors", VirtIOSCSI, parent_obj.conf.max_sectors,
> > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > index b69d517496..a66ea2368b 100644
> > --- a/include/hw/virtio/virtio.h
> > +++ b/include/hw/virtio/virtio.h
> > @@ -48,6 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features,
> >  typedef struct VirtQueue VirtQueue;
> >  
> >  #define VIRTQUEUE_MAX_SIZE 1024
> > +#define VIRTQUEUE_DEFAULT_SIZE 128
> 
> Going from the header only, this looks like a value that is supposed to
> be used for every virtqueue... but from the users, this is only for blk
> and scsi.
> 
> I don't think adding a default for everything makes sense, even if the
> same value makes sense for blk and scsi.

Agreed, this value is too general.  VIRTIO_BLK_VQ_DEFAULT_SIZE and
VIRTIO_SCSI_VQ_DEFAULT_SIZE would make sense to me.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1699 bytes --]

On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
> The goal is to reduce the amount of requests issued by a guest on
> 1M reads/writes. This rises the performance up to 4% on that kind of
> disk access pattern.
> 
> The maximum chunk size to be used for the guest disk accessing is
> limited with seg_max parameter, which represents the max amount of
> pices in the scatter-geather list in one guest disk request.
> 
> Since seg_max is virqueue_size dependent, increasing the virtqueue
> size increases seg_max, which, in turn, increases the maximum size
> of data to be read/write from guest disk.
> 
> More details in the original problem statment:
> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
> 
> Suggested-by: Denis V. Lunev <den@openvz.org>
> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
> ---
>  hw/core/machine.c          | 3 +++
>  include/hw/virtio/virtio.h | 2 +-
>  2 files changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index 3e288bfceb..8bc401d8b7 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -28,6 +28,9 @@
>  #include "hw/mem/nvdimm.h"
>  
>  GlobalProperty hw_compat_4_2[] = {
> +    { "virtio-blk-device", "queue-size", "128"},
> +    { "virtio-scsi-device", "virtqueue_size", "128"},
> +    { "vhost-blk-device", "virtqueue_size", "128"},

vhost-blk-device?!  Who has this?  It's not in qemu.git so please omit
this line. ;-)

On the other hand, do you want to do this for the vhost-user-blk,
vhost-user-scsi, and vhost-scsi devices that exist in qemu.git?  Those
devices would benefit from better performance too.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v1 1/4] virtio: introduce VIRTQUEUE_DEFUALT_SIZE instead of hardcoded constants

[Denis Plotnikov]

On 30.01.2020 17:56, Stefan Hajnoczi wrote:
> On Wed, Jan 29, 2020 at 06:55:18PM +0100, Cornelia Huck wrote:
>> On Wed, 29 Jan 2020 17:06:59 +0300
>> Denis Plotnikov <dplotnikov@virtuozzo.com> wrote:
>>
>>> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>> ---
>>>   hw/block/virtio-blk.c      | 6 ++++--
>>>   hw/scsi/virtio-scsi.c      | 5 +++--
>>>   include/hw/virtio/virtio.h | 1 +
>>>   3 files changed, 8 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
>>> index 09f46ed85f..72f935033f 100644
>>> --- a/hw/block/virtio-blk.c
>>> +++ b/hw/block/virtio-blk.c
>>> @@ -914,7 +914,8 @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
>>>       memset(&blkcfg, 0, sizeof(blkcfg));
>>>       virtio_stq_p(vdev, &blkcfg.capacity, capacity);
>>>       virtio_stl_p(vdev, &blkcfg.seg_max,
>>> -                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 : 128 - 2);
>>> +                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 :
>>> +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
>>>       virtio_stw_p(vdev, &blkcfg.geometry.cylinders, conf->cyls);
>>>       virtio_stl_p(vdev, &blkcfg.blk_size, blk_size);
>>>       virtio_stw_p(vdev, &blkcfg.min_io_size, conf->min_io_size / blk_size);
>>> @@ -1272,7 +1273,8 @@ static Property virtio_blk_properties[] = {
>>>       DEFINE_PROP_BIT("request-merging", VirtIOBlock, conf.request_merging, 0,
>>>                       true),
>>>       DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, 1),
>>> -    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size, 128),
>>> +    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size,
>>> +                       VIRTQUEUE_DEFAULT_SIZE),
>>>       DEFINE_PROP_BOOL("seg-max-adjust", VirtIOBlock, conf.seg_max_adjust, true),
>>>       DEFINE_PROP_LINK("iothread", VirtIOBlock, conf.iothread, TYPE_IOTHREAD,
>>>                        IOThread *),
>>> diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
>>> index 3b61563609..36f66046ae 100644
>>> --- a/hw/scsi/virtio-scsi.c
>>> +++ b/hw/scsi/virtio-scsi.c
>>> @@ -660,7 +660,8 @@ static void virtio_scsi_get_config(VirtIODevice *vdev,
>>>   
>>>       virtio_stl_p(vdev, &scsiconf->num_queues, s->conf.num_queues);
>>>       virtio_stl_p(vdev, &scsiconf->seg_max,
>>> -                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 : 128 - 2);
>>> +                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 :
>>> +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
>>>       virtio_stl_p(vdev, &scsiconf->max_sectors, s->conf.max_sectors);
>>>       virtio_stl_p(vdev, &scsiconf->cmd_per_lun, s->conf.cmd_per_lun);
>>>       virtio_stl_p(vdev, &scsiconf->event_info_size, sizeof(VirtIOSCSIEvent));
>>> @@ -965,7 +966,7 @@ static void virtio_scsi_device_unrealize(DeviceState *dev, Error **errp)
>>>   static Property virtio_scsi_properties[] = {
>>>       DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, 1),
>>>       DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSI,
>>> -                                         parent_obj.conf.virtqueue_size, 128),
>>> +                       parent_obj.conf.virtqueue_size, VIRTQUEUE_DEFAULT_SIZE),
>>>       DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSI,
>>>                         parent_obj.conf.seg_max_adjust, true),
>>>       DEFINE_PROP_UINT32("max_sectors", VirtIOSCSI, parent_obj.conf.max_sectors,
>>> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
>>> index b69d517496..a66ea2368b 100644
>>> --- a/include/hw/virtio/virtio.h
>>> +++ b/include/hw/virtio/virtio.h
>>> @@ -48,6 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features,
>>>   typedef struct VirtQueue VirtQueue;
>>>   
>>>   #define VIRTQUEUE_MAX_SIZE 1024
>>> +#define VIRTQUEUE_DEFAULT_SIZE 128
>> Going from the header only, this looks like a value that is supposed to
>> be used for every virtqueue... but from the users, this is only for blk
>> and scsi.
>>
>> I don't think adding a default for everything makes sense, even if the
>> same value makes sense for blk and scsi.
> Agreed, this value is too general.  VIRTIO_BLK_VQ_DEFAULT_SIZE and
> VIRTIO_SCSI_VQ_DEFAULT_SIZE would make sense to me.
>
> Stefan
agree, that would be better.
Will redo and resend the series.

Denis

Re: [PATCH v1 1/4] virtio: introduce VIRTQUEUE_DEFUALT_SIZE instead of hardcoded constants

[Denis Plotnikov]

On 30.01.2020 16:38, Michael S. Tsirkin wrote:
> On Wed, Jan 29, 2020 at 05:06:59PM +0300, Denis Plotnikov wrote:
>> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
>
> I'm not sure what the point is. It's more or less an accident that
> these two devices share the queue size, this constance
> makes no sense to me.
Ok, then let's just make a separate queue length constant for each type.
(Will redo and send in the next series)
Thanks!

Denis
>
>> ---
>>   hw/block/virtio-blk.c      | 6 ++++--
>>   hw/scsi/virtio-scsi.c      | 5 +++--
>>   include/hw/virtio/virtio.h | 1 +
>>   3 files changed, 8 insertions(+), 4 deletions(-)
>>
>> diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
>> index 09f46ed85f..72f935033f 100644
>> --- a/hw/block/virtio-blk.c
>> +++ b/hw/block/virtio-blk.c
>> @@ -914,7 +914,8 @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
>>       memset(&blkcfg, 0, sizeof(blkcfg));
>>       virtio_stq_p(vdev, &blkcfg.capacity, capacity);
>>       virtio_stl_p(vdev, &blkcfg.seg_max,
>> -                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 : 128 - 2);
>> +                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 :
>> +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
>>       virtio_stw_p(vdev, &blkcfg.geometry.cylinders, conf->cyls);
>>       virtio_stl_p(vdev, &blkcfg.blk_size, blk_size);
>>       virtio_stw_p(vdev, &blkcfg.min_io_size, conf->min_io_size / blk_size);
>> @@ -1272,7 +1273,8 @@ static Property virtio_blk_properties[] = {
>>       DEFINE_PROP_BIT("request-merging", VirtIOBlock, conf.request_merging, 0,
>>                       true),
>>       DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, 1),
>> -    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size, 128),
>> +    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size,
>> +                       VIRTQUEUE_DEFAULT_SIZE),
>>       DEFINE_PROP_BOOL("seg-max-adjust", VirtIOBlock, conf.seg_max_adjust, true),
>>       DEFINE_PROP_LINK("iothread", VirtIOBlock, conf.iothread, TYPE_IOTHREAD,
>>                        IOThread *),
>> diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
>> index 3b61563609..36f66046ae 100644
>> --- a/hw/scsi/virtio-scsi.c
>> +++ b/hw/scsi/virtio-scsi.c
>> @@ -660,7 +660,8 @@ static void virtio_scsi_get_config(VirtIODevice *vdev,
>>   
>>       virtio_stl_p(vdev, &scsiconf->num_queues, s->conf.num_queues);
>>       virtio_stl_p(vdev, &scsiconf->seg_max,
>> -                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 : 128 - 2);
>> +                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 :
>> +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
>>       virtio_stl_p(vdev, &scsiconf->max_sectors, s->conf.max_sectors);
>>       virtio_stl_p(vdev, &scsiconf->cmd_per_lun, s->conf.cmd_per_lun);
>>       virtio_stl_p(vdev, &scsiconf->event_info_size, sizeof(VirtIOSCSIEvent));
>> @@ -965,7 +966,7 @@ static void virtio_scsi_device_unrealize(DeviceState *dev, Error **errp)
>>   static Property virtio_scsi_properties[] = {
>>       DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, 1),
>>       DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSI,
>> -                                         parent_obj.conf.virtqueue_size, 128),
>> +                       parent_obj.conf.virtqueue_size, VIRTQUEUE_DEFAULT_SIZE),
>>       DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSI,
>>                         parent_obj.conf.seg_max_adjust, true),
>>       DEFINE_PROP_UINT32("max_sectors", VirtIOSCSI, parent_obj.conf.max_sectors,
>> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
>> index b69d517496..a66ea2368b 100644
>> --- a/include/hw/virtio/virtio.h
>> +++ b/include/hw/virtio/virtio.h
>> @@ -48,6 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features,
>>   typedef struct VirtQueue VirtQueue;
>>   
>>   #define VIRTQUEUE_MAX_SIZE 1024
>> +#define VIRTQUEUE_DEFAULT_SIZE 128
>>   
>>   typedef struct VirtQueueElement
>>   {
>> -- 
>> 2.17.0

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Denis Plotnikov]

On 30.01.2020 16:40, Michael S. Tsirkin wrote:
> On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
>> The goal is to reduce the amount of requests issued by a guest on
>> 1M reads/writes. This rises the performance up to 4% on that kind of
>> disk access pattern.
>>
>> The maximum chunk size to be used for the guest disk accessing is
>> limited with seg_max parameter, which represents the max amount of
>> pices in the scatter-geather list in one guest disk request.
>>
>> Since seg_max is virqueue_size dependent, increasing the virtqueue
>> size increases seg_max, which, in turn, increases the maximum size
>> of data to be read/write from guest disk.
>>
>> More details in the original problem statment:
>> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
>>
>> Suggested-by: Denis V. Lunev <den@openvz.org>
>> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
>
> looks good but let's just patch each device separately.
Ok
Denis
>
>> ---
>>   hw/core/machine.c          | 3 +++
>>   include/hw/virtio/virtio.h | 2 +-
>>   2 files changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/hw/core/machine.c b/hw/core/machine.c
>> index 3e288bfceb..8bc401d8b7 100644
>> --- a/hw/core/machine.c
>> +++ b/hw/core/machine.c
>> @@ -28,6 +28,9 @@
>>   #include "hw/mem/nvdimm.h"
>>   
>>   GlobalProperty hw_compat_4_2[] = {
>> +    { "virtio-blk-device", "queue-size", "128"},
>> +    { "virtio-scsi-device", "virtqueue_size", "128"},
>> +    { "vhost-blk-device", "virtqueue_size", "128"},
>>       { "virtio-blk-device", "x-enable-wce-if-config-wce", "off" },
>>       { "virtio-blk-device", "seg-max-adjust", "off"},
>>       { "virtio-scsi-device", "seg_max_adjust", "off"},
>> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
>> index a66ea2368b..16d540e390 100644
>> --- a/include/hw/virtio/virtio.h
>> +++ b/include/hw/virtio/virtio.h
>> @@ -48,7 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features,
>>   typedef struct VirtQueue VirtQueue;
>>   
>>   #define VIRTQUEUE_MAX_SIZE 1024
>> -#define VIRTQUEUE_DEFAULT_SIZE 128
>> +#define VIRTQUEUE_DEFAULT_SIZE 256
>>   
>>   typedef struct VirtQueueElement
>>   {
>> -- 
>> 2.17.0

Re: [PATCH v1 3/4] tests: add virtuqueue size checking to virtio_seg_max_adjust test

[Denis Plotnikov]

On 30.01.2020 16:42, Michael S. Tsirkin wrote:
> On Wed, Jan 29, 2020 at 05:07:01PM +0300, Denis Plotnikov wrote:
>> This is due to the change in the default virtqueue_size in the
>> latest machine type to improve guest disks performance.
>>
> Sorry what is due to the change?
>   
>> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
>> ---
>>   tests/acceptance/virtio_seg_max_adjust.py | 33 ++++++++++++++---------
>>   1 file changed, 21 insertions(+), 12 deletions(-)
>>
>> diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
>> index 5458573138..645d69b313 100755
>> --- a/tests/acceptance/virtio_seg_max_adjust.py
>> +++ b/tests/acceptance/virtio_seg_max_adjust.py
>> @@ -27,8 +27,10 @@ from qemu.machine import QEMUMachine
>>   from avocado_qemu import Test
>>   
>>   #list of machine types and virtqueue properties to test
>> -VIRTIO_SCSI_PROPS = {'seg_max_adjust': 'seg_max_adjust'}
>> -VIRTIO_BLK_PROPS = {'seg_max_adjust': 'seg-max-adjust'}
>> +VIRTIO_SCSI_PROPS = {'seg_max_adjust': 'seg_max_adjust',
>> +                     'queue_size': 'virtqueue_size'}
>> +VIRTIO_BLK_PROPS = {'seg_max_adjust': 'seg-max-adjust',
>> +                    'queue_size': 'queue-size'}
>>   
>>   DEV_TYPES = {'virtio-scsi-pci': VIRTIO_SCSI_PROPS,
>>                'virtio-blk-pci': VIRTIO_BLK_PROPS}
>> @@ -40,7 +42,7 @@ VM_DEV_PARAMS = {'virtio-scsi-pci': ['-device', 'virtio-scsi-pci,id=scsi0'],
>>                                       'driver=null-co,id=drive0,if=none']}
>>   
>>   
>> -class VirtioMaxSegSettingsCheck(Test):
>> +class VirtioPramsCheck(Test):
>>       @staticmethod
>>       def make_pattern(props):
>>           pattern_items = ['{0} = \w+'.format(prop) for prop in props]
>> @@ -72,20 +74,24 @@ class VirtioMaxSegSettingsCheck(Test):
>>                   props[p[0]] = p[1]
>>           return query_ok, props, error
>>   
>> -    def check_mt(self, mt, dev_type_name):
>> +    def check_mt(self, mt, expected_props, dev_type_name):
>>           with QEMUMachine(self.qemu_bin) as vm:
>> -            vm.set_machine(mt["name"])
>> +            vm.set_machine(mt)
>>               for s in VM_DEV_PARAMS[dev_type_name]:
>>                   vm.add_args(s)
>>               vm.launch()
>>               query_ok, props, error = self.query_virtqueue(vm, dev_type_name)
>>   
>>           if not query_ok:
>> -            self.fail('machine type {0}: {1}'.format(mt['name'], error))
>> +            self.fail('machine type {0}: {1}'.format(mt, error))
>>   
>>           for prop_name, prop_val in props.items():
>> -            expected_val = mt[prop_name]
>> -            self.assertEqual(expected_val, prop_val)
>> +            expected_val = expected_props[prop_name]
>> +            msg = 'Property value mismatch for (MT: {0}, '\
>> +                  'property name: {1}): expected value: "{2}" '\
>> +                  'actual value: "{3}"'\
>> +                  .format(mt, prop_name, expected_val, prop_val)
>> +            self.assertEqual(expected_val, prop_val, msg)
>
> Looks like an unrelated change, no?
Yep, I'd better split the patches and add the test improvements in the 
separate one.

Denis
>
>>       @staticmethod
>>       def seg_max_adjust_enabled(mt):
>> @@ -120,15 +126,18 @@ class VirtioMaxSegSettingsCheck(Test):
>>   
>>           for dev_type in DEV_TYPES:
>>               # create the list of machine types and their parameters.
>> -            mtypes = list()
>> +            mtypes = dict()
>>               for m in machines:
>>                   if self.seg_max_adjust_enabled(m):
>>                       enabled = 'true'
>> +                    queue_size = '256'
>>                   else:
>>                       enabled = 'false'
>> -                mtypes.append({'name': m,
>> -                               DEV_TYPES[dev_type]['seg_max_adjust']: enabled})
>> +                    queue_size = '128'
>> +                mtypes[m] = {
>> +                    DEV_TYPES[dev_type]['seg_max_adjust']: enabled,
>> +                    DEV_TYPES[dev_type]['queue_size']: queue_size }
>>   
>>               # test each machine type for a device type
>>               for mt in mtypes:
>> -                self.check_mt(mt, dev_type)
>> +                self.check_mt(mt, mtypes[mt], dev_type)
>> -- 
>> 2.17.0

Re: [PATCH v1 1/4] virtio: introduce VIRTQUEUE_DEFUALT_SIZE instead of hardcoded constants

[Michael S. Tsirkin]

On Mon, Feb 03, 2020 at 03:17:07PM +0300, Denis Plotnikov wrote:
> 
> 
> On 30.01.2020 16:38, Michael S. Tsirkin wrote:
> > On Wed, Jan 29, 2020 at 05:06:59PM +0300, Denis Plotnikov wrote:
> > > Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
> > 
> > I'm not sure what the point is. It's more or less an accident that
> > these two devices share the queue size, this constance
> > makes no sense to me.
> Ok, then let's just make a separate queue length constant for each type.

it's just a number, I don't think we need a constant here.
If you feel it needs documentation, add a comment!

> (Will redo and send in the next series)
> Thanks!
> 
> Denis
> > 
> > > ---
> > >   hw/block/virtio-blk.c      | 6 ++++--
> > >   hw/scsi/virtio-scsi.c      | 5 +++--
> > >   include/hw/virtio/virtio.h | 1 +
> > >   3 files changed, 8 insertions(+), 4 deletions(-)
> > > 
> > > diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
> > > index 09f46ed85f..72f935033f 100644
> > > --- a/hw/block/virtio-blk.c
> > > +++ b/hw/block/virtio-blk.c
> > > @@ -914,7 +914,8 @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
> > >       memset(&blkcfg, 0, sizeof(blkcfg));
> > >       virtio_stq_p(vdev, &blkcfg.capacity, capacity);
> > >       virtio_stl_p(vdev, &blkcfg.seg_max,
> > > -                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 : 128 - 2);
> > > +                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 :
> > > +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
> > >       virtio_stw_p(vdev, &blkcfg.geometry.cylinders, conf->cyls);
> > >       virtio_stl_p(vdev, &blkcfg.blk_size, blk_size);
> > >       virtio_stw_p(vdev, &blkcfg.min_io_size, conf->min_io_size / blk_size);
> > > @@ -1272,7 +1273,8 @@ static Property virtio_blk_properties[] = {
> > >       DEFINE_PROP_BIT("request-merging", VirtIOBlock, conf.request_merging, 0,
> > >                       true),
> > >       DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, 1),
> > > -    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size, 128),
> > > +    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size,
> > > +                       VIRTQUEUE_DEFAULT_SIZE),
> > >       DEFINE_PROP_BOOL("seg-max-adjust", VirtIOBlock, conf.seg_max_adjust, true),
> > >       DEFINE_PROP_LINK("iothread", VirtIOBlock, conf.iothread, TYPE_IOTHREAD,
> > >                        IOThread *),
> > > diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
> > > index 3b61563609..36f66046ae 100644
> > > --- a/hw/scsi/virtio-scsi.c
> > > +++ b/hw/scsi/virtio-scsi.c
> > > @@ -660,7 +660,8 @@ static void virtio_scsi_get_config(VirtIODevice *vdev,
> > >       virtio_stl_p(vdev, &scsiconf->num_queues, s->conf.num_queues);
> > >       virtio_stl_p(vdev, &scsiconf->seg_max,
> > > -                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 : 128 - 2);
> > > +                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 :
> > > +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
> > >       virtio_stl_p(vdev, &scsiconf->max_sectors, s->conf.max_sectors);
> > >       virtio_stl_p(vdev, &scsiconf->cmd_per_lun, s->conf.cmd_per_lun);
> > >       virtio_stl_p(vdev, &scsiconf->event_info_size, sizeof(VirtIOSCSIEvent));
> > > @@ -965,7 +966,7 @@ static void virtio_scsi_device_unrealize(DeviceState *dev, Error **errp)
> > >   static Property virtio_scsi_properties[] = {
> > >       DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, 1),
> > >       DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSI,
> > > -                                         parent_obj.conf.virtqueue_size, 128),
> > > +                       parent_obj.conf.virtqueue_size, VIRTQUEUE_DEFAULT_SIZE),
> > >       DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSI,
> > >                         parent_obj.conf.seg_max_adjust, true),
> > >       DEFINE_PROP_UINT32("max_sectors", VirtIOSCSI, parent_obj.conf.max_sectors,
> > > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > > index b69d517496..a66ea2368b 100644
> > > --- a/include/hw/virtio/virtio.h
> > > +++ b/include/hw/virtio/virtio.h
> > > @@ -48,6 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features,
> > >   typedef struct VirtQueue VirtQueue;
> > >   #define VIRTQUEUE_MAX_SIZE 1024
> > > +#define VIRTQUEUE_DEFAULT_SIZE 128
> > >   typedef struct VirtQueueElement
> > >   {
> > > -- 
> > > 2.17.0

Re: [PATCH v1 1/4] virtio: introduce VIRTQUEUE_DEFUALT_SIZE instead of hardcoded constants

[Denis Plotnikov]

On 03.02.2020 15:51, Michael S. Tsirkin wrote:
> On Mon, Feb 03, 2020 at 03:17:07PM +0300, Denis Plotnikov wrote:
>>
>> On 30.01.2020 16:38, Michael S. Tsirkin wrote:
>>> On Wed, Jan 29, 2020 at 05:06:59PM +0300, Denis Plotnikov wrote:
>>>> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>> I'm not sure what the point is. It's more or less an accident that
>>> these two devices share the queue size, this constance
>>> makes no sense to me.
>> Ok, then let's just make a separate queue length constant for each type.
> it's just a number, I don't think we need a constant here.
> If you feel it needs documentation, add a comment!
I just thought that the meaningful name for the number would be better 
for the code understanding.
Anyway, If doesn't improve anything I'll just change the number and add 
a comment what it means.

Denis
>
>> (Will redo and send in the next series)
>> Thanks!
>>
>> Denis
>>>> ---
>>>>    hw/block/virtio-blk.c      | 6 ++++--
>>>>    hw/scsi/virtio-scsi.c      | 5 +++--
>>>>    include/hw/virtio/virtio.h | 1 +
>>>>    3 files changed, 8 insertions(+), 4 deletions(-)
>>>>
>>>> diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
>>>> index 09f46ed85f..72f935033f 100644
>>>> --- a/hw/block/virtio-blk.c
>>>> +++ b/hw/block/virtio-blk.c
>>>> @@ -914,7 +914,8 @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
>>>>        memset(&blkcfg, 0, sizeof(blkcfg));
>>>>        virtio_stq_p(vdev, &blkcfg.capacity, capacity);
>>>>        virtio_stl_p(vdev, &blkcfg.seg_max,
>>>> -                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 : 128 - 2);
>>>> +                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 :
>>>> +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
>>>>        virtio_stw_p(vdev, &blkcfg.geometry.cylinders, conf->cyls);
>>>>        virtio_stl_p(vdev, &blkcfg.blk_size, blk_size);
>>>>        virtio_stw_p(vdev, &blkcfg.min_io_size, conf->min_io_size / blk_size);
>>>> @@ -1272,7 +1273,8 @@ static Property virtio_blk_properties[] = {
>>>>        DEFINE_PROP_BIT("request-merging", VirtIOBlock, conf.request_merging, 0,
>>>>                        true),
>>>>        DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, 1),
>>>> -    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size, 128),
>>>> +    DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size,
>>>> +                       VIRTQUEUE_DEFAULT_SIZE),
>>>>        DEFINE_PROP_BOOL("seg-max-adjust", VirtIOBlock, conf.seg_max_adjust, true),
>>>>        DEFINE_PROP_LINK("iothread", VirtIOBlock, conf.iothread, TYPE_IOTHREAD,
>>>>                         IOThread *),
>>>> diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
>>>> index 3b61563609..36f66046ae 100644
>>>> --- a/hw/scsi/virtio-scsi.c
>>>> +++ b/hw/scsi/virtio-scsi.c
>>>> @@ -660,7 +660,8 @@ static void virtio_scsi_get_config(VirtIODevice *vdev,
>>>>        virtio_stl_p(vdev, &scsiconf->num_queues, s->conf.num_queues);
>>>>        virtio_stl_p(vdev, &scsiconf->seg_max,
>>>> -                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 : 128 - 2);
>>>> +                 s->conf.seg_max_adjust ? s->conf.virtqueue_size - 2 :
>>>> +                                          VIRTQUEUE_DEFAULT_SIZE - 2);
>>>>        virtio_stl_p(vdev, &scsiconf->max_sectors, s->conf.max_sectors);
>>>>        virtio_stl_p(vdev, &scsiconf->cmd_per_lun, s->conf.cmd_per_lun);
>>>>        virtio_stl_p(vdev, &scsiconf->event_info_size, sizeof(VirtIOSCSIEvent));
>>>> @@ -965,7 +966,7 @@ static void virtio_scsi_device_unrealize(DeviceState *dev, Error **errp)
>>>>    static Property virtio_scsi_properties[] = {
>>>>        DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, 1),
>>>>        DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSI,
>>>> -                                         parent_obj.conf.virtqueue_size, 128),
>>>> +                       parent_obj.conf.virtqueue_size, VIRTQUEUE_DEFAULT_SIZE),
>>>>        DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSI,
>>>>                          parent_obj.conf.seg_max_adjust, true),
>>>>        DEFINE_PROP_UINT32("max_sectors", VirtIOSCSI, parent_obj.conf.max_sectors,
>>>> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
>>>> index b69d517496..a66ea2368b 100644
>>>> --- a/include/hw/virtio/virtio.h
>>>> +++ b/include/hw/virtio/virtio.h
>>>> @@ -48,6 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features,
>>>>    typedef struct VirtQueue VirtQueue;
>>>>    #define VIRTQUEUE_MAX_SIZE 1024
>>>> +#define VIRTQUEUE_DEFAULT_SIZE 128
>>>>    typedef struct VirtQueueElement
>>>>    {
>>>> -- 
>>>> 2.17.0

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Denis Plotnikov]

On 30.01.2020 17:58, Stefan Hajnoczi wrote:
> On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
>> The goal is to reduce the amount of requests issued by a guest on
>> 1M reads/writes. This rises the performance up to 4% on that kind of
>> disk access pattern.
>>
>> The maximum chunk size to be used for the guest disk accessing is
>> limited with seg_max parameter, which represents the max amount of
>> pices in the scatter-geather list in one guest disk request.
>>
>> Since seg_max is virqueue_size dependent, increasing the virtqueue
>> size increases seg_max, which, in turn, increases the maximum size
>> of data to be read/write from guest disk.
>>
>> More details in the original problem statment:
>> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
>>
>> Suggested-by: Denis V. Lunev <den@openvz.org>
>> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
>> ---
>>   hw/core/machine.c          | 3 +++
>>   include/hw/virtio/virtio.h | 2 +-
>>   2 files changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/hw/core/machine.c b/hw/core/machine.c
>> index 3e288bfceb..8bc401d8b7 100644
>> --- a/hw/core/machine.c
>> +++ b/hw/core/machine.c
>> @@ -28,6 +28,9 @@
>>   #include "hw/mem/nvdimm.h"
>>   
>>   GlobalProperty hw_compat_4_2[] = {
>> +    { "virtio-blk-device", "queue-size", "128"},
>> +    { "virtio-scsi-device", "virtqueue_size", "128"},
>> +    { "vhost-blk-device", "virtqueue_size", "128"},
> vhost-blk-device?!  Who has this?  It's not in qemu.git so please omit
> this line. ;-)
So in this case the line:

{ "vhost-blk-device", "seg_max_adjust", "off"},

introduced by my patch:

commit 1bf8a989a566b2ba41c197004ec2a02562a766a4
Author: Denis Plotnikov <dplotnikov@virtuozzo.com>
Date:   Fri Dec 20 17:09:04 2019 +0300

     virtio: make seg_max virtqueue size dependent

is also wrong. It should be:

{ "vhost-scsi-device", "seg_max_adjust", "off"},

Am I right?

>
> On the other hand, do you want to do this for the vhost-user-blk,
> vhost-user-scsi, and vhost-scsi devices that exist in qemu.git?  Those
> devices would benefit from better performance too.
It seems to be so. We also have the test checking those settings:
tests/acceptance/virtio_seg_max_adjust.py
For now it checks virtio-scsi-pci and virtio-blk-pci.
I'm going to extend it for the virtqueue size checking.
If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth
to check those devices too. But I don't know how to form a command line
for that 3 devices since they should involve some third party components as
backends (kernel modules, DPDK, etc.) and they seems to be not available 
in the
qemu git.
Is there any way to do it with some qit.qemu available stubs or 
something else?
If so, could you please point out the proper way to do it?

Thanks!
Denis

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 3375 bytes --]

On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote:
> 
> 
> On 30.01.2020 17:58, Stefan Hajnoczi wrote:
> > On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
> > > The goal is to reduce the amount of requests issued by a guest on
> > > 1M reads/writes. This rises the performance up to 4% on that kind of
> > > disk access pattern.
> > > 
> > > The maximum chunk size to be used for the guest disk accessing is
> > > limited with seg_max parameter, which represents the max amount of
> > > pices in the scatter-geather list in one guest disk request.
> > > 
> > > Since seg_max is virqueue_size dependent, increasing the virtqueue
> > > size increases seg_max, which, in turn, increases the maximum size
> > > of data to be read/write from guest disk.
> > > 
> > > More details in the original problem statment:
> > > https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
> > > 
> > > Suggested-by: Denis V. Lunev <den@openvz.org>
> > > Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
> > > ---
> > >   hw/core/machine.c          | 3 +++
> > >   include/hw/virtio/virtio.h | 2 +-
> > >   2 files changed, 4 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > > index 3e288bfceb..8bc401d8b7 100644
> > > --- a/hw/core/machine.c
> > > +++ b/hw/core/machine.c
> > > @@ -28,6 +28,9 @@
> > >   #include "hw/mem/nvdimm.h"
> > >   GlobalProperty hw_compat_4_2[] = {
> > > +    { "virtio-blk-device", "queue-size", "128"},
> > > +    { "virtio-scsi-device", "virtqueue_size", "128"},
> > > +    { "vhost-blk-device", "virtqueue_size", "128"},
> > vhost-blk-device?!  Who has this?  It's not in qemu.git so please omit
> > this line. ;-)
> So in this case the line:
> 
> { "vhost-blk-device", "seg_max_adjust", "off"},
> 
> introduced by my patch:
> 
> commit 1bf8a989a566b2ba41c197004ec2a02562a766a4
> Author: Denis Plotnikov <dplotnikov@virtuozzo.com>
> Date:   Fri Dec 20 17:09:04 2019 +0300
> 
>     virtio: make seg_max virtqueue size dependent
> 
> is also wrong. It should be:
> 
> { "vhost-scsi-device", "seg_max_adjust", "off"},
> 
> Am I right?

It's just called "vhost-scsi":

include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi"

> > 
> > On the other hand, do you want to do this for the vhost-user-blk,
> > vhost-user-scsi, and vhost-scsi devices that exist in qemu.git?  Those
> > devices would benefit from better performance too.
> It seems to be so. We also have the test checking those settings:
> tests/acceptance/virtio_seg_max_adjust.py
> For now it checks virtio-scsi-pci and virtio-blk-pci.
> I'm going to extend it for the virtqueue size checking.
> If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth
> to check those devices too. But I don't know how to form a command line
> for that 3 devices since they should involve some third party components as
> backends (kernel modules, DPDK, etc.) and they seems to be not available in
> the
> qemu git.
> Is there any way to do it with some qit.qemu available stubs or something
> else?
> If so, could you please point out the proper way to do it?

qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if
you need to test those vhost-user devices without external dependencies.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Denis Plotnikov]

On 05.02.2020 14:19, Stefan Hajnoczi wrote:
> On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote:
>>
>> On 30.01.2020 17:58, Stefan Hajnoczi wrote:
>>> On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
>>>> The goal is to reduce the amount of requests issued by a guest on
>>>> 1M reads/writes. This rises the performance up to 4% on that kind of
>>>> disk access pattern.
>>>>
>>>> The maximum chunk size to be used for the guest disk accessing is
>>>> limited with seg_max parameter, which represents the max amount of
>>>> pices in the scatter-geather list in one guest disk request.
>>>>
>>>> Since seg_max is virqueue_size dependent, increasing the virtqueue
>>>> size increases seg_max, which, in turn, increases the maximum size
>>>> of data to be read/write from guest disk.
>>>>
>>>> More details in the original problem statment:
>>>> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
>>>>
>>>> Suggested-by: Denis V. Lunev <den@openvz.org>
>>>> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>>> ---
>>>>    hw/core/machine.c          | 3 +++
>>>>    include/hw/virtio/virtio.h | 2 +-
>>>>    2 files changed, 4 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/hw/core/machine.c b/hw/core/machine.c
>>>> index 3e288bfceb..8bc401d8b7 100644
>>>> --- a/hw/core/machine.c
>>>> +++ b/hw/core/machine.c
>>>> @@ -28,6 +28,9 @@
>>>>    #include "hw/mem/nvdimm.h"
>>>>    GlobalProperty hw_compat_4_2[] = {
>>>> +    { "virtio-blk-device", "queue-size", "128"},
>>>> +    { "virtio-scsi-device", "virtqueue_size", "128"},
>>>> +    { "vhost-blk-device", "virtqueue_size", "128"},
>>> vhost-blk-device?!  Who has this?  It's not in qemu.git so please omit
>>> this line. ;-)
>> So in this case the line:
>>
>> { "vhost-blk-device", "seg_max_adjust", "off"},
>>
>> introduced by my patch:
>>
>> commit 1bf8a989a566b2ba41c197004ec2a02562a766a4
>> Author: Denis Plotnikov <dplotnikov@virtuozzo.com>
>> Date:   Fri Dec 20 17:09:04 2019 +0300
>>
>>      virtio: make seg_max virtqueue size dependent
>>
>> is also wrong. It should be:
>>
>> { "vhost-scsi-device", "seg_max_adjust", "off"},
>>
>> Am I right?
> It's just called "vhost-scsi":
>
> include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi"
>
>>> On the other hand, do you want to do this for the vhost-user-blk,
>>> vhost-user-scsi, and vhost-scsi devices that exist in qemu.git?  Those
>>> devices would benefit from better performance too.
After thinking about that for a while, I think we shouldn't extend queue 
sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi.
This is because increasing the queue sizes seems to be just useless for 
them: the all thing is about increasing the queue sizes for increasing 
seg_max (it limits the max block query size from the guest). For 
virtio-blk-device and virtio-scsi-device it makes sense, since they have 
seg-max-adjust property which, if true, sets seg_max to 
virtqueue_size-2. vhost-scsi also have this property but it seems the 
property just doesn't affect anything (remove it?).
Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any seg_max 
settings. If I understand correctly, their backends are ment to be 
responsible for doing that.
So, what about changing the queue sizes just for virtio-blk-device and 
virtio-scsi-device?

Denis

>> It seems to be so. We also have the test checking those settings:
>> tests/acceptance/virtio_seg_max_adjust.py
>> For now it checks virtio-scsi-pci and virtio-blk-pci.
>> I'm going to extend it for the virtqueue size checking.
>> If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth
>> to check those devices too. But I don't know how to form a command line
>> for that 3 devices since they should involve some third party components as
>> backends (kernel modules, DPDK, etc.) and they seems to be not available in
>> the
>> qemu git.
>> Is there any way to do it with some qit.qemu available stubs or something
>> else?
>> If so, could you please point out the proper way to do it?
> qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if
> you need to test those vhost-user devices without external dependencies.
>
> Stefan

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 4039 bytes --]

On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote:
> 
> 
> On 05.02.2020 14:19, Stefan Hajnoczi wrote:
> > On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote:
> > > 
> > > On 30.01.2020 17:58, Stefan Hajnoczi wrote:
> > > > On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
> > > > > The goal is to reduce the amount of requests issued by a guest on
> > > > > 1M reads/writes. This rises the performance up to 4% on that kind of
> > > > > disk access pattern.
> > > > > 
> > > > > The maximum chunk size to be used for the guest disk accessing is
> > > > > limited with seg_max parameter, which represents the max amount of
> > > > > pices in the scatter-geather list in one guest disk request.
> > > > > 
> > > > > Since seg_max is virqueue_size dependent, increasing the virtqueue
> > > > > size increases seg_max, which, in turn, increases the maximum size
> > > > > of data to be read/write from guest disk.
> > > > > 
> > > > > More details in the original problem statment:
> > > > > https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
> > > > > 
> > > > > Suggested-by: Denis V. Lunev <den@openvz.org>
> > > > > Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
> > > > > ---
> > > > >    hw/core/machine.c          | 3 +++
> > > > >    include/hw/virtio/virtio.h | 2 +-
> > > > >    2 files changed, 4 insertions(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > > > > index 3e288bfceb..8bc401d8b7 100644
> > > > > --- a/hw/core/machine.c
> > > > > +++ b/hw/core/machine.c
> > > > > @@ -28,6 +28,9 @@
> > > > >    #include "hw/mem/nvdimm.h"
> > > > >    GlobalProperty hw_compat_4_2[] = {
> > > > > +    { "virtio-blk-device", "queue-size", "128"},
> > > > > +    { "virtio-scsi-device", "virtqueue_size", "128"},
> > > > > +    { "vhost-blk-device", "virtqueue_size", "128"},
> > > > vhost-blk-device?!  Who has this?  It's not in qemu.git so please omit
> > > > this line. ;-)
> > > So in this case the line:
> > > 
> > > { "vhost-blk-device", "seg_max_adjust", "off"},
> > > 
> > > introduced by my patch:
> > > 
> > > commit 1bf8a989a566b2ba41c197004ec2a02562a766a4
> > > Author: Denis Plotnikov <dplotnikov@virtuozzo.com>
> > > Date:   Fri Dec 20 17:09:04 2019 +0300
> > > 
> > >      virtio: make seg_max virtqueue size dependent
> > > 
> > > is also wrong. It should be:
> > > 
> > > { "vhost-scsi-device", "seg_max_adjust", "off"},
> > > 
> > > Am I right?
> > It's just called "vhost-scsi":
> > 
> > include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi"
> > 
> > > > On the other hand, do you want to do this for the vhost-user-blk,
> > > > vhost-user-scsi, and vhost-scsi devices that exist in qemu.git?  Those
> > > > devices would benefit from better performance too.
> After thinking about that for a while, I think we shouldn't extend queue
> sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi.
> This is because increasing the queue sizes seems to be just useless for
> them: the all thing is about increasing the queue sizes for increasing
> seg_max (it limits the max block query size from the guest). For
> virtio-blk-device and virtio-scsi-device it makes sense, since they have
> seg-max-adjust property which, if true, sets seg_max to virtqueue_size-2.
> vhost-scsi also have this property but it seems the property just doesn't
> affect anything (remove it?).
> Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any seg_max
> settings. If I understand correctly, their backends are ment to be
> responsible for doing that.
> So, what about changing the queue sizes just for virtio-blk-device and
> virtio-scsi-device?

That's fine.  If it's beneficial to extend it to the vhost devices then
it can be done later.  I didn't look into it (and the way that
responsibility for these parameters is shared between QEMU and the
vhost-user device backend is a little complicated).

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Michael S. Tsirkin]

On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote:
> 
> 
> On 05.02.2020 14:19, Stefan Hajnoczi wrote:
> > On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote:
> > > 
> > > On 30.01.2020 17:58, Stefan Hajnoczi wrote:
> > > > On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
> > > > > The goal is to reduce the amount of requests issued by a guest on
> > > > > 1M reads/writes. This rises the performance up to 4% on that kind of
> > > > > disk access pattern.
> > > > > 
> > > > > The maximum chunk size to be used for the guest disk accessing is
> > > > > limited with seg_max parameter, which represents the max amount of
> > > > > pices in the scatter-geather list in one guest disk request.
> > > > > 
> > > > > Since seg_max is virqueue_size dependent, increasing the virtqueue
> > > > > size increases seg_max, which, in turn, increases the maximum size
> > > > > of data to be read/write from guest disk.
> > > > > 
> > > > > More details in the original problem statment:
> > > > > https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
> > > > > 
> > > > > Suggested-by: Denis V. Lunev <den@openvz.org>
> > > > > Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
> > > > > ---
> > > > >    hw/core/machine.c          | 3 +++
> > > > >    include/hw/virtio/virtio.h | 2 +-
> > > > >    2 files changed, 4 insertions(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > > > > index 3e288bfceb..8bc401d8b7 100644
> > > > > --- a/hw/core/machine.c
> > > > > +++ b/hw/core/machine.c
> > > > > @@ -28,6 +28,9 @@
> > > > >    #include "hw/mem/nvdimm.h"
> > > > >    GlobalProperty hw_compat_4_2[] = {
> > > > > +    { "virtio-blk-device", "queue-size", "128"},
> > > > > +    { "virtio-scsi-device", "virtqueue_size", "128"},
> > > > > +    { "vhost-blk-device", "virtqueue_size", "128"},
> > > > vhost-blk-device?!  Who has this?  It's not in qemu.git so please omit
> > > > this line. ;-)
> > > So in this case the line:
> > > 
> > > { "vhost-blk-device", "seg_max_adjust", "off"},
> > > 
> > > introduced by my patch:
> > > 
> > > commit 1bf8a989a566b2ba41c197004ec2a02562a766a4
> > > Author: Denis Plotnikov <dplotnikov@virtuozzo.com>
> > > Date:   Fri Dec 20 17:09:04 2019 +0300
> > > 
> > >      virtio: make seg_max virtqueue size dependent
> > > 
> > > is also wrong. It should be:
> > > 
> > > { "vhost-scsi-device", "seg_max_adjust", "off"},
> > > 
> > > Am I right?
> > It's just called "vhost-scsi":
> > 
> > include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi"
> > 
> > > > On the other hand, do you want to do this for the vhost-user-blk,
> > > > vhost-user-scsi, and vhost-scsi devices that exist in qemu.git?  Those
> > > > devices would benefit from better performance too.
> After thinking about that for a while, I think we shouldn't extend queue
> sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi.
> This is because increasing the queue sizes seems to be just useless for
> them: the all thing is about increasing the queue sizes for increasing
> seg_max (it limits the max block query size from the guest). For
> virtio-blk-device and virtio-scsi-device it makes sense, since they have
> seg-max-adjust property which, if true, sets seg_max to virtqueue_size-2.
> vhost-scsi also have this property but it seems the property just doesn't
> affect anything (remove it?).
> Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any seg_max
> settings. If I understand correctly, their backends are ment to be
> responsible for doing that.

The queue size is set by qemu IIRC.

> So, what about changing the queue sizes just for virtio-blk-device and
> virtio-scsi-device?


Hmm that would break ability to migrate between userspace and vhost
backends, would it not?


> Denis
> 
> > > It seems to be so. We also have the test checking those settings:
> > > tests/acceptance/virtio_seg_max_adjust.py
> > > For now it checks virtio-scsi-pci and virtio-blk-pci.
> > > I'm going to extend it for the virtqueue size checking.
> > > If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth
> > > to check those devices too. But I don't know how to form a command line
> > > for that 3 devices since they should involve some third party components as
> > > backends (kernel modules, DPDK, etc.) and they seems to be not available in
> > > the
> > > qemu git.
> > > Is there any way to do it with some qit.qemu available stubs or something
> > > else?
> > > If so, could you please point out the proper way to do it?
> > qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if
> > you need to test those vhost-user devices without external dependencies.
> > 
> > Stefan

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Denis Plotnikov]

On 09.02.2020 10:49, Michael S. Tsirkin wrote:
> On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote:
>>
>> On 05.02.2020 14:19, Stefan Hajnoczi wrote:
>>> On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote:
>>>> On 30.01.2020 17:58, Stefan Hajnoczi wrote:
>>>>> On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
>>>>>> The goal is to reduce the amount of requests issued by a guest on
>>>>>> 1M reads/writes. This rises the performance up to 4% on that kind of
>>>>>> disk access pattern.
>>>>>>
>>>>>> The maximum chunk size to be used for the guest disk accessing is
>>>>>> limited with seg_max parameter, which represents the max amount of
>>>>>> pices in the scatter-geather list in one guest disk request.
>>>>>>
>>>>>> Since seg_max is virqueue_size dependent, increasing the virtqueue
>>>>>> size increases seg_max, which, in turn, increases the maximum size
>>>>>> of data to be read/write from guest disk.
>>>>>>
>>>>>> More details in the original problem statment:
>>>>>> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
>>>>>>
>>>>>> Suggested-by: Denis V. Lunev <den@openvz.org>
>>>>>> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>>>>> ---
>>>>>>     hw/core/machine.c          | 3 +++
>>>>>>     include/hw/virtio/virtio.h | 2 +-
>>>>>>     2 files changed, 4 insertions(+), 1 deletion(-)
>>>>>>
>>>>>> diff --git a/hw/core/machine.c b/hw/core/machine.c
>>>>>> index 3e288bfceb..8bc401d8b7 100644
>>>>>> --- a/hw/core/machine.c
>>>>>> +++ b/hw/core/machine.c
>>>>>> @@ -28,6 +28,9 @@
>>>>>>     #include "hw/mem/nvdimm.h"
>>>>>>     GlobalProperty hw_compat_4_2[] = {
>>>>>> +    { "virtio-blk-device", "queue-size", "128"},
>>>>>> +    { "virtio-scsi-device", "virtqueue_size", "128"},
>>>>>> +    { "vhost-blk-device", "virtqueue_size", "128"},
>>>>> vhost-blk-device?!  Who has this?  It's not in qemu.git so please omit
>>>>> this line. ;-)
>>>> So in this case the line:
>>>>
>>>> { "vhost-blk-device", "seg_max_adjust", "off"},
>>>>
>>>> introduced by my patch:
>>>>
>>>> commit 1bf8a989a566b2ba41c197004ec2a02562a766a4
>>>> Author: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>>> Date:   Fri Dec 20 17:09:04 2019 +0300
>>>>
>>>>       virtio: make seg_max virtqueue size dependent
>>>>
>>>> is also wrong. It should be:
>>>>
>>>> { "vhost-scsi-device", "seg_max_adjust", "off"},
>>>>
>>>> Am I right?
>>> It's just called "vhost-scsi":
>>>
>>> include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi"
>>>
>>>>> On the other hand, do you want to do this for the vhost-user-blk,
>>>>> vhost-user-scsi, and vhost-scsi devices that exist in qemu.git?  Those
>>>>> devices would benefit from better performance too.
>> After thinking about that for a while, I think we shouldn't extend queue
>> sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi.
>> This is because increasing the queue sizes seems to be just useless for
>> them: the all thing is about increasing the queue sizes for increasing
>> seg_max (it limits the max block query size from the guest). For
>> virtio-blk-device and virtio-scsi-device it makes sense, since they have
>> seg-max-adjust property which, if true, sets seg_max to virtqueue_size-2.
>> vhost-scsi also have this property but it seems the property just doesn't
>> affect anything (remove it?).
>> Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any seg_max
>> settings. If I understand correctly, their backends are ment to be
>> responsible for doing that.
> The queue size is set by qemu IIRC.
>
>> So, what about changing the queue sizes just for virtio-blk-device and
>> virtio-scsi-device?
>
> Hmm that would break ability to migrate between userspace and vhost
> backends, would it not?
I'm not sure I've understood what you meant.
Just for the record, I was going to change virtqueue-size for 
virtio-blk-device and virtio-scsi-device since they can adjust seg_max 
to the specified queue size and I don't want to touch vhost-s and 
vhost-user-s since they don't have adjustable seg_max for now.

Denis
>
>
>> Denis
>>
>>>> It seems to be so. We also have the test checking those settings:
>>>> tests/acceptance/virtio_seg_max_adjust.py
>>>> For now it checks virtio-scsi-pci and virtio-blk-pci.
>>>> I'm going to extend it for the virtqueue size checking.
>>>> If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth
>>>> to check those devices too. But I don't know how to form a command line
>>>> for that 3 devices since they should involve some third party components as
>>>> backends (kernel modules, DPDK, etc.) and they seems to be not available in
>>>> the
>>>> qemu git.
>>>> Is there any way to do it with some qit.qemu available stubs or something
>>>> else?
>>>> If so, could you please point out the proper way to do it?
>>> qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if
>>> you need to test those vhost-user devices without external dependencies.
>>>
>>> Stefan

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 718 bytes --]

On Sun, Feb 09, 2020 at 02:49:09AM -0500, Michael S. Tsirkin wrote:
> On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote:
> > On 05.02.2020 14:19, Stefan Hajnoczi wrote:
> > > On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote:
> > > > On 30.01.2020 17:58, Stefan Hajnoczi wrote:
> > > > > On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
> > So, what about changing the queue sizes just for virtio-blk-device and
> > virtio-scsi-device?
> 
> Hmm that would break ability to migrate between userspace and vhost
> backends, would it not?

I think live and offline migration between them would still work as long
as the queue size is manually specified.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Michael S. Tsirkin]

On Mon, Feb 10, 2020 at 06:34:15PM +0300, Denis Plotnikov wrote:
> 
> 
> On 09.02.2020 10:49, Michael S. Tsirkin wrote:
> > On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote:
> > > 
> > > On 05.02.2020 14:19, Stefan Hajnoczi wrote:
> > > > On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote:
> > > > > On 30.01.2020 17:58, Stefan Hajnoczi wrote:
> > > > > > On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
> > > > > > > The goal is to reduce the amount of requests issued by a guest on
> > > > > > > 1M reads/writes. This rises the performance up to 4% on that kind of
> > > > > > > disk access pattern.
> > > > > > > 
> > > > > > > The maximum chunk size to be used for the guest disk accessing is
> > > > > > > limited with seg_max parameter, which represents the max amount of
> > > > > > > pices in the scatter-geather list in one guest disk request.
> > > > > > > 
> > > > > > > Since seg_max is virqueue_size dependent, increasing the virtqueue
> > > > > > > size increases seg_max, which, in turn, increases the maximum size
> > > > > > > of data to be read/write from guest disk.
> > > > > > > 
> > > > > > > More details in the original problem statment:
> > > > > > > https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
> > > > > > > 
> > > > > > > Suggested-by: Denis V. Lunev <den@openvz.org>
> > > > > > > Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
> > > > > > > ---
> > > > > > >     hw/core/machine.c          | 3 +++
> > > > > > >     include/hw/virtio/virtio.h | 2 +-
> > > > > > >     2 files changed, 4 insertions(+), 1 deletion(-)
> > > > > > > 
> > > > > > > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > > > > > > index 3e288bfceb..8bc401d8b7 100644
> > > > > > > --- a/hw/core/machine.c
> > > > > > > +++ b/hw/core/machine.c
> > > > > > > @@ -28,6 +28,9 @@
> > > > > > >     #include "hw/mem/nvdimm.h"
> > > > > > >     GlobalProperty hw_compat_4_2[] = {
> > > > > > > +    { "virtio-blk-device", "queue-size", "128"},
> > > > > > > +    { "virtio-scsi-device", "virtqueue_size", "128"},
> > > > > > > +    { "vhost-blk-device", "virtqueue_size", "128"},
> > > > > > vhost-blk-device?!  Who has this?  It's not in qemu.git so please omit
> > > > > > this line. ;-)
> > > > > So in this case the line:
> > > > > 
> > > > > { "vhost-blk-device", "seg_max_adjust", "off"},
> > > > > 
> > > > > introduced by my patch:
> > > > > 
> > > > > commit 1bf8a989a566b2ba41c197004ec2a02562a766a4
> > > > > Author: Denis Plotnikov <dplotnikov@virtuozzo.com>
> > > > > Date:   Fri Dec 20 17:09:04 2019 +0300
> > > > > 
> > > > >       virtio: make seg_max virtqueue size dependent
> > > > > 
> > > > > is also wrong. It should be:
> > > > > 
> > > > > { "vhost-scsi-device", "seg_max_adjust", "off"},
> > > > > 
> > > > > Am I right?
> > > > It's just called "vhost-scsi":
> > > > 
> > > > include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi"
> > > > 
> > > > > > On the other hand, do you want to do this for the vhost-user-blk,
> > > > > > vhost-user-scsi, and vhost-scsi devices that exist in qemu.git?  Those
> > > > > > devices would benefit from better performance too.
> > > After thinking about that for a while, I think we shouldn't extend queue
> > > sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi.
> > > This is because increasing the queue sizes seems to be just useless for
> > > them: the all thing is about increasing the queue sizes for increasing
> > > seg_max (it limits the max block query size from the guest). For
> > > virtio-blk-device and virtio-scsi-device it makes sense, since they have
> > > seg-max-adjust property which, if true, sets seg_max to virtqueue_size-2.
> > > vhost-scsi also have this property but it seems the property just doesn't
> > > affect anything (remove it?).
> > > Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any seg_max
> > > settings. If I understand correctly, their backends are ment to be
> > > responsible for doing that.
> > The queue size is set by qemu IIRC.
> > 
> > > So, what about changing the queue sizes just for virtio-blk-device and
> > > virtio-scsi-device?
> > 
> > Hmm that would break ability to migrate between userspace and vhost
> > backends, would it not?
> I'm not sure I've understood what you meant.
> Just for the record, I was going to change virtqueue-size for
> virtio-blk-device and virtio-scsi-device



If virtqueue size is different between virtio-blk-device and
vhost-user-blk then one can not migrate the former to the later.

> since they can adjust seg_max to
> the specified queue size and I don't want to touch vhost-s and vhost-user-s
> since they don't have adjustable seg_max for now.
> 
> Denis



If I just grep seg_max_adjust I see them for vhost devices too.
Is that code ineffective somehow? What's it doing then?

> > 
> > 
> > > Denis
> > > 
> > > > > It seems to be so. We also have the test checking those settings:
> > > > > tests/acceptance/virtio_seg_max_adjust.py
> > > > > For now it checks virtio-scsi-pci and virtio-blk-pci.
> > > > > I'm going to extend it for the virtqueue size checking.
> > > > > If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth
> > > > > to check those devices too. But I don't know how to form a command line
> > > > > for that 3 devices since they should involve some third party components as
> > > > > backends (kernel modules, DPDK, etc.) and they seems to be not available in
> > > > > the
> > > > > qemu git.
> > > > > Is there any way to do it with some qit.qemu available stubs or something
> > > > > else?
> > > > > If so, could you please point out the proper way to do it?
> > > > qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if
> > > > you need to test those vhost-user devices without external dependencies.
> > > > 
> > > > Stefan

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Stefano Garzarella]

Hi Denis,
I just found this discussion since we still have the following line in 
hw/core/machine.c:
     { "vhost-blk-device", "seg_max_adjust", "off"}

IIUC it was a typo, and I think we should fix it since in the future we 
can have `vhost-blk-device`.

So, I think we have 2 options:
1. remove that line since for now is useless
2. replace with "vhost-scsi"

I'm not sure which is the best, what do you suggest?

Thanks,
Stefano

On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote:
>On 05.02.2020 14:19, Stefan Hajnoczi wrote:
>>On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote:
>>>On 30.01.2020 17:58, Stefan Hajnoczi wrote:
>>>>On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
>>>>>The goal is to reduce the amount of requests issued by a guest on
>>>>>1M reads/writes. This rises the performance up to 4% on that kind of
>>>>>disk access pattern.
>>>>>
>>>>>The maximum chunk size to be used for the guest disk accessing is
>>>>>limited with seg_max parameter, which represents the max amount of
>>>>>pices in the scatter-geather list in one guest disk request.
>>>>>
>>>>>Since seg_max is virqueue_size dependent, increasing the virtqueue
>>>>>size increases seg_max, which, in turn, increases the maximum size
>>>>>of data to be read/write from guest disk.
>>>>>
>>>>>More details in the original problem statment:
>>>>>https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
>>>>>
>>>>>Suggested-by: Denis V. Lunev <den@openvz.org>
>>>>>Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>>>>---
>>>>>   hw/core/machine.c          | 3 +++
>>>>>   include/hw/virtio/virtio.h | 2 +-
>>>>>   2 files changed, 4 insertions(+), 1 deletion(-)
>>>>>
>>>>>diff --git a/hw/core/machine.c b/hw/core/machine.c
>>>>>index 3e288bfceb..8bc401d8b7 100644
>>>>>--- a/hw/core/machine.c
>>>>>+++ b/hw/core/machine.c
>>>>>@@ -28,6 +28,9 @@
>>>>>   #include "hw/mem/nvdimm.h"
>>>>>   GlobalProperty hw_compat_4_2[] = {
>>>>>+    { "virtio-blk-device", "queue-size", "128"},
>>>>>+    { "virtio-scsi-device", "virtqueue_size", "128"},
>>>>>+    { "vhost-blk-device", "virtqueue_size", "128"},
>>>>vhost-blk-device?!  Who has this?  It's not in qemu.git so please omit
>>>>this line. ;-)
>>>So in this case the line:
>>>
>>>{ "vhost-blk-device", "seg_max_adjust", "off"},
>>>
>>>introduced by my patch:
>>>
>>>commit 1bf8a989a566b2ba41c197004ec2a02562a766a4
>>>Author: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>>Date:   Fri Dec 20 17:09:04 2019 +0300
>>>
>>>     virtio: make seg_max virtqueue size dependent
>>>
>>>is also wrong. It should be:
>>>
>>>{ "vhost-scsi-device", "seg_max_adjust", "off"},
>>>
>>>Am I right?
>>It's just called "vhost-scsi":
>>
>>include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi"
>>
>>>>On the other hand, do you want to do this for the vhost-user-blk,
>>>>vhost-user-scsi, and vhost-scsi devices that exist in qemu.git?  Those
>>>>devices would benefit from better performance too.
>After thinking about that for a while, I think we shouldn't extend 
>queue sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi.
>This is because increasing the queue sizes seems to be just useless 
>for them: the all thing is about increasing the queue sizes for 
>increasing seg_max (it limits the max block query size from the 
>guest). For virtio-blk-device and virtio-scsi-device it makes sense, 
>since they have seg-max-adjust property which, if true, sets seg_max 
>to virtqueue_size-2. vhost-scsi also have this property but it seems 
>the property just doesn't affect anything (remove it?).
>Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any 
>seg_max settings. If I understand correctly, their backends are ment 
>to be responsible for doing that.
>So, what about changing the queue sizes just for virtio-blk-device and 
>virtio-scsi-device?
>
>Denis
>
>>>It seems to be so. We also have the test checking those settings:
>>>tests/acceptance/virtio_seg_max_adjust.py
>>>For now it checks virtio-scsi-pci and virtio-blk-pci.
>>>I'm going to extend it for the virtqueue size checking.
>>>If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth
>>>to check those devices too. But I don't know how to form a command line
>>>for that 3 devices since they should involve some third party components as
>>>backends (kernel modules, DPDK, etc.) and they seems to be not available in
>>>the
>>>qemu git.
>>>Is there any way to do it with some qit.qemu available stubs or something
>>>else?
>>>If so, could you please point out the proper way to do it?
>>qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if
>>you need to test those vhost-user devices without external dependencies.
>>
>>Stefan
>
>

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Stefano Garzarella]

Message bounced, I use new Denis's email address.

On Wed, Sep 08, 2021 at 03:17:16PM +0200, Stefano Garzarella wrote:
>Hi Denis,
>I just found this discussion since we still have the following line in 
>hw/core/machine.c:
>    { "vhost-blk-device", "seg_max_adjust", "off"}
>
>IIUC it was a typo, and I think we should fix it since in the future 
>we can have `vhost-blk-device`.
>
>So, I think we have 2 options:
>1. remove that line since for now is useless
>2. replace with "vhost-scsi"
>
>I'm not sure which is the best, what do you suggest?
>
>Thanks,
>Stefano
>
>On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote:
>>On 05.02.2020 14:19, Stefan Hajnoczi wrote:
>>>On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote:
>>>>On 30.01.2020 17:58, Stefan Hajnoczi wrote:
>>>>>On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
>>>>>>The goal is to reduce the amount of requests issued by a guest on
>>>>>>1M reads/writes. This rises the performance up to 4% on that kind of
>>>>>>disk access pattern.
>>>>>>
>>>>>>The maximum chunk size to be used for the guest disk accessing is
>>>>>>limited with seg_max parameter, which represents the max amount of
>>>>>>pices in the scatter-geather list in one guest disk request.
>>>>>>
>>>>>>Since seg_max is virqueue_size dependent, increasing the virtqueue
>>>>>>size increases seg_max, which, in turn, increases the maximum size
>>>>>>of data to be read/write from guest disk.
>>>>>>
>>>>>>More details in the original problem statment:
>>>>>>https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
>>>>>>
>>>>>>Suggested-by: Denis V. Lunev <den@openvz.org>
>>>>>>Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>>>>>---
>>>>>>  hw/core/machine.c          | 3 +++
>>>>>>  include/hw/virtio/virtio.h | 2 +-
>>>>>>  2 files changed, 4 insertions(+), 1 deletion(-)
>>>>>>
>>>>>>diff --git a/hw/core/machine.c b/hw/core/machine.c
>>>>>>index 3e288bfceb..8bc401d8b7 100644
>>>>>>--- a/hw/core/machine.c
>>>>>>+++ b/hw/core/machine.c
>>>>>>@@ -28,6 +28,9 @@
>>>>>>  #include "hw/mem/nvdimm.h"
>>>>>>  GlobalProperty hw_compat_4_2[] = {
>>>>>>+    { "virtio-blk-device", "queue-size", "128"},
>>>>>>+    { "virtio-scsi-device", "virtqueue_size", "128"},
>>>>>>+    { "vhost-blk-device", "virtqueue_size", "128"},
>>>>>vhost-blk-device?!  Who has this?  It's not in qemu.git so please omit
>>>>>this line. ;-)
>>>>So in this case the line:
>>>>
>>>>{ "vhost-blk-device", "seg_max_adjust", "off"},
>>>>
>>>>introduced by my patch:
>>>>
>>>>commit 1bf8a989a566b2ba41c197004ec2a02562a766a4
>>>>Author: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>>>Date:   Fri Dec 20 17:09:04 2019 +0300
>>>>
>>>>    virtio: make seg_max virtqueue size dependent
>>>>
>>>>is also wrong. It should be:
>>>>
>>>>{ "vhost-scsi-device", "seg_max_adjust", "off"},
>>>>
>>>>Am I right?
>>>It's just called "vhost-scsi":
>>>
>>>include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi"
>>>
>>>>>On the other hand, do you want to do this for the vhost-user-blk,
>>>>>vhost-user-scsi, and vhost-scsi devices that exist in qemu.git?  Those
>>>>>devices would benefit from better performance too.
>>After thinking about that for a while, I think we shouldn't extend 
>>queue sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi.
>>This is because increasing the queue sizes seems to be just useless 
>>for them: the all thing is about increasing the queue sizes for 
>>increasing seg_max (it limits the max block query size from the 
>>guest). For virtio-blk-device and virtio-scsi-device it makes sense, 
>>since they have seg-max-adjust property which, if true, sets seg_max 
>>to virtqueue_size-2. vhost-scsi also have this property but it seems 
>>the property just doesn't affect anything (remove it?).
>>Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any 
>>seg_max settings. If I understand correctly, their backends are ment 
>>to be responsible for doing that.
>>So, what about changing the queue sizes just for virtio-blk-device 
>>and virtio-scsi-device?
>>
>>Denis
>>
>>>>It seems to be so. We also have the test checking those settings:
>>>>tests/acceptance/virtio_seg_max_adjust.py
>>>>For now it checks virtio-scsi-pci and virtio-blk-pci.
>>>>I'm going to extend it for the virtqueue size checking.
>>>>If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth
>>>>to check those devices too. But I don't know how to form a command line
>>>>for that 3 devices since they should involve some third party components as
>>>>backends (kernel modules, DPDK, etc.) and they seems to be not available in
>>>>the
>>>>qemu git.
>>>>Is there any way to do it with some qit.qemu available stubs or something
>>>>else?
>>>>If so, could you please point out the proper way to do it?
>>>qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if
>>>you need to test those vhost-user devices without external dependencies.
>>>
>>>Stefan
>>
>>

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Denis Plotnikov]

On 08.09.2021 16:22, Stefano Garzarella wrote:
> Message bounced, I use new Denis's email address.
>
> On Wed, Sep 08, 2021 at 03:17:16PM +0200, Stefano Garzarella wrote:
>> Hi Denis,
>> I just found this discussion since we still have the following line 
>> in hw/core/machine.c:
>>    { "vhost-blk-device", "seg_max_adjust", "off"}
>>
>> IIUC it was a typo, and I think we should fix it since in the future 
>> we can have `vhost-blk-device`.
>>
>> So, I think we have 2 options:
>> 1. remove that line since for now is useless
>> 2. replace with "vhost-scsi"
>>
>> I'm not sure which is the best, what do you suggest?
>>
>> Thanks,
>> Stefano

Hi Stefano

I prefer to just remove the line without replacing. This will keep 
things exactly like it is now.

Replacing with "vhost-scsi" will affect seg_max (limit to 126) on newly 
created VMs with machine types using "hw_compat_4_2" and older.

Now, because of the typo (error), their seg-max is queue-size dependent. 
I'm not sure, if replacing the line may cause any problems, for example 
on migration: source (queue-size 256, seg max 254) -> destination 
(queue-size 256, seg max 126). But it will definitely introduce two 
different behaviors for VMs with "hw_compat_4_2" and older. So, I'd 
choose the lesser of two evils and keep the things like it's now.

Thanks!

Denis

>>
>> On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote:
>>> On 05.02.2020 14:19, Stefan Hajnoczi wrote:
>>>> On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote:
>>>>> On 30.01.2020 17:58, Stefan Hajnoczi wrote:
>>>>>> On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
>>>>>>> The goal is to reduce the amount of requests issued by a guest on
>>>>>>> 1M reads/writes. This rises the performance up to 4% on that 
>>>>>>> kind of
>>>>>>> disk access pattern.
>>>>>>>
>>>>>>> The maximum chunk size to be used for the guest disk accessing is
>>>>>>> limited with seg_max parameter, which represents the max amount of
>>>>>>> pices in the scatter-geather list in one guest disk request.
>>>>>>>
>>>>>>> Since seg_max is virqueue_size dependent, increasing the virtqueue
>>>>>>> size increases seg_max, which, in turn, increases the maximum size
>>>>>>> of data to be read/write from guest disk.
>>>>>>>
>>>>>>> More details in the original problem statment:
>>>>>>> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
>>>>>>>
>>>>>>> Suggested-by: Denis V. Lunev <den@openvz.org>
>>>>>>> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>>>>>> ---
>>>>>>>  hw/core/machine.c          | 3 +++
>>>>>>>  include/hw/virtio/virtio.h | 2 +-
>>>>>>>  2 files changed, 4 insertions(+), 1 deletion(-)
>>>>>>>
>>>>>>> diff --git a/hw/core/machine.c b/hw/core/machine.c
>>>>>>> index 3e288bfceb..8bc401d8b7 100644
>>>>>>> --- a/hw/core/machine.c
>>>>>>> +++ b/hw/core/machine.c
>>>>>>> @@ -28,6 +28,9 @@
>>>>>>>  #include "hw/mem/nvdimm.h"
>>>>>>>  GlobalProperty hw_compat_4_2[] = {
>>>>>>> +    { "virtio-blk-device", "queue-size", "128"},
>>>>>>> +    { "virtio-scsi-device", "virtqueue_size", "128"},
>>>>>>> +    { "vhost-blk-device", "virtqueue_size", "128"},
>>>>>> vhost-blk-device?!  Who has this?  It's not in qemu.git so please 
>>>>>> omit
>>>>>> this line. ;-)
>>>>> So in this case the line:
>>>>>
>>>>> { "vhost-blk-device", "seg_max_adjust", "off"},
>>>>>
>>>>> introduced by my patch:
>>>>>
>>>>> commit 1bf8a989a566b2ba41c197004ec2a02562a766a4
>>>>> Author: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>>>> Date:   Fri Dec 20 17:09:04 2019 +0300
>>>>>
>>>>>     virtio: make seg_max virtqueue size dependent
>>>>>
>>>>> is also wrong. It should be:
>>>>>
>>>>> { "vhost-scsi-device", "seg_max_adjust", "off"},
>>>>>
>>>>> Am I right?
>>>> It's just called "vhost-scsi":
>>>>
>>>> include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi"
>>>>
>>>>>> On the other hand, do you want to do this for the vhost-user-blk,
>>>>>> vhost-user-scsi, and vhost-scsi devices that exist in qemu.git?  
>>>>>> Those
>>>>>> devices would benefit from better performance too.
>>> After thinking about that for a while, I think we shouldn't extend 
>>> queue sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi.
>>> This is because increasing the queue sizes seems to be just useless 
>>> for them: the all thing is about increasing the queue sizes for 
>>> increasing seg_max (it limits the max block query size from the 
>>> guest). For virtio-blk-device and virtio-scsi-device it makes sense, 
>>> since they have seg-max-adjust property which, if true, sets seg_max 
>>> to virtqueue_size-2. vhost-scsi also have this property but it seems 
>>> the property just doesn't affect anything (remove it?).
>>> Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any 
>>> seg_max settings. If I understand correctly, their backends are ment 
>>> to be responsible for doing that.
>>> So, what about changing the queue sizes just for virtio-blk-device 
>>> and virtio-scsi-device?
>>>
>>> Denis
>>>
>>>>> It seems to be so. We also have the test checking those settings:
>>>>> tests/acceptance/virtio_seg_max_adjust.py
>>>>> For now it checks virtio-scsi-pci and virtio-blk-pci.
>>>>> I'm going to extend it for the virtqueue size checking.
>>>>> If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth
>>>>> to check those devices too. But I don't know how to form a command 
>>>>> line
>>>>> for that 3 devices since they should involve some third party 
>>>>> components as
>>>>> backends (kernel modules, DPDK, etc.) and they seems to be not 
>>>>> available in
>>>>> the
>>>>> qemu git.
>>>>> Is there any way to do it with some qit.qemu available stubs or 
>>>>> something
>>>>> else?
>>>>> If so, could you please point out the proper way to do it?
>>>> qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if
>>>> you need to test those vhost-user devices without external 
>>>> dependencies.
>>>>
>>>> Stefan
>>>
>>>
>

Re: [PATCH v1 4/4] tests: rename virtio_seg_max_adjust to virtio_check_params

[Philippe Mathieu-Daudé]

On 1/29/20 3:07 PM, Denis Plotnikov wrote:
> Since, virtio_seg_max_adjust checks not only seg_max, but also
> virtqueue_size parameter, let's make the test more general and
> add new parameters to be checked there in the future.
> 
> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
> ---
>  .../{virtio_seg_max_adjust.py => virtio_check_params.py}          | 0
>  1 file changed, 0 insertions(+), 0 deletions(-)
>  rename tests/acceptance/{virtio_seg_max_adjust.py => virtio_check_params.py} (100%)

Old one... reminds me of
https://lore.kernel.org/qemu-devel/20200129212345.20547-1-philmd@redhat.com/
:~(

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Stefano Garzarella]

On Wed, Sep 08, 2021 at 06:20:49PM +0300, Denis Plotnikov wrote:
>
>On 08.09.2021 16:22, Stefano Garzarella wrote:
>>Message bounced, I use new Denis's email address.
>>
>>On Wed, Sep 08, 2021 at 03:17:16PM +0200, Stefano Garzarella wrote:
>>>Hi Denis,
>>>I just found this discussion since we still have the following 
>>>line in hw/core/machine.c:
>>>   { "vhost-blk-device", "seg_max_adjust", "off"}
>>>
>>>IIUC it was a typo, and I think we should fix it since in the 
>>>future we can have `vhost-blk-device`.
>>>
>>>So, I think we have 2 options:
>>>1. remove that line since for now is useless
>>>2. replace with "vhost-scsi"
>>>
>>>I'm not sure which is the best, what do you suggest?
>>>
>>>Thanks,
>>>Stefano
>
>Hi Stefano
>
>I prefer to just remove the line without replacing. This will keep 
>things exactly like it is now.
>
>Replacing with "vhost-scsi" will affect seg_max (limit to 126) on 
>newly created VMs with machine types using "hw_compat_4_2" and older.
>
>Now, because of the typo (error), their seg-max is queue-size 
>dependent. I'm not sure, if replacing the line may cause any problems, 
>for example on migration: source (queue-size 256, seg max 254) -> 
>destination (queue-size 256, seg max 126). But it will definitely 
>introduce two different behaviors for VMs with "hw_compat_4_2" and 
>older. So, I'd choose the lesser of two evils and keep the things like 
>it's now.
>

Yep, make sense. It was the same concern I had.

Do you want to send a patch to remove it with this explanation?

Thanks for the clarification,
Stefano

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Denis Plotnikov]

On 09.09.2021 11:28, Stefano Garzarella wrote:
> On Wed, Sep 08, 2021 at 06:20:49PM +0300, Denis Plotnikov wrote:
>>
>> On 08.09.2021 16:22, Stefano Garzarella wrote:
>>> Message bounced, I use new Denis's email address.
>>>
>>> On Wed, Sep 08, 2021 at 03:17:16PM +0200, Stefano Garzarella wrote:
>>>> Hi Denis,
>>>> I just found this discussion since we still have the following line 
>>>> in hw/core/machine.c:
>>>>    { "vhost-blk-device", "seg_max_adjust", "off"}
>>>>
>>>> IIUC it was a typo, and I think we should fix it since in the 
>>>> future we can have `vhost-blk-device`.
>>>>
>>>> So, I think we have 2 options:
>>>> 1. remove that line since for now is useless
>>>> 2. replace with "vhost-scsi"
>>>>
>>>> I'm not sure which is the best, what do you suggest?
>>>>
>>>> Thanks,
>>>> Stefano
>>
>> Hi Stefano
>>
>> I prefer to just remove the line without replacing. This will keep 
>> things exactly like it is now.
>>
>> Replacing with "vhost-scsi" will affect seg_max (limit to 126) on 
>> newly created VMs with machine types using "hw_compat_4_2" and older.
>>
>> Now, because of the typo (error), their seg-max is queue-size 
>> dependent. I'm not sure, if replacing the line may cause any 
>> problems, for example on migration: source (queue-size 256, seg max 
>> 254) -> destination (queue-size 256, seg max 126). But it will 
>> definitely introduce two different behaviors for VMs with 
>> "hw_compat_4_2" and older. So, I'd choose the lesser of two evils and 
>> keep the things like it's now.
>>
>
> Yep, make sense. It was the same concern I had.
>
> Do you want to send a patch to remove it with this explanation?

Yes, sure.

I'll do it today.

Denis

>
> Thanks for the clarification,
> Stefano
>