[PATCH v9 00/32] virtio pci support VIRTIO_F_RING_RESET (refactor vring)

[PATCH v9 00/32] virtio pci support VIRTIO_F_RING_RESET (refactor vring)

[Xuan Zhuo]

The virtio spec already supports the virtio queue reset function. This patch set
is to add this function to the kernel. The relevant virtio spec information is
here:

    https://github.com/oasis-tcs/virtio-spec/issues/124

Also regarding MMIO support for queue reset, I plan to support it after this
patch is passed.

This patch set implements the refactoring of vring. Finally, the
virtuque_resize() interface is provided based on the reset function of the
transport layer.

Test environment:
    Host: 4.19.91
    Qemu: QEMU emulator version 6.2.50 (with vq reset support)
    Test Cmd:  ethtool -G eth1 rx $1 tx $2; ethtool -g eth1

    The default is split mode, modify Qemu virtio-net to add PACKED feature to test
    packed mode.

Qemu code:
    https://github.com/fengidri/qemu/compare/89f3bfa3265554d1d591ee4d7f1197b6e3397e84...master

In order to simplify the review of this patch set, the function of reusing
the old buffers after resize will be introduced in subsequent patch sets.

Please review. Thanks.

v9:
  1. Provide a virtqueue_resize() interface directly
  2. A patch set including vring resize, virtio pci reset, virtio-net resize
  3. No more separate structs

v8:
  1. Provide a virtqueue_reset() interface directly
  2. Split the two patch sets, this is the first part
  3. Add independent allocation helper for allocating state, extra

v7:
  1. fix #6 subject typo
  2. fix #6 ring_size_in_bytes is uninitialized
  3. check by: make W=12

v6:
  1. virtio_pci: use synchronize_irq(irq) to sync the irq callbacks
  2. Introduce virtqueue_reset_vring() to implement the reset of vring during
     the reset process. May use the old vring if num of the vq not change.
  3. find_vqs() support sizes to special the max size of each vq

v5:
  1. add virtio-net support set_ringparam

v4:
  1. just the code of virtio, without virtio-net
  2. Performing reset on a queue is divided into these steps:
    1. reset_vq: reset one vq
    2. recycle the buffer from vq by virtqueue_detach_unused_buf()
    3. release the ring of the vq by vring_release_virtqueue()
    4. enable_reset_vq: re-enable the reset queue
  3. Simplify the parameters of enable_reset_vq()
  4. add container structures for virtio_pci_common_cfg

v3:
  1. keep vq, irq unreleased

Xuan Zhuo (32):
  virtio: add helper virtqueue_get_vring_max_size()
  virtio: struct virtio_config_ops add callbacks for queue_reset
  virtio_ring: update the document of the virtqueue_detach_unused_buf
    for queue reset
  virtio_ring: remove the arg vq of vring_alloc_desc_extra()
  virtio_ring: extract the logic of freeing vring
  virtio_ring: split: extract the logic of alloc queue
  virtio_ring: split: extract the logic of alloc state and extra
  virtio_ring: split: extract the logic of attach vring
  virtio_ring: split: extract the logic of vq init
  virtio_ring: split: introduce virtqueue_reinit_split()
  virtio_ring: split: introduce virtqueue_resize_split()
  virtio_ring: packed: extract the logic of alloc queue
  virtio_ring: packed: extract the logic of alloc state and extra
  virtio_ring: packed: extract the logic of attach vring
  virtio_ring: packed: extract the logic of vq init
  virtio_ring: packed: introduce virtqueue_reinit_packed()
  virtio_ring: packed: introduce virtqueue_resize_packed()
  virtio_ring: introduce virtqueue_resize()
  virtio_pci: struct virtio_pci_common_cfg add queue_notify_data
  virtio: queue_reset: add VIRTIO_F_RING_RESET
  virtio_pci: queue_reset: update struct virtio_pci_common_cfg and
    option functions
  virtio_pci: queue_reset: extract the logic of active vq for modern pci
  virtio_pci: queue_reset: support VIRTIO_F_RING_RESET
  virtio: find_vqs() add arg sizes
  virtio_pci: support the arg sizes of find_vqs()
  virtio_mmio: support the arg sizes of find_vqs()
  virtio: add helper virtio_find_vqs_ctx_size()
  virtio_net: set the default max ring size by find_vqs()
  virtio_net: get ringparam by virtqueue_get_vring_max_size()
  virtio_net: split free_unused_bufs()
  virtio_net: support rx/tx queue resize
  virtio_net: support set_ringparam

 arch/um/drivers/virtio_uml.c             |   3 +-
 drivers/net/virtio_net.c                 | 219 +++++++-
 drivers/platform/mellanox/mlxbf-tmfifo.c |   3 +
 drivers/remoteproc/remoteproc_virtio.c   |   3 +
 drivers/s390/virtio/virtio_ccw.c         |   4 +
 drivers/virtio/virtio_mmio.c             |  11 +-
 drivers/virtio/virtio_pci_common.c       |  28 +-
 drivers/virtio/virtio_pci_common.h       |   3 +-
 drivers/virtio/virtio_pci_legacy.c       |   8 +-
 drivers/virtio/virtio_pci_modern.c       | 149 +++++-
 drivers/virtio/virtio_pci_modern_dev.c   |  36 ++
 drivers/virtio/virtio_ring.c             | 626 ++++++++++++++++++-----
 drivers/virtio/virtio_vdpa.c             |   3 +
 include/linux/virtio.h                   |   6 +
 include/linux/virtio_config.h            |  38 +-
 include/linux/virtio_pci_modern.h        |   2 +
 include/uapi/linux/virtio_config.h       |   7 +-
 include/uapi/linux/virtio_pci.h          |  14 +
 18 files changed, 964 insertions(+), 199 deletions(-)

--
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 00/32] virtio pci support VIRTIO_F_RING_RESET (refactor vring)

[Xuan Zhuo]

The virtio spec already supports the virtio queue reset function. This patch set
is to add this function to the kernel. The relevant virtio spec information is
here:

    https://github.com/oasis-tcs/virtio-spec/issues/124

Also regarding MMIO support for queue reset, I plan to support it after this
patch is passed.

This patch set implements the refactoring of vring. Finally, the
virtuque_resize() interface is provided based on the reset function of the
transport layer.

Test environment:
    Host: 4.19.91
    Qemu: QEMU emulator version 6.2.50 (with vq reset support)
    Test Cmd:  ethtool -G eth1 rx $1 tx $2; ethtool -g eth1

    The default is split mode, modify Qemu virtio-net to add PACKED feature to test
    packed mode.

Qemu code:
    https://github.com/fengidri/qemu/compare/89f3bfa3265554d1d591ee4d7f1197b6e3397e84...master

In order to simplify the review of this patch set, the function of reusing
the old buffers after resize will be introduced in subsequent patch sets.

Please review. Thanks.

v9:
  1. Provide a virtqueue_resize() interface directly
  2. A patch set including vring resize, virtio pci reset, virtio-net resize
  3. No more separate structs

v8:
  1. Provide a virtqueue_reset() interface directly
  2. Split the two patch sets, this is the first part
  3. Add independent allocation helper for allocating state, extra

v7:
  1. fix #6 subject typo
  2. fix #6 ring_size_in_bytes is uninitialized
  3. check by: make W=12

v6:
  1. virtio_pci: use synchronize_irq(irq) to sync the irq callbacks
  2. Introduce virtqueue_reset_vring() to implement the reset of vring during
     the reset process. May use the old vring if num of the vq not change.
  3. find_vqs() support sizes to special the max size of each vq

v5:
  1. add virtio-net support set_ringparam

v4:
  1. just the code of virtio, without virtio-net
  2. Performing reset on a queue is divided into these steps:
    1. reset_vq: reset one vq
    2. recycle the buffer from vq by virtqueue_detach_unused_buf()
    3. release the ring of the vq by vring_release_virtqueue()
    4. enable_reset_vq: re-enable the reset queue
  3. Simplify the parameters of enable_reset_vq()
  4. add container structures for virtio_pci_common_cfg

v3:
  1. keep vq, irq unreleased

Xuan Zhuo (32):
  virtio: add helper virtqueue_get_vring_max_size()
  virtio: struct virtio_config_ops add callbacks for queue_reset
  virtio_ring: update the document of the virtqueue_detach_unused_buf
    for queue reset
  virtio_ring: remove the arg vq of vring_alloc_desc_extra()
  virtio_ring: extract the logic of freeing vring
  virtio_ring: split: extract the logic of alloc queue
  virtio_ring: split: extract the logic of alloc state and extra
  virtio_ring: split: extract the logic of attach vring
  virtio_ring: split: extract the logic of vq init
  virtio_ring: split: introduce virtqueue_reinit_split()
  virtio_ring: split: introduce virtqueue_resize_split()
  virtio_ring: packed: extract the logic of alloc queue
  virtio_ring: packed: extract the logic of alloc state and extra
  virtio_ring: packed: extract the logic of attach vring
  virtio_ring: packed: extract the logic of vq init
  virtio_ring: packed: introduce virtqueue_reinit_packed()
  virtio_ring: packed: introduce virtqueue_resize_packed()
  virtio_ring: introduce virtqueue_resize()
  virtio_pci: struct virtio_pci_common_cfg add queue_notify_data
  virtio: queue_reset: add VIRTIO_F_RING_RESET
  virtio_pci: queue_reset: update struct virtio_pci_common_cfg and
    option functions
  virtio_pci: queue_reset: extract the logic of active vq for modern pci
  virtio_pci: queue_reset: support VIRTIO_F_RING_RESET
  virtio: find_vqs() add arg sizes
  virtio_pci: support the arg sizes of find_vqs()
  virtio_mmio: support the arg sizes of find_vqs()
  virtio: add helper virtio_find_vqs_ctx_size()
  virtio_net: set the default max ring size by find_vqs()
  virtio_net: get ringparam by virtqueue_get_vring_max_size()
  virtio_net: split free_unused_bufs()
  virtio_net: support rx/tx queue resize
  virtio_net: support set_ringparam

 arch/um/drivers/virtio_uml.c             |   3 +-
 drivers/net/virtio_net.c                 | 219 +++++++-
 drivers/platform/mellanox/mlxbf-tmfifo.c |   3 +
 drivers/remoteproc/remoteproc_virtio.c   |   3 +
 drivers/s390/virtio/virtio_ccw.c         |   4 +
 drivers/virtio/virtio_mmio.c             |  11 +-
 drivers/virtio/virtio_pci_common.c       |  28 +-
 drivers/virtio/virtio_pci_common.h       |   3 +-
 drivers/virtio/virtio_pci_legacy.c       |   8 +-
 drivers/virtio/virtio_pci_modern.c       | 149 +++++-
 drivers/virtio/virtio_pci_modern_dev.c   |  36 ++
 drivers/virtio/virtio_ring.c             | 626 ++++++++++++++++++-----
 drivers/virtio/virtio_vdpa.c             |   3 +
 include/linux/virtio.h                   |   6 +
 include/linux/virtio_config.h            |  38 +-
 include/linux/virtio_pci_modern.h        |   2 +
 include/uapi/linux/virtio_config.h       |   7 +-
 include/uapi/linux/virtio_pci.h          |  14 +
 18 files changed, 964 insertions(+), 199 deletions(-)

--
2.31.0

[PATCH v9 00/32] virtio pci support VIRTIO_F_RING_RESET (refactor vring)

[Xuan Zhuo]

The virtio spec already supports the virtio queue reset function. This patch set
is to add this function to the kernel. The relevant virtio spec information is
here:

    https://github.com/oasis-tcs/virtio-spec/issues/124

Also regarding MMIO support for queue reset, I plan to support it after this
patch is passed.

This patch set implements the refactoring of vring. Finally, the
virtuque_resize() interface is provided based on the reset function of the
transport layer.

Test environment:
    Host: 4.19.91
    Qemu: QEMU emulator version 6.2.50 (with vq reset support)
    Test Cmd:  ethtool -G eth1 rx $1 tx $2; ethtool -g eth1

    The default is split mode, modify Qemu virtio-net to add PACKED feature to test
    packed mode.

Qemu code:
    https://github.com/fengidri/qemu/compare/89f3bfa3265554d1d591ee4d7f1197b6e3397e84...master

In order to simplify the review of this patch set, the function of reusing
the old buffers after resize will be introduced in subsequent patch sets.

Please review. Thanks.

v9:
  1. Provide a virtqueue_resize() interface directly
  2. A patch set including vring resize, virtio pci reset, virtio-net resize
  3. No more separate structs

v8:
  1. Provide a virtqueue_reset() interface directly
  2. Split the two patch sets, this is the first part
  3. Add independent allocation helper for allocating state, extra

v7:
  1. fix #6 subject typo
  2. fix #6 ring_size_in_bytes is uninitialized
  3. check by: make W=12

v6:
  1. virtio_pci: use synchronize_irq(irq) to sync the irq callbacks
  2. Introduce virtqueue_reset_vring() to implement the reset of vring during
     the reset process. May use the old vring if num of the vq not change.
  3. find_vqs() support sizes to special the max size of each vq

v5:
  1. add virtio-net support set_ringparam

v4:
  1. just the code of virtio, without virtio-net
  2. Performing reset on a queue is divided into these steps:
    1. reset_vq: reset one vq
    2. recycle the buffer from vq by virtqueue_detach_unused_buf()
    3. release the ring of the vq by vring_release_virtqueue()
    4. enable_reset_vq: re-enable the reset queue
  3. Simplify the parameters of enable_reset_vq()
  4. add container structures for virtio_pci_common_cfg

v3:
  1. keep vq, irq unreleased

Xuan Zhuo (32):
  virtio: add helper virtqueue_get_vring_max_size()
  virtio: struct virtio_config_ops add callbacks for queue_reset
  virtio_ring: update the document of the virtqueue_detach_unused_buf
    for queue reset
  virtio_ring: remove the arg vq of vring_alloc_desc_extra()
  virtio_ring: extract the logic of freeing vring
  virtio_ring: split: extract the logic of alloc queue
  virtio_ring: split: extract the logic of alloc state and extra
  virtio_ring: split: extract the logic of attach vring
  virtio_ring: split: extract the logic of vq init
  virtio_ring: split: introduce virtqueue_reinit_split()
  virtio_ring: split: introduce virtqueue_resize_split()
  virtio_ring: packed: extract the logic of alloc queue
  virtio_ring: packed: extract the logic of alloc state and extra
  virtio_ring: packed: extract the logic of attach vring
  virtio_ring: packed: extract the logic of vq init
  virtio_ring: packed: introduce virtqueue_reinit_packed()
  virtio_ring: packed: introduce virtqueue_resize_packed()
  virtio_ring: introduce virtqueue_resize()
  virtio_pci: struct virtio_pci_common_cfg add queue_notify_data
  virtio: queue_reset: add VIRTIO_F_RING_RESET
  virtio_pci: queue_reset: update struct virtio_pci_common_cfg and
    option functions
  virtio_pci: queue_reset: extract the logic of active vq for modern pci
  virtio_pci: queue_reset: support VIRTIO_F_RING_RESET
  virtio: find_vqs() add arg sizes
  virtio_pci: support the arg sizes of find_vqs()
  virtio_mmio: support the arg sizes of find_vqs()
  virtio: add helper virtio_find_vqs_ctx_size()
  virtio_net: set the default max ring size by find_vqs()
  virtio_net: get ringparam by virtqueue_get_vring_max_size()
  virtio_net: split free_unused_bufs()
  virtio_net: support rx/tx queue resize
  virtio_net: support set_ringparam

 arch/um/drivers/virtio_uml.c             |   3 +-
 drivers/net/virtio_net.c                 | 219 +++++++-
 drivers/platform/mellanox/mlxbf-tmfifo.c |   3 +
 drivers/remoteproc/remoteproc_virtio.c   |   3 +
 drivers/s390/virtio/virtio_ccw.c         |   4 +
 drivers/virtio/virtio_mmio.c             |  11 +-
 drivers/virtio/virtio_pci_common.c       |  28 +-
 drivers/virtio/virtio_pci_common.h       |   3 +-
 drivers/virtio/virtio_pci_legacy.c       |   8 +-
 drivers/virtio/virtio_pci_modern.c       | 149 +++++-
 drivers/virtio/virtio_pci_modern_dev.c   |  36 ++
 drivers/virtio/virtio_ring.c             | 626 ++++++++++++++++++-----
 drivers/virtio/virtio_vdpa.c             |   3 +
 include/linux/virtio.h                   |   6 +
 include/linux/virtio_config.h            |  38 +-
 include/linux/virtio_pci_modern.h        |   2 +
 include/uapi/linux/virtio_config.h       |   7 +-
 include/uapi/linux/virtio_pci.h          |  14 +
 18 files changed, 964 insertions(+), 199 deletions(-)

--
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 01/32] virtio: add helper virtqueue_get_vring_max_size()

[Xuan Zhuo]

Record the maximum queue num supported by the device.

virtio-net can display the maximum (supported by hardware) ring size in
ethtool -g eth0.

When the subsequent patch implements vring reset, it can judge whether
the ring size passed by the driver is legal based on this.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 arch/um/drivers/virtio_uml.c             |  1 +
 drivers/platform/mellanox/mlxbf-tmfifo.c |  2 ++
 drivers/remoteproc/remoteproc_virtio.c   |  2 ++
 drivers/s390/virtio/virtio_ccw.c         |  3 +++
 drivers/virtio/virtio_mmio.c             |  2 ++
 drivers/virtio/virtio_pci_legacy.c       |  2 ++
 drivers/virtio/virtio_pci_modern.c       |  2 ++
 drivers/virtio/virtio_ring.c             | 14 ++++++++++++++
 drivers/virtio/virtio_vdpa.c             |  2 ++
 include/linux/virtio.h                   |  2 ++
 10 files changed, 32 insertions(+)

diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
index ba562d68dc04..904993d15a85 100644
--- a/arch/um/drivers/virtio_uml.c
+++ b/arch/um/drivers/virtio_uml.c
@@ -945,6 +945,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
 		goto error_create;
 	}
 	vq->priv = info;
+	vq->num_max = num;
 	num = virtqueue_get_vring_size(vq);
 
 	if (vu_dev->protocol_features &
diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
index 38800e86ed8a..1ae3c56b66b0 100644
--- a/drivers/platform/mellanox/mlxbf-tmfifo.c
+++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
@@ -959,6 +959,8 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
 			goto error;
 		}
 
+		vq->num_max = vring->num;
+
 		vqs[i] = vq;
 		vring->vq = vq;
 		vq->priv = vring;
diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index 70ab496d0431..7611755d0ae2 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -125,6 +125,8 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev,
 		return ERR_PTR(-ENOMEM);
 	}
 
+	vq->num_max = len;
+
 	rvring->vq = vq;
 	vq->priv = rvring;
 
diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
index d35e7a3f7067..468da60b56c5 100644
--- a/drivers/s390/virtio/virtio_ccw.c
+++ b/drivers/s390/virtio/virtio_ccw.c
@@ -529,6 +529,9 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,
 		err = -ENOMEM;
 		goto out_err;
 	}
+
+	vq->num_max = info->num;
+
 	/* it may have been reduced */
 	info->num = virtqueue_get_vring_size(vq);
 
diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 56128b9c46eb..a41abc8051b9 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -390,6 +390,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
 		goto error_new_virtqueue;
 	}
 
+	vq->num_max = num;
+
 	/* Activate the queue */
 	writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
 	if (vm_dev->version == 1) {
diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
index 34141b9abe27..b68934fe6b5d 100644
--- a/drivers/virtio/virtio_pci_legacy.c
+++ b/drivers/virtio/virtio_pci_legacy.c
@@ -135,6 +135,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	if (!vq)
 		return ERR_PTR(-ENOMEM);
 
+	vq->num_max = num;
+
 	q_pfn = virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT;
 	if (q_pfn >> 32) {
 		dev_err(&vp_dev->pci_dev->dev,
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index 5455bc041fb6..86d301f272b8 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -218,6 +218,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	if (!vq)
 		return ERR_PTR(-ENOMEM);
 
+	vq->num_max = num;
+
 	/* activate the queue */
 	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
 	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 962f1477b1fa..b87130c8f312 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2371,6 +2371,20 @@ void vring_transport_features(struct virtio_device *vdev)
 }
 EXPORT_SYMBOL_GPL(vring_transport_features);
 
+/**
+ * virtqueue_get_vring_max_size - return the max size of the virtqueue's vring
+ * @_vq: the struct virtqueue containing the vring of interest.
+ *
+ * Returns the max size of the vring.
+ *
+ * Unlike other operations, this need not be serialized.
+ */
+unsigned int virtqueue_get_vring_max_size(struct virtqueue *_vq)
+{
+	return _vq->num_max;
+}
+EXPORT_SYMBOL_GPL(virtqueue_get_vring_max_size);
+
 /**
  * virtqueue_get_vring_size - return the size of the virtqueue's vring
  * @_vq: the struct virtqueue containing the vring of interest.
diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
index 7767a7f0119b..39e4c08eb0f2 100644
--- a/drivers/virtio/virtio_vdpa.c
+++ b/drivers/virtio/virtio_vdpa.c
@@ -183,6 +183,8 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
 		goto error_new_virtqueue;
 	}
 
+	vq->num_max = max_num;
+
 	/* Setup virtqueue callback */
 	cb.callback = virtio_vdpa_virtqueue_cb;
 	cb.private = info;
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index 72292a62cd90..d59adc4be068 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -31,6 +31,7 @@ struct virtqueue {
 	struct virtio_device *vdev;
 	unsigned int index;
 	unsigned int num_free;
+	unsigned int num_max;
 	void *priv;
 };
 
@@ -80,6 +81,7 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *vq);
 
 void *virtqueue_detach_unused_buf(struct virtqueue *vq);
 
+unsigned int virtqueue_get_vring_max_size(struct virtqueue *vq);
 unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
 
 bool virtqueue_is_broken(struct virtqueue *vq);
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 01/32] virtio: add helper virtqueue_get_vring_max_size()

[Xuan Zhuo]

Record the maximum queue num supported by the device.

virtio-net can display the maximum (supported by hardware) ring size in
ethtool -g eth0.

When the subsequent patch implements vring reset, it can judge whether
the ring size passed by the driver is legal based on this.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 arch/um/drivers/virtio_uml.c             |  1 +
 drivers/platform/mellanox/mlxbf-tmfifo.c |  2 ++
 drivers/remoteproc/remoteproc_virtio.c   |  2 ++
 drivers/s390/virtio/virtio_ccw.c         |  3 +++
 drivers/virtio/virtio_mmio.c             |  2 ++
 drivers/virtio/virtio_pci_legacy.c       |  2 ++
 drivers/virtio/virtio_pci_modern.c       |  2 ++
 drivers/virtio/virtio_ring.c             | 14 ++++++++++++++
 drivers/virtio/virtio_vdpa.c             |  2 ++
 include/linux/virtio.h                   |  2 ++
 10 files changed, 32 insertions(+)

diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
index ba562d68dc04..904993d15a85 100644
--- a/arch/um/drivers/virtio_uml.c
+++ b/arch/um/drivers/virtio_uml.c
@@ -945,6 +945,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
 		goto error_create;
 	}
 	vq->priv = info;
+	vq->num_max = num;
 	num = virtqueue_get_vring_size(vq);
 
 	if (vu_dev->protocol_features &
diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
index 38800e86ed8a..1ae3c56b66b0 100644
--- a/drivers/platform/mellanox/mlxbf-tmfifo.c
+++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
@@ -959,6 +959,8 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
 			goto error;
 		}
 
+		vq->num_max = vring->num;
+
 		vqs[i] = vq;
 		vring->vq = vq;
 		vq->priv = vring;
diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index 70ab496d0431..7611755d0ae2 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -125,6 +125,8 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev,
 		return ERR_PTR(-ENOMEM);
 	}
 
+	vq->num_max = len;
+
 	rvring->vq = vq;
 	vq->priv = rvring;
 
diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
index d35e7a3f7067..468da60b56c5 100644
--- a/drivers/s390/virtio/virtio_ccw.c
+++ b/drivers/s390/virtio/virtio_ccw.c
@@ -529,6 +529,9 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,
 		err = -ENOMEM;
 		goto out_err;
 	}
+
+	vq->num_max = info->num;
+
 	/* it may have been reduced */
 	info->num = virtqueue_get_vring_size(vq);
 
diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 56128b9c46eb..a41abc8051b9 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -390,6 +390,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
 		goto error_new_virtqueue;
 	}
 
+	vq->num_max = num;
+
 	/* Activate the queue */
 	writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
 	if (vm_dev->version == 1) {
diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
index 34141b9abe27..b68934fe6b5d 100644
--- a/drivers/virtio/virtio_pci_legacy.c
+++ b/drivers/virtio/virtio_pci_legacy.c
@@ -135,6 +135,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	if (!vq)
 		return ERR_PTR(-ENOMEM);
 
+	vq->num_max = num;
+
 	q_pfn = virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT;
 	if (q_pfn >> 32) {
 		dev_err(&vp_dev->pci_dev->dev,
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index 5455bc041fb6..86d301f272b8 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -218,6 +218,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	if (!vq)
 		return ERR_PTR(-ENOMEM);
 
+	vq->num_max = num;
+
 	/* activate the queue */
 	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
 	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 962f1477b1fa..b87130c8f312 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2371,6 +2371,20 @@ void vring_transport_features(struct virtio_device *vdev)
 }
 EXPORT_SYMBOL_GPL(vring_transport_features);
 
+/**
+ * virtqueue_get_vring_max_size - return the max size of the virtqueue's vring
+ * @_vq: the struct virtqueue containing the vring of interest.
+ *
+ * Returns the max size of the vring.
+ *
+ * Unlike other operations, this need not be serialized.
+ */
+unsigned int virtqueue_get_vring_max_size(struct virtqueue *_vq)
+{
+	return _vq->num_max;
+}
+EXPORT_SYMBOL_GPL(virtqueue_get_vring_max_size);
+
 /**
  * virtqueue_get_vring_size - return the size of the virtqueue's vring
  * @_vq: the struct virtqueue containing the vring of interest.
diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
index 7767a7f0119b..39e4c08eb0f2 100644
--- a/drivers/virtio/virtio_vdpa.c
+++ b/drivers/virtio/virtio_vdpa.c
@@ -183,6 +183,8 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
 		goto error_new_virtqueue;
 	}
 
+	vq->num_max = max_num;
+
 	/* Setup virtqueue callback */
 	cb.callback = virtio_vdpa_virtqueue_cb;
 	cb.private = info;
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index 72292a62cd90..d59adc4be068 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -31,6 +31,7 @@ struct virtqueue {
 	struct virtio_device *vdev;
 	unsigned int index;
 	unsigned int num_free;
+	unsigned int num_max;
 	void *priv;
 };
 
@@ -80,6 +81,7 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *vq);
 
 void *virtqueue_detach_unused_buf(struct virtqueue *vq);
 
+unsigned int virtqueue_get_vring_max_size(struct virtqueue *vq);
 unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
 
 bool virtqueue_is_broken(struct virtqueue *vq);
-- 
2.31.0

[PATCH v9 01/32] virtio: add helper virtqueue_get_vring_max_size()

[Xuan Zhuo]

Record the maximum queue num supported by the device.

virtio-net can display the maximum (supported by hardware) ring size in
ethtool -g eth0.

When the subsequent patch implements vring reset, it can judge whether
the ring size passed by the driver is legal based on this.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 arch/um/drivers/virtio_uml.c             |  1 +
 drivers/platform/mellanox/mlxbf-tmfifo.c |  2 ++
 drivers/remoteproc/remoteproc_virtio.c   |  2 ++
 drivers/s390/virtio/virtio_ccw.c         |  3 +++
 drivers/virtio/virtio_mmio.c             |  2 ++
 drivers/virtio/virtio_pci_legacy.c       |  2 ++
 drivers/virtio/virtio_pci_modern.c       |  2 ++
 drivers/virtio/virtio_ring.c             | 14 ++++++++++++++
 drivers/virtio/virtio_vdpa.c             |  2 ++
 include/linux/virtio.h                   |  2 ++
 10 files changed, 32 insertions(+)

diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
index ba562d68dc04..904993d15a85 100644
--- a/arch/um/drivers/virtio_uml.c
+++ b/arch/um/drivers/virtio_uml.c
@@ -945,6 +945,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
 		goto error_create;
 	}
 	vq->priv = info;
+	vq->num_max = num;
 	num = virtqueue_get_vring_size(vq);
 
 	if (vu_dev->protocol_features &
diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
index 38800e86ed8a..1ae3c56b66b0 100644
--- a/drivers/platform/mellanox/mlxbf-tmfifo.c
+++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
@@ -959,6 +959,8 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
 			goto error;
 		}
 
+		vq->num_max = vring->num;
+
 		vqs[i] = vq;
 		vring->vq = vq;
 		vq->priv = vring;
diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index 70ab496d0431..7611755d0ae2 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -125,6 +125,8 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev,
 		return ERR_PTR(-ENOMEM);
 	}
 
+	vq->num_max = len;
+
 	rvring->vq = vq;
 	vq->priv = rvring;
 
diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
index d35e7a3f7067..468da60b56c5 100644
--- a/drivers/s390/virtio/virtio_ccw.c
+++ b/drivers/s390/virtio/virtio_ccw.c
@@ -529,6 +529,9 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,
 		err = -ENOMEM;
 		goto out_err;
 	}
+
+	vq->num_max = info->num;
+
 	/* it may have been reduced */
 	info->num = virtqueue_get_vring_size(vq);
 
diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 56128b9c46eb..a41abc8051b9 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -390,6 +390,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
 		goto error_new_virtqueue;
 	}
 
+	vq->num_max = num;
+
 	/* Activate the queue */
 	writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
 	if (vm_dev->version == 1) {
diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
index 34141b9abe27..b68934fe6b5d 100644
--- a/drivers/virtio/virtio_pci_legacy.c
+++ b/drivers/virtio/virtio_pci_legacy.c
@@ -135,6 +135,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	if (!vq)
 		return ERR_PTR(-ENOMEM);
 
+	vq->num_max = num;
+
 	q_pfn = virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT;
 	if (q_pfn >> 32) {
 		dev_err(&vp_dev->pci_dev->dev,
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index 5455bc041fb6..86d301f272b8 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -218,6 +218,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	if (!vq)
 		return ERR_PTR(-ENOMEM);
 
+	vq->num_max = num;
+
 	/* activate the queue */
 	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
 	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 962f1477b1fa..b87130c8f312 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2371,6 +2371,20 @@ void vring_transport_features(struct virtio_device *vdev)
 }
 EXPORT_SYMBOL_GPL(vring_transport_features);
 
+/**
+ * virtqueue_get_vring_max_size - return the max size of the virtqueue's vring
+ * @_vq: the struct virtqueue containing the vring of interest.
+ *
+ * Returns the max size of the vring.
+ *
+ * Unlike other operations, this need not be serialized.
+ */
+unsigned int virtqueue_get_vring_max_size(struct virtqueue *_vq)
+{
+	return _vq->num_max;
+}
+EXPORT_SYMBOL_GPL(virtqueue_get_vring_max_size);
+
 /**
  * virtqueue_get_vring_size - return the size of the virtqueue's vring
  * @_vq: the struct virtqueue containing the vring of interest.
diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
index 7767a7f0119b..39e4c08eb0f2 100644
--- a/drivers/virtio/virtio_vdpa.c
+++ b/drivers/virtio/virtio_vdpa.c
@@ -183,6 +183,8 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
 		goto error_new_virtqueue;
 	}
 
+	vq->num_max = max_num;
+
 	/* Setup virtqueue callback */
 	cb.callback = virtio_vdpa_virtqueue_cb;
 	cb.private = info;
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index 72292a62cd90..d59adc4be068 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -31,6 +31,7 @@ struct virtqueue {
 	struct virtio_device *vdev;
 	unsigned int index;
 	unsigned int num_free;
+	unsigned int num_max;
 	void *priv;
 };
 
@@ -80,6 +81,7 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *vq);
 
 void *virtqueue_detach_unused_buf(struct virtqueue *vq);
 
+unsigned int virtqueue_get_vring_max_size(struct virtqueue *vq);
 unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
 
 bool virtqueue_is_broken(struct virtqueue *vq);
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 02/32] virtio: struct virtio_config_ops add callbacks for queue_reset

[Xuan Zhuo]

Performing reset on a queue is divided into four steps:

 1. transport: notify the device to reset the queue
 2. vring:     recycle the buffer submitted
 3. vring:     reset/resize the vring (may re-alloc)
 4. transport: mmap vring to device, and enable the queue

In order to support queue reset, add two callbacks(reset_vq,
enable_reset_vq) in struct virtio_config_ops to implement steps 1 and 4.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 include/linux/virtio_config.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
index 4d107ad31149..d4adcd0e1c57 100644
--- a/include/linux/virtio_config.h
+++ b/include/linux/virtio_config.h
@@ -74,6 +74,16 @@ struct virtio_shm_region {
  * @set_vq_affinity: set the affinity for a virtqueue (optional).
  * @get_vq_affinity: get the affinity for a virtqueue (optional).
  * @get_shm_region: get a shared memory region based on the index.
+ * @reset_vq: reset a queue individually (optional).
+ *	vq: the virtqueue
+ *	Returns 0 on success or error status
+ *	reset_vq will guarantee that the callbacks are disabled and synchronized.
+ *	Except for the callback, the caller should guarantee that the vring is
+ *	not accessed by any functions of virtqueue.
+ * @enable_reset_vq: enable a reset queue
+ *	vq: the virtqueue
+ *	Returns 0 on success or error status
+ *	If reset_vq is set, then enable_reset_vq must also be set.
  */
 typedef void vq_callback_t(struct virtqueue *);
 struct virtio_config_ops {
@@ -100,6 +110,8 @@ struct virtio_config_ops {
 			int index);
 	bool (*get_shm_region)(struct virtio_device *vdev,
 			       struct virtio_shm_region *region, u8 id);
+	int (*reset_vq)(struct virtqueue *vq);
+	int (*enable_reset_vq)(struct virtqueue *vq);
 };
 
 /* If driver didn't advertise the feature, it will never appear. */
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 02/32] virtio: struct virtio_config_ops add callbacks for queue_reset

[Xuan Zhuo]

Performing reset on a queue is divided into four steps:

 1. transport: notify the device to reset the queue
 2. vring:     recycle the buffer submitted
 3. vring:     reset/resize the vring (may re-alloc)
 4. transport: mmap vring to device, and enable the queue

In order to support queue reset, add two callbacks(reset_vq,
enable_reset_vq) in struct virtio_config_ops to implement steps 1 and 4.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 include/linux/virtio_config.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
index 4d107ad31149..d4adcd0e1c57 100644
--- a/include/linux/virtio_config.h
+++ b/include/linux/virtio_config.h
@@ -74,6 +74,16 @@ struct virtio_shm_region {
  * @set_vq_affinity: set the affinity for a virtqueue (optional).
  * @get_vq_affinity: get the affinity for a virtqueue (optional).
  * @get_shm_region: get a shared memory region based on the index.
+ * @reset_vq: reset a queue individually (optional).
+ *	vq: the virtqueue
+ *	Returns 0 on success or error status
+ *	reset_vq will guarantee that the callbacks are disabled and synchronized.
+ *	Except for the callback, the caller should guarantee that the vring is
+ *	not accessed by any functions of virtqueue.
+ * @enable_reset_vq: enable a reset queue
+ *	vq: the virtqueue
+ *	Returns 0 on success or error status
+ *	If reset_vq is set, then enable_reset_vq must also be set.
  */
 typedef void vq_callback_t(struct virtqueue *);
 struct virtio_config_ops {
@@ -100,6 +110,8 @@ struct virtio_config_ops {
 			int index);
 	bool (*get_shm_region)(struct virtio_device *vdev,
 			       struct virtio_shm_region *region, u8 id);
+	int (*reset_vq)(struct virtqueue *vq);
+	int (*enable_reset_vq)(struct virtqueue *vq);
 };
 
 /* If driver didn't advertise the feature, it will never appear. */
-- 
2.31.0

[PATCH v9 02/32] virtio: struct virtio_config_ops add callbacks for queue_reset

[Xuan Zhuo]

Performing reset on a queue is divided into four steps:

 1. transport: notify the device to reset the queue
 2. vring:     recycle the buffer submitted
 3. vring:     reset/resize the vring (may re-alloc)
 4. transport: mmap vring to device, and enable the queue

In order to support queue reset, add two callbacks(reset_vq,
enable_reset_vq) in struct virtio_config_ops to implement steps 1 and 4.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 include/linux/virtio_config.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
index 4d107ad31149..d4adcd0e1c57 100644
--- a/include/linux/virtio_config.h
+++ b/include/linux/virtio_config.h
@@ -74,6 +74,16 @@ struct virtio_shm_region {
  * @set_vq_affinity: set the affinity for a virtqueue (optional).
  * @get_vq_affinity: get the affinity for a virtqueue (optional).
  * @get_shm_region: get a shared memory region based on the index.
+ * @reset_vq: reset a queue individually (optional).
+ *	vq: the virtqueue
+ *	Returns 0 on success or error status
+ *	reset_vq will guarantee that the callbacks are disabled and synchronized.
+ *	Except for the callback, the caller should guarantee that the vring is
+ *	not accessed by any functions of virtqueue.
+ * @enable_reset_vq: enable a reset queue
+ *	vq: the virtqueue
+ *	Returns 0 on success or error status
+ *	If reset_vq is set, then enable_reset_vq must also be set.
  */
 typedef void vq_callback_t(struct virtqueue *);
 struct virtio_config_ops {
@@ -100,6 +110,8 @@ struct virtio_config_ops {
 			int index);
 	bool (*get_shm_region)(struct virtio_device *vdev,
 			       struct virtio_shm_region *region, u8 id);
+	int (*reset_vq)(struct virtqueue *vq);
+	int (*enable_reset_vq)(struct virtqueue *vq);
 };
 
 /* If driver didn't advertise the feature, it will never appear. */
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 03/32] virtio_ring: update the document of the virtqueue_detach_unused_buf for queue reset

[Xuan Zhuo]

Added documentation for virtqueue_detach_unused_buf, allowing it to be
called on queue reset.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index b87130c8f312..f1807f6b06a5 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2127,8 +2127,8 @@ EXPORT_SYMBOL_GPL(virtqueue_enable_cb_delayed);
  * @_vq: the struct virtqueue we're talking about.
  *
  * Returns NULL or the "data" token handed to virtqueue_add_*().
- * This is not valid on an active queue; it is useful only for device
- * shutdown.
+ * This is not valid on an active queue; it is useful for device
+ * shutdown or the reset queue.
  */
 void *virtqueue_detach_unused_buf(struct virtqueue *_vq)
 {
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 03/32] virtio_ring: update the document of the virtqueue_detach_unused_buf for queue reset

[Xuan Zhuo]

Added documentation for virtqueue_detach_unused_buf, allowing it to be
called on queue reset.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index b87130c8f312..f1807f6b06a5 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2127,8 +2127,8 @@ EXPORT_SYMBOL_GPL(virtqueue_enable_cb_delayed);
  * @_vq: the struct virtqueue we're talking about.
  *
  * Returns NULL or the "data" token handed to virtqueue_add_*().
- * This is not valid on an active queue; it is useful only for device
- * shutdown.
+ * This is not valid on an active queue; it is useful for device
+ * shutdown or the reset queue.
  */
 void *virtqueue_detach_unused_buf(struct virtqueue *_vq)
 {
-- 
2.31.0

[PATCH v9 03/32] virtio_ring: update the document of the virtqueue_detach_unused_buf for queue reset

[Xuan Zhuo]

Added documentation for virtqueue_detach_unused_buf, allowing it to be
called on queue reset.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index b87130c8f312..f1807f6b06a5 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2127,8 +2127,8 @@ EXPORT_SYMBOL_GPL(virtqueue_enable_cb_delayed);
  * @_vq: the struct virtqueue we're talking about.
  *
  * Returns NULL or the "data" token handed to virtqueue_add_*().
- * This is not valid on an active queue; it is useful only for device
- * shutdown.
+ * This is not valid on an active queue; it is useful for device
+ * shutdown or the reset queue.
  */
 void *virtqueue_detach_unused_buf(struct virtqueue *_vq)
 {
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 04/32] virtio_ring: remove the arg vq of vring_alloc_desc_extra()

[Xuan Zhuo]

The parameter vq of vring_alloc_desc_extra() is useless. This patch
removes this parameter.

Subsequent patches will call this function to avoid passing useless
arguments.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index f1807f6b06a5..cb6010750a94 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1636,8 +1636,7 @@ static void *virtqueue_detach_unused_buf_packed(struct virtqueue *_vq)
 	return NULL;
 }
 
-static struct vring_desc_extra *vring_alloc_desc_extra(struct vring_virtqueue *vq,
-						       unsigned int num)
+static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
 {
 	struct vring_desc_extra *desc_extra;
 	unsigned int i;
@@ -1755,7 +1754,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	/* Put everything in free lists. */
 	vq->free_head = 0;
 
-	vq->packed.desc_extra = vring_alloc_desc_extra(vq, num);
+	vq->packed.desc_extra = vring_alloc_desc_extra(num);
 	if (!vq->packed.desc_extra)
 		goto err_desc_extra;
 
@@ -2233,7 +2232,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	if (!vq->split.desc_state)
 		goto err_state;
 
-	vq->split.desc_extra = vring_alloc_desc_extra(vq, vring.num);
+	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
 	if (!vq->split.desc_extra)
 		goto err_extra;
 
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 04/32] virtio_ring: remove the arg vq of vring_alloc_desc_extra()

[Xuan Zhuo]

The parameter vq of vring_alloc_desc_extra() is useless. This patch
removes this parameter.

Subsequent patches will call this function to avoid passing useless
arguments.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index f1807f6b06a5..cb6010750a94 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1636,8 +1636,7 @@ static void *virtqueue_detach_unused_buf_packed(struct virtqueue *_vq)
 	return NULL;
 }
 
-static struct vring_desc_extra *vring_alloc_desc_extra(struct vring_virtqueue *vq,
-						       unsigned int num)
+static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
 {
 	struct vring_desc_extra *desc_extra;
 	unsigned int i;
@@ -1755,7 +1754,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	/* Put everything in free lists. */
 	vq->free_head = 0;
 
-	vq->packed.desc_extra = vring_alloc_desc_extra(vq, num);
+	vq->packed.desc_extra = vring_alloc_desc_extra(num);
 	if (!vq->packed.desc_extra)
 		goto err_desc_extra;
 
@@ -2233,7 +2232,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	if (!vq->split.desc_state)
 		goto err_state;
 
-	vq->split.desc_extra = vring_alloc_desc_extra(vq, vring.num);
+	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
 	if (!vq->split.desc_extra)
 		goto err_extra;
 
-- 
2.31.0

[PATCH v9 04/32] virtio_ring: remove the arg vq of vring_alloc_desc_extra()

[Xuan Zhuo]

The parameter vq of vring_alloc_desc_extra() is useless. This patch
removes this parameter.

Subsequent patches will call this function to avoid passing useless
arguments.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index f1807f6b06a5..cb6010750a94 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1636,8 +1636,7 @@ static void *virtqueue_detach_unused_buf_packed(struct virtqueue *_vq)
 	return NULL;
 }
 
-static struct vring_desc_extra *vring_alloc_desc_extra(struct vring_virtqueue *vq,
-						       unsigned int num)
+static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
 {
 	struct vring_desc_extra *desc_extra;
 	unsigned int i;
@@ -1755,7 +1754,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	/* Put everything in free lists. */
 	vq->free_head = 0;
 
-	vq->packed.desc_extra = vring_alloc_desc_extra(vq, num);
+	vq->packed.desc_extra = vring_alloc_desc_extra(num);
 	if (!vq->packed.desc_extra)
 		goto err_desc_extra;
 
@@ -2233,7 +2232,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	if (!vq->split.desc_state)
 		goto err_state;
 
-	vq->split.desc_extra = vring_alloc_desc_extra(vq, vring.num);
+	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
 	if (!vq->split.desc_extra)
 		goto err_extra;
 
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 05/32] virtio_ring: extract the logic of freeing vring

[Xuan Zhuo]

Introduce vring_free() to free the vring of vq.

Subsequent patches will use vring_free() alone.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index cb6010750a94..33fddfb907a6 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2301,14 +2301,10 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
 }
 EXPORT_SYMBOL_GPL(vring_new_virtqueue);
 
-void vring_del_virtqueue(struct virtqueue *_vq)
+static void vring_free(struct virtqueue *_vq)
 {
 	struct vring_virtqueue *vq = to_vvq(_vq);
 
-	spin_lock(&vq->vq.vdev->vqs_list_lock);
-	list_del(&_vq->list);
-	spin_unlock(&vq->vq.vdev->vqs_list_lock);
-
 	if (vq->we_own_ring) {
 		if (vq->packed_ring) {
 			vring_free_queue(vq->vq.vdev,
@@ -2339,6 +2335,18 @@ void vring_del_virtqueue(struct virtqueue *_vq)
 		kfree(vq->split.desc_state);
 		kfree(vq->split.desc_extra);
 	}
+}
+
+void vring_del_virtqueue(struct virtqueue *_vq)
+{
+	struct vring_virtqueue *vq = to_vvq(_vq);
+
+	spin_lock(&vq->vq.vdev->vqs_list_lock);
+	list_del(&_vq->list);
+	spin_unlock(&vq->vq.vdev->vqs_list_lock);
+
+	vring_free(_vq);
+
 	kfree(vq);
 }
 EXPORT_SYMBOL_GPL(vring_del_virtqueue);
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 05/32] virtio_ring: extract the logic of freeing vring

[Xuan Zhuo]

Introduce vring_free() to free the vring of vq.

Subsequent patches will use vring_free() alone.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index cb6010750a94..33fddfb907a6 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2301,14 +2301,10 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
 }
 EXPORT_SYMBOL_GPL(vring_new_virtqueue);
 
-void vring_del_virtqueue(struct virtqueue *_vq)
+static void vring_free(struct virtqueue *_vq)
 {
 	struct vring_virtqueue *vq = to_vvq(_vq);
 
-	spin_lock(&vq->vq.vdev->vqs_list_lock);
-	list_del(&_vq->list);
-	spin_unlock(&vq->vq.vdev->vqs_list_lock);
-
 	if (vq->we_own_ring) {
 		if (vq->packed_ring) {
 			vring_free_queue(vq->vq.vdev,
@@ -2339,6 +2335,18 @@ void vring_del_virtqueue(struct virtqueue *_vq)
 		kfree(vq->split.desc_state);
 		kfree(vq->split.desc_extra);
 	}
+}
+
+void vring_del_virtqueue(struct virtqueue *_vq)
+{
+	struct vring_virtqueue *vq = to_vvq(_vq);
+
+	spin_lock(&vq->vq.vdev->vqs_list_lock);
+	list_del(&_vq->list);
+	spin_unlock(&vq->vq.vdev->vqs_list_lock);
+
+	vring_free(_vq);
+
 	kfree(vq);
 }
 EXPORT_SYMBOL_GPL(vring_del_virtqueue);
-- 
2.31.0

[PATCH v9 05/32] virtio_ring: extract the logic of freeing vring

[Xuan Zhuo]

Introduce vring_free() to free the vring of vq.

Subsequent patches will use vring_free() alone.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index cb6010750a94..33fddfb907a6 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2301,14 +2301,10 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
 }
 EXPORT_SYMBOL_GPL(vring_new_virtqueue);
 
-void vring_del_virtqueue(struct virtqueue *_vq)
+static void vring_free(struct virtqueue *_vq)
 {
 	struct vring_virtqueue *vq = to_vvq(_vq);
 
-	spin_lock(&vq->vq.vdev->vqs_list_lock);
-	list_del(&_vq->list);
-	spin_unlock(&vq->vq.vdev->vqs_list_lock);
-
 	if (vq->we_own_ring) {
 		if (vq->packed_ring) {
 			vring_free_queue(vq->vq.vdev,
@@ -2339,6 +2335,18 @@ void vring_del_virtqueue(struct virtqueue *_vq)
 		kfree(vq->split.desc_state);
 		kfree(vq->split.desc_extra);
 	}
+}
+
+void vring_del_virtqueue(struct virtqueue *_vq)
+{
+	struct vring_virtqueue *vq = to_vvq(_vq);
+
+	spin_lock(&vq->vq.vdev->vqs_list_lock);
+	list_del(&_vq->list);
+	spin_unlock(&vq->vq.vdev->vqs_list_lock);
+
+	vring_free(_vq);
+
 	kfree(vq);
 }
 EXPORT_SYMBOL_GPL(vring_del_virtqueue);
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 06/32] virtio_ring: split: extract the logic of alloc queue

[Xuan Zhuo]

Separate the logic of split to create vring queue.

This feature is required for subsequent virtuqueue reset vring.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++------------
 1 file changed, 36 insertions(+), 17 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 33fddfb907a6..72d5ae063fa0 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -915,23 +915,15 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
 	return NULL;
 }
 
-static struct virtqueue *vring_create_virtqueue_split(
-	unsigned int index,
-	unsigned int num,
-	unsigned int vring_align,
-	struct virtio_device *vdev,
-	bool weak_barriers,
-	bool may_reduce_num,
-	bool context,
-	bool (*notify)(struct virtqueue *),
-	void (*callback)(struct virtqueue *),
-	const char *name)
+static void *vring_alloc_queue_split(struct virtio_device *vdev,
+				     dma_addr_t *dma_addr,
+				     u32 *n,
+				     unsigned int vring_align,
+				     bool weak_barriers,
+				     bool may_reduce_num)
 {
-	struct virtqueue *vq;
 	void *queue = NULL;
-	dma_addr_t dma_addr;
-	size_t queue_size_in_bytes;
-	struct vring vring;
+	u32 num = *n;
 
 	/* We assume num is a power of 2. */
 	if (num & (num - 1)) {
@@ -942,7 +934,7 @@ static struct virtqueue *vring_create_virtqueue_split(
 	/* TODO: allocate each queue chunk individually */
 	for (; num && vring_size(num, vring_align) > PAGE_SIZE; num /= 2) {
 		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
-					  &dma_addr,
+					  dma_addr,
 					  GFP_KERNEL|__GFP_NOWARN|__GFP_ZERO);
 		if (queue)
 			break;
@@ -956,11 +948,38 @@ static struct virtqueue *vring_create_virtqueue_split(
 	if (!queue) {
 		/* Try to get a single page. You are my only hope! */
 		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
-					  &dma_addr, GFP_KERNEL|__GFP_ZERO);
+					  dma_addr, GFP_KERNEL|__GFP_ZERO);
 	}
 	if (!queue)
 		return NULL;
 
+	*n = num;
+	return queue;
+}
+
+static struct virtqueue *vring_create_virtqueue_split(
+	unsigned int index,
+	unsigned int num,
+	unsigned int vring_align,
+	struct virtio_device *vdev,
+	bool weak_barriers,
+	bool may_reduce_num,
+	bool context,
+	bool (*notify)(struct virtqueue *),
+	void (*callback)(struct virtqueue *),
+	const char *name)
+{
+	size_t queue_size_in_bytes;
+	struct virtqueue *vq;
+	dma_addr_t dma_addr;
+	struct vring vring;
+	void *queue;
+
+	queue = vring_alloc_queue_split(vdev, &dma_addr, &num, vring_align,
+					weak_barriers, may_reduce_num);
+	if (!queue)
+		return NULL;
+
 	queue_size_in_bytes = vring_size(num, vring_align);
 	vring_init(&vring, num, queue, vring_align);
 
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 06/32] virtio_ring: split: extract the logic of alloc queue

[Xuan Zhuo]

Separate the logic of split to create vring queue.

This feature is required for subsequent virtuqueue reset vring.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++------------
 1 file changed, 36 insertions(+), 17 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 33fddfb907a6..72d5ae063fa0 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -915,23 +915,15 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
 	return NULL;
 }
 
-static struct virtqueue *vring_create_virtqueue_split(
-	unsigned int index,
-	unsigned int num,
-	unsigned int vring_align,
-	struct virtio_device *vdev,
-	bool weak_barriers,
-	bool may_reduce_num,
-	bool context,
-	bool (*notify)(struct virtqueue *),
-	void (*callback)(struct virtqueue *),
-	const char *name)
+static void *vring_alloc_queue_split(struct virtio_device *vdev,
+				     dma_addr_t *dma_addr,
+				     u32 *n,
+				     unsigned int vring_align,
+				     bool weak_barriers,
+				     bool may_reduce_num)
 {
-	struct virtqueue *vq;
 	void *queue = NULL;
-	dma_addr_t dma_addr;
-	size_t queue_size_in_bytes;
-	struct vring vring;
+	u32 num = *n;
 
 	/* We assume num is a power of 2. */
 	if (num & (num - 1)) {
@@ -942,7 +934,7 @@ static struct virtqueue *vring_create_virtqueue_split(
 	/* TODO: allocate each queue chunk individually */
 	for (; num && vring_size(num, vring_align) > PAGE_SIZE; num /= 2) {
 		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
-					  &dma_addr,
+					  dma_addr,
 					  GFP_KERNEL|__GFP_NOWARN|__GFP_ZERO);
 		if (queue)
 			break;
@@ -956,11 +948,38 @@ static struct virtqueue *vring_create_virtqueue_split(
 	if (!queue) {
 		/* Try to get a single page. You are my only hope! */
 		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
-					  &dma_addr, GFP_KERNEL|__GFP_ZERO);
+					  dma_addr, GFP_KERNEL|__GFP_ZERO);
 	}
 	if (!queue)
 		return NULL;
 
+	*n = num;
+	return queue;
+}
+
+static struct virtqueue *vring_create_virtqueue_split(
+	unsigned int index,
+	unsigned int num,
+	unsigned int vring_align,
+	struct virtio_device *vdev,
+	bool weak_barriers,
+	bool may_reduce_num,
+	bool context,
+	bool (*notify)(struct virtqueue *),
+	void (*callback)(struct virtqueue *),
+	const char *name)
+{
+	size_t queue_size_in_bytes;
+	struct virtqueue *vq;
+	dma_addr_t dma_addr;
+	struct vring vring;
+	void *queue;
+
+	queue = vring_alloc_queue_split(vdev, &dma_addr, &num, vring_align,
+					weak_barriers, may_reduce_num);
+	if (!queue)
+		return NULL;
+
 	queue_size_in_bytes = vring_size(num, vring_align);
 	vring_init(&vring, num, queue, vring_align);
 
-- 
2.31.0

[PATCH v9 06/32] virtio_ring: split: extract the logic of alloc queue

[Xuan Zhuo]

Separate the logic of split to create vring queue.

This feature is required for subsequent virtuqueue reset vring.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++------------
 1 file changed, 36 insertions(+), 17 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 33fddfb907a6..72d5ae063fa0 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -915,23 +915,15 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
 	return NULL;
 }
 
-static struct virtqueue *vring_create_virtqueue_split(
-	unsigned int index,
-	unsigned int num,
-	unsigned int vring_align,
-	struct virtio_device *vdev,
-	bool weak_barriers,
-	bool may_reduce_num,
-	bool context,
-	bool (*notify)(struct virtqueue *),
-	void (*callback)(struct virtqueue *),
-	const char *name)
+static void *vring_alloc_queue_split(struct virtio_device *vdev,
+				     dma_addr_t *dma_addr,
+				     u32 *n,
+				     unsigned int vring_align,
+				     bool weak_barriers,
+				     bool may_reduce_num)
 {
-	struct virtqueue *vq;
 	void *queue = NULL;
-	dma_addr_t dma_addr;
-	size_t queue_size_in_bytes;
-	struct vring vring;
+	u32 num = *n;
 
 	/* We assume num is a power of 2. */
 	if (num & (num - 1)) {
@@ -942,7 +934,7 @@ static struct virtqueue *vring_create_virtqueue_split(
 	/* TODO: allocate each queue chunk individually */
 	for (; num && vring_size(num, vring_align) > PAGE_SIZE; num /= 2) {
 		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
-					  &dma_addr,
+					  dma_addr,
 					  GFP_KERNEL|__GFP_NOWARN|__GFP_ZERO);
 		if (queue)
 			break;
@@ -956,11 +948,38 @@ static struct virtqueue *vring_create_virtqueue_split(
 	if (!queue) {
 		/* Try to get a single page. You are my only hope! */
 		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
-					  &dma_addr, GFP_KERNEL|__GFP_ZERO);
+					  dma_addr, GFP_KERNEL|__GFP_ZERO);
 	}
 	if (!queue)
 		return NULL;
 
+	*n = num;
+	return queue;
+}
+
+static struct virtqueue *vring_create_virtqueue_split(
+	unsigned int index,
+	unsigned int num,
+	unsigned int vring_align,
+	struct virtio_device *vdev,
+	bool weak_barriers,
+	bool may_reduce_num,
+	bool context,
+	bool (*notify)(struct virtqueue *),
+	void (*callback)(struct virtqueue *),
+	const char *name)
+{
+	size_t queue_size_in_bytes;
+	struct virtqueue *vq;
+	dma_addr_t dma_addr;
+	struct vring vring;
+	void *queue;
+
+	queue = vring_alloc_queue_split(vdev, &dma_addr, &num, vring_align,
+					weak_barriers, may_reduce_num);
+	if (!queue)
+		return NULL;
+
 	queue_size_in_bytes = vring_size(num, vring_align);
 	vring_init(&vring, num, queue, vring_align);
 
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 07/32] virtio_ring: split: extract the logic of alloc state and extra

[Xuan Zhuo]

Separate the logic of creating desc_state, desc_extra, and subsequent
patches will call it independently.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++++----------
 1 file changed, 38 insertions(+), 15 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 72d5ae063fa0..6de67439cb57 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -198,6 +198,7 @@ struct vring_virtqueue {
 #endif
 };
 
+static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
 
 /*
  * Helpers.
@@ -915,6 +916,33 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
 	return NULL;
 }
 
+static int vring_alloc_state_extra_split(u32 num,
+					 struct vring_desc_state_split **desc_state,
+					 struct vring_desc_extra **desc_extra)
+{
+	struct vring_desc_state_split *state;
+	struct vring_desc_extra *extra;
+
+	state = kmalloc_array(num, sizeof(struct vring_desc_state_split), GFP_KERNEL);
+	if (!state)
+		goto err_state;
+
+	extra = vring_alloc_desc_extra(num);
+	if (!extra)
+		goto err_extra;
+
+	memset(state, 0, num * sizeof(struct vring_desc_state_split));
+
+	*desc_state = state;
+	*desc_extra = extra;
+	return 0;
+
+err_extra:
+	kfree(state);
+err_state:
+	return -ENOMEM;
+}
+
 static void *vring_alloc_queue_split(struct virtio_device *vdev,
 				     dma_addr_t *dma_addr,
 				     u32 *n,
@@ -2196,7 +2224,10 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 					void (*callback)(struct virtqueue *),
 					const char *name)
 {
+	struct vring_desc_state_split *state;
+	struct vring_desc_extra *extra;
 	struct vring_virtqueue *vq;
+	int err;
 
 	if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED))
 		return NULL;
@@ -2246,30 +2277,22 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 					vq->split.avail_flags_shadow);
 	}
 
-	vq->split.desc_state = kmalloc_array(vring.num,
-			sizeof(struct vring_desc_state_split), GFP_KERNEL);
-	if (!vq->split.desc_state)
-		goto err_state;
+	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
+	if (err) {
+		kfree(vq);
+		return NULL;
+	}
 
-	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
-	if (!vq->split.desc_extra)
-		goto err_extra;
+	vq->split.desc_state = state;
+	vq->split.desc_extra = extra;
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
-	memset(vq->split.desc_state, 0, vring.num *
-			sizeof(struct vring_desc_state_split));
 
 	spin_lock(&vdev->vqs_list_lock);
 	list_add_tail(&vq->vq.list, &vdev->vqs);
 	spin_unlock(&vdev->vqs_list_lock);
 	return &vq->vq;
-
-err_extra:
-	kfree(vq->split.desc_state);
-err_state:
-	kfree(vq);
-	return NULL;
 }
 EXPORT_SYMBOL_GPL(__vring_new_virtqueue);
 
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 07/32] virtio_ring: split: extract the logic of alloc state and extra

[Xuan Zhuo]

Separate the logic of creating desc_state, desc_extra, and subsequent
patches will call it independently.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++++----------
 1 file changed, 38 insertions(+), 15 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 72d5ae063fa0..6de67439cb57 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -198,6 +198,7 @@ struct vring_virtqueue {
 #endif
 };
 
+static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
 
 /*
  * Helpers.
@@ -915,6 +916,33 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
 	return NULL;
 }
 
+static int vring_alloc_state_extra_split(u32 num,
+					 struct vring_desc_state_split **desc_state,
+					 struct vring_desc_extra **desc_extra)
+{
+	struct vring_desc_state_split *state;
+	struct vring_desc_extra *extra;
+
+	state = kmalloc_array(num, sizeof(struct vring_desc_state_split), GFP_KERNEL);
+	if (!state)
+		goto err_state;
+
+	extra = vring_alloc_desc_extra(num);
+	if (!extra)
+		goto err_extra;
+
+	memset(state, 0, num * sizeof(struct vring_desc_state_split));
+
+	*desc_state = state;
+	*desc_extra = extra;
+	return 0;
+
+err_extra:
+	kfree(state);
+err_state:
+	return -ENOMEM;
+}
+
 static void *vring_alloc_queue_split(struct virtio_device *vdev,
 				     dma_addr_t *dma_addr,
 				     u32 *n,
@@ -2196,7 +2224,10 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 					void (*callback)(struct virtqueue *),
 					const char *name)
 {
+	struct vring_desc_state_split *state;
+	struct vring_desc_extra *extra;
 	struct vring_virtqueue *vq;
+	int err;
 
 	if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED))
 		return NULL;
@@ -2246,30 +2277,22 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 					vq->split.avail_flags_shadow);
 	}
 
-	vq->split.desc_state = kmalloc_array(vring.num,
-			sizeof(struct vring_desc_state_split), GFP_KERNEL);
-	if (!vq->split.desc_state)
-		goto err_state;
+	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
+	if (err) {
+		kfree(vq);
+		return NULL;
+	}
 
-	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
-	if (!vq->split.desc_extra)
-		goto err_extra;
+	vq->split.desc_state = state;
+	vq->split.desc_extra = extra;
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
-	memset(vq->split.desc_state, 0, vring.num *
-			sizeof(struct vring_desc_state_split));
 
 	spin_lock(&vdev->vqs_list_lock);
 	list_add_tail(&vq->vq.list, &vdev->vqs);
 	spin_unlock(&vdev->vqs_list_lock);
 	return &vq->vq;
-
-err_extra:
-	kfree(vq->split.desc_state);
-err_state:
-	kfree(vq);
-	return NULL;
 }
 EXPORT_SYMBOL_GPL(__vring_new_virtqueue);
 
-- 
2.31.0

[PATCH v9 07/32] virtio_ring: split: extract the logic of alloc state and extra

[Xuan Zhuo]

Separate the logic of creating desc_state, desc_extra, and subsequent
patches will call it independently.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++++----------
 1 file changed, 38 insertions(+), 15 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 72d5ae063fa0..6de67439cb57 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -198,6 +198,7 @@ struct vring_virtqueue {
 #endif
 };
 
+static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
 
 /*
  * Helpers.
@@ -915,6 +916,33 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
 	return NULL;
 }
 
+static int vring_alloc_state_extra_split(u32 num,
+					 struct vring_desc_state_split **desc_state,
+					 struct vring_desc_extra **desc_extra)
+{
+	struct vring_desc_state_split *state;
+	struct vring_desc_extra *extra;
+
+	state = kmalloc_array(num, sizeof(struct vring_desc_state_split), GFP_KERNEL);
+	if (!state)
+		goto err_state;
+
+	extra = vring_alloc_desc_extra(num);
+	if (!extra)
+		goto err_extra;
+
+	memset(state, 0, num * sizeof(struct vring_desc_state_split));
+
+	*desc_state = state;
+	*desc_extra = extra;
+	return 0;
+
+err_extra:
+	kfree(state);
+err_state:
+	return -ENOMEM;
+}
+
 static void *vring_alloc_queue_split(struct virtio_device *vdev,
 				     dma_addr_t *dma_addr,
 				     u32 *n,
@@ -2196,7 +2224,10 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 					void (*callback)(struct virtqueue *),
 					const char *name)
 {
+	struct vring_desc_state_split *state;
+	struct vring_desc_extra *extra;
 	struct vring_virtqueue *vq;
+	int err;
 
 	if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED))
 		return NULL;
@@ -2246,30 +2277,22 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 					vq->split.avail_flags_shadow);
 	}
 
-	vq->split.desc_state = kmalloc_array(vring.num,
-			sizeof(struct vring_desc_state_split), GFP_KERNEL);
-	if (!vq->split.desc_state)
-		goto err_state;
+	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
+	if (err) {
+		kfree(vq);
+		return NULL;
+	}
 
-	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
-	if (!vq->split.desc_extra)
-		goto err_extra;
+	vq->split.desc_state = state;
+	vq->split.desc_extra = extra;
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
-	memset(vq->split.desc_state, 0, vring.num *
-			sizeof(struct vring_desc_state_split));
 
 	spin_lock(&vdev->vqs_list_lock);
 	list_add_tail(&vq->vq.list, &vdev->vqs);
 	spin_unlock(&vdev->vqs_list_lock);
 	return &vq->vq;
-
-err_extra:
-	kfree(vq->split.desc_state);
-err_state:
-	kfree(vq);
-	return NULL;
 }
 EXPORT_SYMBOL_GPL(__vring_new_virtqueue);
 
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 08/32] virtio_ring: split: extract the logic of attach vring

[Xuan Zhuo]

Separate the logic of attach vring, subsequent patches will call it
separately.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 6de67439cb57..083f2992ba0d 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -916,6 +916,19 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
 	return NULL;
 }
 
+static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
+					 struct vring vring,
+					 struct vring_desc_state_split *desc_state,
+					 struct vring_desc_extra *desc_extra)
+{
+	vq->split.vring = vring;
+	vq->split.queue_dma_addr = 0;
+	vq->split.queue_size_in_bytes = 0;
+
+	vq->split.desc_state = desc_state;
+	vq->split.desc_extra = desc_extra;
+}
+
 static int vring_alloc_state_extra_split(u32 num,
 					 struct vring_desc_state_split **desc_state,
 					 struct vring_desc_extra **desc_extra)
@@ -2262,10 +2275,6 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
 		vq->weak_barriers = false;
 
-	vq->split.queue_dma_addr = 0;
-	vq->split.queue_size_in_bytes = 0;
-
-	vq->split.vring = vring;
 	vq->split.avail_flags_shadow = 0;
 	vq->split.avail_idx_shadow = 0;
 
@@ -2283,8 +2292,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 		return NULL;
 	}
 
-	vq->split.desc_state = state;
-	vq->split.desc_extra = extra;
+	vring_virtqueue_attach_split(vq, vring, state, extra);
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 08/32] virtio_ring: split: extract the logic of attach vring

[Xuan Zhuo]

Separate the logic of attach vring, subsequent patches will call it
separately.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 6de67439cb57..083f2992ba0d 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -916,6 +916,19 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
 	return NULL;
 }
 
+static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
+					 struct vring vring,
+					 struct vring_desc_state_split *desc_state,
+					 struct vring_desc_extra *desc_extra)
+{
+	vq->split.vring = vring;
+	vq->split.queue_dma_addr = 0;
+	vq->split.queue_size_in_bytes = 0;
+
+	vq->split.desc_state = desc_state;
+	vq->split.desc_extra = desc_extra;
+}
+
 static int vring_alloc_state_extra_split(u32 num,
 					 struct vring_desc_state_split **desc_state,
 					 struct vring_desc_extra **desc_extra)
@@ -2262,10 +2275,6 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
 		vq->weak_barriers = false;
 
-	vq->split.queue_dma_addr = 0;
-	vq->split.queue_size_in_bytes = 0;
-
-	vq->split.vring = vring;
 	vq->split.avail_flags_shadow = 0;
 	vq->split.avail_idx_shadow = 0;
 
@@ -2283,8 +2292,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 		return NULL;
 	}
 
-	vq->split.desc_state = state;
-	vq->split.desc_extra = extra;
+	vring_virtqueue_attach_split(vq, vring, state, extra);
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
-- 
2.31.0

[PATCH v9 08/32] virtio_ring: split: extract the logic of attach vring

[Xuan Zhuo]

Separate the logic of attach vring, subsequent patches will call it
separately.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 6de67439cb57..083f2992ba0d 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -916,6 +916,19 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
 	return NULL;
 }
 
+static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
+					 struct vring vring,
+					 struct vring_desc_state_split *desc_state,
+					 struct vring_desc_extra *desc_extra)
+{
+	vq->split.vring = vring;
+	vq->split.queue_dma_addr = 0;
+	vq->split.queue_size_in_bytes = 0;
+
+	vq->split.desc_state = desc_state;
+	vq->split.desc_extra = desc_extra;
+}
+
 static int vring_alloc_state_extra_split(u32 num,
 					 struct vring_desc_state_split **desc_state,
 					 struct vring_desc_extra **desc_extra)
@@ -2262,10 +2275,6 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
 		vq->weak_barriers = false;
 
-	vq->split.queue_dma_addr = 0;
-	vq->split.queue_size_in_bytes = 0;
-
-	vq->split.vring = vring;
 	vq->split.avail_flags_shadow = 0;
 	vq->split.avail_idx_shadow = 0;
 
@@ -2283,8 +2292,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 		return NULL;
 	}
 
-	vq->split.desc_state = state;
-	vq->split.desc_extra = extra;
+	vring_virtqueue_attach_split(vq, vring, state, extra);
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 09/32] virtio_ring: split: extract the logic of vq init

[Xuan Zhuo]

Separate the logic of initializing vq, and subsequent patches will call
it separately.

The feature of this part is that it does not depend on the information
passed by the upper layer and can be called repeatedly.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 68 ++++++++++++++++++++----------------
 1 file changed, 38 insertions(+), 30 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 083f2992ba0d..874f878087a3 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -916,6 +916,43 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
 	return NULL;
 }
 
+static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
+				       struct virtio_device *vdev,
+				       bool own_ring)
+{
+	vq->packed_ring = false;
+	vq->vq.num_free = vq->split.vring.num;
+	vq->we_own_ring = own_ring;
+	vq->broken = false;
+	vq->last_used_idx = 0;
+	vq->event_triggered = false;
+	vq->num_added = 0;
+	vq->use_dma_api = vring_use_dma_api(vdev);
+#ifdef DEBUG
+	vq->in_use = false;
+	vq->last_add_time_valid = false;
+#endif
+
+	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
+
+	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
+		vq->weak_barriers = false;
+
+	vq->split.avail_flags_shadow = 0;
+	vq->split.avail_idx_shadow = 0;
+
+	/* No callback?  Tell other side not to bother us. */
+	if (!vq->vq.callback) {
+		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
+		if (!vq->event)
+			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
+					vq->split.avail_flags_shadow);
+	}
+
+	/* Put everything in free lists. */
+	vq->free_head = 0;
+}
+
 static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
 					 struct vring vring,
 					 struct vring_desc_state_split *desc_state,
@@ -2249,42 +2286,15 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	if (!vq)
 		return NULL;
 
-	vq->packed_ring = false;
 	vq->vq.callback = callback;
 	vq->vq.vdev = vdev;
 	vq->vq.name = name;
-	vq->vq.num_free = vring.num;
 	vq->vq.index = index;
-	vq->we_own_ring = false;
 	vq->notify = notify;
 	vq->weak_barriers = weak_barriers;
-	vq->broken = false;
-	vq->last_used_idx = 0;
-	vq->event_triggered = false;
-	vq->num_added = 0;
-	vq->use_dma_api = vring_use_dma_api(vdev);
-#ifdef DEBUG
-	vq->in_use = false;
-	vq->last_add_time_valid = false;
-#endif
 
 	vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) &&
 		!context;
-	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
-
-	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
-		vq->weak_barriers = false;
-
-	vq->split.avail_flags_shadow = 0;
-	vq->split.avail_idx_shadow = 0;
-
-	/* No callback?  Tell other side not to bother us. */
-	if (!callback) {
-		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
-		if (!vq->event)
-			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
-					vq->split.avail_flags_shadow);
-	}
 
 	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
 	if (err) {
@@ -2293,9 +2303,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	}
 
 	vring_virtqueue_attach_split(vq, vring, state, extra);
-
-	/* Put everything in free lists. */
-	vq->free_head = 0;
+	vring_virtqueue_init_split(vq, vdev, false);
 
 	spin_lock(&vdev->vqs_list_lock);
 	list_add_tail(&vq->vq.list, &vdev->vqs);
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 09/32] virtio_ring: split: extract the logic of vq init

[Xuan Zhuo]

Separate the logic of initializing vq, and subsequent patches will call
it separately.

The feature of this part is that it does not depend on the information
passed by the upper layer and can be called repeatedly.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 68 ++++++++++++++++++++----------------
 1 file changed, 38 insertions(+), 30 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 083f2992ba0d..874f878087a3 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -916,6 +916,43 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
 	return NULL;
 }
 
+static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
+				       struct virtio_device *vdev,
+				       bool own_ring)
+{
+	vq->packed_ring = false;
+	vq->vq.num_free = vq->split.vring.num;
+	vq->we_own_ring = own_ring;
+	vq->broken = false;
+	vq->last_used_idx = 0;
+	vq->event_triggered = false;
+	vq->num_added = 0;
+	vq->use_dma_api = vring_use_dma_api(vdev);
+#ifdef DEBUG
+	vq->in_use = false;
+	vq->last_add_time_valid = false;
+#endif
+
+	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
+
+	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
+		vq->weak_barriers = false;
+
+	vq->split.avail_flags_shadow = 0;
+	vq->split.avail_idx_shadow = 0;
+
+	/* No callback?  Tell other side not to bother us. */
+	if (!vq->vq.callback) {
+		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
+		if (!vq->event)
+			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
+					vq->split.avail_flags_shadow);
+	}
+
+	/* Put everything in free lists. */
+	vq->free_head = 0;
+}
+
 static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
 					 struct vring vring,
 					 struct vring_desc_state_split *desc_state,
@@ -2249,42 +2286,15 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	if (!vq)
 		return NULL;
 
-	vq->packed_ring = false;
 	vq->vq.callback = callback;
 	vq->vq.vdev = vdev;
 	vq->vq.name = name;
-	vq->vq.num_free = vring.num;
 	vq->vq.index = index;
-	vq->we_own_ring = false;
 	vq->notify = notify;
 	vq->weak_barriers = weak_barriers;
-	vq->broken = false;
-	vq->last_used_idx = 0;
-	vq->event_triggered = false;
-	vq->num_added = 0;
-	vq->use_dma_api = vring_use_dma_api(vdev);
-#ifdef DEBUG
-	vq->in_use = false;
-	vq->last_add_time_valid = false;
-#endif
 
 	vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) &&
 		!context;
-	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
-
-	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
-		vq->weak_barriers = false;
-
-	vq->split.avail_flags_shadow = 0;
-	vq->split.avail_idx_shadow = 0;
-
-	/* No callback?  Tell other side not to bother us. */
-	if (!callback) {
-		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
-		if (!vq->event)
-			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
-					vq->split.avail_flags_shadow);
-	}
 
 	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
 	if (err) {
@@ -2293,9 +2303,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	}
 
 	vring_virtqueue_attach_split(vq, vring, state, extra);
-
-	/* Put everything in free lists. */
-	vq->free_head = 0;
+	vring_virtqueue_init_split(vq, vdev, false);
 
 	spin_lock(&vdev->vqs_list_lock);
 	list_add_tail(&vq->vq.list, &vdev->vqs);
-- 
2.31.0

[PATCH v9 09/32] virtio_ring: split: extract the logic of vq init

[Xuan Zhuo]

Separate the logic of initializing vq, and subsequent patches will call
it separately.

The feature of this part is that it does not depend on the information
passed by the upper layer and can be called repeatedly.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 68 ++++++++++++++++++++----------------
 1 file changed, 38 insertions(+), 30 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 083f2992ba0d..874f878087a3 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -916,6 +916,43 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
 	return NULL;
 }
 
+static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
+				       struct virtio_device *vdev,
+				       bool own_ring)
+{
+	vq->packed_ring = false;
+	vq->vq.num_free = vq->split.vring.num;
+	vq->we_own_ring = own_ring;
+	vq->broken = false;
+	vq->last_used_idx = 0;
+	vq->event_triggered = false;
+	vq->num_added = 0;
+	vq->use_dma_api = vring_use_dma_api(vdev);
+#ifdef DEBUG
+	vq->in_use = false;
+	vq->last_add_time_valid = false;
+#endif
+
+	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
+
+	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
+		vq->weak_barriers = false;
+
+	vq->split.avail_flags_shadow = 0;
+	vq->split.avail_idx_shadow = 0;
+
+	/* No callback?  Tell other side not to bother us. */
+	if (!vq->vq.callback) {
+		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
+		if (!vq->event)
+			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
+					vq->split.avail_flags_shadow);
+	}
+
+	/* Put everything in free lists. */
+	vq->free_head = 0;
+}
+
 static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
 					 struct vring vring,
 					 struct vring_desc_state_split *desc_state,
@@ -2249,42 +2286,15 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	if (!vq)
 		return NULL;
 
-	vq->packed_ring = false;
 	vq->vq.callback = callback;
 	vq->vq.vdev = vdev;
 	vq->vq.name = name;
-	vq->vq.num_free = vring.num;
 	vq->vq.index = index;
-	vq->we_own_ring = false;
 	vq->notify = notify;
 	vq->weak_barriers = weak_barriers;
-	vq->broken = false;
-	vq->last_used_idx = 0;
-	vq->event_triggered = false;
-	vq->num_added = 0;
-	vq->use_dma_api = vring_use_dma_api(vdev);
-#ifdef DEBUG
-	vq->in_use = false;
-	vq->last_add_time_valid = false;
-#endif
 
 	vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) &&
 		!context;
-	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
-
-	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
-		vq->weak_barriers = false;
-
-	vq->split.avail_flags_shadow = 0;
-	vq->split.avail_idx_shadow = 0;
-
-	/* No callback?  Tell other side not to bother us. */
-	if (!callback) {
-		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
-		if (!vq->event)
-			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
-					vq->split.avail_flags_shadow);
-	}
 
 	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
 	if (err) {
@@ -2293,9 +2303,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	}
 
 	vring_virtqueue_attach_split(vq, vring, state, extra);
-
-	/* Put everything in free lists. */
-	vq->free_head = 0;
+	vring_virtqueue_init_split(vq, vdev, false);
 
 	spin_lock(&vdev->vqs_list_lock);
 	list_add_tail(&vq->vq.list, &vdev->vqs);
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 10/32] virtio_ring: split: introduce virtqueue_reinit_split()

[Xuan Zhuo]

Introduce a function to initialize vq without allocating new ring,
desc_state, desc_extra.

Subsequent patches will call this function after reset vq to
reinitialize vq.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 874f878087a3..3dc6ace2ba7a 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -953,6 +953,25 @@ static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
 	vq->free_head = 0;
 }
 
+static void virtqueue_reinit_split(struct vring_virtqueue *vq)
+{
+	struct virtio_device *vdev = vq->vq.vdev;
+	int size, i;
+
+	memset(vq->split.vring.desc, 0, vq->split.queue_size_in_bytes);
+
+	size = sizeof(struct vring_desc_state_split) * vq->split.vring.num;
+	memset(vq->split.desc_state, 0, size);
+
+	size = sizeof(struct vring_desc_extra) * vq->split.vring.num;
+	memset(vq->split.desc_extra, 0, size);
+
+	for (i = 0; i < vq->split.vring.num - 1; i++)
+		vq->split.desc_extra[i].next = i + 1;
+
+	vring_virtqueue_init_split(vq, vdev, true);
+}
+
 static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
 					 struct vring vring,
 					 struct vring_desc_state_split *desc_state,
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 10/32] virtio_ring: split: introduce virtqueue_reinit_split()

[Xuan Zhuo]

Introduce a function to initialize vq without allocating new ring,
desc_state, desc_extra.

Subsequent patches will call this function after reset vq to
reinitialize vq.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 874f878087a3..3dc6ace2ba7a 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -953,6 +953,25 @@ static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
 	vq->free_head = 0;
 }
 
+static void virtqueue_reinit_split(struct vring_virtqueue *vq)
+{
+	struct virtio_device *vdev = vq->vq.vdev;
+	int size, i;
+
+	memset(vq->split.vring.desc, 0, vq->split.queue_size_in_bytes);
+
+	size = sizeof(struct vring_desc_state_split) * vq->split.vring.num;
+	memset(vq->split.desc_state, 0, size);
+
+	size = sizeof(struct vring_desc_extra) * vq->split.vring.num;
+	memset(vq->split.desc_extra, 0, size);
+
+	for (i = 0; i < vq->split.vring.num - 1; i++)
+		vq->split.desc_extra[i].next = i + 1;
+
+	vring_virtqueue_init_split(vq, vdev, true);
+}
+
 static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
 					 struct vring vring,
 					 struct vring_desc_state_split *desc_state,
-- 
2.31.0

[PATCH v9 10/32] virtio_ring: split: introduce virtqueue_reinit_split()

[Xuan Zhuo]

Introduce a function to initialize vq without allocating new ring,
desc_state, desc_extra.

Subsequent patches will call this function after reset vq to
reinitialize vq.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 874f878087a3..3dc6ace2ba7a 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -953,6 +953,25 @@ static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
 	vq->free_head = 0;
 }
 
+static void virtqueue_reinit_split(struct vring_virtqueue *vq)
+{
+	struct virtio_device *vdev = vq->vq.vdev;
+	int size, i;
+
+	memset(vq->split.vring.desc, 0, vq->split.queue_size_in_bytes);
+
+	size = sizeof(struct vring_desc_state_split) * vq->split.vring.num;
+	memset(vq->split.desc_state, 0, size);
+
+	size = sizeof(struct vring_desc_extra) * vq->split.vring.num;
+	memset(vq->split.desc_extra, 0, size);
+
+	for (i = 0; i < vq->split.vring.num - 1; i++)
+		vq->split.desc_extra[i].next = i + 1;
+
+	vring_virtqueue_init_split(vq, vdev, true);
+}
+
 static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
 					 struct vring vring,
 					 struct vring_desc_state_split *desc_state,
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 11/32] virtio_ring: split: introduce virtqueue_resize_split()

[Xuan Zhuo]

virtio ring split supports resize.

Only after the new vring is successfully allocated based on the new num,
we will release the old vring. In any case, an error is returned,
indicating that the vring still points to the old vring.

In the case of an error, the caller must
re-initialize(virtqueue_reinit_split()) the virtqueue to ensure that the
vring can be used.

In addition, vring_align, may_reduce_num are necessary for reallocating
vring, so they are retained for creating vq.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 47 ++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 3dc6ace2ba7a..33864134a744 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -139,6 +139,12 @@ struct vring_virtqueue {
 			/* DMA address and size information */
 			dma_addr_t queue_dma_addr;
 			size_t queue_size_in_bytes;
+
+			/* The parameters for creating vrings are reserved for
+			 * creating new vring.
+			 */
+			u32 vring_align;
+			bool may_reduce_num;
 		} split;
 
 		/* Available for packed ring */
@@ -199,6 +205,7 @@ struct vring_virtqueue {
 };
 
 static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
+static void vring_free(struct virtqueue *_vq);
 
 /*
  * Helpers.
@@ -1088,6 +1095,8 @@ static struct virtqueue *vring_create_virtqueue_split(
 		return NULL;
 	}
 
+	to_vvq(vq)->split.vring_align = vring_align;
+	to_vvq(vq)->split.may_reduce_num = may_reduce_num;
 	to_vvq(vq)->split.queue_dma_addr = dma_addr;
 	to_vvq(vq)->split.queue_size_in_bytes = queue_size_in_bytes;
 	to_vvq(vq)->we_own_ring = true;
@@ -1095,6 +1104,44 @@ static struct virtqueue *vring_create_virtqueue_split(
 	return vq;
 }
 
+static int virtqueue_resize_split(struct virtqueue *_vq, u32 num)
+{
+	struct vring_virtqueue *vq = to_vvq(_vq);
+	struct virtio_device *vdev = _vq->vdev;
+	struct vring_desc_state_split *state;
+	struct vring_desc_extra *extra;
+	size_t queue_size_in_bytes;
+	dma_addr_t dma_addr;
+	struct vring vring;
+	int err = -ENOMEM;
+	void *queue;
+
+	queue = vring_alloc_queue_split(vdev, &dma_addr, &num,
+					vq->split.vring_align,
+					vq->weak_barriers,
+					vq->split.may_reduce_num);
+	if (!queue)
+		return -ENOMEM;
+
+	queue_size_in_bytes = vring_size(num, vq->split.vring_align);
+
+	err = vring_alloc_state_extra_split(num, &state, &extra);
+	if (err) {
+		vring_free_queue(vdev, queue_size_in_bytes, queue, dma_addr);
+		return -ENOMEM;
+	}
+
+	vring_free(&vq->vq);
+
+	vring_init(&vring, num, queue, vq->split.vring_align);
+	vring_virtqueue_attach_split(vq, vring, state, extra);
+	vq->split.queue_dma_addr = dma_addr;
+	vq->split.queue_size_in_bytes = queue_size_in_bytes;
+
+	vring_virtqueue_init_split(vq, vdev, true);
+	return 0;
+}
+
 
 /*
  * Packed ring specific functions - *_packed().
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 11/32] virtio_ring: split: introduce virtqueue_resize_split()

[Xuan Zhuo]

virtio ring split supports resize.

Only after the new vring is successfully allocated based on the new num,
we will release the old vring. In any case, an error is returned,
indicating that the vring still points to the old vring.

In the case of an error, the caller must
re-initialize(virtqueue_reinit_split()) the virtqueue to ensure that the
vring can be used.

In addition, vring_align, may_reduce_num are necessary for reallocating
vring, so they are retained for creating vq.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 47 ++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 3dc6ace2ba7a..33864134a744 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -139,6 +139,12 @@ struct vring_virtqueue {
 			/* DMA address and size information */
 			dma_addr_t queue_dma_addr;
 			size_t queue_size_in_bytes;
+
+			/* The parameters for creating vrings are reserved for
+			 * creating new vring.
+			 */
+			u32 vring_align;
+			bool may_reduce_num;
 		} split;
 
 		/* Available for packed ring */
@@ -199,6 +205,7 @@ struct vring_virtqueue {
 };
 
 static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
+static void vring_free(struct virtqueue *_vq);
 
 /*
  * Helpers.
@@ -1088,6 +1095,8 @@ static struct virtqueue *vring_create_virtqueue_split(
 		return NULL;
 	}
 
+	to_vvq(vq)->split.vring_align = vring_align;
+	to_vvq(vq)->split.may_reduce_num = may_reduce_num;
 	to_vvq(vq)->split.queue_dma_addr = dma_addr;
 	to_vvq(vq)->split.queue_size_in_bytes = queue_size_in_bytes;
 	to_vvq(vq)->we_own_ring = true;
@@ -1095,6 +1104,44 @@ static struct virtqueue *vring_create_virtqueue_split(
 	return vq;
 }
 
+static int virtqueue_resize_split(struct virtqueue *_vq, u32 num)
+{
+	struct vring_virtqueue *vq = to_vvq(_vq);
+	struct virtio_device *vdev = _vq->vdev;
+	struct vring_desc_state_split *state;
+	struct vring_desc_extra *extra;
+	size_t queue_size_in_bytes;
+	dma_addr_t dma_addr;
+	struct vring vring;
+	int err = -ENOMEM;
+	void *queue;
+
+	queue = vring_alloc_queue_split(vdev, &dma_addr, &num,
+					vq->split.vring_align,
+					vq->weak_barriers,
+					vq->split.may_reduce_num);
+	if (!queue)
+		return -ENOMEM;
+
+	queue_size_in_bytes = vring_size(num, vq->split.vring_align);
+
+	err = vring_alloc_state_extra_split(num, &state, &extra);
+	if (err) {
+		vring_free_queue(vdev, queue_size_in_bytes, queue, dma_addr);
+		return -ENOMEM;
+	}
+
+	vring_free(&vq->vq);
+
+	vring_init(&vring, num, queue, vq->split.vring_align);
+	vring_virtqueue_attach_split(vq, vring, state, extra);
+	vq->split.queue_dma_addr = dma_addr;
+	vq->split.queue_size_in_bytes = queue_size_in_bytes;
+
+	vring_virtqueue_init_split(vq, vdev, true);
+	return 0;
+}
+
 
 /*
  * Packed ring specific functions - *_packed().
-- 
2.31.0

[PATCH v9 11/32] virtio_ring: split: introduce virtqueue_resize_split()

[Xuan Zhuo]

virtio ring split supports resize.

Only after the new vring is successfully allocated based on the new num,
we will release the old vring. In any case, an error is returned,
indicating that the vring still points to the old vring.

In the case of an error, the caller must
re-initialize(virtqueue_reinit_split()) the virtqueue to ensure that the
vring can be used.

In addition, vring_align, may_reduce_num are necessary for reallocating
vring, so they are retained for creating vq.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 47 ++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 3dc6ace2ba7a..33864134a744 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -139,6 +139,12 @@ struct vring_virtqueue {
 			/* DMA address and size information */
 			dma_addr_t queue_dma_addr;
 			size_t queue_size_in_bytes;
+
+			/* The parameters for creating vrings are reserved for
+			 * creating new vring.
+			 */
+			u32 vring_align;
+			bool may_reduce_num;
 		} split;
 
 		/* Available for packed ring */
@@ -199,6 +205,7 @@ struct vring_virtqueue {
 };
 
 static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
+static void vring_free(struct virtqueue *_vq);
 
 /*
  * Helpers.
@@ -1088,6 +1095,8 @@ static struct virtqueue *vring_create_virtqueue_split(
 		return NULL;
 	}
 
+	to_vvq(vq)->split.vring_align = vring_align;
+	to_vvq(vq)->split.may_reduce_num = may_reduce_num;
 	to_vvq(vq)->split.queue_dma_addr = dma_addr;
 	to_vvq(vq)->split.queue_size_in_bytes = queue_size_in_bytes;
 	to_vvq(vq)->we_own_ring = true;
@@ -1095,6 +1104,44 @@ static struct virtqueue *vring_create_virtqueue_split(
 	return vq;
 }
 
+static int virtqueue_resize_split(struct virtqueue *_vq, u32 num)
+{
+	struct vring_virtqueue *vq = to_vvq(_vq);
+	struct virtio_device *vdev = _vq->vdev;
+	struct vring_desc_state_split *state;
+	struct vring_desc_extra *extra;
+	size_t queue_size_in_bytes;
+	dma_addr_t dma_addr;
+	struct vring vring;
+	int err = -ENOMEM;
+	void *queue;
+
+	queue = vring_alloc_queue_split(vdev, &dma_addr, &num,
+					vq->split.vring_align,
+					vq->weak_barriers,
+					vq->split.may_reduce_num);
+	if (!queue)
+		return -ENOMEM;
+
+	queue_size_in_bytes = vring_size(num, vq->split.vring_align);
+
+	err = vring_alloc_state_extra_split(num, &state, &extra);
+	if (err) {
+		vring_free_queue(vdev, queue_size_in_bytes, queue, dma_addr);
+		return -ENOMEM;
+	}
+
+	vring_free(&vq->vq);
+
+	vring_init(&vring, num, queue, vq->split.vring_align);
+	vring_virtqueue_attach_split(vq, vring, state, extra);
+	vq->split.queue_dma_addr = dma_addr;
+	vq->split.queue_size_in_bytes = queue_size_in_bytes;
+
+	vring_virtqueue_init_split(vq, vdev, true);
+	return 0;
+}
+
 
 /*
  * Packed ring specific functions - *_packed().
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 12/32] virtio_ring: packed: extract the logic of alloc queue

[Xuan Zhuo]

Separate the logic of packed to create vring queue.

For the convenience of passing parameters, add a structure
vring_packed.

This feature is required for subsequent virtuqueue reset vring.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 70 ++++++++++++++++++++++++++++--------
 1 file changed, 56 insertions(+), 14 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 33864134a744..ea451ae2aaef 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1817,19 +1817,17 @@ static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
 	return desc_extra;
 }
 
-static struct virtqueue *vring_create_virtqueue_packed(
-	unsigned int index,
-	unsigned int num,
-	unsigned int vring_align,
-	struct virtio_device *vdev,
-	bool weak_barriers,
-	bool may_reduce_num,
-	bool context,
-	bool (*notify)(struct virtqueue *),
-	void (*callback)(struct virtqueue *),
-	const char *name)
+static int vring_alloc_queue_packed(struct virtio_device *vdev,
+				    u32 num,
+				    struct vring_packed_desc **_ring,
+				    struct vring_packed_desc_event **_driver,
+				    struct vring_packed_desc_event **_device,
+				    dma_addr_t *_ring_dma_addr,
+				    dma_addr_t *_driver_event_dma_addr,
+				    dma_addr_t *_device_event_dma_addr,
+				    size_t *_ring_size_in_bytes,
+				    size_t *_event_size_in_bytes)
 {
-	struct vring_virtqueue *vq;
 	struct vring_packed_desc *ring;
 	struct vring_packed_desc_event *driver, *device;
 	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
@@ -1857,6 +1855,52 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	if (!device)
 		goto err_device;
 
+	*_ring                   = ring;
+	*_driver                 = driver;
+	*_device                 = device;
+	*_ring_dma_addr          = ring_dma_addr;
+	*_driver_event_dma_addr  = driver_event_dma_addr;
+	*_device_event_dma_addr  = device_event_dma_addr;
+	*_ring_size_in_bytes     = ring_size_in_bytes;
+	*_event_size_in_bytes    = event_size_in_bytes;
+
+	return 0;
+
+err_device:
+	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
+
+err_driver:
+	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
+
+err_ring:
+	return -ENOMEM;
+}
+
+static struct virtqueue *vring_create_virtqueue_packed(
+	unsigned int index,
+	unsigned int num,
+	unsigned int vring_align,
+	struct virtio_device *vdev,
+	bool weak_barriers,
+	bool may_reduce_num,
+	bool context,
+	bool (*notify)(struct virtqueue *),
+	void (*callback)(struct virtqueue *),
+	const char *name)
+{
+	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
+	struct vring_packed_desc_event *driver, *device;
+	size_t ring_size_in_bytes, event_size_in_bytes;
+	struct vring_packed_desc *ring;
+	struct vring_virtqueue *vq;
+
+	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
+				     &ring_dma_addr, &driver_event_dma_addr,
+				     &device_event_dma_addr,
+				     &ring_size_in_bytes,
+				     &event_size_in_bytes))
+		goto err_ring;
+
 	vq = kmalloc(sizeof(*vq), GFP_KERNEL);
 	if (!vq)
 		goto err_vq;
@@ -1939,9 +1983,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	kfree(vq);
 err_vq:
 	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
-err_device:
 	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
-err_driver:
 	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
 err_ring:
 	return NULL;
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 12/32] virtio_ring: packed: extract the logic of alloc queue

[Xuan Zhuo]

Separate the logic of packed to create vring queue.

For the convenience of passing parameters, add a structure
vring_packed.

This feature is required for subsequent virtuqueue reset vring.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 70 ++++++++++++++++++++++++++++--------
 1 file changed, 56 insertions(+), 14 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 33864134a744..ea451ae2aaef 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1817,19 +1817,17 @@ static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
 	return desc_extra;
 }
 
-static struct virtqueue *vring_create_virtqueue_packed(
-	unsigned int index,
-	unsigned int num,
-	unsigned int vring_align,
-	struct virtio_device *vdev,
-	bool weak_barriers,
-	bool may_reduce_num,
-	bool context,
-	bool (*notify)(struct virtqueue *),
-	void (*callback)(struct virtqueue *),
-	const char *name)
+static int vring_alloc_queue_packed(struct virtio_device *vdev,
+				    u32 num,
+				    struct vring_packed_desc **_ring,
+				    struct vring_packed_desc_event **_driver,
+				    struct vring_packed_desc_event **_device,
+				    dma_addr_t *_ring_dma_addr,
+				    dma_addr_t *_driver_event_dma_addr,
+				    dma_addr_t *_device_event_dma_addr,
+				    size_t *_ring_size_in_bytes,
+				    size_t *_event_size_in_bytes)
 {
-	struct vring_virtqueue *vq;
 	struct vring_packed_desc *ring;
 	struct vring_packed_desc_event *driver, *device;
 	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
@@ -1857,6 +1855,52 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	if (!device)
 		goto err_device;
 
+	*_ring                   = ring;
+	*_driver                 = driver;
+	*_device                 = device;
+	*_ring_dma_addr          = ring_dma_addr;
+	*_driver_event_dma_addr  = driver_event_dma_addr;
+	*_device_event_dma_addr  = device_event_dma_addr;
+	*_ring_size_in_bytes     = ring_size_in_bytes;
+	*_event_size_in_bytes    = event_size_in_bytes;
+
+	return 0;
+
+err_device:
+	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
+
+err_driver:
+	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
+
+err_ring:
+	return -ENOMEM;
+}
+
+static struct virtqueue *vring_create_virtqueue_packed(
+	unsigned int index,
+	unsigned int num,
+	unsigned int vring_align,
+	struct virtio_device *vdev,
+	bool weak_barriers,
+	bool may_reduce_num,
+	bool context,
+	bool (*notify)(struct virtqueue *),
+	void (*callback)(struct virtqueue *),
+	const char *name)
+{
+	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
+	struct vring_packed_desc_event *driver, *device;
+	size_t ring_size_in_bytes, event_size_in_bytes;
+	struct vring_packed_desc *ring;
+	struct vring_virtqueue *vq;
+
+	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
+				     &ring_dma_addr, &driver_event_dma_addr,
+				     &device_event_dma_addr,
+				     &ring_size_in_bytes,
+				     &event_size_in_bytes))
+		goto err_ring;
+
 	vq = kmalloc(sizeof(*vq), GFP_KERNEL);
 	if (!vq)
 		goto err_vq;
@@ -1939,9 +1983,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	kfree(vq);
 err_vq:
 	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
-err_device:
 	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
-err_driver:
 	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
 err_ring:
 	return NULL;
-- 
2.31.0

[PATCH v9 12/32] virtio_ring: packed: extract the logic of alloc queue

[Xuan Zhuo]

Separate the logic of packed to create vring queue.

For the convenience of passing parameters, add a structure
vring_packed.

This feature is required for subsequent virtuqueue reset vring.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 70 ++++++++++++++++++++++++++++--------
 1 file changed, 56 insertions(+), 14 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 33864134a744..ea451ae2aaef 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1817,19 +1817,17 @@ static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
 	return desc_extra;
 }
 
-static struct virtqueue *vring_create_virtqueue_packed(
-	unsigned int index,
-	unsigned int num,
-	unsigned int vring_align,
-	struct virtio_device *vdev,
-	bool weak_barriers,
-	bool may_reduce_num,
-	bool context,
-	bool (*notify)(struct virtqueue *),
-	void (*callback)(struct virtqueue *),
-	const char *name)
+static int vring_alloc_queue_packed(struct virtio_device *vdev,
+				    u32 num,
+				    struct vring_packed_desc **_ring,
+				    struct vring_packed_desc_event **_driver,
+				    struct vring_packed_desc_event **_device,
+				    dma_addr_t *_ring_dma_addr,
+				    dma_addr_t *_driver_event_dma_addr,
+				    dma_addr_t *_device_event_dma_addr,
+				    size_t *_ring_size_in_bytes,
+				    size_t *_event_size_in_bytes)
 {
-	struct vring_virtqueue *vq;
 	struct vring_packed_desc *ring;
 	struct vring_packed_desc_event *driver, *device;
 	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
@@ -1857,6 +1855,52 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	if (!device)
 		goto err_device;
 
+	*_ring                   = ring;
+	*_driver                 = driver;
+	*_device                 = device;
+	*_ring_dma_addr          = ring_dma_addr;
+	*_driver_event_dma_addr  = driver_event_dma_addr;
+	*_device_event_dma_addr  = device_event_dma_addr;
+	*_ring_size_in_bytes     = ring_size_in_bytes;
+	*_event_size_in_bytes    = event_size_in_bytes;
+
+	return 0;
+
+err_device:
+	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
+
+err_driver:
+	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
+
+err_ring:
+	return -ENOMEM;
+}
+
+static struct virtqueue *vring_create_virtqueue_packed(
+	unsigned int index,
+	unsigned int num,
+	unsigned int vring_align,
+	struct virtio_device *vdev,
+	bool weak_barriers,
+	bool may_reduce_num,
+	bool context,
+	bool (*notify)(struct virtqueue *),
+	void (*callback)(struct virtqueue *),
+	const char *name)
+{
+	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
+	struct vring_packed_desc_event *driver, *device;
+	size_t ring_size_in_bytes, event_size_in_bytes;
+	struct vring_packed_desc *ring;
+	struct vring_virtqueue *vq;
+
+	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
+				     &ring_dma_addr, &driver_event_dma_addr,
+				     &device_event_dma_addr,
+				     &ring_size_in_bytes,
+				     &event_size_in_bytes))
+		goto err_ring;
+
 	vq = kmalloc(sizeof(*vq), GFP_KERNEL);
 	if (!vq)
 		goto err_vq;
@@ -1939,9 +1983,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	kfree(vq);
 err_vq:
 	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
-err_device:
 	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
-err_driver:
 	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
 err_ring:
 	return NULL;
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 13/32] virtio_ring: packed: extract the logic of alloc state and extra

[Xuan Zhuo]

Separate the logic for alloc desc_state and desc_extra, which will
be called separately by subsequent patches.

Use struct vring_packed to pass desc_state, desc_extra.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 51 ++++++++++++++++++++++++++----------
 1 file changed, 37 insertions(+), 14 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index ea451ae2aaef..5b5976c5742e 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1876,6 +1876,34 @@ static int vring_alloc_queue_packed(struct virtio_device *vdev,
 	return -ENOMEM;
 }
 
+static int vring_alloc_state_extra_packed(u32 num,
+					  struct vring_desc_state_packed **desc_state,
+					  struct vring_desc_extra **desc_extra)
+{
+	struct vring_desc_state_packed *state;
+	struct vring_desc_extra *extra;
+
+	state = kmalloc_array(num, sizeof(struct vring_desc_state_packed), GFP_KERNEL);
+	if (!state)
+		goto err_desc_state;
+
+	memset(state, 0, num * sizeof(struct vring_desc_state_packed));
+
+	extra = vring_alloc_desc_extra(num);
+	if (!extra)
+		goto err_desc_extra;
+
+	*desc_state = state;
+	*desc_extra = extra;
+
+	return 0;
+
+err_desc_extra:
+	kfree(state);
+err_desc_state:
+	return -ENOMEM;
+}
+
 static struct virtqueue *vring_create_virtqueue_packed(
 	unsigned int index,
 	unsigned int num,
@@ -1891,8 +1919,11 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
 	struct vring_packed_desc_event *driver, *device;
 	size_t ring_size_in_bytes, event_size_in_bytes;
+	struct vring_desc_state_packed *state;
+	struct vring_desc_extra *extra;
 	struct vring_packed_desc *ring;
 	struct vring_virtqueue *vq;
+	int err;
 
 	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
 				     &ring_dma_addr, &driver_event_dma_addr,
@@ -1949,22 +1980,16 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	vq->packed.event_flags_shadow = 0;
 	vq->packed.avail_used_flags = 1 << VRING_PACKED_DESC_F_AVAIL;
 
-	vq->packed.desc_state = kmalloc_array(num,
-			sizeof(struct vring_desc_state_packed),
-			GFP_KERNEL);
-	if (!vq->packed.desc_state)
-		goto err_desc_state;
+	err = vring_alloc_state_extra_packed(num, &state, &extra);
+	if (err)
+		goto err_state_extra;
 
-	memset(vq->packed.desc_state, 0,
-		num * sizeof(struct vring_desc_state_packed));
+	vq->packed.desc_state = state;
+	vq->packed.desc_extra = extra;
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
 
-	vq->packed.desc_extra = vring_alloc_desc_extra(num);
-	if (!vq->packed.desc_extra)
-		goto err_desc_extra;
-
 	/* No callback?  Tell other side not to bother us. */
 	if (!callback) {
 		vq->packed.event_flags_shadow = VRING_PACKED_EVENT_FLAG_DISABLE;
@@ -1977,9 +2002,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	spin_unlock(&vdev->vqs_list_lock);
 	return &vq->vq;
 
-err_desc_extra:
-	kfree(vq->packed.desc_state);
-err_desc_state:
+err_state_extra:
 	kfree(vq);
 err_vq:
 	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 13/32] virtio_ring: packed: extract the logic of alloc state and extra

[Xuan Zhuo]

Separate the logic for alloc desc_state and desc_extra, which will
be called separately by subsequent patches.

Use struct vring_packed to pass desc_state, desc_extra.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 51 ++++++++++++++++++++++++++----------
 1 file changed, 37 insertions(+), 14 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index ea451ae2aaef..5b5976c5742e 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1876,6 +1876,34 @@ static int vring_alloc_queue_packed(struct virtio_device *vdev,
 	return -ENOMEM;
 }
 
+static int vring_alloc_state_extra_packed(u32 num,
+					  struct vring_desc_state_packed **desc_state,
+					  struct vring_desc_extra **desc_extra)
+{
+	struct vring_desc_state_packed *state;
+	struct vring_desc_extra *extra;
+
+	state = kmalloc_array(num, sizeof(struct vring_desc_state_packed), GFP_KERNEL);
+	if (!state)
+		goto err_desc_state;
+
+	memset(state, 0, num * sizeof(struct vring_desc_state_packed));
+
+	extra = vring_alloc_desc_extra(num);
+	if (!extra)
+		goto err_desc_extra;
+
+	*desc_state = state;
+	*desc_extra = extra;
+
+	return 0;
+
+err_desc_extra:
+	kfree(state);
+err_desc_state:
+	return -ENOMEM;
+}
+
 static struct virtqueue *vring_create_virtqueue_packed(
 	unsigned int index,
 	unsigned int num,
@@ -1891,8 +1919,11 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
 	struct vring_packed_desc_event *driver, *device;
 	size_t ring_size_in_bytes, event_size_in_bytes;
+	struct vring_desc_state_packed *state;
+	struct vring_desc_extra *extra;
 	struct vring_packed_desc *ring;
 	struct vring_virtqueue *vq;
+	int err;
 
 	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
 				     &ring_dma_addr, &driver_event_dma_addr,
@@ -1949,22 +1980,16 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	vq->packed.event_flags_shadow = 0;
 	vq->packed.avail_used_flags = 1 << VRING_PACKED_DESC_F_AVAIL;
 
-	vq->packed.desc_state = kmalloc_array(num,
-			sizeof(struct vring_desc_state_packed),
-			GFP_KERNEL);
-	if (!vq->packed.desc_state)
-		goto err_desc_state;
+	err = vring_alloc_state_extra_packed(num, &state, &extra);
+	if (err)
+		goto err_state_extra;
 
-	memset(vq->packed.desc_state, 0,
-		num * sizeof(struct vring_desc_state_packed));
+	vq->packed.desc_state = state;
+	vq->packed.desc_extra = extra;
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
 
-	vq->packed.desc_extra = vring_alloc_desc_extra(num);
-	if (!vq->packed.desc_extra)
-		goto err_desc_extra;
-
 	/* No callback?  Tell other side not to bother us. */
 	if (!callback) {
 		vq->packed.event_flags_shadow = VRING_PACKED_EVENT_FLAG_DISABLE;
@@ -1977,9 +2002,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	spin_unlock(&vdev->vqs_list_lock);
 	return &vq->vq;
 
-err_desc_extra:
-	kfree(vq->packed.desc_state);
-err_desc_state:
+err_state_extra:
 	kfree(vq);
 err_vq:
 	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
-- 
2.31.0

[PATCH v9 13/32] virtio_ring: packed: extract the logic of alloc state and extra

[Xuan Zhuo]

Separate the logic for alloc desc_state and desc_extra, which will
be called separately by subsequent patches.

Use struct vring_packed to pass desc_state, desc_extra.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 51 ++++++++++++++++++++++++++----------
 1 file changed, 37 insertions(+), 14 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index ea451ae2aaef..5b5976c5742e 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1876,6 +1876,34 @@ static int vring_alloc_queue_packed(struct virtio_device *vdev,
 	return -ENOMEM;
 }
 
+static int vring_alloc_state_extra_packed(u32 num,
+					  struct vring_desc_state_packed **desc_state,
+					  struct vring_desc_extra **desc_extra)
+{
+	struct vring_desc_state_packed *state;
+	struct vring_desc_extra *extra;
+
+	state = kmalloc_array(num, sizeof(struct vring_desc_state_packed), GFP_KERNEL);
+	if (!state)
+		goto err_desc_state;
+
+	memset(state, 0, num * sizeof(struct vring_desc_state_packed));
+
+	extra = vring_alloc_desc_extra(num);
+	if (!extra)
+		goto err_desc_extra;
+
+	*desc_state = state;
+	*desc_extra = extra;
+
+	return 0;
+
+err_desc_extra:
+	kfree(state);
+err_desc_state:
+	return -ENOMEM;
+}
+
 static struct virtqueue *vring_create_virtqueue_packed(
 	unsigned int index,
 	unsigned int num,
@@ -1891,8 +1919,11 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
 	struct vring_packed_desc_event *driver, *device;
 	size_t ring_size_in_bytes, event_size_in_bytes;
+	struct vring_desc_state_packed *state;
+	struct vring_desc_extra *extra;
 	struct vring_packed_desc *ring;
 	struct vring_virtqueue *vq;
+	int err;
 
 	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
 				     &ring_dma_addr, &driver_event_dma_addr,
@@ -1949,22 +1980,16 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	vq->packed.event_flags_shadow = 0;
 	vq->packed.avail_used_flags = 1 << VRING_PACKED_DESC_F_AVAIL;
 
-	vq->packed.desc_state = kmalloc_array(num,
-			sizeof(struct vring_desc_state_packed),
-			GFP_KERNEL);
-	if (!vq->packed.desc_state)
-		goto err_desc_state;
+	err = vring_alloc_state_extra_packed(num, &state, &extra);
+	if (err)
+		goto err_state_extra;
 
-	memset(vq->packed.desc_state, 0,
-		num * sizeof(struct vring_desc_state_packed));
+	vq->packed.desc_state = state;
+	vq->packed.desc_extra = extra;
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
 
-	vq->packed.desc_extra = vring_alloc_desc_extra(num);
-	if (!vq->packed.desc_extra)
-		goto err_desc_extra;
-
 	/* No callback?  Tell other side not to bother us. */
 	if (!callback) {
 		vq->packed.event_flags_shadow = VRING_PACKED_EVENT_FLAG_DISABLE;
@@ -1977,9 +2002,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	spin_unlock(&vdev->vqs_list_lock);
 	return &vq->vq;
 
-err_desc_extra:
-	kfree(vq->packed.desc_state);
-err_desc_state:
+err_state_extra:
 	kfree(vq);
 err_vq:
 	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 14/32] virtio_ring: packed: extract the logic of attach vring

[Xuan Zhuo]

Separate the logic of attach vring, the subsequent patch will call it
separately.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 47 +++++++++++++++++++++++++-----------
 1 file changed, 33 insertions(+), 14 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 5b5976c5742e..80d446fa8d16 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1904,6 +1904,35 @@ static int vring_alloc_state_extra_packed(u32 num,
 	return -ENOMEM;
 }
 
+static void vring_virtqueue_attach_packed(struct vring_virtqueue *vq,
+					  u32 num,
+					  struct vring_packed_desc *ring,
+					  struct vring_packed_desc_event *driver,
+					  struct vring_packed_desc_event *device,
+					  dma_addr_t ring_dma_addr,
+					  dma_addr_t driver_event_dma_addr,
+					  dma_addr_t device_event_dma_addr,
+					  size_t ring_size_in_bytes,
+					  size_t event_size_in_bytes,
+					  struct vring_desc_state_packed *state,
+					  struct vring_desc_extra *extra)
+{
+	vq->packed.ring_dma_addr = ring_dma_addr;
+	vq->packed.driver_event_dma_addr = driver_event_dma_addr;
+	vq->packed.device_event_dma_addr = device_event_dma_addr;
+
+	vq->packed.ring_size_in_bytes = ring_size_in_bytes;
+	vq->packed.event_size_in_bytes = event_size_in_bytes;
+
+	vq->packed.vring.num = num;
+	vq->packed.vring.desc = ring;
+	vq->packed.vring.driver = driver;
+	vq->packed.vring.device = device;
+
+	vq->packed.desc_state = state;
+	vq->packed.desc_extra = extra;
+}
+
 static struct virtqueue *vring_create_virtqueue_packed(
 	unsigned int index,
 	unsigned int num,
@@ -1962,18 +1991,6 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
 		vq->weak_barriers = false;
 
-	vq->packed.ring_dma_addr = ring_dma_addr;
-	vq->packed.driver_event_dma_addr = driver_event_dma_addr;
-	vq->packed.device_event_dma_addr = device_event_dma_addr;
-
-	vq->packed.ring_size_in_bytes = ring_size_in_bytes;
-	vq->packed.event_size_in_bytes = event_size_in_bytes;
-
-	vq->packed.vring.num = num;
-	vq->packed.vring.desc = ring;
-	vq->packed.vring.driver = driver;
-	vq->packed.vring.device = device;
-
 	vq->packed.next_avail_idx = 0;
 	vq->packed.avail_wrap_counter = 1;
 	vq->packed.used_wrap_counter = 1;
@@ -1984,8 +2001,10 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	if (err)
 		goto err_state_extra;
 
-	vq->packed.desc_state = state;
-	vq->packed.desc_extra = extra;
+	vring_virtqueue_attach_packed(vq, num, ring, driver, device,
+				      ring_dma_addr, driver_event_dma_addr,
+				      device_event_dma_addr, ring_size_in_bytes,
+				      event_size_in_bytes, state, extra);
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 14/32] virtio_ring: packed: extract the logic of attach vring

[Xuan Zhuo]

Separate the logic of attach vring, the subsequent patch will call it
separately.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 47 +++++++++++++++++++++++++-----------
 1 file changed, 33 insertions(+), 14 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 5b5976c5742e..80d446fa8d16 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1904,6 +1904,35 @@ static int vring_alloc_state_extra_packed(u32 num,
 	return -ENOMEM;
 }
 
+static void vring_virtqueue_attach_packed(struct vring_virtqueue *vq,
+					  u32 num,
+					  struct vring_packed_desc *ring,
+					  struct vring_packed_desc_event *driver,
+					  struct vring_packed_desc_event *device,
+					  dma_addr_t ring_dma_addr,
+					  dma_addr_t driver_event_dma_addr,
+					  dma_addr_t device_event_dma_addr,
+					  size_t ring_size_in_bytes,
+					  size_t event_size_in_bytes,
+					  struct vring_desc_state_packed *state,
+					  struct vring_desc_extra *extra)
+{
+	vq->packed.ring_dma_addr = ring_dma_addr;
+	vq->packed.driver_event_dma_addr = driver_event_dma_addr;
+	vq->packed.device_event_dma_addr = device_event_dma_addr;
+
+	vq->packed.ring_size_in_bytes = ring_size_in_bytes;
+	vq->packed.event_size_in_bytes = event_size_in_bytes;
+
+	vq->packed.vring.num = num;
+	vq->packed.vring.desc = ring;
+	vq->packed.vring.driver = driver;
+	vq->packed.vring.device = device;
+
+	vq->packed.desc_state = state;
+	vq->packed.desc_extra = extra;
+}
+
 static struct virtqueue *vring_create_virtqueue_packed(
 	unsigned int index,
 	unsigned int num,
@@ -1962,18 +1991,6 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
 		vq->weak_barriers = false;
 
-	vq->packed.ring_dma_addr = ring_dma_addr;
-	vq->packed.driver_event_dma_addr = driver_event_dma_addr;
-	vq->packed.device_event_dma_addr = device_event_dma_addr;
-
-	vq->packed.ring_size_in_bytes = ring_size_in_bytes;
-	vq->packed.event_size_in_bytes = event_size_in_bytes;
-
-	vq->packed.vring.num = num;
-	vq->packed.vring.desc = ring;
-	vq->packed.vring.driver = driver;
-	vq->packed.vring.device = device;
-
 	vq->packed.next_avail_idx = 0;
 	vq->packed.avail_wrap_counter = 1;
 	vq->packed.used_wrap_counter = 1;
@@ -1984,8 +2001,10 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	if (err)
 		goto err_state_extra;
 
-	vq->packed.desc_state = state;
-	vq->packed.desc_extra = extra;
+	vring_virtqueue_attach_packed(vq, num, ring, driver, device,
+				      ring_dma_addr, driver_event_dma_addr,
+				      device_event_dma_addr, ring_size_in_bytes,
+				      event_size_in_bytes, state, extra);
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
-- 
2.31.0

[PATCH v9 14/32] virtio_ring: packed: extract the logic of attach vring

[Xuan Zhuo]

Separate the logic of attach vring, the subsequent patch will call it
separately.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 47 +++++++++++++++++++++++++-----------
 1 file changed, 33 insertions(+), 14 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 5b5976c5742e..80d446fa8d16 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1904,6 +1904,35 @@ static int vring_alloc_state_extra_packed(u32 num,
 	return -ENOMEM;
 }
 
+static void vring_virtqueue_attach_packed(struct vring_virtqueue *vq,
+					  u32 num,
+					  struct vring_packed_desc *ring,
+					  struct vring_packed_desc_event *driver,
+					  struct vring_packed_desc_event *device,
+					  dma_addr_t ring_dma_addr,
+					  dma_addr_t driver_event_dma_addr,
+					  dma_addr_t device_event_dma_addr,
+					  size_t ring_size_in_bytes,
+					  size_t event_size_in_bytes,
+					  struct vring_desc_state_packed *state,
+					  struct vring_desc_extra *extra)
+{
+	vq->packed.ring_dma_addr = ring_dma_addr;
+	vq->packed.driver_event_dma_addr = driver_event_dma_addr;
+	vq->packed.device_event_dma_addr = device_event_dma_addr;
+
+	vq->packed.ring_size_in_bytes = ring_size_in_bytes;
+	vq->packed.event_size_in_bytes = event_size_in_bytes;
+
+	vq->packed.vring.num = num;
+	vq->packed.vring.desc = ring;
+	vq->packed.vring.driver = driver;
+	vq->packed.vring.device = device;
+
+	vq->packed.desc_state = state;
+	vq->packed.desc_extra = extra;
+}
+
 static struct virtqueue *vring_create_virtqueue_packed(
 	unsigned int index,
 	unsigned int num,
@@ -1962,18 +1991,6 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
 		vq->weak_barriers = false;
 
-	vq->packed.ring_dma_addr = ring_dma_addr;
-	vq->packed.driver_event_dma_addr = driver_event_dma_addr;
-	vq->packed.device_event_dma_addr = device_event_dma_addr;
-
-	vq->packed.ring_size_in_bytes = ring_size_in_bytes;
-	vq->packed.event_size_in_bytes = event_size_in_bytes;
-
-	vq->packed.vring.num = num;
-	vq->packed.vring.desc = ring;
-	vq->packed.vring.driver = driver;
-	vq->packed.vring.device = device;
-
 	vq->packed.next_avail_idx = 0;
 	vq->packed.avail_wrap_counter = 1;
 	vq->packed.used_wrap_counter = 1;
@@ -1984,8 +2001,10 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	if (err)
 		goto err_state_extra;
 
-	vq->packed.desc_state = state;
-	vq->packed.desc_extra = extra;
+	vring_virtqueue_attach_packed(vq, num, ring, driver, device,
+				      ring_dma_addr, driver_event_dma_addr,
+				      device_event_dma_addr, ring_size_in_bytes,
+				      event_size_in_bytes, state, extra);
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 15/32] virtio_ring: packed: extract the logic of vq init

[Xuan Zhuo]

Separate the logic of initializing vq, and subsequent patches will call
it separately.

The characteristic of this part of the logic is that it does not depend
on the information passed by the upper layer, and can be called
repeatedly.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 71 ++++++++++++++++++++----------------
 1 file changed, 39 insertions(+), 32 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 80d446fa8d16..c783eb272468 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1933,6 +1933,44 @@ static void vring_virtqueue_attach_packed(struct vring_virtqueue *vq,
 	vq->packed.desc_extra = extra;
 }
 
+static void vring_virtqueue_init_packed(struct vring_virtqueue *vq,
+					struct virtio_device *vdev)
+{
+	vq->vq.num_free = vq->packed.vring.num;
+	vq->we_own_ring = true;
+	vq->broken = false;
+	vq->last_used_idx = 0;
+	vq->event_triggered = false;
+	vq->num_added = 0;
+	vq->packed_ring = true;
+	vq->use_dma_api = vring_use_dma_api(vdev);
+#ifdef DEBUG
+	vq->in_use = false;
+	vq->last_add_time_valid = false;
+#endif
+
+	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
+
+	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
+		vq->weak_barriers = false;
+
+	vq->packed.next_avail_idx = 0;
+	vq->packed.avail_wrap_counter = 1;
+	vq->packed.used_wrap_counter = 1;
+	vq->packed.event_flags_shadow = 0;
+	vq->packed.avail_used_flags = 1 << VRING_PACKED_DESC_F_AVAIL;
+
+	/* Put everything in free lists. */
+	vq->free_head = 0;
+
+	/* No callback?  Tell other side not to bother us. */
+	if (!vq->vq.callback) {
+		vq->packed.event_flags_shadow = VRING_PACKED_EVENT_FLAG_DISABLE;
+		vq->packed.vring.driver->flags =
+			cpu_to_le16(vq->packed.event_flags_shadow);
+	}
+}
+
 static struct virtqueue *vring_create_virtqueue_packed(
 	unsigned int index,
 	unsigned int num,
@@ -1968,34 +2006,12 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	vq->vq.callback = callback;
 	vq->vq.vdev = vdev;
 	vq->vq.name = name;
-	vq->vq.num_free = num;
 	vq->vq.index = index;
-	vq->we_own_ring = true;
 	vq->notify = notify;
 	vq->weak_barriers = weak_barriers;
-	vq->broken = false;
-	vq->last_used_idx = 0;
-	vq->event_triggered = false;
-	vq->num_added = 0;
-	vq->packed_ring = true;
-	vq->use_dma_api = vring_use_dma_api(vdev);
-#ifdef DEBUG
-	vq->in_use = false;
-	vq->last_add_time_valid = false;
-#endif
 
 	vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) &&
 		!context;
-	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
-
-	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
-		vq->weak_barriers = false;
-
-	vq->packed.next_avail_idx = 0;
-	vq->packed.avail_wrap_counter = 1;
-	vq->packed.used_wrap_counter = 1;
-	vq->packed.event_flags_shadow = 0;
-	vq->packed.avail_used_flags = 1 << VRING_PACKED_DESC_F_AVAIL;
 
 	err = vring_alloc_state_extra_packed(num, &state, &extra);
 	if (err)
@@ -2005,16 +2021,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 				      ring_dma_addr, driver_event_dma_addr,
 				      device_event_dma_addr, ring_size_in_bytes,
 				      event_size_in_bytes, state, extra);
-
-	/* Put everything in free lists. */
-	vq->free_head = 0;
-
-	/* No callback?  Tell other side not to bother us. */
-	if (!callback) {
-		vq->packed.event_flags_shadow = VRING_PACKED_EVENT_FLAG_DISABLE;
-		vq->packed.vring.driver->flags =
-			cpu_to_le16(vq->packed.event_flags_shadow);
-	}
+	vring_virtqueue_init_packed(vq, vdev);
 
 	spin_lock(&vdev->vqs_list_lock);
 	list_add_tail(&vq->vq.list, &vdev->vqs);
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 15/32] virtio_ring: packed: extract the logic of vq init

[Xuan Zhuo]

Separate the logic of initializing vq, and subsequent patches will call
it separately.

The characteristic of this part of the logic is that it does not depend
on the information passed by the upper layer, and can be called
repeatedly.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 71 ++++++++++++++++++++----------------
 1 file changed, 39 insertions(+), 32 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 80d446fa8d16..c783eb272468 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1933,6 +1933,44 @@ static void vring_virtqueue_attach_packed(struct vring_virtqueue *vq,
 	vq->packed.desc_extra = extra;
 }
 
+static void vring_virtqueue_init_packed(struct vring_virtqueue *vq,
+					struct virtio_device *vdev)
+{
+	vq->vq.num_free = vq->packed.vring.num;
+	vq->we_own_ring = true;
+	vq->broken = false;
+	vq->last_used_idx = 0;
+	vq->event_triggered = false;
+	vq->num_added = 0;
+	vq->packed_ring = true;
+	vq->use_dma_api = vring_use_dma_api(vdev);
+#ifdef DEBUG
+	vq->in_use = false;
+	vq->last_add_time_valid = false;
+#endif
+
+	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
+
+	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
+		vq->weak_barriers = false;
+
+	vq->packed.next_avail_idx = 0;
+	vq->packed.avail_wrap_counter = 1;
+	vq->packed.used_wrap_counter = 1;
+	vq->packed.event_flags_shadow = 0;
+	vq->packed.avail_used_flags = 1 << VRING_PACKED_DESC_F_AVAIL;
+
+	/* Put everything in free lists. */
+	vq->free_head = 0;
+
+	/* No callback?  Tell other side not to bother us. */
+	if (!vq->vq.callback) {
+		vq->packed.event_flags_shadow = VRING_PACKED_EVENT_FLAG_DISABLE;
+		vq->packed.vring.driver->flags =
+			cpu_to_le16(vq->packed.event_flags_shadow);
+	}
+}
+
 static struct virtqueue *vring_create_virtqueue_packed(
 	unsigned int index,
 	unsigned int num,
@@ -1968,34 +2006,12 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	vq->vq.callback = callback;
 	vq->vq.vdev = vdev;
 	vq->vq.name = name;
-	vq->vq.num_free = num;
 	vq->vq.index = index;
-	vq->we_own_ring = true;
 	vq->notify = notify;
 	vq->weak_barriers = weak_barriers;
-	vq->broken = false;
-	vq->last_used_idx = 0;
-	vq->event_triggered = false;
-	vq->num_added = 0;
-	vq->packed_ring = true;
-	vq->use_dma_api = vring_use_dma_api(vdev);
-#ifdef DEBUG
-	vq->in_use = false;
-	vq->last_add_time_valid = false;
-#endif
 
 	vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) &&
 		!context;
-	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
-
-	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
-		vq->weak_barriers = false;
-
-	vq->packed.next_avail_idx = 0;
-	vq->packed.avail_wrap_counter = 1;
-	vq->packed.used_wrap_counter = 1;
-	vq->packed.event_flags_shadow = 0;
-	vq->packed.avail_used_flags = 1 << VRING_PACKED_DESC_F_AVAIL;
 
 	err = vring_alloc_state_extra_packed(num, &state, &extra);
 	if (err)
@@ -2005,16 +2021,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 				      ring_dma_addr, driver_event_dma_addr,
 				      device_event_dma_addr, ring_size_in_bytes,
 				      event_size_in_bytes, state, extra);
-
-	/* Put everything in free lists. */
-	vq->free_head = 0;
-
-	/* No callback?  Tell other side not to bother us. */
-	if (!callback) {
-		vq->packed.event_flags_shadow = VRING_PACKED_EVENT_FLAG_DISABLE;
-		vq->packed.vring.driver->flags =
-			cpu_to_le16(vq->packed.event_flags_shadow);
-	}
+	vring_virtqueue_init_packed(vq, vdev);
 
 	spin_lock(&vdev->vqs_list_lock);
 	list_add_tail(&vq->vq.list, &vdev->vqs);
-- 
2.31.0

[PATCH v9 15/32] virtio_ring: packed: extract the logic of vq init

[Xuan Zhuo]

Separate the logic of initializing vq, and subsequent patches will call
it separately.

The characteristic of this part of the logic is that it does not depend
on the information passed by the upper layer, and can be called
repeatedly.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 71 ++++++++++++++++++++----------------
 1 file changed, 39 insertions(+), 32 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 80d446fa8d16..c783eb272468 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1933,6 +1933,44 @@ static void vring_virtqueue_attach_packed(struct vring_virtqueue *vq,
 	vq->packed.desc_extra = extra;
 }
 
+static void vring_virtqueue_init_packed(struct vring_virtqueue *vq,
+					struct virtio_device *vdev)
+{
+	vq->vq.num_free = vq->packed.vring.num;
+	vq->we_own_ring = true;
+	vq->broken = false;
+	vq->last_used_idx = 0;
+	vq->event_triggered = false;
+	vq->num_added = 0;
+	vq->packed_ring = true;
+	vq->use_dma_api = vring_use_dma_api(vdev);
+#ifdef DEBUG
+	vq->in_use = false;
+	vq->last_add_time_valid = false;
+#endif
+
+	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
+
+	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
+		vq->weak_barriers = false;
+
+	vq->packed.next_avail_idx = 0;
+	vq->packed.avail_wrap_counter = 1;
+	vq->packed.used_wrap_counter = 1;
+	vq->packed.event_flags_shadow = 0;
+	vq->packed.avail_used_flags = 1 << VRING_PACKED_DESC_F_AVAIL;
+
+	/* Put everything in free lists. */
+	vq->free_head = 0;
+
+	/* No callback?  Tell other side not to bother us. */
+	if (!vq->vq.callback) {
+		vq->packed.event_flags_shadow = VRING_PACKED_EVENT_FLAG_DISABLE;
+		vq->packed.vring.driver->flags =
+			cpu_to_le16(vq->packed.event_flags_shadow);
+	}
+}
+
 static struct virtqueue *vring_create_virtqueue_packed(
 	unsigned int index,
 	unsigned int num,
@@ -1968,34 +2006,12 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	vq->vq.callback = callback;
 	vq->vq.vdev = vdev;
 	vq->vq.name = name;
-	vq->vq.num_free = num;
 	vq->vq.index = index;
-	vq->we_own_ring = true;
 	vq->notify = notify;
 	vq->weak_barriers = weak_barriers;
-	vq->broken = false;
-	vq->last_used_idx = 0;
-	vq->event_triggered = false;
-	vq->num_added = 0;
-	vq->packed_ring = true;
-	vq->use_dma_api = vring_use_dma_api(vdev);
-#ifdef DEBUG
-	vq->in_use = false;
-	vq->last_add_time_valid = false;
-#endif
 
 	vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) &&
 		!context;
-	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
-
-	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
-		vq->weak_barriers = false;
-
-	vq->packed.next_avail_idx = 0;
-	vq->packed.avail_wrap_counter = 1;
-	vq->packed.used_wrap_counter = 1;
-	vq->packed.event_flags_shadow = 0;
-	vq->packed.avail_used_flags = 1 << VRING_PACKED_DESC_F_AVAIL;
 
 	err = vring_alloc_state_extra_packed(num, &state, &extra);
 	if (err)
@@ -2005,16 +2021,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 				      ring_dma_addr, driver_event_dma_addr,
 				      device_event_dma_addr, ring_size_in_bytes,
 				      event_size_in_bytes, state, extra);
-
-	/* Put everything in free lists. */
-	vq->free_head = 0;
-
-	/* No callback?  Tell other side not to bother us. */
-	if (!callback) {
-		vq->packed.event_flags_shadow = VRING_PACKED_EVENT_FLAG_DISABLE;
-		vq->packed.vring.driver->flags =
-			cpu_to_le16(vq->packed.event_flags_shadow);
-	}
+	vring_virtqueue_init_packed(vq, vdev);
 
 	spin_lock(&vdev->vqs_list_lock);
 	list_add_tail(&vq->vq.list, &vdev->vqs);
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 16/32] virtio_ring: packed: introduce virtqueue_reinit_packed()

[Xuan Zhuo]

Introduce a function to initialize vq without allocating new ring,
desc_state, desc_extra.

Subsequent patches will call this function after reset vq to
reinitialize vq.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index c783eb272468..9a4f2db718bd 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1971,6 +1971,27 @@ static void vring_virtqueue_init_packed(struct vring_virtqueue *vq,
 	}
 }
 
+static void virtqueue_reinit_packed(struct vring_virtqueue *vq)
+{
+	struct virtio_device *vdev = vq->vq.vdev;
+	int size, i;
+
+	memset(vq->packed.vring.device, 0, vq->packed.event_size_in_bytes);
+	memset(vq->packed.vring.driver, 0, vq->packed.event_size_in_bytes);
+	memset(vq->packed.vring.desc, 0, vq->packed.ring_size_in_bytes);
+
+	size = sizeof(struct vring_desc_state_packed) * vq->packed.vring.num;
+	memset(vq->packed.desc_state, 0, size);
+
+	size = sizeof(struct vring_desc_extra) * vq->packed.vring.num;
+	memset(vq->packed.desc_extra, 0, size);
+
+	for (i = 0; i < vq->packed.vring.num - 1; i++)
+		vq->packed.desc_extra[i].next = i + 1;
+
+	vring_virtqueue_init_packed(vq, vdev);
+}
+
 static struct virtqueue *vring_create_virtqueue_packed(
 	unsigned int index,
 	unsigned int num,
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 16/32] virtio_ring: packed: introduce virtqueue_reinit_packed()

[Xuan Zhuo]

Introduce a function to initialize vq without allocating new ring,
desc_state, desc_extra.

Subsequent patches will call this function after reset vq to
reinitialize vq.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index c783eb272468..9a4f2db718bd 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1971,6 +1971,27 @@ static void vring_virtqueue_init_packed(struct vring_virtqueue *vq,
 	}
 }
 
+static void virtqueue_reinit_packed(struct vring_virtqueue *vq)
+{
+	struct virtio_device *vdev = vq->vq.vdev;
+	int size, i;
+
+	memset(vq->packed.vring.device, 0, vq->packed.event_size_in_bytes);
+	memset(vq->packed.vring.driver, 0, vq->packed.event_size_in_bytes);
+	memset(vq->packed.vring.desc, 0, vq->packed.ring_size_in_bytes);
+
+	size = sizeof(struct vring_desc_state_packed) * vq->packed.vring.num;
+	memset(vq->packed.desc_state, 0, size);
+
+	size = sizeof(struct vring_desc_extra) * vq->packed.vring.num;
+	memset(vq->packed.desc_extra, 0, size);
+
+	for (i = 0; i < vq->packed.vring.num - 1; i++)
+		vq->packed.desc_extra[i].next = i + 1;
+
+	vring_virtqueue_init_packed(vq, vdev);
+}
+
 static struct virtqueue *vring_create_virtqueue_packed(
 	unsigned int index,
 	unsigned int num,
-- 
2.31.0

[PATCH v9 16/32] virtio_ring: packed: introduce virtqueue_reinit_packed()

[Xuan Zhuo]

Introduce a function to initialize vq without allocating new ring,
desc_state, desc_extra.

Subsequent patches will call this function after reset vq to
reinitialize vq.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index c783eb272468..9a4f2db718bd 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1971,6 +1971,27 @@ static void vring_virtqueue_init_packed(struct vring_virtqueue *vq,
 	}
 }
 
+static void virtqueue_reinit_packed(struct vring_virtqueue *vq)
+{
+	struct virtio_device *vdev = vq->vq.vdev;
+	int size, i;
+
+	memset(vq->packed.vring.device, 0, vq->packed.event_size_in_bytes);
+	memset(vq->packed.vring.driver, 0, vq->packed.event_size_in_bytes);
+	memset(vq->packed.vring.desc, 0, vq->packed.ring_size_in_bytes);
+
+	size = sizeof(struct vring_desc_state_packed) * vq->packed.vring.num;
+	memset(vq->packed.desc_state, 0, size);
+
+	size = sizeof(struct vring_desc_extra) * vq->packed.vring.num;
+	memset(vq->packed.desc_extra, 0, size);
+
+	for (i = 0; i < vq->packed.vring.num - 1; i++)
+		vq->packed.desc_extra[i].next = i + 1;
+
+	vring_virtqueue_init_packed(vq, vdev);
+}
+
 static struct virtqueue *vring_create_virtqueue_packed(
 	unsigned int index,
 	unsigned int num,
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 17/32] virtio_ring: packed: introduce virtqueue_resize_packed()

[Xuan Zhuo]

virtio ring packed supports resize.

Only after the new vring is successfully allocated based on the new num,
we will release the old vring. In any case, an error is returned,
indicating that the vring still points to the old vring.

In the case of an error, the caller must re-initialize(by
virtqueue_reinit_packed()) the virtqueue to ensure that the vring can be
used.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 40 ++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 9a4f2db718bd..06f66b15c86c 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2059,6 +2059,46 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	return NULL;
 }
 
+static int virtqueue_resize_packed(struct virtqueue *_vq, u32 num)
+{
+	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
+	struct vring_packed_desc_event *driver, *device;
+	size_t ring_size_in_bytes, event_size_in_bytes;
+	struct vring_virtqueue *vq = to_vvq(_vq);
+	struct virtio_device *vdev = _vq->vdev;
+	struct vring_desc_state_packed *state;
+	struct vring_desc_extra *extra;
+	struct vring_packed_desc *ring;
+	int err;
+
+	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
+				     &ring_dma_addr, &driver_event_dma_addr,
+				     &device_event_dma_addr,
+				     &ring_size_in_bytes, &event_size_in_bytes))
+		goto err_ring;
+
+
+	err = vring_alloc_state_extra_packed(num, &state, &extra);
+	if (err)
+		goto err_state_extra;
+
+	vring_free(&vq->vq);
+
+	vring_virtqueue_attach_packed(vq, num, ring, driver, device,
+				      ring_dma_addr, driver_event_dma_addr,
+				      device_event_dma_addr, ring_size_in_bytes,
+				      event_size_in_bytes, state, extra);
+	vring_virtqueue_init_packed(vq, vdev);
+	return 0;
+
+err_state_extra:
+	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
+	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
+	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
+err_ring:
+	return -ENOMEM;
+}
+
 
 /*
  * Generic functions and exported symbols.
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 17/32] virtio_ring: packed: introduce virtqueue_resize_packed()

[Xuan Zhuo]

virtio ring packed supports resize.

Only after the new vring is successfully allocated based on the new num,
we will release the old vring. In any case, an error is returned,
indicating that the vring still points to the old vring.

In the case of an error, the caller must re-initialize(by
virtqueue_reinit_packed()) the virtqueue to ensure that the vring can be
used.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 40 ++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 9a4f2db718bd..06f66b15c86c 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2059,6 +2059,46 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	return NULL;
 }
 
+static int virtqueue_resize_packed(struct virtqueue *_vq, u32 num)
+{
+	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
+	struct vring_packed_desc_event *driver, *device;
+	size_t ring_size_in_bytes, event_size_in_bytes;
+	struct vring_virtqueue *vq = to_vvq(_vq);
+	struct virtio_device *vdev = _vq->vdev;
+	struct vring_desc_state_packed *state;
+	struct vring_desc_extra *extra;
+	struct vring_packed_desc *ring;
+	int err;
+
+	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
+				     &ring_dma_addr, &driver_event_dma_addr,
+				     &device_event_dma_addr,
+				     &ring_size_in_bytes, &event_size_in_bytes))
+		goto err_ring;
+
+
+	err = vring_alloc_state_extra_packed(num, &state, &extra);
+	if (err)
+		goto err_state_extra;
+
+	vring_free(&vq->vq);
+
+	vring_virtqueue_attach_packed(vq, num, ring, driver, device,
+				      ring_dma_addr, driver_event_dma_addr,
+				      device_event_dma_addr, ring_size_in_bytes,
+				      event_size_in_bytes, state, extra);
+	vring_virtqueue_init_packed(vq, vdev);
+	return 0;
+
+err_state_extra:
+	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
+	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
+	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
+err_ring:
+	return -ENOMEM;
+}
+
 
 /*
  * Generic functions and exported symbols.
-- 
2.31.0

[PATCH v9 17/32] virtio_ring: packed: introduce virtqueue_resize_packed()

[Xuan Zhuo]

virtio ring packed supports resize.

Only after the new vring is successfully allocated based on the new num,
we will release the old vring. In any case, an error is returned,
indicating that the vring still points to the old vring.

In the case of an error, the caller must re-initialize(by
virtqueue_reinit_packed()) the virtqueue to ensure that the vring can be
used.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 40 ++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 9a4f2db718bd..06f66b15c86c 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2059,6 +2059,46 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	return NULL;
 }
 
+static int virtqueue_resize_packed(struct virtqueue *_vq, u32 num)
+{
+	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
+	struct vring_packed_desc_event *driver, *device;
+	size_t ring_size_in_bytes, event_size_in_bytes;
+	struct vring_virtqueue *vq = to_vvq(_vq);
+	struct virtio_device *vdev = _vq->vdev;
+	struct vring_desc_state_packed *state;
+	struct vring_desc_extra *extra;
+	struct vring_packed_desc *ring;
+	int err;
+
+	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
+				     &ring_dma_addr, &driver_event_dma_addr,
+				     &device_event_dma_addr,
+				     &ring_size_in_bytes, &event_size_in_bytes))
+		goto err_ring;
+
+
+	err = vring_alloc_state_extra_packed(num, &state, &extra);
+	if (err)
+		goto err_state_extra;
+
+	vring_free(&vq->vq);
+
+	vring_virtqueue_attach_packed(vq, num, ring, driver, device,
+				      ring_dma_addr, driver_event_dma_addr,
+				      device_event_dma_addr, ring_size_in_bytes,
+				      event_size_in_bytes, state, extra);
+	vring_virtqueue_init_packed(vq, vdev);
+	return 0;
+
+err_state_extra:
+	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
+	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
+	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
+err_ring:
+	return -ENOMEM;
+}
+
 
 /*
  * Generic functions and exported symbols.
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 18/32] virtio_ring: introduce virtqueue_resize()

[Xuan Zhuo]

Introduce virtqueue_resize() to implement the resize of vring.
Based on these, the driver can dynamically adjust the size of the vring.
For example: ethtool -G.

virtqueue_resize() implements resize based on the vq reset function. In
case of failure to allocate a new vring, it will give up resize and use
the original vring.

During this process, if the re-enable reset vq fails, the vq can no
longer be used. Although the probability of this situation is not high.

The parameter recycle is used to recycle the buffer that is no longer
used.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++++++++++
 include/linux/virtio.h       |  3 ++
 2 files changed, 72 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 06f66b15c86c..6250e19fc5bf 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2554,6 +2554,75 @@ struct virtqueue *vring_create_virtqueue(
 }
 EXPORT_SYMBOL_GPL(vring_create_virtqueue);
 
+/**
+ * virtqueue_resize - resize the vring of vq
+ * @_vq: the struct virtqueue we're talking about.
+ * @num: new ring num
+ * @recycle: callback for recycle the useless buffer
+ *
+ * When it is really necessary to create a new vring, it will set the current vq
+ * into the reset state. Then call the passed callback to recycle the buffer
+ * that is no longer used. Only after the new vring is successfully created, the
+ * old vring will be released.
+ *
+ * Caller must ensure we don't call this with other virtqueue operations
+ * at the same time (except where noted).
+ *
+ * Returns zero or a negative error.
+ */
+int virtqueue_resize(struct virtqueue *_vq, u32 num,
+		     void (*recycle)(struct virtqueue *vq, void *buf))
+{
+	struct vring_virtqueue *vq = to_vvq(_vq);
+	struct virtio_device *vdev = vq->vq.vdev;
+	bool packed;
+	void *buf;
+	int err;
+
+	if (!vq->we_own_ring)
+		return -EINVAL;
+
+	if (num > vq->vq.num_max)
+		return -E2BIG;
+
+	if (!num)
+		return -EINVAL;
+
+	packed = virtio_has_feature(vdev, VIRTIO_F_RING_PACKED) ? true : false;
+
+	if ((packed ? vq->packed.vring.num : vq->split.vring.num) == num)
+		return 0;
+
+	if (!vdev->config->reset_vq)
+		return -ENOENT;
+
+	if (!vdev->config->enable_reset_vq)
+		return -ENOENT;
+
+	err = vdev->config->reset_vq(_vq);
+	if (err)
+		return err;
+
+	while ((buf = virtqueue_detach_unused_buf(_vq)) != NULL)
+		recycle(_vq, buf);
+
+	if (packed) {
+		err = virtqueue_resize_packed(_vq, num);
+		if (err)
+			virtqueue_reinit_packed(vq);
+	} else {
+		err = virtqueue_resize_split(_vq, num);
+		if (err)
+			virtqueue_reinit_split(vq);
+	}
+
+	if (vdev->config->enable_reset_vq(_vq))
+		return -EBUSY;
+
+	return err;
+}
+EXPORT_SYMBOL_GPL(virtqueue_resize);
+
 /* Only available for split ring */
 struct virtqueue *vring_new_virtqueue(unsigned int index,
 				      unsigned int num,
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index d59adc4be068..c86ff02e0ca0 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -91,6 +91,9 @@ dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
 dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
 dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
 
+int virtqueue_resize(struct virtqueue *vq, u32 num,
+		     void (*recycle)(struct virtqueue *vq, void *buf));
+
 /**
  * virtio_device - representation of a device using virtio
  * @index: unique position on the virtio bus
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 18/32] virtio_ring: introduce virtqueue_resize()

[Xuan Zhuo]

Introduce virtqueue_resize() to implement the resize of vring.
Based on these, the driver can dynamically adjust the size of the vring.
For example: ethtool -G.

virtqueue_resize() implements resize based on the vq reset function. In
case of failure to allocate a new vring, it will give up resize and use
the original vring.

During this process, if the re-enable reset vq fails, the vq can no
longer be used. Although the probability of this situation is not high.

The parameter recycle is used to recycle the buffer that is no longer
used.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++++++++++
 include/linux/virtio.h       |  3 ++
 2 files changed, 72 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 06f66b15c86c..6250e19fc5bf 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2554,6 +2554,75 @@ struct virtqueue *vring_create_virtqueue(
 }
 EXPORT_SYMBOL_GPL(vring_create_virtqueue);
 
+/**
+ * virtqueue_resize - resize the vring of vq
+ * @_vq: the struct virtqueue we're talking about.
+ * @num: new ring num
+ * @recycle: callback for recycle the useless buffer
+ *
+ * When it is really necessary to create a new vring, it will set the current vq
+ * into the reset state. Then call the passed callback to recycle the buffer
+ * that is no longer used. Only after the new vring is successfully created, the
+ * old vring will be released.
+ *
+ * Caller must ensure we don't call this with other virtqueue operations
+ * at the same time (except where noted).
+ *
+ * Returns zero or a negative error.
+ */
+int virtqueue_resize(struct virtqueue *_vq, u32 num,
+		     void (*recycle)(struct virtqueue *vq, void *buf))
+{
+	struct vring_virtqueue *vq = to_vvq(_vq);
+	struct virtio_device *vdev = vq->vq.vdev;
+	bool packed;
+	void *buf;
+	int err;
+
+	if (!vq->we_own_ring)
+		return -EINVAL;
+
+	if (num > vq->vq.num_max)
+		return -E2BIG;
+
+	if (!num)
+		return -EINVAL;
+
+	packed = virtio_has_feature(vdev, VIRTIO_F_RING_PACKED) ? true : false;
+
+	if ((packed ? vq->packed.vring.num : vq->split.vring.num) == num)
+		return 0;
+
+	if (!vdev->config->reset_vq)
+		return -ENOENT;
+
+	if (!vdev->config->enable_reset_vq)
+		return -ENOENT;
+
+	err = vdev->config->reset_vq(_vq);
+	if (err)
+		return err;
+
+	while ((buf = virtqueue_detach_unused_buf(_vq)) != NULL)
+		recycle(_vq, buf);
+
+	if (packed) {
+		err = virtqueue_resize_packed(_vq, num);
+		if (err)
+			virtqueue_reinit_packed(vq);
+	} else {
+		err = virtqueue_resize_split(_vq, num);
+		if (err)
+			virtqueue_reinit_split(vq);
+	}
+
+	if (vdev->config->enable_reset_vq(_vq))
+		return -EBUSY;
+
+	return err;
+}
+EXPORT_SYMBOL_GPL(virtqueue_resize);
+
 /* Only available for split ring */
 struct virtqueue *vring_new_virtqueue(unsigned int index,
 				      unsigned int num,
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index d59adc4be068..c86ff02e0ca0 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -91,6 +91,9 @@ dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
 dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
 dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
 
+int virtqueue_resize(struct virtqueue *vq, u32 num,
+		     void (*recycle)(struct virtqueue *vq, void *buf));
+
 /**
  * virtio_device - representation of a device using virtio
  * @index: unique position on the virtio bus
-- 
2.31.0

[PATCH v9 18/32] virtio_ring: introduce virtqueue_resize()

[Xuan Zhuo]

Introduce virtqueue_resize() to implement the resize of vring.
Based on these, the driver can dynamically adjust the size of the vring.
For example: ethtool -G.

virtqueue_resize() implements resize based on the vq reset function. In
case of failure to allocate a new vring, it will give up resize and use
the original vring.

During this process, if the re-enable reset vq fails, the vq can no
longer be used. Although the probability of this situation is not high.

The parameter recycle is used to recycle the buffer that is no longer
used.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++++++++++
 include/linux/virtio.h       |  3 ++
 2 files changed, 72 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 06f66b15c86c..6250e19fc5bf 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2554,6 +2554,75 @@ struct virtqueue *vring_create_virtqueue(
 }
 EXPORT_SYMBOL_GPL(vring_create_virtqueue);
 
+/**
+ * virtqueue_resize - resize the vring of vq
+ * @_vq: the struct virtqueue we're talking about.
+ * @num: new ring num
+ * @recycle: callback for recycle the useless buffer
+ *
+ * When it is really necessary to create a new vring, it will set the current vq
+ * into the reset state. Then call the passed callback to recycle the buffer
+ * that is no longer used. Only after the new vring is successfully created, the
+ * old vring will be released.
+ *
+ * Caller must ensure we don't call this with other virtqueue operations
+ * at the same time (except where noted).
+ *
+ * Returns zero or a negative error.
+ */
+int virtqueue_resize(struct virtqueue *_vq, u32 num,
+		     void (*recycle)(struct virtqueue *vq, void *buf))
+{
+	struct vring_virtqueue *vq = to_vvq(_vq);
+	struct virtio_device *vdev = vq->vq.vdev;
+	bool packed;
+	void *buf;
+	int err;
+
+	if (!vq->we_own_ring)
+		return -EINVAL;
+
+	if (num > vq->vq.num_max)
+		return -E2BIG;
+
+	if (!num)
+		return -EINVAL;
+
+	packed = virtio_has_feature(vdev, VIRTIO_F_RING_PACKED) ? true : false;
+
+	if ((packed ? vq->packed.vring.num : vq->split.vring.num) == num)
+		return 0;
+
+	if (!vdev->config->reset_vq)
+		return -ENOENT;
+
+	if (!vdev->config->enable_reset_vq)
+		return -ENOENT;
+
+	err = vdev->config->reset_vq(_vq);
+	if (err)
+		return err;
+
+	while ((buf = virtqueue_detach_unused_buf(_vq)) != NULL)
+		recycle(_vq, buf);
+
+	if (packed) {
+		err = virtqueue_resize_packed(_vq, num);
+		if (err)
+			virtqueue_reinit_packed(vq);
+	} else {
+		err = virtqueue_resize_split(_vq, num);
+		if (err)
+			virtqueue_reinit_split(vq);
+	}
+
+	if (vdev->config->enable_reset_vq(_vq))
+		return -EBUSY;
+
+	return err;
+}
+EXPORT_SYMBOL_GPL(virtqueue_resize);
+
 /* Only available for split ring */
 struct virtqueue *vring_new_virtqueue(unsigned int index,
 				      unsigned int num,
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index d59adc4be068..c86ff02e0ca0 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -91,6 +91,9 @@ dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
 dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
 dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
 
+int virtqueue_resize(struct virtqueue *vq, u32 num,
+		     void (*recycle)(struct virtqueue *vq, void *buf));
+
 /**
  * virtio_device - representation of a device using virtio
  * @index: unique position on the virtio bus
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 19/32] virtio_pci: struct virtio_pci_common_cfg add queue_notify_data

[Xuan Zhuo]

Add queue_notify_data in struct virtio_pci_common_cfg, which comes from
here https://github.com/oasis-tcs/virtio-spec/issues/89

For not breaks uABI, add a new struct virtio_pci_common_cfg_notify.

Since I want to add queue_reset after queue_notify_data, I submitted
this patch first.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
 include/uapi/linux/virtio_pci.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/include/uapi/linux/virtio_pci.h b/include/uapi/linux/virtio_pci.h
index 3a86f36d7e3d..22bec9bd0dfc 100644
--- a/include/uapi/linux/virtio_pci.h
+++ b/include/uapi/linux/virtio_pci.h
@@ -166,6 +166,13 @@ struct virtio_pci_common_cfg {
 	__le32 queue_used_hi;		/* read-write */
 };
 
+struct virtio_pci_common_cfg_notify {
+	struct virtio_pci_common_cfg cfg;
+
+	__le16 queue_notify_data;	/* read-write */
+	__le16 padding;
+};
+
 /* Fields in VIRTIO_PCI_CAP_PCI_CFG: */
 struct virtio_pci_cfg_cap {
 	struct virtio_pci_cap cap;
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 19/32] virtio_pci: struct virtio_pci_common_cfg add queue_notify_data

[Xuan Zhuo]

Add queue_notify_data in struct virtio_pci_common_cfg, which comes from
here https://github.com/oasis-tcs/virtio-spec/issues/89

For not breaks uABI, add a new struct virtio_pci_common_cfg_notify.

Since I want to add queue_reset after queue_notify_data, I submitted
this patch first.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
 include/uapi/linux/virtio_pci.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/include/uapi/linux/virtio_pci.h b/include/uapi/linux/virtio_pci.h
index 3a86f36d7e3d..22bec9bd0dfc 100644
--- a/include/uapi/linux/virtio_pci.h
+++ b/include/uapi/linux/virtio_pci.h
@@ -166,6 +166,13 @@ struct virtio_pci_common_cfg {
 	__le32 queue_used_hi;		/* read-write */
 };
 
+struct virtio_pci_common_cfg_notify {
+	struct virtio_pci_common_cfg cfg;
+
+	__le16 queue_notify_data;	/* read-write */
+	__le16 padding;
+};
+
 /* Fields in VIRTIO_PCI_CAP_PCI_CFG: */
 struct virtio_pci_cfg_cap {
 	struct virtio_pci_cap cap;
-- 
2.31.0

[PATCH v9 19/32] virtio_pci: struct virtio_pci_common_cfg add queue_notify_data

[Xuan Zhuo]

Add queue_notify_data in struct virtio_pci_common_cfg, which comes from
here https://github.com/oasis-tcs/virtio-spec/issues/89

For not breaks uABI, add a new struct virtio_pci_common_cfg_notify.

Since I want to add queue_reset after queue_notify_data, I submitted
this patch first.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
 include/uapi/linux/virtio_pci.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/include/uapi/linux/virtio_pci.h b/include/uapi/linux/virtio_pci.h
index 3a86f36d7e3d..22bec9bd0dfc 100644
--- a/include/uapi/linux/virtio_pci.h
+++ b/include/uapi/linux/virtio_pci.h
@@ -166,6 +166,13 @@ struct virtio_pci_common_cfg {
 	__le32 queue_used_hi;		/* read-write */
 };
 
+struct virtio_pci_common_cfg_notify {
+	struct virtio_pci_common_cfg cfg;
+
+	__le16 queue_notify_data;	/* read-write */
+	__le16 padding;
+};
+
 /* Fields in VIRTIO_PCI_CAP_PCI_CFG: */
 struct virtio_pci_cfg_cap {
 	struct virtio_pci_cap cap;
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 20/32] virtio: queue_reset: add VIRTIO_F_RING_RESET

[Xuan Zhuo]

Added VIRTIO_F_RING_RESET, it came from here
https://github.com/oasis-tcs/virtio-spec/issues/124

This feature indicates that the driver can reset a queue individually.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
 include/uapi/linux/virtio_config.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/include/uapi/linux/virtio_config.h b/include/uapi/linux/virtio_config.h
index b5eda06f0d57..0862be802ff8 100644
--- a/include/uapi/linux/virtio_config.h
+++ b/include/uapi/linux/virtio_config.h
@@ -52,7 +52,7 @@
  * rest are per-device feature bits.
  */
 #define VIRTIO_TRANSPORT_F_START	28
-#define VIRTIO_TRANSPORT_F_END		38
+#define VIRTIO_TRANSPORT_F_END		41
 
 #ifndef VIRTIO_CONFIG_NO_LEGACY
 /* Do we get callbacks when the ring is completely used, even if we've
@@ -92,4 +92,9 @@
  * Does the device support Single Root I/O Virtualization?
  */
 #define VIRTIO_F_SR_IOV			37
+
+/*
+ * This feature indicates that the driver can reset a queue individually.
+ */
+#define VIRTIO_F_RING_RESET		40
 #endif /* _UAPI_LINUX_VIRTIO_CONFIG_H */
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 20/32] virtio: queue_reset: add VIRTIO_F_RING_RESET

[Xuan Zhuo]

Added VIRTIO_F_RING_RESET, it came from here
https://github.com/oasis-tcs/virtio-spec/issues/124

This feature indicates that the driver can reset a queue individually.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
 include/uapi/linux/virtio_config.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/include/uapi/linux/virtio_config.h b/include/uapi/linux/virtio_config.h
index b5eda06f0d57..0862be802ff8 100644
--- a/include/uapi/linux/virtio_config.h
+++ b/include/uapi/linux/virtio_config.h
@@ -52,7 +52,7 @@
  * rest are per-device feature bits.
  */
 #define VIRTIO_TRANSPORT_F_START	28
-#define VIRTIO_TRANSPORT_F_END		38
+#define VIRTIO_TRANSPORT_F_END		41
 
 #ifndef VIRTIO_CONFIG_NO_LEGACY
 /* Do we get callbacks when the ring is completely used, even if we've
@@ -92,4 +92,9 @@
  * Does the device support Single Root I/O Virtualization?
  */
 #define VIRTIO_F_SR_IOV			37
+
+/*
+ * This feature indicates that the driver can reset a queue individually.
+ */
+#define VIRTIO_F_RING_RESET		40
 #endif /* _UAPI_LINUX_VIRTIO_CONFIG_H */
-- 
2.31.0

[PATCH v9 20/32] virtio: queue_reset: add VIRTIO_F_RING_RESET

[Xuan Zhuo]

Added VIRTIO_F_RING_RESET, it came from here
https://github.com/oasis-tcs/virtio-spec/issues/124

This feature indicates that the driver can reset a queue individually.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
 include/uapi/linux/virtio_config.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/include/uapi/linux/virtio_config.h b/include/uapi/linux/virtio_config.h
index b5eda06f0d57..0862be802ff8 100644
--- a/include/uapi/linux/virtio_config.h
+++ b/include/uapi/linux/virtio_config.h
@@ -52,7 +52,7 @@
  * rest are per-device feature bits.
  */
 #define VIRTIO_TRANSPORT_F_START	28
-#define VIRTIO_TRANSPORT_F_END		38
+#define VIRTIO_TRANSPORT_F_END		41
 
 #ifndef VIRTIO_CONFIG_NO_LEGACY
 /* Do we get callbacks when the ring is completely used, even if we've
@@ -92,4 +92,9 @@
  * Does the device support Single Root I/O Virtualization?
  */
 #define VIRTIO_F_SR_IOV			37
+
+/*
+ * This feature indicates that the driver can reset a queue individually.
+ */
+#define VIRTIO_F_RING_RESET		40
 #endif /* _UAPI_LINUX_VIRTIO_CONFIG_H */
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 21/32] virtio_pci: queue_reset: update struct virtio_pci_common_cfg and option functions

[Xuan Zhuo]

Add queue_reset in virtio_pci_common_cfg, and add related operation
functions.

For not breaks uABI, add a new struct virtio_pci_common_cfg_reset.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_pci_modern_dev.c | 36 ++++++++++++++++++++++++++
 include/linux/virtio_pci_modern.h      |  2 ++
 include/uapi/linux/virtio_pci.h        |  7 +++++
 3 files changed, 45 insertions(+)

diff --git a/drivers/virtio/virtio_pci_modern_dev.c b/drivers/virtio/virtio_pci_modern_dev.c
index e8b3ff2b9fbc..8c74b00bc511 100644
--- a/drivers/virtio/virtio_pci_modern_dev.c
+++ b/drivers/virtio/virtio_pci_modern_dev.c
@@ -3,6 +3,7 @@
 #include <linux/virtio_pci_modern.h>
 #include <linux/module.h>
 #include <linux/pci.h>
+#include <linux/delay.h>
 
 /*
  * vp_modern_map_capability - map a part of virtio pci capability
@@ -463,6 +464,41 @@ void vp_modern_set_status(struct virtio_pci_modern_device *mdev,
 }
 EXPORT_SYMBOL_GPL(vp_modern_set_status);
 
+/*
+ * vp_modern_get_queue_reset - get the queue reset status
+ * @mdev: the modern virtio-pci device
+ * @index: queue index
+ */
+int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index)
+{
+	struct virtio_pci_common_cfg_reset __iomem *cfg;
+
+	cfg = (struct virtio_pci_common_cfg_reset __iomem *)mdev->common;
+
+	vp_iowrite16(index, &cfg->cfg.queue_select);
+	return vp_ioread16(&cfg->queue_reset);
+}
+EXPORT_SYMBOL_GPL(vp_modern_get_queue_reset);
+
+/*
+ * vp_modern_set_queue_reset - reset the queue
+ * @mdev: the modern virtio-pci device
+ * @index: queue index
+ */
+void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index)
+{
+	struct virtio_pci_common_cfg_reset __iomem *cfg;
+
+	cfg = (struct virtio_pci_common_cfg_reset __iomem *)mdev->common;
+
+	vp_iowrite16(index, &cfg->cfg.queue_select);
+	vp_iowrite16(1, &cfg->queue_reset);
+
+	while (vp_ioread16(&cfg->queue_reset) != 1)
+		msleep(1);
+}
+EXPORT_SYMBOL_GPL(vp_modern_set_queue_reset);
+
 /*
  * vp_modern_queue_vector - set the MSIX vector for a specific virtqueue
  * @mdev: the modern virtio-pci device
diff --git a/include/linux/virtio_pci_modern.h b/include/linux/virtio_pci_modern.h
index eb2bd9b4077d..cc4154dd7b28 100644
--- a/include/linux/virtio_pci_modern.h
+++ b/include/linux/virtio_pci_modern.h
@@ -106,4 +106,6 @@ void __iomem * vp_modern_map_vq_notify(struct virtio_pci_modern_device *mdev,
 				       u16 index, resource_size_t *pa);
 int vp_modern_probe(struct virtio_pci_modern_device *mdev);
 void vp_modern_remove(struct virtio_pci_modern_device *mdev);
+int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index);
+void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index);
 #endif
diff --git a/include/uapi/linux/virtio_pci.h b/include/uapi/linux/virtio_pci.h
index 22bec9bd0dfc..d9462efd6ce8 100644
--- a/include/uapi/linux/virtio_pci.h
+++ b/include/uapi/linux/virtio_pci.h
@@ -173,6 +173,13 @@ struct virtio_pci_common_cfg_notify {
 	__le16 padding;
 };
 
+struct virtio_pci_common_cfg_reset {
+	struct virtio_pci_common_cfg cfg;
+
+	__le16 queue_notify_data;	/* read-write */
+	__le16 queue_reset;		/* read-write */
+};
+
 /* Fields in VIRTIO_PCI_CAP_PCI_CFG: */
 struct virtio_pci_cfg_cap {
 	struct virtio_pci_cap cap;
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 21/32] virtio_pci: queue_reset: update struct virtio_pci_common_cfg and option functions

[Xuan Zhuo]

Add queue_reset in virtio_pci_common_cfg, and add related operation
functions.

For not breaks uABI, add a new struct virtio_pci_common_cfg_reset.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_pci_modern_dev.c | 36 ++++++++++++++++++++++++++
 include/linux/virtio_pci_modern.h      |  2 ++
 include/uapi/linux/virtio_pci.h        |  7 +++++
 3 files changed, 45 insertions(+)

diff --git a/drivers/virtio/virtio_pci_modern_dev.c b/drivers/virtio/virtio_pci_modern_dev.c
index e8b3ff2b9fbc..8c74b00bc511 100644
--- a/drivers/virtio/virtio_pci_modern_dev.c
+++ b/drivers/virtio/virtio_pci_modern_dev.c
@@ -3,6 +3,7 @@
 #include <linux/virtio_pci_modern.h>
 #include <linux/module.h>
 #include <linux/pci.h>
+#include <linux/delay.h>
 
 /*
  * vp_modern_map_capability - map a part of virtio pci capability
@@ -463,6 +464,41 @@ void vp_modern_set_status(struct virtio_pci_modern_device *mdev,
 }
 EXPORT_SYMBOL_GPL(vp_modern_set_status);
 
+/*
+ * vp_modern_get_queue_reset - get the queue reset status
+ * @mdev: the modern virtio-pci device
+ * @index: queue index
+ */
+int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index)
+{
+	struct virtio_pci_common_cfg_reset __iomem *cfg;
+
+	cfg = (struct virtio_pci_common_cfg_reset __iomem *)mdev->common;
+
+	vp_iowrite16(index, &cfg->cfg.queue_select);
+	return vp_ioread16(&cfg->queue_reset);
+}
+EXPORT_SYMBOL_GPL(vp_modern_get_queue_reset);
+
+/*
+ * vp_modern_set_queue_reset - reset the queue
+ * @mdev: the modern virtio-pci device
+ * @index: queue index
+ */
+void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index)
+{
+	struct virtio_pci_common_cfg_reset __iomem *cfg;
+
+	cfg = (struct virtio_pci_common_cfg_reset __iomem *)mdev->common;
+
+	vp_iowrite16(index, &cfg->cfg.queue_select);
+	vp_iowrite16(1, &cfg->queue_reset);
+
+	while (vp_ioread16(&cfg->queue_reset) != 1)
+		msleep(1);
+}
+EXPORT_SYMBOL_GPL(vp_modern_set_queue_reset);
+
 /*
  * vp_modern_queue_vector - set the MSIX vector for a specific virtqueue
  * @mdev: the modern virtio-pci device
diff --git a/include/linux/virtio_pci_modern.h b/include/linux/virtio_pci_modern.h
index eb2bd9b4077d..cc4154dd7b28 100644
--- a/include/linux/virtio_pci_modern.h
+++ b/include/linux/virtio_pci_modern.h
@@ -106,4 +106,6 @@ void __iomem * vp_modern_map_vq_notify(struct virtio_pci_modern_device *mdev,
 				       u16 index, resource_size_t *pa);
 int vp_modern_probe(struct virtio_pci_modern_device *mdev);
 void vp_modern_remove(struct virtio_pci_modern_device *mdev);
+int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index);
+void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index);
 #endif
diff --git a/include/uapi/linux/virtio_pci.h b/include/uapi/linux/virtio_pci.h
index 22bec9bd0dfc..d9462efd6ce8 100644
--- a/include/uapi/linux/virtio_pci.h
+++ b/include/uapi/linux/virtio_pci.h
@@ -173,6 +173,13 @@ struct virtio_pci_common_cfg_notify {
 	__le16 padding;
 };
 
+struct virtio_pci_common_cfg_reset {
+	struct virtio_pci_common_cfg cfg;
+
+	__le16 queue_notify_data;	/* read-write */
+	__le16 queue_reset;		/* read-write */
+};
+
 /* Fields in VIRTIO_PCI_CAP_PCI_CFG: */
 struct virtio_pci_cfg_cap {
 	struct virtio_pci_cap cap;
-- 
2.31.0

[PATCH v9 21/32] virtio_pci: queue_reset: update struct virtio_pci_common_cfg and option functions

[Xuan Zhuo]

Add queue_reset in virtio_pci_common_cfg, and add related operation
functions.

For not breaks uABI, add a new struct virtio_pci_common_cfg_reset.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_pci_modern_dev.c | 36 ++++++++++++++++++++++++++
 include/linux/virtio_pci_modern.h      |  2 ++
 include/uapi/linux/virtio_pci.h        |  7 +++++
 3 files changed, 45 insertions(+)

diff --git a/drivers/virtio/virtio_pci_modern_dev.c b/drivers/virtio/virtio_pci_modern_dev.c
index e8b3ff2b9fbc..8c74b00bc511 100644
--- a/drivers/virtio/virtio_pci_modern_dev.c
+++ b/drivers/virtio/virtio_pci_modern_dev.c
@@ -3,6 +3,7 @@
 #include <linux/virtio_pci_modern.h>
 #include <linux/module.h>
 #include <linux/pci.h>
+#include <linux/delay.h>
 
 /*
  * vp_modern_map_capability - map a part of virtio pci capability
@@ -463,6 +464,41 @@ void vp_modern_set_status(struct virtio_pci_modern_device *mdev,
 }
 EXPORT_SYMBOL_GPL(vp_modern_set_status);
 
+/*
+ * vp_modern_get_queue_reset - get the queue reset status
+ * @mdev: the modern virtio-pci device
+ * @index: queue index
+ */
+int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index)
+{
+	struct virtio_pci_common_cfg_reset __iomem *cfg;
+
+	cfg = (struct virtio_pci_common_cfg_reset __iomem *)mdev->common;
+
+	vp_iowrite16(index, &cfg->cfg.queue_select);
+	return vp_ioread16(&cfg->queue_reset);
+}
+EXPORT_SYMBOL_GPL(vp_modern_get_queue_reset);
+
+/*
+ * vp_modern_set_queue_reset - reset the queue
+ * @mdev: the modern virtio-pci device
+ * @index: queue index
+ */
+void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index)
+{
+	struct virtio_pci_common_cfg_reset __iomem *cfg;
+
+	cfg = (struct virtio_pci_common_cfg_reset __iomem *)mdev->common;
+
+	vp_iowrite16(index, &cfg->cfg.queue_select);
+	vp_iowrite16(1, &cfg->queue_reset);
+
+	while (vp_ioread16(&cfg->queue_reset) != 1)
+		msleep(1);
+}
+EXPORT_SYMBOL_GPL(vp_modern_set_queue_reset);
+
 /*
  * vp_modern_queue_vector - set the MSIX vector for a specific virtqueue
  * @mdev: the modern virtio-pci device
diff --git a/include/linux/virtio_pci_modern.h b/include/linux/virtio_pci_modern.h
index eb2bd9b4077d..cc4154dd7b28 100644
--- a/include/linux/virtio_pci_modern.h
+++ b/include/linux/virtio_pci_modern.h
@@ -106,4 +106,6 @@ void __iomem * vp_modern_map_vq_notify(struct virtio_pci_modern_device *mdev,
 				       u16 index, resource_size_t *pa);
 int vp_modern_probe(struct virtio_pci_modern_device *mdev);
 void vp_modern_remove(struct virtio_pci_modern_device *mdev);
+int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index);
+void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index);
 #endif
diff --git a/include/uapi/linux/virtio_pci.h b/include/uapi/linux/virtio_pci.h
index 22bec9bd0dfc..d9462efd6ce8 100644
--- a/include/uapi/linux/virtio_pci.h
+++ b/include/uapi/linux/virtio_pci.h
@@ -173,6 +173,13 @@ struct virtio_pci_common_cfg_notify {
 	__le16 padding;
 };
 
+struct virtio_pci_common_cfg_reset {
+	struct virtio_pci_common_cfg cfg;
+
+	__le16 queue_notify_data;	/* read-write */
+	__le16 queue_reset;		/* read-write */
+};
+
 /* Fields in VIRTIO_PCI_CAP_PCI_CFG: */
 struct virtio_pci_cfg_cap {
 	struct virtio_pci_cap cap;
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 22/32] virtio_pci: queue_reset: extract the logic of active vq for modern pci

[Xuan Zhuo]

Introduce vp_active_vq() to configure vring to backend after vq attach
vring. And configure vq vector if necessary.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_pci_modern.c | 46 ++++++++++++++++++------------
 1 file changed, 28 insertions(+), 18 deletions(-)

diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index 86d301f272b8..49a4493732cf 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -176,6 +176,29 @@ static void vp_reset(struct virtio_device *vdev)
 	vp_disable_cbs(vdev);
 }
 
+static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
+{
+	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
+	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
+	unsigned long index;
+
+	index = vq->index;
+
+	/* activate the queue */
+	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
+	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
+				virtqueue_get_avail_addr(vq),
+				virtqueue_get_used_addr(vq));
+
+	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
+		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
+		if (msix_vec == VIRTIO_MSI_NO_VECTOR)
+			return -EBUSY;
+	}
+
+	return 0;
+}
+
 static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
 {
 	return vp_modern_config_vector(&vp_dev->mdev, vector);
@@ -220,32 +243,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 
 	vq->num_max = num;
 
-	/* activate the queue */
-	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
-	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
-				virtqueue_get_avail_addr(vq),
-				virtqueue_get_used_addr(vq));
+	err = vp_active_vq(vq, msix_vec);
+	if (err)
+		goto err;
 
 	vq->priv = (void __force *)vp_modern_map_vq_notify(mdev, index, NULL);
 	if (!vq->priv) {
 		err = -ENOMEM;
-		goto err_map_notify;
-	}
-
-	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
-		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
-		if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
-			err = -EBUSY;
-			goto err_assign_vector;
-		}
+		goto err;
 	}
 
 	return vq;
 
-err_assign_vector:
-	if (!mdev->notify_base)
-		pci_iounmap(mdev->pci_dev, (void __iomem __force *)vq->priv);
-err_map_notify:
+err:
 	vring_del_virtqueue(vq);
 	return ERR_PTR(err);
 }
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 22/32] virtio_pci: queue_reset: extract the logic of active vq for modern pci

[Xuan Zhuo]

Introduce vp_active_vq() to configure vring to backend after vq attach
vring. And configure vq vector if necessary.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_pci_modern.c | 46 ++++++++++++++++++------------
 1 file changed, 28 insertions(+), 18 deletions(-)

diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index 86d301f272b8..49a4493732cf 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -176,6 +176,29 @@ static void vp_reset(struct virtio_device *vdev)
 	vp_disable_cbs(vdev);
 }
 
+static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
+{
+	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
+	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
+	unsigned long index;
+
+	index = vq->index;
+
+	/* activate the queue */
+	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
+	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
+				virtqueue_get_avail_addr(vq),
+				virtqueue_get_used_addr(vq));
+
+	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
+		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
+		if (msix_vec == VIRTIO_MSI_NO_VECTOR)
+			return -EBUSY;
+	}
+
+	return 0;
+}
+
 static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
 {
 	return vp_modern_config_vector(&vp_dev->mdev, vector);
@@ -220,32 +243,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 
 	vq->num_max = num;
 
-	/* activate the queue */
-	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
-	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
-				virtqueue_get_avail_addr(vq),
-				virtqueue_get_used_addr(vq));
+	err = vp_active_vq(vq, msix_vec);
+	if (err)
+		goto err;
 
 	vq->priv = (void __force *)vp_modern_map_vq_notify(mdev, index, NULL);
 	if (!vq->priv) {
 		err = -ENOMEM;
-		goto err_map_notify;
-	}
-
-	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
-		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
-		if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
-			err = -EBUSY;
-			goto err_assign_vector;
-		}
+		goto err;
 	}
 
 	return vq;
 
-err_assign_vector:
-	if (!mdev->notify_base)
-		pci_iounmap(mdev->pci_dev, (void __iomem __force *)vq->priv);
-err_map_notify:
+err:
 	vring_del_virtqueue(vq);
 	return ERR_PTR(err);
 }
-- 
2.31.0

[PATCH v9 22/32] virtio_pci: queue_reset: extract the logic of active vq for modern pci

[Xuan Zhuo]

Introduce vp_active_vq() to configure vring to backend after vq attach
vring. And configure vq vector if necessary.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_pci_modern.c | 46 ++++++++++++++++++------------
 1 file changed, 28 insertions(+), 18 deletions(-)

diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index 86d301f272b8..49a4493732cf 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -176,6 +176,29 @@ static void vp_reset(struct virtio_device *vdev)
 	vp_disable_cbs(vdev);
 }
 
+static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
+{
+	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
+	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
+	unsigned long index;
+
+	index = vq->index;
+
+	/* activate the queue */
+	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
+	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
+				virtqueue_get_avail_addr(vq),
+				virtqueue_get_used_addr(vq));
+
+	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
+		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
+		if (msix_vec == VIRTIO_MSI_NO_VECTOR)
+			return -EBUSY;
+	}
+
+	return 0;
+}
+
 static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
 {
 	return vp_modern_config_vector(&vp_dev->mdev, vector);
@@ -220,32 +243,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 
 	vq->num_max = num;
 
-	/* activate the queue */
-	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
-	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
-				virtqueue_get_avail_addr(vq),
-				virtqueue_get_used_addr(vq));
+	err = vp_active_vq(vq, msix_vec);
+	if (err)
+		goto err;
 
 	vq->priv = (void __force *)vp_modern_map_vq_notify(mdev, index, NULL);
 	if (!vq->priv) {
 		err = -ENOMEM;
-		goto err_map_notify;
-	}
-
-	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
-		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
-		if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
-			err = -EBUSY;
-			goto err_assign_vector;
-		}
+		goto err;
 	}
 
 	return vq;
 
-err_assign_vector:
-	if (!mdev->notify_base)
-		pci_iounmap(mdev->pci_dev, (void __iomem __force *)vq->priv);
-err_map_notify:
+err:
 	vring_del_virtqueue(vq);
 	return ERR_PTR(err);
 }
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Xuan Zhuo]

This patch implements virtio pci support for QUEUE RESET.

Performing reset on a queue is divided into these steps:

 1. notify the device to reset the queue
 2. recycle the buffer submitted
 3. reset the vring (may re-alloc)
 4. mmap vring to device, and enable the queue

This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
pci scenario.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_pci_common.c |  8 +--
 drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
 drivers/virtio/virtio_ring.c       |  2 +
 include/linux/virtio.h             |  1 +
 4 files changed, 92 insertions(+), 3 deletions(-)

diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index fdbde1db5ec5..863d3a8a0956 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
 	struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
 	unsigned long flags;
 
-	spin_lock_irqsave(&vp_dev->lock, flags);
-	list_del(&info->node);
-	spin_unlock_irqrestore(&vp_dev->lock, flags);
+	if (!vq->reset) {
+		spin_lock_irqsave(&vp_dev->lock, flags);
+		list_del(&info->node);
+		spin_unlock_irqrestore(&vp_dev->lock, flags);
+	}
 
 	vp_dev->del_vq(info);
 	kfree(info);
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index 49a4493732cf..cb5d38f1c9c8 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
 	if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
 			pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
 		__virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
+
+	if (features & BIT_ULL(VIRTIO_F_RING_RESET))
+		__virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
 }
 
 /* virtio config->finalize_features() implementation */
@@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
 	return 0;
 }
 
+static int vp_modern_reset_vq(struct virtqueue *vq)
+{
+	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
+	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
+	struct virtio_pci_vq_info *info;
+	unsigned long flags;
+
+	if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
+		return -ENOENT;
+
+	vp_modern_set_queue_reset(mdev, vq->index);
+
+	info = vp_dev->vqs[vq->index];
+
+	/* delete vq from irq handler */
+	spin_lock_irqsave(&vp_dev->lock, flags);
+	list_del(&info->node);
+	spin_unlock_irqrestore(&vp_dev->lock, flags);
+
+	INIT_LIST_HEAD(&info->node);
+
+	/* For the case where vq has an exclusive irq, to prevent the irq from
+	 * being received again and the pending irq, call disable_irq().
+	 *
+	 * In the scenario based on shared interrupts, vq will be searched from
+	 * the queue virtqueues. Since the previous list_del() has been deleted
+	 * from the queue, it is impossible for vq to be called in this case.
+	 * There is no need to close the corresponding interrupt.
+	 */
+	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
+		disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
+
+	vq->reset = true;
+
+	return 0;
+}
+
+static int vp_modern_enable_reset_vq(struct virtqueue *vq)
+{
+	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
+	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
+	struct virtio_pci_vq_info *info;
+	unsigned long flags, index;
+	int err;
+
+	if (!vq->reset)
+		return -EBUSY;
+
+	index = vq->index;
+	info = vp_dev->vqs[index];
+
+	/* check queue reset status */
+	if (vp_modern_get_queue_reset(mdev, index) != 1)
+		return -EBUSY;
+
+	err = vp_active_vq(vq, info->msix_vector);
+	if (err)
+		return err;
+
+	if (vq->callback) {
+		spin_lock_irqsave(&vp_dev->lock, flags);
+		list_add(&info->node, &vp_dev->virtqueues);
+		spin_unlock_irqrestore(&vp_dev->lock, flags);
+	} else {
+		INIT_LIST_HEAD(&info->node);
+	}
+
+	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
+
+	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
+		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
+
+	vq->reset = false;
+
+	return 0;
+}
+
 static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
 {
 	return vp_modern_config_vector(&vp_dev->mdev, vector);
@@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
 	.set_vq_affinity = vp_set_vq_affinity,
 	.get_vq_affinity = vp_get_vq_affinity,
 	.get_shm_region  = vp_get_shm_region,
+	.reset_vq	 = vp_modern_reset_vq,
+	.enable_reset_vq = vp_modern_enable_reset_vq,
 };
 
 static const struct virtio_config_ops virtio_pci_config_ops = {
@@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
 	.set_vq_affinity = vp_set_vq_affinity,
 	.get_vq_affinity = vp_get_vq_affinity,
 	.get_shm_region  = vp_get_shm_region,
+	.reset_vq	 = vp_modern_reset_vq,
+	.enable_reset_vq = vp_modern_enable_reset_vq,
 };
 
 /* the PCI probing function */
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 6250e19fc5bf..91937e21edca 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	vq->vq.vdev = vdev;
 	vq->vq.name = name;
 	vq->vq.index = index;
+	vq->vq.reset = false;
 	vq->notify = notify;
 	vq->weak_barriers = weak_barriers;
 
@@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	vq->vq.vdev = vdev;
 	vq->vq.name = name;
 	vq->vq.index = index;
+	vq->vq.reset = false;
 	vq->notify = notify;
 	vq->weak_barriers = weak_barriers;
 
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index c86ff02e0ca0..33ab003c5100 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -33,6 +33,7 @@ struct virtqueue {
 	unsigned int num_free;
 	unsigned int num_max;
 	void *priv;
+	bool reset;
 };
 
 int virtqueue_add_outbuf(struct virtqueue *vq,
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Xuan Zhuo]

This patch implements virtio pci support for QUEUE RESET.

Performing reset on a queue is divided into these steps:

 1. notify the device to reset the queue
 2. recycle the buffer submitted
 3. reset the vring (may re-alloc)
 4. mmap vring to device, and enable the queue

This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
pci scenario.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_pci_common.c |  8 +--
 drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
 drivers/virtio/virtio_ring.c       |  2 +
 include/linux/virtio.h             |  1 +
 4 files changed, 92 insertions(+), 3 deletions(-)

diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index fdbde1db5ec5..863d3a8a0956 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
 	struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
 	unsigned long flags;
 
-	spin_lock_irqsave(&vp_dev->lock, flags);
-	list_del(&info->node);
-	spin_unlock_irqrestore(&vp_dev->lock, flags);
+	if (!vq->reset) {
+		spin_lock_irqsave(&vp_dev->lock, flags);
+		list_del(&info->node);
+		spin_unlock_irqrestore(&vp_dev->lock, flags);
+	}
 
 	vp_dev->del_vq(info);
 	kfree(info);
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index 49a4493732cf..cb5d38f1c9c8 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
 	if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
 			pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
 		__virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
+
+	if (features & BIT_ULL(VIRTIO_F_RING_RESET))
+		__virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
 }
 
 /* virtio config->finalize_features() implementation */
@@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
 	return 0;
 }
 
+static int vp_modern_reset_vq(struct virtqueue *vq)
+{
+	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
+	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
+	struct virtio_pci_vq_info *info;
+	unsigned long flags;
+
+	if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
+		return -ENOENT;
+
+	vp_modern_set_queue_reset(mdev, vq->index);
+
+	info = vp_dev->vqs[vq->index];
+
+	/* delete vq from irq handler */
+	spin_lock_irqsave(&vp_dev->lock, flags);
+	list_del(&info->node);
+	spin_unlock_irqrestore(&vp_dev->lock, flags);
+
+	INIT_LIST_HEAD(&info->node);
+
+	/* For the case where vq has an exclusive irq, to prevent the irq from
+	 * being received again and the pending irq, call disable_irq().
+	 *
+	 * In the scenario based on shared interrupts, vq will be searched from
+	 * the queue virtqueues. Since the previous list_del() has been deleted
+	 * from the queue, it is impossible for vq to be called in this case.
+	 * There is no need to close the corresponding interrupt.
+	 */
+	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
+		disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
+
+	vq->reset = true;
+
+	return 0;
+}
+
+static int vp_modern_enable_reset_vq(struct virtqueue *vq)
+{
+	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
+	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
+	struct virtio_pci_vq_info *info;
+	unsigned long flags, index;
+	int err;
+
+	if (!vq->reset)
+		return -EBUSY;
+
+	index = vq->index;
+	info = vp_dev->vqs[index];
+
+	/* check queue reset status */
+	if (vp_modern_get_queue_reset(mdev, index) != 1)
+		return -EBUSY;
+
+	err = vp_active_vq(vq, info->msix_vector);
+	if (err)
+		return err;
+
+	if (vq->callback) {
+		spin_lock_irqsave(&vp_dev->lock, flags);
+		list_add(&info->node, &vp_dev->virtqueues);
+		spin_unlock_irqrestore(&vp_dev->lock, flags);
+	} else {
+		INIT_LIST_HEAD(&info->node);
+	}
+
+	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
+
+	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
+		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
+
+	vq->reset = false;
+
+	return 0;
+}
+
 static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
 {
 	return vp_modern_config_vector(&vp_dev->mdev, vector);
@@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
 	.set_vq_affinity = vp_set_vq_affinity,
 	.get_vq_affinity = vp_get_vq_affinity,
 	.get_shm_region  = vp_get_shm_region,
+	.reset_vq	 = vp_modern_reset_vq,
+	.enable_reset_vq = vp_modern_enable_reset_vq,
 };
 
 static const struct virtio_config_ops virtio_pci_config_ops = {
@@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
 	.set_vq_affinity = vp_set_vq_affinity,
 	.get_vq_affinity = vp_get_vq_affinity,
 	.get_shm_region  = vp_get_shm_region,
+	.reset_vq	 = vp_modern_reset_vq,
+	.enable_reset_vq = vp_modern_enable_reset_vq,
 };
 
 /* the PCI probing function */
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 6250e19fc5bf..91937e21edca 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	vq->vq.vdev = vdev;
 	vq->vq.name = name;
 	vq->vq.index = index;
+	vq->vq.reset = false;
 	vq->notify = notify;
 	vq->weak_barriers = weak_barriers;
 
@@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	vq->vq.vdev = vdev;
 	vq->vq.name = name;
 	vq->vq.index = index;
+	vq->vq.reset = false;
 	vq->notify = notify;
 	vq->weak_barriers = weak_barriers;
 
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index c86ff02e0ca0..33ab003c5100 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -33,6 +33,7 @@ struct virtqueue {
 	unsigned int num_free;
 	unsigned int num_max;
 	void *priv;
+	bool reset;
 };
 
 int virtqueue_add_outbuf(struct virtqueue *vq,
-- 
2.31.0

[PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Xuan Zhuo]

This patch implements virtio pci support for QUEUE RESET.

Performing reset on a queue is divided into these steps:

 1. notify the device to reset the queue
 2. recycle the buffer submitted
 3. reset the vring (may re-alloc)
 4. mmap vring to device, and enable the queue

This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
pci scenario.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_pci_common.c |  8 +--
 drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
 drivers/virtio/virtio_ring.c       |  2 +
 include/linux/virtio.h             |  1 +
 4 files changed, 92 insertions(+), 3 deletions(-)

diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index fdbde1db5ec5..863d3a8a0956 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
 	struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
 	unsigned long flags;
 
-	spin_lock_irqsave(&vp_dev->lock, flags);
-	list_del(&info->node);
-	spin_unlock_irqrestore(&vp_dev->lock, flags);
+	if (!vq->reset) {
+		spin_lock_irqsave(&vp_dev->lock, flags);
+		list_del(&info->node);
+		spin_unlock_irqrestore(&vp_dev->lock, flags);
+	}
 
 	vp_dev->del_vq(info);
 	kfree(info);
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index 49a4493732cf..cb5d38f1c9c8 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
 	if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
 			pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
 		__virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
+
+	if (features & BIT_ULL(VIRTIO_F_RING_RESET))
+		__virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
 }
 
 /* virtio config->finalize_features() implementation */
@@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
 	return 0;
 }
 
+static int vp_modern_reset_vq(struct virtqueue *vq)
+{
+	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
+	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
+	struct virtio_pci_vq_info *info;
+	unsigned long flags;
+
+	if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
+		return -ENOENT;
+
+	vp_modern_set_queue_reset(mdev, vq->index);
+
+	info = vp_dev->vqs[vq->index];
+
+	/* delete vq from irq handler */
+	spin_lock_irqsave(&vp_dev->lock, flags);
+	list_del(&info->node);
+	spin_unlock_irqrestore(&vp_dev->lock, flags);
+
+	INIT_LIST_HEAD(&info->node);
+
+	/* For the case where vq has an exclusive irq, to prevent the irq from
+	 * being received again and the pending irq, call disable_irq().
+	 *
+	 * In the scenario based on shared interrupts, vq will be searched from
+	 * the queue virtqueues. Since the previous list_del() has been deleted
+	 * from the queue, it is impossible for vq to be called in this case.
+	 * There is no need to close the corresponding interrupt.
+	 */
+	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
+		disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
+
+	vq->reset = true;
+
+	return 0;
+}
+
+static int vp_modern_enable_reset_vq(struct virtqueue *vq)
+{
+	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
+	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
+	struct virtio_pci_vq_info *info;
+	unsigned long flags, index;
+	int err;
+
+	if (!vq->reset)
+		return -EBUSY;
+
+	index = vq->index;
+	info = vp_dev->vqs[index];
+
+	/* check queue reset status */
+	if (vp_modern_get_queue_reset(mdev, index) != 1)
+		return -EBUSY;
+
+	err = vp_active_vq(vq, info->msix_vector);
+	if (err)
+		return err;
+
+	if (vq->callback) {
+		spin_lock_irqsave(&vp_dev->lock, flags);
+		list_add(&info->node, &vp_dev->virtqueues);
+		spin_unlock_irqrestore(&vp_dev->lock, flags);
+	} else {
+		INIT_LIST_HEAD(&info->node);
+	}
+
+	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
+
+	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
+		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
+
+	vq->reset = false;
+
+	return 0;
+}
+
 static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
 {
 	return vp_modern_config_vector(&vp_dev->mdev, vector);
@@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
 	.set_vq_affinity = vp_set_vq_affinity,
 	.get_vq_affinity = vp_get_vq_affinity,
 	.get_shm_region  = vp_get_shm_region,
+	.reset_vq	 = vp_modern_reset_vq,
+	.enable_reset_vq = vp_modern_enable_reset_vq,
 };
 
 static const struct virtio_config_ops virtio_pci_config_ops = {
@@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
 	.set_vq_affinity = vp_set_vq_affinity,
 	.get_vq_affinity = vp_get_vq_affinity,
 	.get_shm_region  = vp_get_shm_region,
+	.reset_vq	 = vp_modern_reset_vq,
+	.enable_reset_vq = vp_modern_enable_reset_vq,
 };
 
 /* the PCI probing function */
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 6250e19fc5bf..91937e21edca 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
 	vq->vq.vdev = vdev;
 	vq->vq.name = name;
 	vq->vq.index = index;
+	vq->vq.reset = false;
 	vq->notify = notify;
 	vq->weak_barriers = weak_barriers;
 
@@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
 	vq->vq.vdev = vdev;
 	vq->vq.name = name;
 	vq->vq.index = index;
+	vq->vq.reset = false;
 	vq->notify = notify;
 	vq->weak_barriers = weak_barriers;
 
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index c86ff02e0ca0..33ab003c5100 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -33,6 +33,7 @@ struct virtqueue {
 	unsigned int num_free;
 	unsigned int num_max;
 	void *priv;
+	bool reset;
 };
 
 int virtqueue_add_outbuf(struct virtqueue *vq,
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 24/32] virtio: find_vqs() add arg sizes

[Xuan Zhuo]

find_vqs() adds a new parameter sizes to specify the size of each vq
vring.

0 means use the maximum size supported by the backend.

In the split scenario, the meaning of size is the largest size, because
it may be limited by memory, the virtio core will try a smaller size.
And the size is power of 2.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Hans de Goede <hdegoede@redhat.com>
Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 arch/um/drivers/virtio_uml.c             |  2 +-
 drivers/platform/mellanox/mlxbf-tmfifo.c |  1 +
 drivers/remoteproc/remoteproc_virtio.c   |  1 +
 drivers/s390/virtio/virtio_ccw.c         |  1 +
 drivers/virtio/virtio_mmio.c             |  1 +
 drivers/virtio/virtio_pci_common.c       |  2 +-
 drivers/virtio/virtio_pci_common.h       |  2 +-
 drivers/virtio/virtio_pci_modern.c       |  7 +++++--
 drivers/virtio/virtio_vdpa.c             |  1 +
 include/linux/virtio_config.h            | 14 +++++++++-----
 10 files changed, 22 insertions(+), 10 deletions(-)

diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
index 904993d15a85..6af98d130840 100644
--- a/arch/um/drivers/virtio_uml.c
+++ b/arch/um/drivers/virtio_uml.c
@@ -998,7 +998,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
 
 static int vu_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		       struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		       const char * const names[], const bool *ctx,
+		       const char * const names[], u32 sizes[], const bool *ctx,
 		       struct irq_affinity *desc)
 {
 	struct virtio_uml_device *vu_dev = to_virtio_uml_device(vdev);
diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
index 1ae3c56b66b0..8be13d416f48 100644
--- a/drivers/platform/mellanox/mlxbf-tmfifo.c
+++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
@@ -928,6 +928,7 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
 					struct virtqueue *vqs[],
 					vq_callback_t *callbacks[],
 					const char * const names[],
+					u32 sizes[],
 					const bool *ctx,
 					struct irq_affinity *desc)
 {
diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index 7611755d0ae2..baad31c9da45 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -158,6 +158,7 @@ static int rproc_virtio_find_vqs(struct virtio_device *vdev, unsigned int nvqs,
 				 struct virtqueue *vqs[],
 				 vq_callback_t *callbacks[],
 				 const char * const names[],
+				 u32 sizes[],
 				 const bool * ctx,
 				 struct irq_affinity *desc)
 {
diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
index 468da60b56c5..f0c814a54e78 100644
--- a/drivers/s390/virtio/virtio_ccw.c
+++ b/drivers/s390/virtio/virtio_ccw.c
@@ -634,6 +634,7 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 			       struct virtqueue *vqs[],
 			       vq_callback_t *callbacks[],
 			       const char * const names[],
+			       u32 sizes[],
 			       const bool *ctx,
 			       struct irq_affinity *desc)
 {
diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index a41abc8051b9..9d5a674bdeec 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -461,6 +461,7 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		       struct virtqueue *vqs[],
 		       vq_callback_t *callbacks[],
 		       const char * const names[],
+		       u32 sizes[],
 		       const bool *ctx,
 		       struct irq_affinity *desc)
 {
diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index 863d3a8a0956..826ea2e35d54 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -427,7 +427,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
 /* the config->find_vqs() implementation */
 int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		const char * const names[], const bool *ctx,
+		const char * const names[], u32 sizes[], const bool *ctx,
 		struct irq_affinity *desc)
 {
 	int err;
diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
index 23f6c5c678d5..859eed559e10 100644
--- a/drivers/virtio/virtio_pci_common.h
+++ b/drivers/virtio/virtio_pci_common.h
@@ -113,7 +113,7 @@ void vp_del_vqs(struct virtio_device *vdev);
 /* the config->find_vqs() implementation */
 int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		const char * const names[], const bool *ctx,
+		const char * const names[], u32 sizes[], const bool *ctx,
 		struct irq_affinity *desc);
 const char *vp_bus_name(struct virtio_device *vdev);
 
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index cb5d38f1c9c8..3b35e5056165 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -343,12 +343,15 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 static int vp_modern_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 			      struct virtqueue *vqs[],
 			      vq_callback_t *callbacks[],
-			      const char * const names[], const bool *ctx,
+			      const char * const names[],
+			      u32 sizes[],
+			      const bool *ctx,
 			      struct irq_affinity *desc)
 {
 	struct virtio_pci_device *vp_dev = to_vp_device(vdev);
 	struct virtqueue *vq;
-	int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, ctx, desc);
+	int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, sizes, ctx,
+			     desc);
 
 	if (rc)
 		return rc;
diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
index 39e4c08eb0f2..ac86d3a6633c 100644
--- a/drivers/virtio/virtio_vdpa.c
+++ b/drivers/virtio/virtio_vdpa.c
@@ -269,6 +269,7 @@ static int virtio_vdpa_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 				struct virtqueue *vqs[],
 				vq_callback_t *callbacks[],
 				const char * const names[],
+				u32 sizes[],
 				const bool *ctx,
 				struct irq_affinity *desc)
 {
diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
index d4adcd0e1c57..0f7def7ddfd2 100644
--- a/include/linux/virtio_config.h
+++ b/include/linux/virtio_config.h
@@ -57,6 +57,7 @@ struct virtio_shm_region {
  *		include a NULL entry for vqs that do not need a callback
  *	names: array of virtqueue names (mainly for debugging)
  *		include a NULL entry for vqs unused by driver
+ *	sizes: array of virtqueue sizes
  *	Returns 0 on success or error status
  * @del_vqs: free virtqueues found by find_vqs().
  * @get_features: get the array of feature bits for this device.
@@ -98,7 +99,9 @@ struct virtio_config_ops {
 	void (*reset)(struct virtio_device *vdev);
 	int (*find_vqs)(struct virtio_device *, unsigned nvqs,
 			struct virtqueue *vqs[], vq_callback_t *callbacks[],
-			const char * const names[], const bool *ctx,
+			const char * const names[],
+			u32 sizes[],
+			const bool *ctx,
 			struct irq_affinity *desc);
 	void (*del_vqs)(struct virtio_device *);
 	u64 (*get_features)(struct virtio_device *vdev);
@@ -206,7 +209,7 @@ struct virtqueue *virtio_find_single_vq(struct virtio_device *vdev,
 	const char *names[] = { n };
 	struct virtqueue *vq;
 	int err = vdev->config->find_vqs(vdev, 1, &vq, callbacks, names, NULL,
-					 NULL);
+					 NULL, NULL);
 	if (err < 0)
 		return ERR_PTR(err);
 	return vq;
@@ -218,7 +221,8 @@ int virtio_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 			const char * const names[],
 			struct irq_affinity *desc)
 {
-	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL, desc);
+	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL,
+				      NULL, desc);
 }
 
 static inline
@@ -227,8 +231,8 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
 			const char * const names[], const bool *ctx,
 			struct irq_affinity *desc)
 {
-	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, ctx,
-				      desc);
+	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL,
+				      ctx, desc);
 }
 
 /**
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 24/32] virtio: find_vqs() add arg sizes

[Xuan Zhuo]

find_vqs() adds a new parameter sizes to specify the size of each vq
vring.

0 means use the maximum size supported by the backend.

In the split scenario, the meaning of size is the largest size, because
it may be limited by memory, the virtio core will try a smaller size.
And the size is power of 2.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Hans de Goede <hdegoede@redhat.com>
Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 arch/um/drivers/virtio_uml.c             |  2 +-
 drivers/platform/mellanox/mlxbf-tmfifo.c |  1 +
 drivers/remoteproc/remoteproc_virtio.c   |  1 +
 drivers/s390/virtio/virtio_ccw.c         |  1 +
 drivers/virtio/virtio_mmio.c             |  1 +
 drivers/virtio/virtio_pci_common.c       |  2 +-
 drivers/virtio/virtio_pci_common.h       |  2 +-
 drivers/virtio/virtio_pci_modern.c       |  7 +++++--
 drivers/virtio/virtio_vdpa.c             |  1 +
 include/linux/virtio_config.h            | 14 +++++++++-----
 10 files changed, 22 insertions(+), 10 deletions(-)

diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
index 904993d15a85..6af98d130840 100644
--- a/arch/um/drivers/virtio_uml.c
+++ b/arch/um/drivers/virtio_uml.c
@@ -998,7 +998,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
 
 static int vu_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		       struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		       const char * const names[], const bool *ctx,
+		       const char * const names[], u32 sizes[], const bool *ctx,
 		       struct irq_affinity *desc)
 {
 	struct virtio_uml_device *vu_dev = to_virtio_uml_device(vdev);
diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
index 1ae3c56b66b0..8be13d416f48 100644
--- a/drivers/platform/mellanox/mlxbf-tmfifo.c
+++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
@@ -928,6 +928,7 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
 					struct virtqueue *vqs[],
 					vq_callback_t *callbacks[],
 					const char * const names[],
+					u32 sizes[],
 					const bool *ctx,
 					struct irq_affinity *desc)
 {
diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index 7611755d0ae2..baad31c9da45 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -158,6 +158,7 @@ static int rproc_virtio_find_vqs(struct virtio_device *vdev, unsigned int nvqs,
 				 struct virtqueue *vqs[],
 				 vq_callback_t *callbacks[],
 				 const char * const names[],
+				 u32 sizes[],
 				 const bool * ctx,
 				 struct irq_affinity *desc)
 {
diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
index 468da60b56c5..f0c814a54e78 100644
--- a/drivers/s390/virtio/virtio_ccw.c
+++ b/drivers/s390/virtio/virtio_ccw.c
@@ -634,6 +634,7 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 			       struct virtqueue *vqs[],
 			       vq_callback_t *callbacks[],
 			       const char * const names[],
+			       u32 sizes[],
 			       const bool *ctx,
 			       struct irq_affinity *desc)
 {
diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index a41abc8051b9..9d5a674bdeec 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -461,6 +461,7 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		       struct virtqueue *vqs[],
 		       vq_callback_t *callbacks[],
 		       const char * const names[],
+		       u32 sizes[],
 		       const bool *ctx,
 		       struct irq_affinity *desc)
 {
diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index 863d3a8a0956..826ea2e35d54 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -427,7 +427,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
 /* the config->find_vqs() implementation */
 int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		const char * const names[], const bool *ctx,
+		const char * const names[], u32 sizes[], const bool *ctx,
 		struct irq_affinity *desc)
 {
 	int err;
diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
index 23f6c5c678d5..859eed559e10 100644
--- a/drivers/virtio/virtio_pci_common.h
+++ b/drivers/virtio/virtio_pci_common.h
@@ -113,7 +113,7 @@ void vp_del_vqs(struct virtio_device *vdev);
 /* the config->find_vqs() implementation */
 int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		const char * const names[], const bool *ctx,
+		const char * const names[], u32 sizes[], const bool *ctx,
 		struct irq_affinity *desc);
 const char *vp_bus_name(struct virtio_device *vdev);
 
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index cb5d38f1c9c8..3b35e5056165 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -343,12 +343,15 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 static int vp_modern_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 			      struct virtqueue *vqs[],
 			      vq_callback_t *callbacks[],
-			      const char * const names[], const bool *ctx,
+			      const char * const names[],
+			      u32 sizes[],
+			      const bool *ctx,
 			      struct irq_affinity *desc)
 {
 	struct virtio_pci_device *vp_dev = to_vp_device(vdev);
 	struct virtqueue *vq;
-	int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, ctx, desc);
+	int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, sizes, ctx,
+			     desc);
 
 	if (rc)
 		return rc;
diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
index 39e4c08eb0f2..ac86d3a6633c 100644
--- a/drivers/virtio/virtio_vdpa.c
+++ b/drivers/virtio/virtio_vdpa.c
@@ -269,6 +269,7 @@ static int virtio_vdpa_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 				struct virtqueue *vqs[],
 				vq_callback_t *callbacks[],
 				const char * const names[],
+				u32 sizes[],
 				const bool *ctx,
 				struct irq_affinity *desc)
 {
diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
index d4adcd0e1c57..0f7def7ddfd2 100644
--- a/include/linux/virtio_config.h
+++ b/include/linux/virtio_config.h
@@ -57,6 +57,7 @@ struct virtio_shm_region {
  *		include a NULL entry for vqs that do not need a callback
  *	names: array of virtqueue names (mainly for debugging)
  *		include a NULL entry for vqs unused by driver
+ *	sizes: array of virtqueue sizes
  *	Returns 0 on success or error status
  * @del_vqs: free virtqueues found by find_vqs().
  * @get_features: get the array of feature bits for this device.
@@ -98,7 +99,9 @@ struct virtio_config_ops {
 	void (*reset)(struct virtio_device *vdev);
 	int (*find_vqs)(struct virtio_device *, unsigned nvqs,
 			struct virtqueue *vqs[], vq_callback_t *callbacks[],
-			const char * const names[], const bool *ctx,
+			const char * const names[],
+			u32 sizes[],
+			const bool *ctx,
 			struct irq_affinity *desc);
 	void (*del_vqs)(struct virtio_device *);
 	u64 (*get_features)(struct virtio_device *vdev);
@@ -206,7 +209,7 @@ struct virtqueue *virtio_find_single_vq(struct virtio_device *vdev,
 	const char *names[] = { n };
 	struct virtqueue *vq;
 	int err = vdev->config->find_vqs(vdev, 1, &vq, callbacks, names, NULL,
-					 NULL);
+					 NULL, NULL);
 	if (err < 0)
 		return ERR_PTR(err);
 	return vq;
@@ -218,7 +221,8 @@ int virtio_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 			const char * const names[],
 			struct irq_affinity *desc)
 {
-	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL, desc);
+	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL,
+				      NULL, desc);
 }
 
 static inline
@@ -227,8 +231,8 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
 			const char * const names[], const bool *ctx,
 			struct irq_affinity *desc)
 {
-	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, ctx,
-				      desc);
+	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL,
+				      ctx, desc);
 }
 
 /**
-- 
2.31.0

[PATCH v9 24/32] virtio: find_vqs() add arg sizes

[Xuan Zhuo]

find_vqs() adds a new parameter sizes to specify the size of each vq
vring.

0 means use the maximum size supported by the backend.

In the split scenario, the meaning of size is the largest size, because
it may be limited by memory, the virtio core will try a smaller size.
And the size is power of 2.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Hans de Goede <hdegoede@redhat.com>
Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 arch/um/drivers/virtio_uml.c             |  2 +-
 drivers/platform/mellanox/mlxbf-tmfifo.c |  1 +
 drivers/remoteproc/remoteproc_virtio.c   |  1 +
 drivers/s390/virtio/virtio_ccw.c         |  1 +
 drivers/virtio/virtio_mmio.c             |  1 +
 drivers/virtio/virtio_pci_common.c       |  2 +-
 drivers/virtio/virtio_pci_common.h       |  2 +-
 drivers/virtio/virtio_pci_modern.c       |  7 +++++--
 drivers/virtio/virtio_vdpa.c             |  1 +
 include/linux/virtio_config.h            | 14 +++++++++-----
 10 files changed, 22 insertions(+), 10 deletions(-)

diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
index 904993d15a85..6af98d130840 100644
--- a/arch/um/drivers/virtio_uml.c
+++ b/arch/um/drivers/virtio_uml.c
@@ -998,7 +998,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
 
 static int vu_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		       struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		       const char * const names[], const bool *ctx,
+		       const char * const names[], u32 sizes[], const bool *ctx,
 		       struct irq_affinity *desc)
 {
 	struct virtio_uml_device *vu_dev = to_virtio_uml_device(vdev);
diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
index 1ae3c56b66b0..8be13d416f48 100644
--- a/drivers/platform/mellanox/mlxbf-tmfifo.c
+++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
@@ -928,6 +928,7 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
 					struct virtqueue *vqs[],
 					vq_callback_t *callbacks[],
 					const char * const names[],
+					u32 sizes[],
 					const bool *ctx,
 					struct irq_affinity *desc)
 {
diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index 7611755d0ae2..baad31c9da45 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -158,6 +158,7 @@ static int rproc_virtio_find_vqs(struct virtio_device *vdev, unsigned int nvqs,
 				 struct virtqueue *vqs[],
 				 vq_callback_t *callbacks[],
 				 const char * const names[],
+				 u32 sizes[],
 				 const bool * ctx,
 				 struct irq_affinity *desc)
 {
diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
index 468da60b56c5..f0c814a54e78 100644
--- a/drivers/s390/virtio/virtio_ccw.c
+++ b/drivers/s390/virtio/virtio_ccw.c
@@ -634,6 +634,7 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 			       struct virtqueue *vqs[],
 			       vq_callback_t *callbacks[],
 			       const char * const names[],
+			       u32 sizes[],
 			       const bool *ctx,
 			       struct irq_affinity *desc)
 {
diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index a41abc8051b9..9d5a674bdeec 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -461,6 +461,7 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		       struct virtqueue *vqs[],
 		       vq_callback_t *callbacks[],
 		       const char * const names[],
+		       u32 sizes[],
 		       const bool *ctx,
 		       struct irq_affinity *desc)
 {
diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index 863d3a8a0956..826ea2e35d54 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -427,7 +427,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
 /* the config->find_vqs() implementation */
 int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		const char * const names[], const bool *ctx,
+		const char * const names[], u32 sizes[], const bool *ctx,
 		struct irq_affinity *desc)
 {
 	int err;
diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
index 23f6c5c678d5..859eed559e10 100644
--- a/drivers/virtio/virtio_pci_common.h
+++ b/drivers/virtio/virtio_pci_common.h
@@ -113,7 +113,7 @@ void vp_del_vqs(struct virtio_device *vdev);
 /* the config->find_vqs() implementation */
 int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		const char * const names[], const bool *ctx,
+		const char * const names[], u32 sizes[], const bool *ctx,
 		struct irq_affinity *desc);
 const char *vp_bus_name(struct virtio_device *vdev);
 
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index cb5d38f1c9c8..3b35e5056165 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -343,12 +343,15 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 static int vp_modern_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 			      struct virtqueue *vqs[],
 			      vq_callback_t *callbacks[],
-			      const char * const names[], const bool *ctx,
+			      const char * const names[],
+			      u32 sizes[],
+			      const bool *ctx,
 			      struct irq_affinity *desc)
 {
 	struct virtio_pci_device *vp_dev = to_vp_device(vdev);
 	struct virtqueue *vq;
-	int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, ctx, desc);
+	int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, sizes, ctx,
+			     desc);
 
 	if (rc)
 		return rc;
diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
index 39e4c08eb0f2..ac86d3a6633c 100644
--- a/drivers/virtio/virtio_vdpa.c
+++ b/drivers/virtio/virtio_vdpa.c
@@ -269,6 +269,7 @@ static int virtio_vdpa_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 				struct virtqueue *vqs[],
 				vq_callback_t *callbacks[],
 				const char * const names[],
+				u32 sizes[],
 				const bool *ctx,
 				struct irq_affinity *desc)
 {
diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
index d4adcd0e1c57..0f7def7ddfd2 100644
--- a/include/linux/virtio_config.h
+++ b/include/linux/virtio_config.h
@@ -57,6 +57,7 @@ struct virtio_shm_region {
  *		include a NULL entry for vqs that do not need a callback
  *	names: array of virtqueue names (mainly for debugging)
  *		include a NULL entry for vqs unused by driver
+ *	sizes: array of virtqueue sizes
  *	Returns 0 on success or error status
  * @del_vqs: free virtqueues found by find_vqs().
  * @get_features: get the array of feature bits for this device.
@@ -98,7 +99,9 @@ struct virtio_config_ops {
 	void (*reset)(struct virtio_device *vdev);
 	int (*find_vqs)(struct virtio_device *, unsigned nvqs,
 			struct virtqueue *vqs[], vq_callback_t *callbacks[],
-			const char * const names[], const bool *ctx,
+			const char * const names[],
+			u32 sizes[],
+			const bool *ctx,
 			struct irq_affinity *desc);
 	void (*del_vqs)(struct virtio_device *);
 	u64 (*get_features)(struct virtio_device *vdev);
@@ -206,7 +209,7 @@ struct virtqueue *virtio_find_single_vq(struct virtio_device *vdev,
 	const char *names[] = { n };
 	struct virtqueue *vq;
 	int err = vdev->config->find_vqs(vdev, 1, &vq, callbacks, names, NULL,
-					 NULL);
+					 NULL, NULL);
 	if (err < 0)
 		return ERR_PTR(err);
 	return vq;
@@ -218,7 +221,8 @@ int virtio_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 			const char * const names[],
 			struct irq_affinity *desc)
 {
-	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL, desc);
+	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL,
+				      NULL, desc);
 }
 
 static inline
@@ -227,8 +231,8 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
 			const char * const names[], const bool *ctx,
 			struct irq_affinity *desc)
 {
-	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, ctx,
-				      desc);
+	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL,
+				      ctx, desc);
 }
 
 /**
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 25/32] virtio_pci: support the arg sizes of find_vqs()

[Xuan Zhuo]

Virtio PCI supports new parameter sizes of find_vqs().

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_pci_common.c | 18 ++++++++++--------
 drivers/virtio/virtio_pci_common.h |  1 +
 drivers/virtio/virtio_pci_legacy.c |  6 +++++-
 drivers/virtio/virtio_pci_modern.c | 10 +++++++---
 4 files changed, 23 insertions(+), 12 deletions(-)

diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index 826ea2e35d54..23976c61583f 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -208,6 +208,7 @@ static int vp_request_msix_vectors(struct virtio_device *vdev, int nvectors,
 static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index,
 				     void (*callback)(struct virtqueue *vq),
 				     const char *name,
+				     u32 size,
 				     bool ctx,
 				     u16 msix_vec)
 {
@@ -220,7 +221,7 @@ static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index,
 	if (!info)
 		return ERR_PTR(-ENOMEM);
 
-	vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, ctx,
+	vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, size, ctx,
 			      msix_vec);
 	if (IS_ERR(vq))
 		goto out_info;
@@ -314,7 +315,7 @@ void vp_del_vqs(struct virtio_device *vdev)
 
 static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
 		struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		const char * const names[], bool per_vq_vectors,
+		const char * const names[], u32 sizes[], bool per_vq_vectors,
 		const bool *ctx,
 		struct irq_affinity *desc)
 {
@@ -357,8 +358,8 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
 		else
 			msix_vec = VP_MSIX_VQ_VECTOR;
 		vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
-				     ctx ? ctx[i] : false,
-				     msix_vec);
+				     sizes ? sizes[i] : 0,
+				     ctx ? ctx[i] : false, msix_vec);
 		if (IS_ERR(vqs[i])) {
 			err = PTR_ERR(vqs[i]);
 			goto error_find;
@@ -388,7 +389,7 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
 
 static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
 		struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		const char * const names[], const bool *ctx)
+		const char * const names[], u32 sizes[], const bool *ctx)
 {
 	struct virtio_pci_device *vp_dev = to_vp_device(vdev);
 	int i, err, queue_idx = 0;
@@ -410,6 +411,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
 			continue;
 		}
 		vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
+				     sizes ? sizes[i] : 0,
 				     ctx ? ctx[i] : false,
 				     VIRTIO_MSI_NO_VECTOR);
 		if (IS_ERR(vqs[i])) {
@@ -433,15 +435,15 @@ int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 	int err;
 
 	/* Try MSI-X with one vector per queue. */
-	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, true, ctx, desc);
+	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, sizes, true, ctx, desc);
 	if (!err)
 		return 0;
 	/* Fallback: MSI-X with one vector for config, one shared for queues. */
-	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, false, ctx, desc);
+	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, sizes, false, ctx, desc);
 	if (!err)
 		return 0;
 	/* Finally fall back to regular interrupts. */
-	return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, ctx);
+	return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, sizes, ctx);
 }
 
 const char *vp_bus_name(struct virtio_device *vdev)
diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
index 859eed559e10..fbf5a6d4b164 100644
--- a/drivers/virtio/virtio_pci_common.h
+++ b/drivers/virtio/virtio_pci_common.h
@@ -81,6 +81,7 @@ struct virtio_pci_device {
 				      unsigned idx,
 				      void (*callback)(struct virtqueue *vq),
 				      const char *name,
+				      u32 size,
 				      bool ctx,
 				      u16 msix_vec);
 	void (*del_vq)(struct virtio_pci_vq_info *info);
diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
index b68934fe6b5d..2c4ade5fb420 100644
--- a/drivers/virtio/virtio_pci_legacy.c
+++ b/drivers/virtio/virtio_pci_legacy.c
@@ -112,6 +112,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 				  unsigned index,
 				  void (*callback)(struct virtqueue *vq),
 				  const char *name,
+				  u32 size,
 				  bool ctx,
 				  u16 msix_vec)
 {
@@ -125,10 +126,13 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	if (!num || vp_legacy_get_queue_enable(&vp_dev->ldev, index))
 		return ERR_PTR(-ENOENT);
 
+	if (!size || size > num)
+		size = num;
+
 	info->msix_vector = msix_vec;
 
 	/* create the vring */
-	vq = vring_create_virtqueue(index, num,
+	vq = vring_create_virtqueue(index, size,
 				    VIRTIO_PCI_VRING_ALIGN, &vp_dev->vdev,
 				    true, false, ctx,
 				    vp_notify, callback, name);
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index 3b35e5056165..a17c47d4435a 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -289,6 +289,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 				  unsigned index,
 				  void (*callback)(struct virtqueue *vq),
 				  const char *name,
+				  u32 size,
 				  bool ctx,
 				  u16 msix_vec)
 {
@@ -306,15 +307,18 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	if (!num || vp_modern_get_queue_enable(mdev, index))
 		return ERR_PTR(-ENOENT);
 
-	if (num & (num - 1)) {
-		dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", num);
+	if (!size || size > num)
+		size = num;
+
+	if (size & (size - 1)) {
+		dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", size);
 		return ERR_PTR(-EINVAL);
 	}
 
 	info->msix_vector = msix_vec;
 
 	/* create the vring */
-	vq = vring_create_virtqueue(index, num,
+	vq = vring_create_virtqueue(index, size,
 				    SMP_CACHE_BYTES, &vp_dev->vdev,
 				    true, true, ctx,
 				    vp_notify, callback, name);
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 25/32] virtio_pci: support the arg sizes of find_vqs()

[Xuan Zhuo]

Virtio PCI supports new parameter sizes of find_vqs().

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_pci_common.c | 18 ++++++++++--------
 drivers/virtio/virtio_pci_common.h |  1 +
 drivers/virtio/virtio_pci_legacy.c |  6 +++++-
 drivers/virtio/virtio_pci_modern.c | 10 +++++++---
 4 files changed, 23 insertions(+), 12 deletions(-)

diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index 826ea2e35d54..23976c61583f 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -208,6 +208,7 @@ static int vp_request_msix_vectors(struct virtio_device *vdev, int nvectors,
 static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index,
 				     void (*callback)(struct virtqueue *vq),
 				     const char *name,
+				     u32 size,
 				     bool ctx,
 				     u16 msix_vec)
 {
@@ -220,7 +221,7 @@ static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index,
 	if (!info)
 		return ERR_PTR(-ENOMEM);
 
-	vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, ctx,
+	vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, size, ctx,
 			      msix_vec);
 	if (IS_ERR(vq))
 		goto out_info;
@@ -314,7 +315,7 @@ void vp_del_vqs(struct virtio_device *vdev)
 
 static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
 		struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		const char * const names[], bool per_vq_vectors,
+		const char * const names[], u32 sizes[], bool per_vq_vectors,
 		const bool *ctx,
 		struct irq_affinity *desc)
 {
@@ -357,8 +358,8 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
 		else
 			msix_vec = VP_MSIX_VQ_VECTOR;
 		vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
-				     ctx ? ctx[i] : false,
-				     msix_vec);
+				     sizes ? sizes[i] : 0,
+				     ctx ? ctx[i] : false, msix_vec);
 		if (IS_ERR(vqs[i])) {
 			err = PTR_ERR(vqs[i]);
 			goto error_find;
@@ -388,7 +389,7 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
 
 static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
 		struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		const char * const names[], const bool *ctx)
+		const char * const names[], u32 sizes[], const bool *ctx)
 {
 	struct virtio_pci_device *vp_dev = to_vp_device(vdev);
 	int i, err, queue_idx = 0;
@@ -410,6 +411,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
 			continue;
 		}
 		vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
+				     sizes ? sizes[i] : 0,
 				     ctx ? ctx[i] : false,
 				     VIRTIO_MSI_NO_VECTOR);
 		if (IS_ERR(vqs[i])) {
@@ -433,15 +435,15 @@ int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 	int err;
 
 	/* Try MSI-X with one vector per queue. */
-	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, true, ctx, desc);
+	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, sizes, true, ctx, desc);
 	if (!err)
 		return 0;
 	/* Fallback: MSI-X with one vector for config, one shared for queues. */
-	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, false, ctx, desc);
+	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, sizes, false, ctx, desc);
 	if (!err)
 		return 0;
 	/* Finally fall back to regular interrupts. */
-	return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, ctx);
+	return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, sizes, ctx);
 }
 
 const char *vp_bus_name(struct virtio_device *vdev)
diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
index 859eed559e10..fbf5a6d4b164 100644
--- a/drivers/virtio/virtio_pci_common.h
+++ b/drivers/virtio/virtio_pci_common.h
@@ -81,6 +81,7 @@ struct virtio_pci_device {
 				      unsigned idx,
 				      void (*callback)(struct virtqueue *vq),
 				      const char *name,
+				      u32 size,
 				      bool ctx,
 				      u16 msix_vec);
 	void (*del_vq)(struct virtio_pci_vq_info *info);
diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
index b68934fe6b5d..2c4ade5fb420 100644
--- a/drivers/virtio/virtio_pci_legacy.c
+++ b/drivers/virtio/virtio_pci_legacy.c
@@ -112,6 +112,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 				  unsigned index,
 				  void (*callback)(struct virtqueue *vq),
 				  const char *name,
+				  u32 size,
 				  bool ctx,
 				  u16 msix_vec)
 {
@@ -125,10 +126,13 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	if (!num || vp_legacy_get_queue_enable(&vp_dev->ldev, index))
 		return ERR_PTR(-ENOENT);
 
+	if (!size || size > num)
+		size = num;
+
 	info->msix_vector = msix_vec;
 
 	/* create the vring */
-	vq = vring_create_virtqueue(index, num,
+	vq = vring_create_virtqueue(index, size,
 				    VIRTIO_PCI_VRING_ALIGN, &vp_dev->vdev,
 				    true, false, ctx,
 				    vp_notify, callback, name);
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index 3b35e5056165..a17c47d4435a 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -289,6 +289,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 				  unsigned index,
 				  void (*callback)(struct virtqueue *vq),
 				  const char *name,
+				  u32 size,
 				  bool ctx,
 				  u16 msix_vec)
 {
@@ -306,15 +307,18 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	if (!num || vp_modern_get_queue_enable(mdev, index))
 		return ERR_PTR(-ENOENT);
 
-	if (num & (num - 1)) {
-		dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", num);
+	if (!size || size > num)
+		size = num;
+
+	if (size & (size - 1)) {
+		dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", size);
 		return ERR_PTR(-EINVAL);
 	}
 
 	info->msix_vector = msix_vec;
 
 	/* create the vring */
-	vq = vring_create_virtqueue(index, num,
+	vq = vring_create_virtqueue(index, size,
 				    SMP_CACHE_BYTES, &vp_dev->vdev,
 				    true, true, ctx,
 				    vp_notify, callback, name);
-- 
2.31.0

[PATCH v9 25/32] virtio_pci: support the arg sizes of find_vqs()

[Xuan Zhuo]

Virtio PCI supports new parameter sizes of find_vqs().

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_pci_common.c | 18 ++++++++++--------
 drivers/virtio/virtio_pci_common.h |  1 +
 drivers/virtio/virtio_pci_legacy.c |  6 +++++-
 drivers/virtio/virtio_pci_modern.c | 10 +++++++---
 4 files changed, 23 insertions(+), 12 deletions(-)

diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index 826ea2e35d54..23976c61583f 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -208,6 +208,7 @@ static int vp_request_msix_vectors(struct virtio_device *vdev, int nvectors,
 static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index,
 				     void (*callback)(struct virtqueue *vq),
 				     const char *name,
+				     u32 size,
 				     bool ctx,
 				     u16 msix_vec)
 {
@@ -220,7 +221,7 @@ static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index,
 	if (!info)
 		return ERR_PTR(-ENOMEM);
 
-	vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, ctx,
+	vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, size, ctx,
 			      msix_vec);
 	if (IS_ERR(vq))
 		goto out_info;
@@ -314,7 +315,7 @@ void vp_del_vqs(struct virtio_device *vdev)
 
 static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
 		struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		const char * const names[], bool per_vq_vectors,
+		const char * const names[], u32 sizes[], bool per_vq_vectors,
 		const bool *ctx,
 		struct irq_affinity *desc)
 {
@@ -357,8 +358,8 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
 		else
 			msix_vec = VP_MSIX_VQ_VECTOR;
 		vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
-				     ctx ? ctx[i] : false,
-				     msix_vec);
+				     sizes ? sizes[i] : 0,
+				     ctx ? ctx[i] : false, msix_vec);
 		if (IS_ERR(vqs[i])) {
 			err = PTR_ERR(vqs[i]);
 			goto error_find;
@@ -388,7 +389,7 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
 
 static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
 		struct virtqueue *vqs[], vq_callback_t *callbacks[],
-		const char * const names[], const bool *ctx)
+		const char * const names[], u32 sizes[], const bool *ctx)
 {
 	struct virtio_pci_device *vp_dev = to_vp_device(vdev);
 	int i, err, queue_idx = 0;
@@ -410,6 +411,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
 			continue;
 		}
 		vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
+				     sizes ? sizes[i] : 0,
 				     ctx ? ctx[i] : false,
 				     VIRTIO_MSI_NO_VECTOR);
 		if (IS_ERR(vqs[i])) {
@@ -433,15 +435,15 @@ int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 	int err;
 
 	/* Try MSI-X with one vector per queue. */
-	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, true, ctx, desc);
+	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, sizes, true, ctx, desc);
 	if (!err)
 		return 0;
 	/* Fallback: MSI-X with one vector for config, one shared for queues. */
-	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, false, ctx, desc);
+	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, sizes, false, ctx, desc);
 	if (!err)
 		return 0;
 	/* Finally fall back to regular interrupts. */
-	return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, ctx);
+	return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, sizes, ctx);
 }
 
 const char *vp_bus_name(struct virtio_device *vdev)
diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
index 859eed559e10..fbf5a6d4b164 100644
--- a/drivers/virtio/virtio_pci_common.h
+++ b/drivers/virtio/virtio_pci_common.h
@@ -81,6 +81,7 @@ struct virtio_pci_device {
 				      unsigned idx,
 				      void (*callback)(struct virtqueue *vq),
 				      const char *name,
+				      u32 size,
 				      bool ctx,
 				      u16 msix_vec);
 	void (*del_vq)(struct virtio_pci_vq_info *info);
diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
index b68934fe6b5d..2c4ade5fb420 100644
--- a/drivers/virtio/virtio_pci_legacy.c
+++ b/drivers/virtio/virtio_pci_legacy.c
@@ -112,6 +112,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 				  unsigned index,
 				  void (*callback)(struct virtqueue *vq),
 				  const char *name,
+				  u32 size,
 				  bool ctx,
 				  u16 msix_vec)
 {
@@ -125,10 +126,13 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	if (!num || vp_legacy_get_queue_enable(&vp_dev->ldev, index))
 		return ERR_PTR(-ENOENT);
 
+	if (!size || size > num)
+		size = num;
+
 	info->msix_vector = msix_vec;
 
 	/* create the vring */
-	vq = vring_create_virtqueue(index, num,
+	vq = vring_create_virtqueue(index, size,
 				    VIRTIO_PCI_VRING_ALIGN, &vp_dev->vdev,
 				    true, false, ctx,
 				    vp_notify, callback, name);
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index 3b35e5056165..a17c47d4435a 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -289,6 +289,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 				  unsigned index,
 				  void (*callback)(struct virtqueue *vq),
 				  const char *name,
+				  u32 size,
 				  bool ctx,
 				  u16 msix_vec)
 {
@@ -306,15 +307,18 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	if (!num || vp_modern_get_queue_enable(mdev, index))
 		return ERR_PTR(-ENOENT);
 
-	if (num & (num - 1)) {
-		dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", num);
+	if (!size || size > num)
+		size = num;
+
+	if (size & (size - 1)) {
+		dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", size);
 		return ERR_PTR(-EINVAL);
 	}
 
 	info->msix_vector = msix_vec;
 
 	/* create the vring */
-	vq = vring_create_virtqueue(index, num,
+	vq = vring_create_virtqueue(index, size,
 				    SMP_CACHE_BYTES, &vp_dev->vdev,
 				    true, true, ctx,
 				    vp_notify, callback, name);
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 26/32] virtio_mmio: support the arg sizes of find_vqs()

[Xuan Zhuo]

Virtio MMIO support the new parameter sizes of find_vqs().

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_mmio.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 9d5a674bdeec..51cf51764a92 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -347,7 +347,7 @@ static void vm_del_vqs(struct virtio_device *vdev)
 
 static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
 				  void (*callback)(struct virtqueue *vq),
-				  const char *name, bool ctx)
+				  const char *name, u32 size, bool ctx)
 {
 	struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
 	struct virtio_mmio_vq_info *info;
@@ -382,8 +382,11 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
 		goto error_new_virtqueue;
 	}
 
+	if (!size || size > num)
+		size = num;
+
 	/* Create the vring */
-	vq = vring_create_virtqueue(index, num, VIRTIO_MMIO_VRING_ALIGN, vdev,
+	vq = vring_create_virtqueue(index, size, VIRTIO_MMIO_VRING_ALIGN, vdev,
 				 true, true, ctx, vm_notify, callback, name);
 	if (!vq) {
 		err = -ENOMEM;
@@ -484,6 +487,7 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		}
 
 		vqs[i] = vm_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
+				     sizes ? sizes[i] : 0,
 				     ctx ? ctx[i] : false);
 		if (IS_ERR(vqs[i])) {
 			vm_del_vqs(vdev);
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 26/32] virtio_mmio: support the arg sizes of find_vqs()

[Xuan Zhuo]

Virtio MMIO support the new parameter sizes of find_vqs().

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_mmio.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 9d5a674bdeec..51cf51764a92 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -347,7 +347,7 @@ static void vm_del_vqs(struct virtio_device *vdev)
 
 static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
 				  void (*callback)(struct virtqueue *vq),
-				  const char *name, bool ctx)
+				  const char *name, u32 size, bool ctx)
 {
 	struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
 	struct virtio_mmio_vq_info *info;
@@ -382,8 +382,11 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
 		goto error_new_virtqueue;
 	}
 
+	if (!size || size > num)
+		size = num;
+
 	/* Create the vring */
-	vq = vring_create_virtqueue(index, num, VIRTIO_MMIO_VRING_ALIGN, vdev,
+	vq = vring_create_virtqueue(index, size, VIRTIO_MMIO_VRING_ALIGN, vdev,
 				 true, true, ctx, vm_notify, callback, name);
 	if (!vq) {
 		err = -ENOMEM;
@@ -484,6 +487,7 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		}
 
 		vqs[i] = vm_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
+				     sizes ? sizes[i] : 0,
 				     ctx ? ctx[i] : false);
 		if (IS_ERR(vqs[i])) {
 			vm_del_vqs(vdev);
-- 
2.31.0

[PATCH v9 26/32] virtio_mmio: support the arg sizes of find_vqs()

[Xuan Zhuo]

Virtio MMIO support the new parameter sizes of find_vqs().

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/virtio/virtio_mmio.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 9d5a674bdeec..51cf51764a92 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -347,7 +347,7 @@ static void vm_del_vqs(struct virtio_device *vdev)
 
 static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
 				  void (*callback)(struct virtqueue *vq),
-				  const char *name, bool ctx)
+				  const char *name, u32 size, bool ctx)
 {
 	struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
 	struct virtio_mmio_vq_info *info;
@@ -382,8 +382,11 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
 		goto error_new_virtqueue;
 	}
 
+	if (!size || size > num)
+		size = num;
+
 	/* Create the vring */
-	vq = vring_create_virtqueue(index, num, VIRTIO_MMIO_VRING_ALIGN, vdev,
+	vq = vring_create_virtqueue(index, size, VIRTIO_MMIO_VRING_ALIGN, vdev,
 				 true, true, ctx, vm_notify, callback, name);
 	if (!vq) {
 		err = -ENOMEM;
@@ -484,6 +487,7 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		}
 
 		vqs[i] = vm_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
+				     sizes ? sizes[i] : 0,
 				     ctx ? ctx[i] : false);
 		if (IS_ERR(vqs[i])) {
 			vm_del_vqs(vdev);
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 27/32] virtio: add helper virtio_find_vqs_ctx_size()

[Xuan Zhuo]

Introduce helper virtio_find_vqs_ctx_size() to call find_vqs and specify
the maximum size of each vq ring.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 include/linux/virtio_config.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
index 0f7def7ddfd2..22e29c926946 100644
--- a/include/linux/virtio_config.h
+++ b/include/linux/virtio_config.h
@@ -235,6 +235,18 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
 				      ctx, desc);
 }
 
+static inline
+int virtio_find_vqs_ctx_size(struct virtio_device *vdev, u32 nvqs,
+				 struct virtqueue *vqs[],
+				 vq_callback_t *callbacks[],
+				 const char * const names[],
+				 u32 sizes[],
+				 const bool *ctx, struct irq_affinity *desc)
+{
+	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, sizes,
+				      ctx, desc);
+}
+
 /**
  * virtio_device_ready - enable vq use in probe function
  * @vdev: the device
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 27/32] virtio: add helper virtio_find_vqs_ctx_size()

[Xuan Zhuo]

Introduce helper virtio_find_vqs_ctx_size() to call find_vqs and specify
the maximum size of each vq ring.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 include/linux/virtio_config.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
index 0f7def7ddfd2..22e29c926946 100644
--- a/include/linux/virtio_config.h
+++ b/include/linux/virtio_config.h
@@ -235,6 +235,18 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
 				      ctx, desc);
 }
 
+static inline
+int virtio_find_vqs_ctx_size(struct virtio_device *vdev, u32 nvqs,
+				 struct virtqueue *vqs[],
+				 vq_callback_t *callbacks[],
+				 const char * const names[],
+				 u32 sizes[],
+				 const bool *ctx, struct irq_affinity *desc)
+{
+	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, sizes,
+				      ctx, desc);
+}
+
 /**
  * virtio_device_ready - enable vq use in probe function
  * @vdev: the device
-- 
2.31.0

[PATCH v9 27/32] virtio: add helper virtio_find_vqs_ctx_size()

[Xuan Zhuo]

Introduce helper virtio_find_vqs_ctx_size() to call find_vqs and specify
the maximum size of each vq ring.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 include/linux/virtio_config.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
index 0f7def7ddfd2..22e29c926946 100644
--- a/include/linux/virtio_config.h
+++ b/include/linux/virtio_config.h
@@ -235,6 +235,18 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
 				      ctx, desc);
 }
 
+static inline
+int virtio_find_vqs_ctx_size(struct virtio_device *vdev, u32 nvqs,
+				 struct virtqueue *vqs[],
+				 vq_callback_t *callbacks[],
+				 const char * const names[],
+				 u32 sizes[],
+				 const bool *ctx, struct irq_affinity *desc)
+{
+	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, sizes,
+				      ctx, desc);
+}
+
 /**
  * virtio_device_ready - enable vq use in probe function
  * @vdev: the device
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 28/32] virtio_net: set the default max ring size by find_vqs()

[Xuan Zhuo]

Use virtio_find_vqs_ctx_size() to specify the maximum ring size of tx,
rx at the same time.

                         | rx/tx ring size
-------------------------------------------
speed == UNKNOWN or < 10G| 1024
speed < 40G              | 4096
speed >= 40G             | 8192

Call virtnet_update_settings() once before calling init_vqs() to update
speed.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/net/virtio_net.c | 42 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 38 insertions(+), 4 deletions(-)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index a801ea40908f..dad497a47b3a 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -2861,6 +2861,29 @@ static unsigned int mergeable_min_buf_len(struct virtnet_info *vi, struct virtqu
 		   (unsigned int)GOOD_PACKET_LEN);
 }
 
+static void virtnet_config_sizes(struct virtnet_info *vi, u32 *sizes)
+{
+	u32 i, rx_size, tx_size;
+
+	if (vi->speed == SPEED_UNKNOWN || vi->speed < SPEED_10000) {
+		rx_size = 1024;
+		tx_size = 1024;
+
+	} else if (vi->speed < SPEED_40000) {
+		rx_size = 1024 * 4;
+		tx_size = 1024 * 4;
+
+	} else {
+		rx_size = 1024 * 8;
+		tx_size = 1024 * 8;
+	}
+
+	for (i = 0; i < vi->max_queue_pairs; i++) {
+		sizes[rxq2vq(i)] = rx_size;
+		sizes[txq2vq(i)] = tx_size;
+	}
+}
+
 static int virtnet_find_vqs(struct virtnet_info *vi)
 {
 	vq_callback_t **callbacks;
@@ -2868,6 +2891,7 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
 	int ret = -ENOMEM;
 	int i, total_vqs;
 	const char **names;
+	u32 *sizes;
 	bool *ctx;
 
 	/* We expect 1 RX virtqueue followed by 1 TX virtqueue, followed by
@@ -2895,10 +2919,15 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
 		ctx = NULL;
 	}
 
+	sizes = kmalloc_array(total_vqs, sizeof(*sizes), GFP_KERNEL);
+	if (!sizes)
+		goto err_sizes;
+
 	/* Parameters for control virtqueue, if any */
 	if (vi->has_cvq) {
 		callbacks[total_vqs - 1] = NULL;
 		names[total_vqs - 1] = "control";
+		sizes[total_vqs - 1] = 64;
 	}
 
 	/* Allocate/initialize parameters for send/receive virtqueues */
@@ -2913,8 +2942,10 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
 			ctx[rxq2vq(i)] = true;
 	}
 
-	ret = virtio_find_vqs_ctx(vi->vdev, total_vqs, vqs, callbacks,
-				  names, ctx, NULL);
+	virtnet_config_sizes(vi, sizes);
+
+	ret = virtio_find_vqs_ctx_size(vi->vdev, total_vqs, vqs, callbacks,
+				       names, sizes, ctx, NULL);
 	if (ret)
 		goto err_find;
 
@@ -2934,6 +2965,8 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
 
 
 err_find:
+	kfree(sizes);
+err_sizes:
 	kfree(ctx);
 err_ctx:
 	kfree(names);
@@ -3252,6 +3285,9 @@ static int virtnet_probe(struct virtio_device *vdev)
 		vi->curr_queue_pairs = num_online_cpus();
 	vi->max_queue_pairs = max_queue_pairs;
 
+	virtnet_init_settings(dev);
+	virtnet_update_settings(vi);
+
 	/* Allocate/initialize the rx/tx queues, and invoke find_vqs */
 	err = init_vqs(vi);
 	if (err)
@@ -3264,8 +3300,6 @@ static int virtnet_probe(struct virtio_device *vdev)
 	netif_set_real_num_tx_queues(dev, vi->curr_queue_pairs);
 	netif_set_real_num_rx_queues(dev, vi->curr_queue_pairs);
 
-	virtnet_init_settings(dev);
-
 	if (virtio_has_feature(vdev, VIRTIO_NET_F_STANDBY)) {
 		vi->failover = net_failover_create(vi->dev);
 		if (IS_ERR(vi->failover)) {
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 28/32] virtio_net: set the default max ring size by find_vqs()

[Xuan Zhuo]

Use virtio_find_vqs_ctx_size() to specify the maximum ring size of tx,
rx at the same time.

                         | rx/tx ring size
-------------------------------------------
speed == UNKNOWN or < 10G| 1024
speed < 40G              | 4096
speed >= 40G             | 8192

Call virtnet_update_settings() once before calling init_vqs() to update
speed.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/net/virtio_net.c | 42 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 38 insertions(+), 4 deletions(-)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index a801ea40908f..dad497a47b3a 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -2861,6 +2861,29 @@ static unsigned int mergeable_min_buf_len(struct virtnet_info *vi, struct virtqu
 		   (unsigned int)GOOD_PACKET_LEN);
 }
 
+static void virtnet_config_sizes(struct virtnet_info *vi, u32 *sizes)
+{
+	u32 i, rx_size, tx_size;
+
+	if (vi->speed == SPEED_UNKNOWN || vi->speed < SPEED_10000) {
+		rx_size = 1024;
+		tx_size = 1024;
+
+	} else if (vi->speed < SPEED_40000) {
+		rx_size = 1024 * 4;
+		tx_size = 1024 * 4;
+
+	} else {
+		rx_size = 1024 * 8;
+		tx_size = 1024 * 8;
+	}
+
+	for (i = 0; i < vi->max_queue_pairs; i++) {
+		sizes[rxq2vq(i)] = rx_size;
+		sizes[txq2vq(i)] = tx_size;
+	}
+}
+
 static int virtnet_find_vqs(struct virtnet_info *vi)
 {
 	vq_callback_t **callbacks;
@@ -2868,6 +2891,7 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
 	int ret = -ENOMEM;
 	int i, total_vqs;
 	const char **names;
+	u32 *sizes;
 	bool *ctx;
 
 	/* We expect 1 RX virtqueue followed by 1 TX virtqueue, followed by
@@ -2895,10 +2919,15 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
 		ctx = NULL;
 	}
 
+	sizes = kmalloc_array(total_vqs, sizeof(*sizes), GFP_KERNEL);
+	if (!sizes)
+		goto err_sizes;
+
 	/* Parameters for control virtqueue, if any */
 	if (vi->has_cvq) {
 		callbacks[total_vqs - 1] = NULL;
 		names[total_vqs - 1] = "control";
+		sizes[total_vqs - 1] = 64;
 	}
 
 	/* Allocate/initialize parameters for send/receive virtqueues */
@@ -2913,8 +2942,10 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
 			ctx[rxq2vq(i)] = true;
 	}
 
-	ret = virtio_find_vqs_ctx(vi->vdev, total_vqs, vqs, callbacks,
-				  names, ctx, NULL);
+	virtnet_config_sizes(vi, sizes);
+
+	ret = virtio_find_vqs_ctx_size(vi->vdev, total_vqs, vqs, callbacks,
+				       names, sizes, ctx, NULL);
 	if (ret)
 		goto err_find;
 
@@ -2934,6 +2965,8 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
 
 
 err_find:
+	kfree(sizes);
+err_sizes:
 	kfree(ctx);
 err_ctx:
 	kfree(names);
@@ -3252,6 +3285,9 @@ static int virtnet_probe(struct virtio_device *vdev)
 		vi->curr_queue_pairs = num_online_cpus();
 	vi->max_queue_pairs = max_queue_pairs;
 
+	virtnet_init_settings(dev);
+	virtnet_update_settings(vi);
+
 	/* Allocate/initialize the rx/tx queues, and invoke find_vqs */
 	err = init_vqs(vi);
 	if (err)
@@ -3264,8 +3300,6 @@ static int virtnet_probe(struct virtio_device *vdev)
 	netif_set_real_num_tx_queues(dev, vi->curr_queue_pairs);
 	netif_set_real_num_rx_queues(dev, vi->curr_queue_pairs);
 
-	virtnet_init_settings(dev);
-
 	if (virtio_has_feature(vdev, VIRTIO_NET_F_STANDBY)) {
 		vi->failover = net_failover_create(vi->dev);
 		if (IS_ERR(vi->failover)) {
-- 
2.31.0

[PATCH v9 28/32] virtio_net: set the default max ring size by find_vqs()

[Xuan Zhuo]

Use virtio_find_vqs_ctx_size() to specify the maximum ring size of tx,
rx at the same time.

                         | rx/tx ring size
-------------------------------------------
speed == UNKNOWN or < 10G| 1024
speed < 40G              | 4096
speed >= 40G             | 8192

Call virtnet_update_settings() once before calling init_vqs() to update
speed.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/net/virtio_net.c | 42 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 38 insertions(+), 4 deletions(-)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index a801ea40908f..dad497a47b3a 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -2861,6 +2861,29 @@ static unsigned int mergeable_min_buf_len(struct virtnet_info *vi, struct virtqu
 		   (unsigned int)GOOD_PACKET_LEN);
 }
 
+static void virtnet_config_sizes(struct virtnet_info *vi, u32 *sizes)
+{
+	u32 i, rx_size, tx_size;
+
+	if (vi->speed == SPEED_UNKNOWN || vi->speed < SPEED_10000) {
+		rx_size = 1024;
+		tx_size = 1024;
+
+	} else if (vi->speed < SPEED_40000) {
+		rx_size = 1024 * 4;
+		tx_size = 1024 * 4;
+
+	} else {
+		rx_size = 1024 * 8;
+		tx_size = 1024 * 8;
+	}
+
+	for (i = 0; i < vi->max_queue_pairs; i++) {
+		sizes[rxq2vq(i)] = rx_size;
+		sizes[txq2vq(i)] = tx_size;
+	}
+}
+
 static int virtnet_find_vqs(struct virtnet_info *vi)
 {
 	vq_callback_t **callbacks;
@@ -2868,6 +2891,7 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
 	int ret = -ENOMEM;
 	int i, total_vqs;
 	const char **names;
+	u32 *sizes;
 	bool *ctx;
 
 	/* We expect 1 RX virtqueue followed by 1 TX virtqueue, followed by
@@ -2895,10 +2919,15 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
 		ctx = NULL;
 	}
 
+	sizes = kmalloc_array(total_vqs, sizeof(*sizes), GFP_KERNEL);
+	if (!sizes)
+		goto err_sizes;
+
 	/* Parameters for control virtqueue, if any */
 	if (vi->has_cvq) {
 		callbacks[total_vqs - 1] = NULL;
 		names[total_vqs - 1] = "control";
+		sizes[total_vqs - 1] = 64;
 	}
 
 	/* Allocate/initialize parameters for send/receive virtqueues */
@@ -2913,8 +2942,10 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
 			ctx[rxq2vq(i)] = true;
 	}
 
-	ret = virtio_find_vqs_ctx(vi->vdev, total_vqs, vqs, callbacks,
-				  names, ctx, NULL);
+	virtnet_config_sizes(vi, sizes);
+
+	ret = virtio_find_vqs_ctx_size(vi->vdev, total_vqs, vqs, callbacks,
+				       names, sizes, ctx, NULL);
 	if (ret)
 		goto err_find;
 
@@ -2934,6 +2965,8 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
 
 
 err_find:
+	kfree(sizes);
+err_sizes:
 	kfree(ctx);
 err_ctx:
 	kfree(names);
@@ -3252,6 +3285,9 @@ static int virtnet_probe(struct virtio_device *vdev)
 		vi->curr_queue_pairs = num_online_cpus();
 	vi->max_queue_pairs = max_queue_pairs;
 
+	virtnet_init_settings(dev);
+	virtnet_update_settings(vi);
+
 	/* Allocate/initialize the rx/tx queues, and invoke find_vqs */
 	err = init_vqs(vi);
 	if (err)
@@ -3264,8 +3300,6 @@ static int virtnet_probe(struct virtio_device *vdev)
 	netif_set_real_num_tx_queues(dev, vi->curr_queue_pairs);
 	netif_set_real_num_rx_queues(dev, vi->curr_queue_pairs);
 
-	virtnet_init_settings(dev);
-
 	if (virtio_has_feature(vdev, VIRTIO_NET_F_STANDBY)) {
 		vi->failover = net_failover_create(vi->dev);
 		if (IS_ERR(vi->failover)) {
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 29/32] virtio_net: get ringparam by virtqueue_get_vring_max_size()

[Xuan Zhuo]

Use virtqueue_get_vring_max_size() in virtnet_get_ringparam() to set
tx,rx_max_pending.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/net/virtio_net.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index dad497a47b3a..96d96c666c8c 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -2177,10 +2177,10 @@ static void virtnet_get_ringparam(struct net_device *dev,
 {
 	struct virtnet_info *vi = netdev_priv(dev);
 
-	ring->rx_max_pending = virtqueue_get_vring_size(vi->rq[0].vq);
-	ring->tx_max_pending = virtqueue_get_vring_size(vi->sq[0].vq);
-	ring->rx_pending = ring->rx_max_pending;
-	ring->tx_pending = ring->tx_max_pending;
+	ring->rx_max_pending = virtqueue_get_vring_max_size(vi->rq[0].vq);
+	ring->tx_max_pending = virtqueue_get_vring_max_size(vi->sq[0].vq);
+	ring->rx_pending = virtqueue_get_vring_size(vi->rq[0].vq);
+	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
 }
 
 
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 29/32] virtio_net: get ringparam by virtqueue_get_vring_max_size()

[Xuan Zhuo]

Use virtqueue_get_vring_max_size() in virtnet_get_ringparam() to set
tx,rx_max_pending.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/net/virtio_net.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index dad497a47b3a..96d96c666c8c 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -2177,10 +2177,10 @@ static void virtnet_get_ringparam(struct net_device *dev,
 {
 	struct virtnet_info *vi = netdev_priv(dev);
 
-	ring->rx_max_pending = virtqueue_get_vring_size(vi->rq[0].vq);
-	ring->tx_max_pending = virtqueue_get_vring_size(vi->sq[0].vq);
-	ring->rx_pending = ring->rx_max_pending;
-	ring->tx_pending = ring->tx_max_pending;
+	ring->rx_max_pending = virtqueue_get_vring_max_size(vi->rq[0].vq);
+	ring->tx_max_pending = virtqueue_get_vring_max_size(vi->sq[0].vq);
+	ring->rx_pending = virtqueue_get_vring_size(vi->rq[0].vq);
+	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
 }
 
 
-- 
2.31.0

[PATCH v9 29/32] virtio_net: get ringparam by virtqueue_get_vring_max_size()

[Xuan Zhuo]

Use virtqueue_get_vring_max_size() in virtnet_get_ringparam() to set
tx,rx_max_pending.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/net/virtio_net.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index dad497a47b3a..96d96c666c8c 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -2177,10 +2177,10 @@ static void virtnet_get_ringparam(struct net_device *dev,
 {
 	struct virtnet_info *vi = netdev_priv(dev);
 
-	ring->rx_max_pending = virtqueue_get_vring_size(vi->rq[0].vq);
-	ring->tx_max_pending = virtqueue_get_vring_size(vi->sq[0].vq);
-	ring->rx_pending = ring->rx_max_pending;
-	ring->tx_pending = ring->tx_max_pending;
+	ring->rx_max_pending = virtqueue_get_vring_max_size(vi->rq[0].vq);
+	ring->tx_max_pending = virtqueue_get_vring_max_size(vi->sq[0].vq);
+	ring->rx_pending = virtqueue_get_vring_size(vi->rq[0].vq);
+	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
 }
 
 
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 30/32] virtio_net: split free_unused_bufs()

[Xuan Zhuo]

This patch separates two functions for freeing sq buf and rq buf from
free_unused_bufs().

When supporting the enable/disable tx/rq queue in the future, it is
necessary to support separate recovery of a sq buf or a rq buf.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
 drivers/net/virtio_net.c | 41 ++++++++++++++++++++++++----------------
 1 file changed, 25 insertions(+), 16 deletions(-)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 96d96c666c8c..b8bf00525177 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -2804,6 +2804,27 @@ static void free_receive_page_frags(struct virtnet_info *vi)
 			put_page(vi->rq[i].alloc_frag.page);
 }
 
+static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf)
+{
+	if (!is_xdp_frame(buf))
+		dev_kfree_skb(buf);
+	else
+		xdp_return_frame(ptr_to_xdp(buf));
+}
+
+static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf)
+{
+	struct virtnet_info *vi = vq->vdev->priv;
+	int i = vq2rxq(vq);
+
+	if (vi->mergeable_rx_bufs)
+		put_page(virt_to_head_page(buf));
+	else if (vi->big_packets)
+		give_pages(&vi->rq[i], buf);
+	else
+		put_page(virt_to_head_page(buf));
+}
+
 static void free_unused_bufs(struct virtnet_info *vi)
 {
 	void *buf;
@@ -2811,26 +2832,14 @@ static void free_unused_bufs(struct virtnet_info *vi)
 
 	for (i = 0; i < vi->max_queue_pairs; i++) {
 		struct virtqueue *vq = vi->sq[i].vq;
-		while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) {
-			if (!is_xdp_frame(buf))
-				dev_kfree_skb(buf);
-			else
-				xdp_return_frame(ptr_to_xdp(buf));
-		}
+		while ((buf = virtqueue_detach_unused_buf(vq)) != NULL)
+			virtnet_sq_free_unused_buf(vq, buf);
 	}
 
 	for (i = 0; i < vi->max_queue_pairs; i++) {
 		struct virtqueue *vq = vi->rq[i].vq;
-
-		while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) {
-			if (vi->mergeable_rx_bufs) {
-				put_page(virt_to_head_page(buf));
-			} else if (vi->big_packets) {
-				give_pages(&vi->rq[i], buf);
-			} else {
-				put_page(virt_to_head_page(buf));
-			}
-		}
+		while ((buf = virtqueue_detach_unused_buf(vq)) != NULL)
+			virtnet_rq_free_unused_buf(vq, buf);
 	}
 }
 
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 30/32] virtio_net: split free_unused_bufs()

[Xuan Zhuo]

This patch separates two functions for freeing sq buf and rq buf from
free_unused_bufs().

When supporting the enable/disable tx/rq queue in the future, it is
necessary to support separate recovery of a sq buf or a rq buf.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
 drivers/net/virtio_net.c | 41 ++++++++++++++++++++++++----------------
 1 file changed, 25 insertions(+), 16 deletions(-)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 96d96c666c8c..b8bf00525177 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -2804,6 +2804,27 @@ static void free_receive_page_frags(struct virtnet_info *vi)
 			put_page(vi->rq[i].alloc_frag.page);
 }
 
+static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf)
+{
+	if (!is_xdp_frame(buf))
+		dev_kfree_skb(buf);
+	else
+		xdp_return_frame(ptr_to_xdp(buf));
+}
+
+static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf)
+{
+	struct virtnet_info *vi = vq->vdev->priv;
+	int i = vq2rxq(vq);
+
+	if (vi->mergeable_rx_bufs)
+		put_page(virt_to_head_page(buf));
+	else if (vi->big_packets)
+		give_pages(&vi->rq[i], buf);
+	else
+		put_page(virt_to_head_page(buf));
+}
+
 static void free_unused_bufs(struct virtnet_info *vi)
 {
 	void *buf;
@@ -2811,26 +2832,14 @@ static void free_unused_bufs(struct virtnet_info *vi)
 
 	for (i = 0; i < vi->max_queue_pairs; i++) {
 		struct virtqueue *vq = vi->sq[i].vq;
-		while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) {
-			if (!is_xdp_frame(buf))
-				dev_kfree_skb(buf);
-			else
-				xdp_return_frame(ptr_to_xdp(buf));
-		}
+		while ((buf = virtqueue_detach_unused_buf(vq)) != NULL)
+			virtnet_sq_free_unused_buf(vq, buf);
 	}
 
 	for (i = 0; i < vi->max_queue_pairs; i++) {
 		struct virtqueue *vq = vi->rq[i].vq;
-
-		while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) {
-			if (vi->mergeable_rx_bufs) {
-				put_page(virt_to_head_page(buf));
-			} else if (vi->big_packets) {
-				give_pages(&vi->rq[i], buf);
-			} else {
-				put_page(virt_to_head_page(buf));
-			}
-		}
+		while ((buf = virtqueue_detach_unused_buf(vq)) != NULL)
+			virtnet_rq_free_unused_buf(vq, buf);
 	}
 }
 
-- 
2.31.0

[PATCH v9 30/32] virtio_net: split free_unused_bufs()

[Xuan Zhuo]

This patch separates two functions for freeing sq buf and rq buf from
free_unused_bufs().

When supporting the enable/disable tx/rq queue in the future, it is
necessary to support separate recovery of a sq buf or a rq buf.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
 drivers/net/virtio_net.c | 41 ++++++++++++++++++++++++----------------
 1 file changed, 25 insertions(+), 16 deletions(-)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 96d96c666c8c..b8bf00525177 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -2804,6 +2804,27 @@ static void free_receive_page_frags(struct virtnet_info *vi)
 			put_page(vi->rq[i].alloc_frag.page);
 }
 
+static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf)
+{
+	if (!is_xdp_frame(buf))
+		dev_kfree_skb(buf);
+	else
+		xdp_return_frame(ptr_to_xdp(buf));
+}
+
+static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf)
+{
+	struct virtnet_info *vi = vq->vdev->priv;
+	int i = vq2rxq(vq);
+
+	if (vi->mergeable_rx_bufs)
+		put_page(virt_to_head_page(buf));
+	else if (vi->big_packets)
+		give_pages(&vi->rq[i], buf);
+	else
+		put_page(virt_to_head_page(buf));
+}
+
 static void free_unused_bufs(struct virtnet_info *vi)
 {
 	void *buf;
@@ -2811,26 +2832,14 @@ static void free_unused_bufs(struct virtnet_info *vi)
 
 	for (i = 0; i < vi->max_queue_pairs; i++) {
 		struct virtqueue *vq = vi->sq[i].vq;
-		while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) {
-			if (!is_xdp_frame(buf))
-				dev_kfree_skb(buf);
-			else
-				xdp_return_frame(ptr_to_xdp(buf));
-		}
+		while ((buf = virtqueue_detach_unused_buf(vq)) != NULL)
+			virtnet_sq_free_unused_buf(vq, buf);
 	}
 
 	for (i = 0; i < vi->max_queue_pairs; i++) {
 		struct virtqueue *vq = vi->rq[i].vq;
-
-		while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) {
-			if (vi->mergeable_rx_bufs) {
-				put_page(virt_to_head_page(buf));
-			} else if (vi->big_packets) {
-				give_pages(&vi->rq[i], buf);
-			} else {
-				put_page(virt_to_head_page(buf));
-			}
-		}
+		while ((buf = virtqueue_detach_unused_buf(vq)) != NULL)
+			virtnet_rq_free_unused_buf(vq, buf);
 	}
 }
 
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

This patch implements the resize function of the rx, tx queues.
Based on this function, it is possible to modify the ring num of the
queue.

There may be an exception during the resize process, the resize may
fail, or the vq can no longer be used. Either way, we must execute
napi_enable(). Because napi_disable is similar to a lock, napi_enable
must be called after calling napi_disable.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index b8bf00525177..ba6859f305f7 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -251,6 +251,9 @@ struct padded_vnet_hdr {
 	char padding[4];
 };
 
+static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
+static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
+
 static bool is_xdp_frame(void *ptr)
 {
 	return (unsigned long)ptr & VIRTIO_XDP_FLAG;
@@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
 {
 	napi_enable(napi);
 
+	/* Check if vq is in reset state. The normal reset/resize process will
+	 * be protected by napi. However, the protection of napi is only enabled
+	 * during the operation, and the protection of napi will end after the
+	 * operation is completed. If re-enable fails during the process, vq
+	 * will remain unavailable with reset state.
+	 */
+	if (vq->reset)
+		return;
+
 	/* If all buffers were filled by other side before we napi_enabled, we
 	 * won't get another interrupt, so process any outstanding packets now.
 	 * Call local_bh_enable after to trigger softIRQ processing.
@@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
 		struct receive_queue *rq = &vi->rq[i];
 
 		napi_disable(&rq->napi);
+
+		/* Check if vq is in reset state. See more in
+		 * virtnet_napi_enable()
+		 */
+		if (rq->vq->reset) {
+			virtnet_napi_enable(rq->vq, &rq->napi);
+			continue;
+		}
+
 		still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
 		virtnet_napi_enable(rq->vq, &rq->napi);
 
@@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
 	if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
 		return;
 
+	/* Check if vq is in reset state. See more in virtnet_napi_enable() */
+	if (sq->vq->reset)
+		return;
+
 	if (__netif_tx_trylock(txq)) {
 		do {
 			virtqueue_disable_cb(sq->vq);
@@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
 	return NETDEV_TX_OK;
 }
 
+static int virtnet_rx_resize(struct virtnet_info *vi,
+			     struct receive_queue *rq, u32 ring_num)
+{
+	int err;
+
+	napi_disable(&rq->napi);
+
+	err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
+	if (err)
+		goto err;
+
+	if (!try_fill_recv(vi, rq, GFP_KERNEL))
+		schedule_delayed_work(&vi->refill, 0);
+
+	virtnet_napi_enable(rq->vq, &rq->napi);
+	return 0;
+
+err:
+	netdev_err(vi->dev,
+		   "reset rx reset vq fail: rx queue index: %td err: %d\n",
+		   rq - vi->rq, err);
+	virtnet_napi_enable(rq->vq, &rq->napi);
+	return err;
+}
+
+static int virtnet_tx_resize(struct virtnet_info *vi,
+			     struct send_queue *sq, u32 ring_num)
+{
+	struct netdev_queue *txq;
+	int err, qindex;
+
+	qindex = sq - vi->sq;
+
+	virtnet_napi_tx_disable(&sq->napi);
+
+	txq = netdev_get_tx_queue(vi->dev, qindex);
+	__netif_tx_lock_bh(txq);
+	netif_stop_subqueue(vi->dev, qindex);
+	__netif_tx_unlock_bh(txq);
+
+	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
+	if (err)
+		goto err;
+
+	netif_start_subqueue(vi->dev, qindex);
+	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
+	return 0;
+
+err:
+	netdev_err(vi->dev,
+		   "reset tx reset vq fail: tx queue index: %td err: %d\n",
+		   sq - vi->sq, err);
+	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
+	return err;
+}
+
 /*
  * Send command via the control virtqueue and check status.  Commands
  * supported by the hypervisor, as indicated by feature bits, should
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

This patch implements the resize function of the rx, tx queues.
Based on this function, it is possible to modify the ring num of the
queue.

There may be an exception during the resize process, the resize may
fail, or the vq can no longer be used. Either way, we must execute
napi_enable(). Because napi_disable is similar to a lock, napi_enable
must be called after calling napi_disable.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index b8bf00525177..ba6859f305f7 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -251,6 +251,9 @@ struct padded_vnet_hdr {
 	char padding[4];
 };
 
+static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
+static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
+
 static bool is_xdp_frame(void *ptr)
 {
 	return (unsigned long)ptr & VIRTIO_XDP_FLAG;
@@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
 {
 	napi_enable(napi);
 
+	/* Check if vq is in reset state. The normal reset/resize process will
+	 * be protected by napi. However, the protection of napi is only enabled
+	 * during the operation, and the protection of napi will end after the
+	 * operation is completed. If re-enable fails during the process, vq
+	 * will remain unavailable with reset state.
+	 */
+	if (vq->reset)
+		return;
+
 	/* If all buffers were filled by other side before we napi_enabled, we
 	 * won't get another interrupt, so process any outstanding packets now.
 	 * Call local_bh_enable after to trigger softIRQ processing.
@@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
 		struct receive_queue *rq = &vi->rq[i];
 
 		napi_disable(&rq->napi);
+
+		/* Check if vq is in reset state. See more in
+		 * virtnet_napi_enable()
+		 */
+		if (rq->vq->reset) {
+			virtnet_napi_enable(rq->vq, &rq->napi);
+			continue;
+		}
+
 		still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
 		virtnet_napi_enable(rq->vq, &rq->napi);
 
@@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
 	if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
 		return;
 
+	/* Check if vq is in reset state. See more in virtnet_napi_enable() */
+	if (sq->vq->reset)
+		return;
+
 	if (__netif_tx_trylock(txq)) {
 		do {
 			virtqueue_disable_cb(sq->vq);
@@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
 	return NETDEV_TX_OK;
 }
 
+static int virtnet_rx_resize(struct virtnet_info *vi,
+			     struct receive_queue *rq, u32 ring_num)
+{
+	int err;
+
+	napi_disable(&rq->napi);
+
+	err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
+	if (err)
+		goto err;
+
+	if (!try_fill_recv(vi, rq, GFP_KERNEL))
+		schedule_delayed_work(&vi->refill, 0);
+
+	virtnet_napi_enable(rq->vq, &rq->napi);
+	return 0;
+
+err:
+	netdev_err(vi->dev,
+		   "reset rx reset vq fail: rx queue index: %td err: %d\n",
+		   rq - vi->rq, err);
+	virtnet_napi_enable(rq->vq, &rq->napi);
+	return err;
+}
+
+static int virtnet_tx_resize(struct virtnet_info *vi,
+			     struct send_queue *sq, u32 ring_num)
+{
+	struct netdev_queue *txq;
+	int err, qindex;
+
+	qindex = sq - vi->sq;
+
+	virtnet_napi_tx_disable(&sq->napi);
+
+	txq = netdev_get_tx_queue(vi->dev, qindex);
+	__netif_tx_lock_bh(txq);
+	netif_stop_subqueue(vi->dev, qindex);
+	__netif_tx_unlock_bh(txq);
+
+	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
+	if (err)
+		goto err;
+
+	netif_start_subqueue(vi->dev, qindex);
+	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
+	return 0;
+
+err:
+	netdev_err(vi->dev,
+		   "reset tx reset vq fail: tx queue index: %td err: %d\n",
+		   sq - vi->sq, err);
+	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
+	return err;
+}
+
 /*
  * Send command via the control virtqueue and check status.  Commands
  * supported by the hypervisor, as indicated by feature bits, should
-- 
2.31.0

[PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

This patch implements the resize function of the rx, tx queues.
Based on this function, it is possible to modify the ring num of the
queue.

There may be an exception during the resize process, the resize may
fail, or the vq can no longer be used. Either way, we must execute
napi_enable(). Because napi_disable is similar to a lock, napi_enable
must be called after calling napi_disable.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index b8bf00525177..ba6859f305f7 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -251,6 +251,9 @@ struct padded_vnet_hdr {
 	char padding[4];
 };
 
+static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
+static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
+
 static bool is_xdp_frame(void *ptr)
 {
 	return (unsigned long)ptr & VIRTIO_XDP_FLAG;
@@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
 {
 	napi_enable(napi);
 
+	/* Check if vq is in reset state. The normal reset/resize process will
+	 * be protected by napi. However, the protection of napi is only enabled
+	 * during the operation, and the protection of napi will end after the
+	 * operation is completed. If re-enable fails during the process, vq
+	 * will remain unavailable with reset state.
+	 */
+	if (vq->reset)
+		return;
+
 	/* If all buffers were filled by other side before we napi_enabled, we
 	 * won't get another interrupt, so process any outstanding packets now.
 	 * Call local_bh_enable after to trigger softIRQ processing.
@@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
 		struct receive_queue *rq = &vi->rq[i];
 
 		napi_disable(&rq->napi);
+
+		/* Check if vq is in reset state. See more in
+		 * virtnet_napi_enable()
+		 */
+		if (rq->vq->reset) {
+			virtnet_napi_enable(rq->vq, &rq->napi);
+			continue;
+		}
+
 		still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
 		virtnet_napi_enable(rq->vq, &rq->napi);
 
@@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
 	if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
 		return;
 
+	/* Check if vq is in reset state. See more in virtnet_napi_enable() */
+	if (sq->vq->reset)
+		return;
+
 	if (__netif_tx_trylock(txq)) {
 		do {
 			virtqueue_disable_cb(sq->vq);
@@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
 	return NETDEV_TX_OK;
 }
 
+static int virtnet_rx_resize(struct virtnet_info *vi,
+			     struct receive_queue *rq, u32 ring_num)
+{
+	int err;
+
+	napi_disable(&rq->napi);
+
+	err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
+	if (err)
+		goto err;
+
+	if (!try_fill_recv(vi, rq, GFP_KERNEL))
+		schedule_delayed_work(&vi->refill, 0);
+
+	virtnet_napi_enable(rq->vq, &rq->napi);
+	return 0;
+
+err:
+	netdev_err(vi->dev,
+		   "reset rx reset vq fail: rx queue index: %td err: %d\n",
+		   rq - vi->rq, err);
+	virtnet_napi_enable(rq->vq, &rq->napi);
+	return err;
+}
+
+static int virtnet_tx_resize(struct virtnet_info *vi,
+			     struct send_queue *sq, u32 ring_num)
+{
+	struct netdev_queue *txq;
+	int err, qindex;
+
+	qindex = sq - vi->sq;
+
+	virtnet_napi_tx_disable(&sq->napi);
+
+	txq = netdev_get_tx_queue(vi->dev, qindex);
+	__netif_tx_lock_bh(txq);
+	netif_stop_subqueue(vi->dev, qindex);
+	__netif_tx_unlock_bh(txq);
+
+	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
+	if (err)
+		goto err;
+
+	netif_start_subqueue(vi->dev, qindex);
+	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
+	return 0;
+
+err:
+	netdev_err(vi->dev,
+		   "reset tx reset vq fail: tx queue index: %td err: %d\n",
+		   sq - vi->sq, err);
+	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
+	return err;
+}
+
 /*
  * Send command via the control virtqueue and check status.  Commands
  * supported by the hypervisor, as indicated by feature bits, should
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

[PATCH v9 32/32] virtio_net: support set_ringparam

[Xuan Zhuo]

Support set_ringparam based on virtio queue reset.

Users can use ethtool -G eth0 <ring_num> to modify the ring size of
virtio-net.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/net/virtio_net.c | 47 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index ba6859f305f7..37e4e27f1e4e 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -2264,6 +2264,52 @@ static void virtnet_get_ringparam(struct net_device *dev,
 	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
 }
 
+static int virtnet_set_ringparam(struct net_device *dev,
+				 struct ethtool_ringparam *ring,
+				 struct kernel_ethtool_ringparam *kernel_ring,
+				 struct netlink_ext_ack *extack)
+{
+	struct virtnet_info *vi = netdev_priv(dev);
+	u32 rx_pending, tx_pending;
+	struct receive_queue *rq;
+	struct send_queue *sq;
+	int i, err;
+
+	if (ring->rx_mini_pending || ring->rx_jumbo_pending)
+		return -EINVAL;
+
+	rx_pending = virtqueue_get_vring_size(vi->rq[0].vq);
+	tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
+
+	if (ring->rx_pending == rx_pending &&
+	    ring->tx_pending == tx_pending)
+		return 0;
+
+	if (ring->rx_pending > virtqueue_get_vring_max_size(vi->rq[0].vq))
+		return -EINVAL;
+
+	if (ring->tx_pending > virtqueue_get_vring_max_size(vi->sq[0].vq))
+		return -EINVAL;
+
+	for (i = 0; i < vi->max_queue_pairs; i++) {
+		rq = vi->rq + i;
+		sq = vi->sq + i;
+
+		if (ring->tx_pending != tx_pending) {
+			err = virtnet_tx_resize(vi, sq, ring->tx_pending);
+			if (err)
+				return err;
+		}
+
+		if (ring->rx_pending != rx_pending) {
+			err = virtnet_rx_resize(vi, rq, ring->rx_pending);
+			if (err)
+				return err;
+		}
+	}
+
+	return 0;
+}
 
 static void virtnet_get_drvinfo(struct net_device *dev,
 				struct ethtool_drvinfo *info)
@@ -2497,6 +2543,7 @@ static const struct ethtool_ops virtnet_ethtool_ops = {
 	.get_drvinfo = virtnet_get_drvinfo,
 	.get_link = ethtool_op_get_link,
 	.get_ringparam = virtnet_get_ringparam,
+	.set_ringparam = virtnet_set_ringparam,
 	.get_strings = virtnet_get_strings,
 	.get_sset_count = virtnet_get_sset_count,
 	.get_ethtool_stats = virtnet_get_ethtool_stats,
-- 
2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v9 32/32] virtio_net: support set_ringparam

[Xuan Zhuo]

Support set_ringparam based on virtio queue reset.

Users can use ethtool -G eth0 <ring_num> to modify the ring size of
virtio-net.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/net/virtio_net.c | 47 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index ba6859f305f7..37e4e27f1e4e 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -2264,6 +2264,52 @@ static void virtnet_get_ringparam(struct net_device *dev,
 	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
 }
 
+static int virtnet_set_ringparam(struct net_device *dev,
+				 struct ethtool_ringparam *ring,
+				 struct kernel_ethtool_ringparam *kernel_ring,
+				 struct netlink_ext_ack *extack)
+{
+	struct virtnet_info *vi = netdev_priv(dev);
+	u32 rx_pending, tx_pending;
+	struct receive_queue *rq;
+	struct send_queue *sq;
+	int i, err;
+
+	if (ring->rx_mini_pending || ring->rx_jumbo_pending)
+		return -EINVAL;
+
+	rx_pending = virtqueue_get_vring_size(vi->rq[0].vq);
+	tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
+
+	if (ring->rx_pending == rx_pending &&
+	    ring->tx_pending == tx_pending)
+		return 0;
+
+	if (ring->rx_pending > virtqueue_get_vring_max_size(vi->rq[0].vq))
+		return -EINVAL;
+
+	if (ring->tx_pending > virtqueue_get_vring_max_size(vi->sq[0].vq))
+		return -EINVAL;
+
+	for (i = 0; i < vi->max_queue_pairs; i++) {
+		rq = vi->rq + i;
+		sq = vi->sq + i;
+
+		if (ring->tx_pending != tx_pending) {
+			err = virtnet_tx_resize(vi, sq, ring->tx_pending);
+			if (err)
+				return err;
+		}
+
+		if (ring->rx_pending != rx_pending) {
+			err = virtnet_rx_resize(vi, rq, ring->rx_pending);
+			if (err)
+				return err;
+		}
+	}
+
+	return 0;
+}
 
 static void virtnet_get_drvinfo(struct net_device *dev,
 				struct ethtool_drvinfo *info)
@@ -2497,6 +2543,7 @@ static const struct ethtool_ops virtnet_ethtool_ops = {
 	.get_drvinfo = virtnet_get_drvinfo,
 	.get_link = ethtool_op_get_link,
 	.get_ringparam = virtnet_get_ringparam,
+	.set_ringparam = virtnet_set_ringparam,
 	.get_strings = virtnet_get_strings,
 	.get_sset_count = virtnet_get_sset_count,
 	.get_ethtool_stats = virtnet_get_ethtool_stats,
-- 
2.31.0

[PATCH v9 32/32] virtio_net: support set_ringparam

[Xuan Zhuo]

Support set_ringparam based on virtio queue reset.

Users can use ethtool -G eth0 <ring_num> to modify the ring size of
virtio-net.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 drivers/net/virtio_net.c | 47 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index ba6859f305f7..37e4e27f1e4e 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -2264,6 +2264,52 @@ static void virtnet_get_ringparam(struct net_device *dev,
 	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
 }
 
+static int virtnet_set_ringparam(struct net_device *dev,
+				 struct ethtool_ringparam *ring,
+				 struct kernel_ethtool_ringparam *kernel_ring,
+				 struct netlink_ext_ack *extack)
+{
+	struct virtnet_info *vi = netdev_priv(dev);
+	u32 rx_pending, tx_pending;
+	struct receive_queue *rq;
+	struct send_queue *sq;
+	int i, err;
+
+	if (ring->rx_mini_pending || ring->rx_jumbo_pending)
+		return -EINVAL;
+
+	rx_pending = virtqueue_get_vring_size(vi->rq[0].vq);
+	tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
+
+	if (ring->rx_pending == rx_pending &&
+	    ring->tx_pending == tx_pending)
+		return 0;
+
+	if (ring->rx_pending > virtqueue_get_vring_max_size(vi->rq[0].vq))
+		return -EINVAL;
+
+	if (ring->tx_pending > virtqueue_get_vring_max_size(vi->sq[0].vq))
+		return -EINVAL;
+
+	for (i = 0; i < vi->max_queue_pairs; i++) {
+		rq = vi->rq + i;
+		sq = vi->sq + i;
+
+		if (ring->tx_pending != tx_pending) {
+			err = virtnet_tx_resize(vi, sq, ring->tx_pending);
+			if (err)
+				return err;
+		}
+
+		if (ring->rx_pending != rx_pending) {
+			err = virtnet_rx_resize(vi, rq, ring->rx_pending);
+			if (err)
+				return err;
+		}
+	}
+
+	return 0;
+}
 
 static void virtnet_get_drvinfo(struct net_device *dev,
 				struct ethtool_drvinfo *info)
@@ -2497,6 +2543,7 @@ static const struct ethtool_ops virtnet_ethtool_ops = {
 	.get_drvinfo = virtnet_get_drvinfo,
 	.get_link = ethtool_op_get_link,
 	.get_ringparam = virtnet_get_ringparam,
+	.set_ringparam = virtnet_set_ringparam,
 	.get_strings = virtnet_get_strings,
 	.get_sset_count = virtnet_get_sset_count,
 	.get_ethtool_stats = virtnet_get_ethtool_stats,
-- 
2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 01/32] virtio: add helper virtqueue_get_vring_max_size()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Record the maximum queue num supported by the device.
>
> virtio-net can display the maximum (supported by hardware) ring size in
> ethtool -g eth0.
>
> When the subsequent patch implements vring reset, it can judge whether
> the ring size passed by the driver is legal based on this.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   arch/um/drivers/virtio_uml.c             |  1 +
>   drivers/platform/mellanox/mlxbf-tmfifo.c |  2 ++
>   drivers/remoteproc/remoteproc_virtio.c   |  2 ++
>   drivers/s390/virtio/virtio_ccw.c         |  3 +++
>   drivers/virtio/virtio_mmio.c             |  2 ++
>   drivers/virtio/virtio_pci_legacy.c       |  2 ++
>   drivers/virtio/virtio_pci_modern.c       |  2 ++
>   drivers/virtio/virtio_ring.c             | 14 ++++++++++++++
>   drivers/virtio/virtio_vdpa.c             |  2 ++
>   include/linux/virtio.h                   |  2 ++
>   10 files changed, 32 insertions(+)
>
> diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
> index ba562d68dc04..904993d15a85 100644
> --- a/arch/um/drivers/virtio_uml.c
> +++ b/arch/um/drivers/virtio_uml.c
> @@ -945,6 +945,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
>   		goto error_create;
>   	}
>   	vq->priv = info;
> +	vq->num_max = num;
>   	num = virtqueue_get_vring_size(vq);
>   
>   	if (vu_dev->protocol_features &
> diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
> index 38800e86ed8a..1ae3c56b66b0 100644
> --- a/drivers/platform/mellanox/mlxbf-tmfifo.c
> +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
> @@ -959,6 +959,8 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
>   			goto error;
>   		}
>   
> +		vq->num_max = vring->num;
> +
>   		vqs[i] = vq;
>   		vring->vq = vq;
>   		vq->priv = vring;
> diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
> index 70ab496d0431..7611755d0ae2 100644
> --- a/drivers/remoteproc/remoteproc_virtio.c
> +++ b/drivers/remoteproc/remoteproc_virtio.c
> @@ -125,6 +125,8 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev,
>   		return ERR_PTR(-ENOMEM);
>   	}
>   
> +	vq->num_max = len;


I wonder if this is correct.

It looks to me len is counted in bytes:

/**
  * struct rproc_vring - remoteproc vring state
  * @va: virtual address
  * @len: length, in bytes
  * @da: device address
  * @align: vring alignment
  * @notifyid: rproc-specific unique vring index
  * @rvdev: remote vdev
  * @vq: the virtqueue of this vring
  */
struct rproc_vring {
         void *va;
         int len;
         u32 da;
         u32 align;
         int notifyid;
         struct rproc_vdev *rvdev;
         struct virtqueue *vq;
};


Other looks good.

Thanks


> +
>   	rvring->vq = vq;
>   	vq->priv = rvring;
>   
> diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
> index d35e7a3f7067..468da60b56c5 100644
> --- a/drivers/s390/virtio/virtio_ccw.c
> +++ b/drivers/s390/virtio/virtio_ccw.c
> @@ -529,6 +529,9 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,
>   		err = -ENOMEM;
>   		goto out_err;
>   	}
> +
> +	vq->num_max = info->num;
> +
>   	/* it may have been reduced */
>   	info->num = virtqueue_get_vring_size(vq);
>   
> diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> index 56128b9c46eb..a41abc8051b9 100644
> --- a/drivers/virtio/virtio_mmio.c
> +++ b/drivers/virtio/virtio_mmio.c
> @@ -390,6 +390,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
>   		goto error_new_virtqueue;
>   	}
>   
> +	vq->num_max = num;
> +
>   	/* Activate the queue */
>   	writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
>   	if (vm_dev->version == 1) {
> diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
> index 34141b9abe27..b68934fe6b5d 100644
> --- a/drivers/virtio/virtio_pci_legacy.c
> +++ b/drivers/virtio/virtio_pci_legacy.c
> @@ -135,6 +135,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   	if (!vq)
>   		return ERR_PTR(-ENOMEM);
>   
> +	vq->num_max = num;
> +
>   	q_pfn = virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT;
>   	if (q_pfn >> 32) {
>   		dev_err(&vp_dev->pci_dev->dev,
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 5455bc041fb6..86d301f272b8 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -218,6 +218,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   	if (!vq)
>   		return ERR_PTR(-ENOMEM);
>   
> +	vq->num_max = num;
> +
>   	/* activate the queue */
>   	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
>   	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 962f1477b1fa..b87130c8f312 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2371,6 +2371,20 @@ void vring_transport_features(struct virtio_device *vdev)
>   }
>   EXPORT_SYMBOL_GPL(vring_transport_features);
>   
> +/**
> + * virtqueue_get_vring_max_size - return the max size of the virtqueue's vring
> + * @_vq: the struct virtqueue containing the vring of interest.
> + *
> + * Returns the max size of the vring.
> + *
> + * Unlike other operations, this need not be serialized.
> + */
> +unsigned int virtqueue_get_vring_max_size(struct virtqueue *_vq)
> +{
> +	return _vq->num_max;
> +}
> +EXPORT_SYMBOL_GPL(virtqueue_get_vring_max_size);
> +
>   /**
>    * virtqueue_get_vring_size - return the size of the virtqueue's vring
>    * @_vq: the struct virtqueue containing the vring of interest.
> diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
> index 7767a7f0119b..39e4c08eb0f2 100644
> --- a/drivers/virtio/virtio_vdpa.c
> +++ b/drivers/virtio/virtio_vdpa.c
> @@ -183,6 +183,8 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
>   		goto error_new_virtqueue;
>   	}
>   
> +	vq->num_max = max_num;
> +
>   	/* Setup virtqueue callback */
>   	cb.callback = virtio_vdpa_virtqueue_cb;
>   	cb.private = info;
> diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> index 72292a62cd90..d59adc4be068 100644
> --- a/include/linux/virtio.h
> +++ b/include/linux/virtio.h
> @@ -31,6 +31,7 @@ struct virtqueue {
>   	struct virtio_device *vdev;
>   	unsigned int index;
>   	unsigned int num_free;
> +	unsigned int num_max;
>   	void *priv;
>   };
>   
> @@ -80,6 +81,7 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *vq);
>   
>   void *virtqueue_detach_unused_buf(struct virtqueue *vq);
>   
> +unsigned int virtqueue_get_vring_max_size(struct virtqueue *vq);
>   unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
>   
>   bool virtqueue_is_broken(struct virtqueue *vq);

Re: [PATCH v9 01/32] virtio: add helper virtqueue_get_vring_max_size()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Record the maximum queue num supported by the device.
>
> virtio-net can display the maximum (supported by hardware) ring size in
> ethtool -g eth0.
>
> When the subsequent patch implements vring reset, it can judge whether
> the ring size passed by the driver is legal based on this.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   arch/um/drivers/virtio_uml.c             |  1 +
>   drivers/platform/mellanox/mlxbf-tmfifo.c |  2 ++
>   drivers/remoteproc/remoteproc_virtio.c   |  2 ++
>   drivers/s390/virtio/virtio_ccw.c         |  3 +++
>   drivers/virtio/virtio_mmio.c             |  2 ++
>   drivers/virtio/virtio_pci_legacy.c       |  2 ++
>   drivers/virtio/virtio_pci_modern.c       |  2 ++
>   drivers/virtio/virtio_ring.c             | 14 ++++++++++++++
>   drivers/virtio/virtio_vdpa.c             |  2 ++
>   include/linux/virtio.h                   |  2 ++
>   10 files changed, 32 insertions(+)
>
> diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
> index ba562d68dc04..904993d15a85 100644
> --- a/arch/um/drivers/virtio_uml.c
> +++ b/arch/um/drivers/virtio_uml.c
> @@ -945,6 +945,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
>   		goto error_create;
>   	}
>   	vq->priv = info;
> +	vq->num_max = num;
>   	num = virtqueue_get_vring_size(vq);
>   
>   	if (vu_dev->protocol_features &
> diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
> index 38800e86ed8a..1ae3c56b66b0 100644
> --- a/drivers/platform/mellanox/mlxbf-tmfifo.c
> +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
> @@ -959,6 +959,8 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
>   			goto error;
>   		}
>   
> +		vq->num_max = vring->num;
> +
>   		vqs[i] = vq;
>   		vring->vq = vq;
>   		vq->priv = vring;
> diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
> index 70ab496d0431..7611755d0ae2 100644
> --- a/drivers/remoteproc/remoteproc_virtio.c
> +++ b/drivers/remoteproc/remoteproc_virtio.c
> @@ -125,6 +125,8 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev,
>   		return ERR_PTR(-ENOMEM);
>   	}
>   
> +	vq->num_max = len;


I wonder if this is correct.

It looks to me len is counted in bytes:

/**
  * struct rproc_vring - remoteproc vring state
  * @va: virtual address
  * @len: length, in bytes
  * @da: device address
  * @align: vring alignment
  * @notifyid: rproc-specific unique vring index
  * @rvdev: remote vdev
  * @vq: the virtqueue of this vring
  */
struct rproc_vring {
         void *va;
         int len;
         u32 da;
         u32 align;
         int notifyid;
         struct rproc_vdev *rvdev;
         struct virtqueue *vq;
};


Other looks good.

Thanks


> +
>   	rvring->vq = vq;
>   	vq->priv = rvring;
>   
> diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
> index d35e7a3f7067..468da60b56c5 100644
> --- a/drivers/s390/virtio/virtio_ccw.c
> +++ b/drivers/s390/virtio/virtio_ccw.c
> @@ -529,6 +529,9 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,
>   		err = -ENOMEM;
>   		goto out_err;
>   	}
> +
> +	vq->num_max = info->num;
> +
>   	/* it may have been reduced */
>   	info->num = virtqueue_get_vring_size(vq);
>   
> diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> index 56128b9c46eb..a41abc8051b9 100644
> --- a/drivers/virtio/virtio_mmio.c
> +++ b/drivers/virtio/virtio_mmio.c
> @@ -390,6 +390,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
>   		goto error_new_virtqueue;
>   	}
>   
> +	vq->num_max = num;
> +
>   	/* Activate the queue */
>   	writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
>   	if (vm_dev->version == 1) {
> diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
> index 34141b9abe27..b68934fe6b5d 100644
> --- a/drivers/virtio/virtio_pci_legacy.c
> +++ b/drivers/virtio/virtio_pci_legacy.c
> @@ -135,6 +135,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   	if (!vq)
>   		return ERR_PTR(-ENOMEM);
>   
> +	vq->num_max = num;
> +
>   	q_pfn = virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT;
>   	if (q_pfn >> 32) {
>   		dev_err(&vp_dev->pci_dev->dev,
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 5455bc041fb6..86d301f272b8 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -218,6 +218,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   	if (!vq)
>   		return ERR_PTR(-ENOMEM);
>   
> +	vq->num_max = num;
> +
>   	/* activate the queue */
>   	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
>   	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 962f1477b1fa..b87130c8f312 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2371,6 +2371,20 @@ void vring_transport_features(struct virtio_device *vdev)
>   }
>   EXPORT_SYMBOL_GPL(vring_transport_features);
>   
> +/**
> + * virtqueue_get_vring_max_size - return the max size of the virtqueue's vring
> + * @_vq: the struct virtqueue containing the vring of interest.
> + *
> + * Returns the max size of the vring.
> + *
> + * Unlike other operations, this need not be serialized.
> + */
> +unsigned int virtqueue_get_vring_max_size(struct virtqueue *_vq)
> +{
> +	return _vq->num_max;
> +}
> +EXPORT_SYMBOL_GPL(virtqueue_get_vring_max_size);
> +
>   /**
>    * virtqueue_get_vring_size - return the size of the virtqueue's vring
>    * @_vq: the struct virtqueue containing the vring of interest.
> diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
> index 7767a7f0119b..39e4c08eb0f2 100644
> --- a/drivers/virtio/virtio_vdpa.c
> +++ b/drivers/virtio/virtio_vdpa.c
> @@ -183,6 +183,8 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
>   		goto error_new_virtqueue;
>   	}
>   
> +	vq->num_max = max_num;
> +
>   	/* Setup virtqueue callback */
>   	cb.callback = virtio_vdpa_virtqueue_cb;
>   	cb.private = info;
> diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> index 72292a62cd90..d59adc4be068 100644
> --- a/include/linux/virtio.h
> +++ b/include/linux/virtio.h
> @@ -31,6 +31,7 @@ struct virtqueue {
>   	struct virtio_device *vdev;
>   	unsigned int index;
>   	unsigned int num_free;
> +	unsigned int num_max;
>   	void *priv;
>   };
>   
> @@ -80,6 +81,7 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *vq);
>   
>   void *virtqueue_detach_unused_buf(struct virtqueue *vq);
>   
> +unsigned int virtqueue_get_vring_max_size(struct virtqueue *vq);
>   unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
>   
>   bool virtqueue_is_broken(struct virtqueue *vq);

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 01/32] virtio: add helper virtqueue_get_vring_max_size()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Record the maximum queue num supported by the device.
>
> virtio-net can display the maximum (supported by hardware) ring size in
> ethtool -g eth0.
>
> When the subsequent patch implements vring reset, it can judge whether
> the ring size passed by the driver is legal based on this.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   arch/um/drivers/virtio_uml.c             |  1 +
>   drivers/platform/mellanox/mlxbf-tmfifo.c |  2 ++
>   drivers/remoteproc/remoteproc_virtio.c   |  2 ++
>   drivers/s390/virtio/virtio_ccw.c         |  3 +++
>   drivers/virtio/virtio_mmio.c             |  2 ++
>   drivers/virtio/virtio_pci_legacy.c       |  2 ++
>   drivers/virtio/virtio_pci_modern.c       |  2 ++
>   drivers/virtio/virtio_ring.c             | 14 ++++++++++++++
>   drivers/virtio/virtio_vdpa.c             |  2 ++
>   include/linux/virtio.h                   |  2 ++
>   10 files changed, 32 insertions(+)
>
> diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
> index ba562d68dc04..904993d15a85 100644
> --- a/arch/um/drivers/virtio_uml.c
> +++ b/arch/um/drivers/virtio_uml.c
> @@ -945,6 +945,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
>   		goto error_create;
>   	}
>   	vq->priv = info;
> +	vq->num_max = num;
>   	num = virtqueue_get_vring_size(vq);
>   
>   	if (vu_dev->protocol_features &
> diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
> index 38800e86ed8a..1ae3c56b66b0 100644
> --- a/drivers/platform/mellanox/mlxbf-tmfifo.c
> +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
> @@ -959,6 +959,8 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
>   			goto error;
>   		}
>   
> +		vq->num_max = vring->num;
> +
>   		vqs[i] = vq;
>   		vring->vq = vq;
>   		vq->priv = vring;
> diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
> index 70ab496d0431..7611755d0ae2 100644
> --- a/drivers/remoteproc/remoteproc_virtio.c
> +++ b/drivers/remoteproc/remoteproc_virtio.c
> @@ -125,6 +125,8 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev,
>   		return ERR_PTR(-ENOMEM);
>   	}
>   
> +	vq->num_max = len;


I wonder if this is correct.

It looks to me len is counted in bytes:

/**
  * struct rproc_vring - remoteproc vring state
  * @va: virtual address
  * @len: length, in bytes
  * @da: device address
  * @align: vring alignment
  * @notifyid: rproc-specific unique vring index
  * @rvdev: remote vdev
  * @vq: the virtqueue of this vring
  */
struct rproc_vring {
         void *va;
         int len;
         u32 da;
         u32 align;
         int notifyid;
         struct rproc_vdev *rvdev;
         struct virtqueue *vq;
};


Other looks good.

Thanks


> +
>   	rvring->vq = vq;
>   	vq->priv = rvring;
>   
> diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
> index d35e7a3f7067..468da60b56c5 100644
> --- a/drivers/s390/virtio/virtio_ccw.c
> +++ b/drivers/s390/virtio/virtio_ccw.c
> @@ -529,6 +529,9 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,
>   		err = -ENOMEM;
>   		goto out_err;
>   	}
> +
> +	vq->num_max = info->num;
> +
>   	/* it may have been reduced */
>   	info->num = virtqueue_get_vring_size(vq);
>   
> diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> index 56128b9c46eb..a41abc8051b9 100644
> --- a/drivers/virtio/virtio_mmio.c
> +++ b/drivers/virtio/virtio_mmio.c
> @@ -390,6 +390,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
>   		goto error_new_virtqueue;
>   	}
>   
> +	vq->num_max = num;
> +
>   	/* Activate the queue */
>   	writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
>   	if (vm_dev->version == 1) {
> diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
> index 34141b9abe27..b68934fe6b5d 100644
> --- a/drivers/virtio/virtio_pci_legacy.c
> +++ b/drivers/virtio/virtio_pci_legacy.c
> @@ -135,6 +135,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   	if (!vq)
>   		return ERR_PTR(-ENOMEM);
>   
> +	vq->num_max = num;
> +
>   	q_pfn = virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT;
>   	if (q_pfn >> 32) {
>   		dev_err(&vp_dev->pci_dev->dev,
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 5455bc041fb6..86d301f272b8 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -218,6 +218,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   	if (!vq)
>   		return ERR_PTR(-ENOMEM);
>   
> +	vq->num_max = num;
> +
>   	/* activate the queue */
>   	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
>   	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 962f1477b1fa..b87130c8f312 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2371,6 +2371,20 @@ void vring_transport_features(struct virtio_device *vdev)
>   }
>   EXPORT_SYMBOL_GPL(vring_transport_features);
>   
> +/**
> + * virtqueue_get_vring_max_size - return the max size of the virtqueue's vring
> + * @_vq: the struct virtqueue containing the vring of interest.
> + *
> + * Returns the max size of the vring.
> + *
> + * Unlike other operations, this need not be serialized.
> + */
> +unsigned int virtqueue_get_vring_max_size(struct virtqueue *_vq)
> +{
> +	return _vq->num_max;
> +}
> +EXPORT_SYMBOL_GPL(virtqueue_get_vring_max_size);
> +
>   /**
>    * virtqueue_get_vring_size - return the size of the virtqueue's vring
>    * @_vq: the struct virtqueue containing the vring of interest.
> diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
> index 7767a7f0119b..39e4c08eb0f2 100644
> --- a/drivers/virtio/virtio_vdpa.c
> +++ b/drivers/virtio/virtio_vdpa.c
> @@ -183,6 +183,8 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
>   		goto error_new_virtqueue;
>   	}
>   
> +	vq->num_max = max_num;
> +
>   	/* Setup virtqueue callback */
>   	cb.callback = virtio_vdpa_virtqueue_cb;
>   	cb.private = info;
> diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> index 72292a62cd90..d59adc4be068 100644
> --- a/include/linux/virtio.h
> +++ b/include/linux/virtio.h
> @@ -31,6 +31,7 @@ struct virtqueue {
>   	struct virtio_device *vdev;
>   	unsigned int index;
>   	unsigned int num_free;
> +	unsigned int num_max;
>   	void *priv;
>   };
>   
> @@ -80,6 +81,7 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *vq);
>   
>   void *virtqueue_detach_unused_buf(struct virtqueue *vq);
>   
> +unsigned int virtqueue_get_vring_max_size(struct virtqueue *vq);
>   unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
>   
>   bool virtqueue_is_broken(struct virtqueue *vq);


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 02/32] virtio: struct virtio_config_ops add callbacks for queue_reset

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Performing reset on a queue is divided into four steps:
>
>   1. transport: notify the device to reset the queue
>   2. vring:     recycle the buffer submitted
>   3. vring:     reset/resize the vring (may re-alloc)
>   4. transport: mmap vring to device, and enable the queue


Nit: it looks to me we'd better say it's an example (since step 3 or 
even 2 is not a must).


>
> In order to support queue reset, add two callbacks(reset_vq,
> enable_reset_vq) in struct virtio_config_ops to implement steps 1 and 4.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   include/linux/virtio_config.h | 12 ++++++++++++
>   1 file changed, 12 insertions(+)
>
> diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> index 4d107ad31149..d4adcd0e1c57 100644
> --- a/include/linux/virtio_config.h
> +++ b/include/linux/virtio_config.h
> @@ -74,6 +74,16 @@ struct virtio_shm_region {
>    * @set_vq_affinity: set the affinity for a virtqueue (optional).
>    * @get_vq_affinity: get the affinity for a virtqueue (optional).
>    * @get_shm_region: get a shared memory region based on the index.
> + * @reset_vq: reset a queue individually (optional).
> + *	vq: the virtqueue
> + *	Returns 0 on success or error status
> + *	reset_vq will guarantee that the callbacks are disabled and synchronized.
> + *	Except for the callback, the caller should guarantee that the vring is


I wonder what's the implications for virtio hardening[1]. In that 
series, we agree to have a synchronize_vqs() config ops to make sure 
callbacks are synchronized.

It uses a global flag and a device wise synchronization mechanism. It 
looks to me we need to switch to

1) per virtqueue flag
2) per virtqueue synchronization

Thanks


> + *	not accessed by any functions of virtqueue.
> + * @enable_reset_vq: enable a reset queue
> + *	vq: the virtqueue
> + *	Returns 0 on success or error status
> + *	If reset_vq is set, then enable_reset_vq must also be set.
>    */
>   typedef void vq_callback_t(struct virtqueue *);
>   struct virtio_config_ops {
> @@ -100,6 +110,8 @@ struct virtio_config_ops {
>   			int index);
>   	bool (*get_shm_region)(struct virtio_device *vdev,
>   			       struct virtio_shm_region *region, u8 id);
> +	int (*reset_vq)(struct virtqueue *vq);
> +	int (*enable_reset_vq)(struct virtqueue *vq);
>   };
>   
>   /* If driver didn't advertise the feature, it will never appear. */

Re: [PATCH v9 02/32] virtio: struct virtio_config_ops add callbacks for queue_reset

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Performing reset on a queue is divided into four steps:
>
>   1. transport: notify the device to reset the queue
>   2. vring:     recycle the buffer submitted
>   3. vring:     reset/resize the vring (may re-alloc)
>   4. transport: mmap vring to device, and enable the queue


Nit: it looks to me we'd better say it's an example (since step 3 or 
even 2 is not a must).


>
> In order to support queue reset, add two callbacks(reset_vq,
> enable_reset_vq) in struct virtio_config_ops to implement steps 1 and 4.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   include/linux/virtio_config.h | 12 ++++++++++++
>   1 file changed, 12 insertions(+)
>
> diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> index 4d107ad31149..d4adcd0e1c57 100644
> --- a/include/linux/virtio_config.h
> +++ b/include/linux/virtio_config.h
> @@ -74,6 +74,16 @@ struct virtio_shm_region {
>    * @set_vq_affinity: set the affinity for a virtqueue (optional).
>    * @get_vq_affinity: get the affinity for a virtqueue (optional).
>    * @get_shm_region: get a shared memory region based on the index.
> + * @reset_vq: reset a queue individually (optional).
> + *	vq: the virtqueue
> + *	Returns 0 on success or error status
> + *	reset_vq will guarantee that the callbacks are disabled and synchronized.
> + *	Except for the callback, the caller should guarantee that the vring is


I wonder what's the implications for virtio hardening[1]. In that 
series, we agree to have a synchronize_vqs() config ops to make sure 
callbacks are synchronized.

It uses a global flag and a device wise synchronization mechanism. It 
looks to me we need to switch to

1) per virtqueue flag
2) per virtqueue synchronization

Thanks


> + *	not accessed by any functions of virtqueue.
> + * @enable_reset_vq: enable a reset queue
> + *	vq: the virtqueue
> + *	Returns 0 on success or error status
> + *	If reset_vq is set, then enable_reset_vq must also be set.
>    */
>   typedef void vq_callback_t(struct virtqueue *);
>   struct virtio_config_ops {
> @@ -100,6 +110,8 @@ struct virtio_config_ops {
>   			int index);
>   	bool (*get_shm_region)(struct virtio_device *vdev,
>   			       struct virtio_shm_region *region, u8 id);
> +	int (*reset_vq)(struct virtqueue *vq);
> +	int (*enable_reset_vq)(struct virtqueue *vq);
>   };
>   
>   /* If driver didn't advertise the feature, it will never appear. */

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 02/32] virtio: struct virtio_config_ops add callbacks for queue_reset

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Performing reset on a queue is divided into four steps:
>
>   1. transport: notify the device to reset the queue
>   2. vring:     recycle the buffer submitted
>   3. vring:     reset/resize the vring (may re-alloc)
>   4. transport: mmap vring to device, and enable the queue


Nit: it looks to me we'd better say it's an example (since step 3 or 
even 2 is not a must).


>
> In order to support queue reset, add two callbacks(reset_vq,
> enable_reset_vq) in struct virtio_config_ops to implement steps 1 and 4.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   include/linux/virtio_config.h | 12 ++++++++++++
>   1 file changed, 12 insertions(+)
>
> diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> index 4d107ad31149..d4adcd0e1c57 100644
> --- a/include/linux/virtio_config.h
> +++ b/include/linux/virtio_config.h
> @@ -74,6 +74,16 @@ struct virtio_shm_region {
>    * @set_vq_affinity: set the affinity for a virtqueue (optional).
>    * @get_vq_affinity: get the affinity for a virtqueue (optional).
>    * @get_shm_region: get a shared memory region based on the index.
> + * @reset_vq: reset a queue individually (optional).
> + *	vq: the virtqueue
> + *	Returns 0 on success or error status
> + *	reset_vq will guarantee that the callbacks are disabled and synchronized.
> + *	Except for the callback, the caller should guarantee that the vring is


I wonder what's the implications for virtio hardening[1]. In that 
series, we agree to have a synchronize_vqs() config ops to make sure 
callbacks are synchronized.

It uses a global flag and a device wise synchronization mechanism. It 
looks to me we need to switch to

1) per virtqueue flag
2) per virtqueue synchronization

Thanks


> + *	not accessed by any functions of virtqueue.
> + * @enable_reset_vq: enable a reset queue
> + *	vq: the virtqueue
> + *	Returns 0 on success or error status
> + *	If reset_vq is set, then enable_reset_vq must also be set.
>    */
>   typedef void vq_callback_t(struct virtqueue *);
>   struct virtio_config_ops {
> @@ -100,6 +110,8 @@ struct virtio_config_ops {
>   			int index);
>   	bool (*get_shm_region)(struct virtio_device *vdev,
>   			       struct virtio_shm_region *region, u8 id);
> +	int (*reset_vq)(struct virtqueue *vq);
> +	int (*enable_reset_vq)(struct virtqueue *vq);
>   };
>   
>   /* If driver didn't advertise the feature, it will never appear. */


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 02/32] virtio: struct virtio_config_ops add callbacks for queue_reset

[Jason Wang]

On Tue, Apr 12, 2022 at 10:47 AM Jason Wang <jasowang@redhat.com> wrote:
>
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Performing reset on a queue is divided into four steps:
> >
> >   1. transport: notify the device to reset the queue
> >   2. vring:     recycle the buffer submitted
> >   3. vring:     reset/resize the vring (may re-alloc)
> >   4. transport: mmap vring to device, and enable the queue
>
>
> Nit: it looks to me we'd better say it's an example (since step 3 or
> even 2 is not a must).
>
>
> >
> > In order to support queue reset, add two callbacks(reset_vq,
> > enable_reset_vq) in struct virtio_config_ops to implement steps 1 and 4.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   include/linux/virtio_config.h | 12 ++++++++++++
> >   1 file changed, 12 insertions(+)
> >
> > diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> > index 4d107ad31149..d4adcd0e1c57 100644
> > --- a/include/linux/virtio_config.h
> > +++ b/include/linux/virtio_config.h
> > @@ -74,6 +74,16 @@ struct virtio_shm_region {
> >    * @set_vq_affinity: set the affinity for a virtqueue (optional).
> >    * @get_vq_affinity: get the affinity for a virtqueue (optional).
> >    * @get_shm_region: get a shared memory region based on the index.
> > + * @reset_vq: reset a queue individually (optional).
> > + *   vq: the virtqueue
> > + *   Returns 0 on success or error status
> > + *   reset_vq will guarantee that the callbacks are disabled and synchronized.
> > + *   Except for the callback, the caller should guarantee that the vring is
>
>
> I wonder what's the implications for virtio hardening[1]. In that

For [1] I meant
https://lore.kernel.org/lkml/20220406083538.16274-1-jasowang@redhat.com/

Thanks


> series, we agree to have a synchronize_vqs() config ops to make sure
> callbacks are synchronized.
>
> It uses a global flag and a device wise synchronization mechanism. It
> looks to me we need to switch to
>
> 1) per virtqueue flag
> 2) per virtqueue synchronization
>
> Thanks
>
>
> > + *   not accessed by any functions of virtqueue.
> > + * @enable_reset_vq: enable a reset queue
> > + *   vq: the virtqueue
> > + *   Returns 0 on success or error status
> > + *   If reset_vq is set, then enable_reset_vq must also be set.
> >    */
> >   typedef void vq_callback_t(struct virtqueue *);
> >   struct virtio_config_ops {
> > @@ -100,6 +110,8 @@ struct virtio_config_ops {
> >                       int index);
> >       bool (*get_shm_region)(struct virtio_device *vdev,
> >                              struct virtio_shm_region *region, u8 id);
> > +     int (*reset_vq)(struct virtqueue *vq);
> > +     int (*enable_reset_vq)(struct virtqueue *vq);
> >   };
> >
> >   /* If driver didn't advertise the feature, it will never appear. */

Re: [PATCH v9 02/32] virtio: struct virtio_config_ops add callbacks for queue_reset

[Jason Wang]

On Tue, Apr 12, 2022 at 10:47 AM Jason Wang <jasowang@redhat.com> wrote:
>
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Performing reset on a queue is divided into four steps:
> >
> >   1. transport: notify the device to reset the queue
> >   2. vring:     recycle the buffer submitted
> >   3. vring:     reset/resize the vring (may re-alloc)
> >   4. transport: mmap vring to device, and enable the queue
>
>
> Nit: it looks to me we'd better say it's an example (since step 3 or
> even 2 is not a must).
>
>
> >
> > In order to support queue reset, add two callbacks(reset_vq,
> > enable_reset_vq) in struct virtio_config_ops to implement steps 1 and 4.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   include/linux/virtio_config.h | 12 ++++++++++++
> >   1 file changed, 12 insertions(+)
> >
> > diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> > index 4d107ad31149..d4adcd0e1c57 100644
> > --- a/include/linux/virtio_config.h
> > +++ b/include/linux/virtio_config.h
> > @@ -74,6 +74,16 @@ struct virtio_shm_region {
> >    * @set_vq_affinity: set the affinity for a virtqueue (optional).
> >    * @get_vq_affinity: get the affinity for a virtqueue (optional).
> >    * @get_shm_region: get a shared memory region based on the index.
> > + * @reset_vq: reset a queue individually (optional).
> > + *   vq: the virtqueue
> > + *   Returns 0 on success or error status
> > + *   reset_vq will guarantee that the callbacks are disabled and synchronized.
> > + *   Except for the callback, the caller should guarantee that the vring is
>
>
> I wonder what's the implications for virtio hardening[1]. In that

For [1] I meant
https://lore.kernel.org/lkml/20220406083538.16274-1-jasowang@redhat.com/

Thanks


> series, we agree to have a synchronize_vqs() config ops to make sure
> callbacks are synchronized.
>
> It uses a global flag and a device wise synchronization mechanism. It
> looks to me we need to switch to
>
> 1) per virtqueue flag
> 2) per virtqueue synchronization
>
> Thanks
>
>
> > + *   not accessed by any functions of virtqueue.
> > + * @enable_reset_vq: enable a reset queue
> > + *   vq: the virtqueue
> > + *   Returns 0 on success or error status
> > + *   If reset_vq is set, then enable_reset_vq must also be set.
> >    */
> >   typedef void vq_callback_t(struct virtqueue *);
> >   struct virtio_config_ops {
> > @@ -100,6 +110,8 @@ struct virtio_config_ops {
> >                       int index);
> >       bool (*get_shm_region)(struct virtio_device *vdev,
> >                              struct virtio_shm_region *region, u8 id);
> > +     int (*reset_vq)(struct virtqueue *vq);
> > +     int (*enable_reset_vq)(struct virtqueue *vq);
> >   };
> >
> >   /* If driver didn't advertise the feature, it will never appear. */

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 02/32] virtio: struct virtio_config_ops add callbacks for queue_reset

[Jason Wang]

On Tue, Apr 12, 2022 at 10:47 AM Jason Wang <jasowang@redhat.com> wrote:
>
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Performing reset on a queue is divided into four steps:
> >
> >   1. transport: notify the device to reset the queue
> >   2. vring:     recycle the buffer submitted
> >   3. vring:     reset/resize the vring (may re-alloc)
> >   4. transport: mmap vring to device, and enable the queue
>
>
> Nit: it looks to me we'd better say it's an example (since step 3 or
> even 2 is not a must).
>
>
> >
> > In order to support queue reset, add two callbacks(reset_vq,
> > enable_reset_vq) in struct virtio_config_ops to implement steps 1 and 4.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   include/linux/virtio_config.h | 12 ++++++++++++
> >   1 file changed, 12 insertions(+)
> >
> > diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> > index 4d107ad31149..d4adcd0e1c57 100644
> > --- a/include/linux/virtio_config.h
> > +++ b/include/linux/virtio_config.h
> > @@ -74,6 +74,16 @@ struct virtio_shm_region {
> >    * @set_vq_affinity: set the affinity for a virtqueue (optional).
> >    * @get_vq_affinity: get the affinity for a virtqueue (optional).
> >    * @get_shm_region: get a shared memory region based on the index.
> > + * @reset_vq: reset a queue individually (optional).
> > + *   vq: the virtqueue
> > + *   Returns 0 on success or error status
> > + *   reset_vq will guarantee that the callbacks are disabled and synchronized.
> > + *   Except for the callback, the caller should guarantee that the vring is
>
>
> I wonder what's the implications for virtio hardening[1]. In that

For [1] I meant
https://lore.kernel.org/lkml/20220406083538.16274-1-jasowang@redhat.com/

Thanks


> series, we agree to have a synchronize_vqs() config ops to make sure
> callbacks are synchronized.
>
> It uses a global flag and a device wise synchronization mechanism. It
> looks to me we need to switch to
>
> 1) per virtqueue flag
> 2) per virtqueue synchronization
>
> Thanks
>
>
> > + *   not accessed by any functions of virtqueue.
> > + * @enable_reset_vq: enable a reset queue
> > + *   vq: the virtqueue
> > + *   Returns 0 on success or error status
> > + *   If reset_vq is set, then enable_reset_vq must also be set.
> >    */
> >   typedef void vq_callback_t(struct virtqueue *);
> >   struct virtio_config_ops {
> > @@ -100,6 +110,8 @@ struct virtio_config_ops {
> >                       int index);
> >       bool (*get_shm_region)(struct virtio_device *vdev,
> >                              struct virtio_shm_region *region, u8 id);
> > +     int (*reset_vq)(struct virtqueue *vq);
> > +     int (*enable_reset_vq)(struct virtqueue *vq);
> >   };
> >
> >   /* If driver didn't advertise the feature, it will never appear. */


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 03/32] virtio_ring: update the document of the virtqueue_detach_unused_buf for queue reset

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Added documentation for virtqueue_detach_unused_buf, allowing it to be
> called on queue reset.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_ring.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index b87130c8f312..f1807f6b06a5 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2127,8 +2127,8 @@ EXPORT_SYMBOL_GPL(virtqueue_enable_cb_delayed);
>    * @_vq: the struct virtqueue we're talking about.
>    *
>    * Returns NULL or the "data" token handed to virtqueue_add_*().
> - * This is not valid on an active queue; it is useful only for device
> - * shutdown.
> + * This is not valid on an active queue; it is useful for device
> + * shutdown or the reset queue.
>    */
>   void *virtqueue_detach_unused_buf(struct virtqueue *_vq)
>   {

Re: [PATCH v9 03/32] virtio_ring: update the document of the virtqueue_detach_unused_buf for queue reset

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Added documentation for virtqueue_detach_unused_buf, allowing it to be
> called on queue reset.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_ring.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index b87130c8f312..f1807f6b06a5 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2127,8 +2127,8 @@ EXPORT_SYMBOL_GPL(virtqueue_enable_cb_delayed);
>    * @_vq: the struct virtqueue we're talking about.
>    *
>    * Returns NULL or the "data" token handed to virtqueue_add_*().
> - * This is not valid on an active queue; it is useful only for device
> - * shutdown.
> + * This is not valid on an active queue; it is useful for device
> + * shutdown or the reset queue.
>    */
>   void *virtqueue_detach_unused_buf(struct virtqueue *_vq)
>   {

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 03/32] virtio_ring: update the document of the virtqueue_detach_unused_buf for queue reset

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Added documentation for virtqueue_detach_unused_buf, allowing it to be
> called on queue reset.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_ring.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index b87130c8f312..f1807f6b06a5 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2127,8 +2127,8 @@ EXPORT_SYMBOL_GPL(virtqueue_enable_cb_delayed);
>    * @_vq: the struct virtqueue we're talking about.
>    *
>    * Returns NULL or the "data" token handed to virtqueue_add_*().
> - * This is not valid on an active queue; it is useful only for device
> - * shutdown.
> + * This is not valid on an active queue; it is useful for device
> + * shutdown or the reset queue.
>    */
>   void *virtqueue_detach_unused_buf(struct virtqueue *_vq)
>   {


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 04/32] virtio_ring: remove the arg vq of vring_alloc_desc_extra()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> The parameter vq of vring_alloc_desc_extra() is useless. This patch
> removes this parameter.
>
> Subsequent patches will call this function to avoid passing useless
> arguments.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_ring.c | 7 +++----
>   1 file changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index f1807f6b06a5..cb6010750a94 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -1636,8 +1636,7 @@ static void *virtqueue_detach_unused_buf_packed(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> -static struct vring_desc_extra *vring_alloc_desc_extra(struct vring_virtqueue *vq,
> -						       unsigned int num)
> +static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
>   {
>   	struct vring_desc_extra *desc_extra;
>   	unsigned int i;
> @@ -1755,7 +1754,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
>   	/* Put everything in free lists. */
>   	vq->free_head = 0;
>   
> -	vq->packed.desc_extra = vring_alloc_desc_extra(vq, num);
> +	vq->packed.desc_extra = vring_alloc_desc_extra(num);
>   	if (!vq->packed.desc_extra)
>   		goto err_desc_extra;
>   
> @@ -2233,7 +2232,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	if (!vq->split.desc_state)
>   		goto err_state;
>   
> -	vq->split.desc_extra = vring_alloc_desc_extra(vq, vring.num);
> +	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
>   	if (!vq->split.desc_extra)
>   		goto err_extra;
>   

Re: [PATCH v9 04/32] virtio_ring: remove the arg vq of vring_alloc_desc_extra()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> The parameter vq of vring_alloc_desc_extra() is useless. This patch
> removes this parameter.
>
> Subsequent patches will call this function to avoid passing useless
> arguments.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_ring.c | 7 +++----
>   1 file changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index f1807f6b06a5..cb6010750a94 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -1636,8 +1636,7 @@ static void *virtqueue_detach_unused_buf_packed(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> -static struct vring_desc_extra *vring_alloc_desc_extra(struct vring_virtqueue *vq,
> -						       unsigned int num)
> +static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
>   {
>   	struct vring_desc_extra *desc_extra;
>   	unsigned int i;
> @@ -1755,7 +1754,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
>   	/* Put everything in free lists. */
>   	vq->free_head = 0;
>   
> -	vq->packed.desc_extra = vring_alloc_desc_extra(vq, num);
> +	vq->packed.desc_extra = vring_alloc_desc_extra(num);
>   	if (!vq->packed.desc_extra)
>   		goto err_desc_extra;
>   
> @@ -2233,7 +2232,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	if (!vq->split.desc_state)
>   		goto err_state;
>   
> -	vq->split.desc_extra = vring_alloc_desc_extra(vq, vring.num);
> +	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
>   	if (!vq->split.desc_extra)
>   		goto err_extra;
>   

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 04/32] virtio_ring: remove the arg vq of vring_alloc_desc_extra()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> The parameter vq of vring_alloc_desc_extra() is useless. This patch
> removes this parameter.
>
> Subsequent patches will call this function to avoid passing useless
> arguments.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_ring.c | 7 +++----
>   1 file changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index f1807f6b06a5..cb6010750a94 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -1636,8 +1636,7 @@ static void *virtqueue_detach_unused_buf_packed(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> -static struct vring_desc_extra *vring_alloc_desc_extra(struct vring_virtqueue *vq,
> -						       unsigned int num)
> +static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
>   {
>   	struct vring_desc_extra *desc_extra;
>   	unsigned int i;
> @@ -1755,7 +1754,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
>   	/* Put everything in free lists. */
>   	vq->free_head = 0;
>   
> -	vq->packed.desc_extra = vring_alloc_desc_extra(vq, num);
> +	vq->packed.desc_extra = vring_alloc_desc_extra(num);
>   	if (!vq->packed.desc_extra)
>   		goto err_desc_extra;
>   
> @@ -2233,7 +2232,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	if (!vq->split.desc_state)
>   		goto err_state;
>   
> -	vq->split.desc_extra = vring_alloc_desc_extra(vq, vring.num);
> +	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
>   	if (!vq->split.desc_extra)
>   		goto err_extra;
>   


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 05/32] virtio_ring: extract the logic of freeing vring

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce vring_free() to free the vring of vq.
>
> Subsequent patches will use vring_free() alone.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_ring.c | 18 +++++++++++++-----
>   1 file changed, 13 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index cb6010750a94..33fddfb907a6 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2301,14 +2301,10 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
>   }
>   EXPORT_SYMBOL_GPL(vring_new_virtqueue);
>   
> -void vring_del_virtqueue(struct virtqueue *_vq)
> +static void vring_free(struct virtqueue *_vq)
>   {
>   	struct vring_virtqueue *vq = to_vvq(_vq);
>   
> -	spin_lock(&vq->vq.vdev->vqs_list_lock);
> -	list_del(&_vq->list);
> -	spin_unlock(&vq->vq.vdev->vqs_list_lock);
> -
>   	if (vq->we_own_ring) {
>   		if (vq->packed_ring) {
>   			vring_free_queue(vq->vq.vdev,
> @@ -2339,6 +2335,18 @@ void vring_del_virtqueue(struct virtqueue *_vq)
>   		kfree(vq->split.desc_state);
>   		kfree(vq->split.desc_extra);
>   	}
> +}
> +
> +void vring_del_virtqueue(struct virtqueue *_vq)
> +{
> +	struct vring_virtqueue *vq = to_vvq(_vq);
> +
> +	spin_lock(&vq->vq.vdev->vqs_list_lock);
> +	list_del(&_vq->list);
> +	spin_unlock(&vq->vq.vdev->vqs_list_lock);
> +
> +	vring_free(_vq);
> +
>   	kfree(vq);
>   }
>   EXPORT_SYMBOL_GPL(vring_del_virtqueue);

Re: [PATCH v9 05/32] virtio_ring: extract the logic of freeing vring

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce vring_free() to free the vring of vq.
>
> Subsequent patches will use vring_free() alone.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_ring.c | 18 +++++++++++++-----
>   1 file changed, 13 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index cb6010750a94..33fddfb907a6 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2301,14 +2301,10 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
>   }
>   EXPORT_SYMBOL_GPL(vring_new_virtqueue);
>   
> -void vring_del_virtqueue(struct virtqueue *_vq)
> +static void vring_free(struct virtqueue *_vq)
>   {
>   	struct vring_virtqueue *vq = to_vvq(_vq);
>   
> -	spin_lock(&vq->vq.vdev->vqs_list_lock);
> -	list_del(&_vq->list);
> -	spin_unlock(&vq->vq.vdev->vqs_list_lock);
> -
>   	if (vq->we_own_ring) {
>   		if (vq->packed_ring) {
>   			vring_free_queue(vq->vq.vdev,
> @@ -2339,6 +2335,18 @@ void vring_del_virtqueue(struct virtqueue *_vq)
>   		kfree(vq->split.desc_state);
>   		kfree(vq->split.desc_extra);
>   	}
> +}
> +
> +void vring_del_virtqueue(struct virtqueue *_vq)
> +{
> +	struct vring_virtqueue *vq = to_vvq(_vq);
> +
> +	spin_lock(&vq->vq.vdev->vqs_list_lock);
> +	list_del(&_vq->list);
> +	spin_unlock(&vq->vq.vdev->vqs_list_lock);
> +
> +	vring_free(_vq);
> +
>   	kfree(vq);
>   }
>   EXPORT_SYMBOL_GPL(vring_del_virtqueue);

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 05/32] virtio_ring: extract the logic of freeing vring

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce vring_free() to free the vring of vq.
>
> Subsequent patches will use vring_free() alone.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_ring.c | 18 +++++++++++++-----
>   1 file changed, 13 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index cb6010750a94..33fddfb907a6 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2301,14 +2301,10 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
>   }
>   EXPORT_SYMBOL_GPL(vring_new_virtqueue);
>   
> -void vring_del_virtqueue(struct virtqueue *_vq)
> +static void vring_free(struct virtqueue *_vq)
>   {
>   	struct vring_virtqueue *vq = to_vvq(_vq);
>   
> -	spin_lock(&vq->vq.vdev->vqs_list_lock);
> -	list_del(&_vq->list);
> -	spin_unlock(&vq->vq.vdev->vqs_list_lock);
> -
>   	if (vq->we_own_ring) {
>   		if (vq->packed_ring) {
>   			vring_free_queue(vq->vq.vdev,
> @@ -2339,6 +2335,18 @@ void vring_del_virtqueue(struct virtqueue *_vq)
>   		kfree(vq->split.desc_state);
>   		kfree(vq->split.desc_extra);
>   	}
> +}
> +
> +void vring_del_virtqueue(struct virtqueue *_vq)
> +{
> +	struct vring_virtqueue *vq = to_vvq(_vq);
> +
> +	spin_lock(&vq->vq.vdev->vqs_list_lock);
> +	list_del(&_vq->list);
> +	spin_unlock(&vq->vq.vdev->vqs_list_lock);
> +
> +	vring_free(_vq);
> +
>   	kfree(vq);
>   }
>   EXPORT_SYMBOL_GPL(vring_del_virtqueue);


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 06/32] virtio_ring: split: extract the logic of alloc queue

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of split to create vring queue.
>
> This feature is required for subsequent virtuqueue reset vring.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++------------
>   1 file changed, 36 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 33fddfb907a6..72d5ae063fa0 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -915,23 +915,15 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> -static struct virtqueue *vring_create_virtqueue_split(
> -	unsigned int index,
> -	unsigned int num,
> -	unsigned int vring_align,
> -	struct virtio_device *vdev,
> -	bool weak_barriers,
> -	bool may_reduce_num,
> -	bool context,
> -	bool (*notify)(struct virtqueue *),
> -	void (*callback)(struct virtqueue *),
> -	const char *name)
> +static void *vring_alloc_queue_split(struct virtio_device *vdev,
> +				     dma_addr_t *dma_addr,
> +				     u32 *n,
> +				     unsigned int vring_align,
> +				     bool weak_barriers,


This is not used in this function.

Thanks


> +				     bool may_reduce_num)
>   {
> -	struct virtqueue *vq;
>   	void *queue = NULL;
> -	dma_addr_t dma_addr;
> -	size_t queue_size_in_bytes;
> -	struct vring vring;
> +	u32 num = *n;
>   
>   	/* We assume num is a power of 2. */
>   	if (num & (num - 1)) {
> @@ -942,7 +934,7 @@ static struct virtqueue *vring_create_virtqueue_split(
>   	/* TODO: allocate each queue chunk individually */
>   	for (; num && vring_size(num, vring_align) > PAGE_SIZE; num /= 2) {
>   		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
> -					  &dma_addr,
> +					  dma_addr,
>   					  GFP_KERNEL|__GFP_NOWARN|__GFP_ZERO);
>   		if (queue)
>   			break;
> @@ -956,11 +948,38 @@ static struct virtqueue *vring_create_virtqueue_split(
>   	if (!queue) {
>   		/* Try to get a single page. You are my only hope! */
>   		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
> -					  &dma_addr, GFP_KERNEL|__GFP_ZERO);
> +					  dma_addr, GFP_KERNEL|__GFP_ZERO);
>   	}
>   	if (!queue)
>   		return NULL;
>   
> +	*n = num;
> +	return queue;
> +}
> +
> +static struct virtqueue *vring_create_virtqueue_split(
> +	unsigned int index,
> +	unsigned int num,
> +	unsigned int vring_align,
> +	struct virtio_device *vdev,
> +	bool weak_barriers,
> +	bool may_reduce_num,
> +	bool context,
> +	bool (*notify)(struct virtqueue *),
> +	void (*callback)(struct virtqueue *),
> +	const char *name)
> +{
> +	size_t queue_size_in_bytes;
> +	struct virtqueue *vq;
> +	dma_addr_t dma_addr;
> +	struct vring vring;
> +	void *queue;
> +
> +	queue = vring_alloc_queue_split(vdev, &dma_addr, &num, vring_align,
> +					weak_barriers, may_reduce_num);
> +	if (!queue)
> +		return NULL;
> +
>   	queue_size_in_bytes = vring_size(num, vring_align);
>   	vring_init(&vring, num, queue, vring_align);
>   

Re: [PATCH v9 06/32] virtio_ring: split: extract the logic of alloc queue

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of split to create vring queue.
>
> This feature is required for subsequent virtuqueue reset vring.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++------------
>   1 file changed, 36 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 33fddfb907a6..72d5ae063fa0 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -915,23 +915,15 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> -static struct virtqueue *vring_create_virtqueue_split(
> -	unsigned int index,
> -	unsigned int num,
> -	unsigned int vring_align,
> -	struct virtio_device *vdev,
> -	bool weak_barriers,
> -	bool may_reduce_num,
> -	bool context,
> -	bool (*notify)(struct virtqueue *),
> -	void (*callback)(struct virtqueue *),
> -	const char *name)
> +static void *vring_alloc_queue_split(struct virtio_device *vdev,
> +				     dma_addr_t *dma_addr,
> +				     u32 *n,
> +				     unsigned int vring_align,
> +				     bool weak_barriers,


This is not used in this function.

Thanks


> +				     bool may_reduce_num)
>   {
> -	struct virtqueue *vq;
>   	void *queue = NULL;
> -	dma_addr_t dma_addr;
> -	size_t queue_size_in_bytes;
> -	struct vring vring;
> +	u32 num = *n;
>   
>   	/* We assume num is a power of 2. */
>   	if (num & (num - 1)) {
> @@ -942,7 +934,7 @@ static struct virtqueue *vring_create_virtqueue_split(
>   	/* TODO: allocate each queue chunk individually */
>   	for (; num && vring_size(num, vring_align) > PAGE_SIZE; num /= 2) {
>   		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
> -					  &dma_addr,
> +					  dma_addr,
>   					  GFP_KERNEL|__GFP_NOWARN|__GFP_ZERO);
>   		if (queue)
>   			break;
> @@ -956,11 +948,38 @@ static struct virtqueue *vring_create_virtqueue_split(
>   	if (!queue) {
>   		/* Try to get a single page. You are my only hope! */
>   		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
> -					  &dma_addr, GFP_KERNEL|__GFP_ZERO);
> +					  dma_addr, GFP_KERNEL|__GFP_ZERO);
>   	}
>   	if (!queue)
>   		return NULL;
>   
> +	*n = num;
> +	return queue;
> +}
> +
> +static struct virtqueue *vring_create_virtqueue_split(
> +	unsigned int index,
> +	unsigned int num,
> +	unsigned int vring_align,
> +	struct virtio_device *vdev,
> +	bool weak_barriers,
> +	bool may_reduce_num,
> +	bool context,
> +	bool (*notify)(struct virtqueue *),
> +	void (*callback)(struct virtqueue *),
> +	const char *name)
> +{
> +	size_t queue_size_in_bytes;
> +	struct virtqueue *vq;
> +	dma_addr_t dma_addr;
> +	struct vring vring;
> +	void *queue;
> +
> +	queue = vring_alloc_queue_split(vdev, &dma_addr, &num, vring_align,
> +					weak_barriers, may_reduce_num);
> +	if (!queue)
> +		return NULL;
> +
>   	queue_size_in_bytes = vring_size(num, vring_align);
>   	vring_init(&vring, num, queue, vring_align);
>   

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 06/32] virtio_ring: split: extract the logic of alloc queue

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of split to create vring queue.
>
> This feature is required for subsequent virtuqueue reset vring.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++------------
>   1 file changed, 36 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 33fddfb907a6..72d5ae063fa0 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -915,23 +915,15 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> -static struct virtqueue *vring_create_virtqueue_split(
> -	unsigned int index,
> -	unsigned int num,
> -	unsigned int vring_align,
> -	struct virtio_device *vdev,
> -	bool weak_barriers,
> -	bool may_reduce_num,
> -	bool context,
> -	bool (*notify)(struct virtqueue *),
> -	void (*callback)(struct virtqueue *),
> -	const char *name)
> +static void *vring_alloc_queue_split(struct virtio_device *vdev,
> +				     dma_addr_t *dma_addr,
> +				     u32 *n,
> +				     unsigned int vring_align,
> +				     bool weak_barriers,


This is not used in this function.

Thanks


> +				     bool may_reduce_num)
>   {
> -	struct virtqueue *vq;
>   	void *queue = NULL;
> -	dma_addr_t dma_addr;
> -	size_t queue_size_in_bytes;
> -	struct vring vring;
> +	u32 num = *n;
>   
>   	/* We assume num is a power of 2. */
>   	if (num & (num - 1)) {
> @@ -942,7 +934,7 @@ static struct virtqueue *vring_create_virtqueue_split(
>   	/* TODO: allocate each queue chunk individually */
>   	for (; num && vring_size(num, vring_align) > PAGE_SIZE; num /= 2) {
>   		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
> -					  &dma_addr,
> +					  dma_addr,
>   					  GFP_KERNEL|__GFP_NOWARN|__GFP_ZERO);
>   		if (queue)
>   			break;
> @@ -956,11 +948,38 @@ static struct virtqueue *vring_create_virtqueue_split(
>   	if (!queue) {
>   		/* Try to get a single page. You are my only hope! */
>   		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
> -					  &dma_addr, GFP_KERNEL|__GFP_ZERO);
> +					  dma_addr, GFP_KERNEL|__GFP_ZERO);
>   	}
>   	if (!queue)
>   		return NULL;
>   
> +	*n = num;
> +	return queue;
> +}
> +
> +static struct virtqueue *vring_create_virtqueue_split(
> +	unsigned int index,
> +	unsigned int num,
> +	unsigned int vring_align,
> +	struct virtio_device *vdev,
> +	bool weak_barriers,
> +	bool may_reduce_num,
> +	bool context,
> +	bool (*notify)(struct virtqueue *),
> +	void (*callback)(struct virtqueue *),
> +	const char *name)
> +{
> +	size_t queue_size_in_bytes;
> +	struct virtqueue *vq;
> +	dma_addr_t dma_addr;
> +	struct vring vring;
> +	void *queue;
> +
> +	queue = vring_alloc_queue_split(vdev, &dma_addr, &num, vring_align,
> +					weak_barriers, may_reduce_num);
> +	if (!queue)
> +		return NULL;
> +
>   	queue_size_in_bytes = vring_size(num, vring_align);
>   	vring_init(&vring, num, queue, vring_align);
>   


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 07/32] virtio_ring: split: extract the logic of alloc state and extra

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of creating desc_state, desc_extra, and subsequent
> patches will call it independently.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++++----------
>   1 file changed, 38 insertions(+), 15 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 72d5ae063fa0..6de67439cb57 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -198,6 +198,7 @@ struct vring_virtqueue {
>   #endif
>   };
>   
> +static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
>   
>   /*
>    * Helpers.
> @@ -915,6 +916,33 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> +static int vring_alloc_state_extra_split(u32 num,
> +					 struct vring_desc_state_split **desc_state,
> +					 struct vring_desc_extra **desc_extra)
> +{
> +	struct vring_desc_state_split *state;
> +	struct vring_desc_extra *extra;
> +
> +	state = kmalloc_array(num, sizeof(struct vring_desc_state_split), GFP_KERNEL);
> +	if (!state)
> +		goto err_state;
> +
> +	extra = vring_alloc_desc_extra(num);
> +	if (!extra)
> +		goto err_extra;
> +
> +	memset(state, 0, num * sizeof(struct vring_desc_state_split));
> +
> +	*desc_state = state;
> +	*desc_extra = extra;
> +	return 0;
> +
> +err_extra:
> +	kfree(state);
> +err_state:
> +	return -ENOMEM;
> +}
> +
>   static void *vring_alloc_queue_split(struct virtio_device *vdev,
>   				     dma_addr_t *dma_addr,
>   				     u32 *n,
> @@ -2196,7 +2224,10 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   					void (*callback)(struct virtqueue *),
>   					const char *name)
>   {
> +	struct vring_desc_state_split *state;
> +	struct vring_desc_extra *extra;
>   	struct vring_virtqueue *vq;
> +	int err;
>   
>   	if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED))
>   		return NULL;
> @@ -2246,30 +2277,22 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   					vq->split.avail_flags_shadow);
>   	}
>   
> -	vq->split.desc_state = kmalloc_array(vring.num,
> -			sizeof(struct vring_desc_state_split), GFP_KERNEL);
> -	if (!vq->split.desc_state)
> -		goto err_state;
> +	err = vring_alloc_state_extra_split(vring.num, &state, &extra);


Nit: we can pass e.g &vq->split.desc_state here to avoid extra temp 
variable and assignment.

Other looks good.

Thanks


> +	if (err) {
> +		kfree(vq);
> +		return NULL;
> +	}
>   
> -	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
> -	if (!vq->split.desc_extra)
> -		goto err_extra;
> +	vq->split.desc_state = state;
> +	vq->split.desc_extra = extra;
>   
>   	/* Put everything in free lists. */
>   	vq->free_head = 0;
> -	memset(vq->split.desc_state, 0, vring.num *
> -			sizeof(struct vring_desc_state_split));
>   
>   	spin_lock(&vdev->vqs_list_lock);
>   	list_add_tail(&vq->vq.list, &vdev->vqs);
>   	spin_unlock(&vdev->vqs_list_lock);
>   	return &vq->vq;
> -
> -err_extra:
> -	kfree(vq->split.desc_state);
> -err_state:
> -	kfree(vq);
> -	return NULL;
>   }
>   EXPORT_SYMBOL_GPL(__vring_new_virtqueue);
>   

Re: [PATCH v9 07/32] virtio_ring: split: extract the logic of alloc state and extra

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of creating desc_state, desc_extra, and subsequent
> patches will call it independently.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++++----------
>   1 file changed, 38 insertions(+), 15 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 72d5ae063fa0..6de67439cb57 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -198,6 +198,7 @@ struct vring_virtqueue {
>   #endif
>   };
>   
> +static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
>   
>   /*
>    * Helpers.
> @@ -915,6 +916,33 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> +static int vring_alloc_state_extra_split(u32 num,
> +					 struct vring_desc_state_split **desc_state,
> +					 struct vring_desc_extra **desc_extra)
> +{
> +	struct vring_desc_state_split *state;
> +	struct vring_desc_extra *extra;
> +
> +	state = kmalloc_array(num, sizeof(struct vring_desc_state_split), GFP_KERNEL);
> +	if (!state)
> +		goto err_state;
> +
> +	extra = vring_alloc_desc_extra(num);
> +	if (!extra)
> +		goto err_extra;
> +
> +	memset(state, 0, num * sizeof(struct vring_desc_state_split));
> +
> +	*desc_state = state;
> +	*desc_extra = extra;
> +	return 0;
> +
> +err_extra:
> +	kfree(state);
> +err_state:
> +	return -ENOMEM;
> +}
> +
>   static void *vring_alloc_queue_split(struct virtio_device *vdev,
>   				     dma_addr_t *dma_addr,
>   				     u32 *n,
> @@ -2196,7 +2224,10 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   					void (*callback)(struct virtqueue *),
>   					const char *name)
>   {
> +	struct vring_desc_state_split *state;
> +	struct vring_desc_extra *extra;
>   	struct vring_virtqueue *vq;
> +	int err;
>   
>   	if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED))
>   		return NULL;
> @@ -2246,30 +2277,22 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   					vq->split.avail_flags_shadow);
>   	}
>   
> -	vq->split.desc_state = kmalloc_array(vring.num,
> -			sizeof(struct vring_desc_state_split), GFP_KERNEL);
> -	if (!vq->split.desc_state)
> -		goto err_state;
> +	err = vring_alloc_state_extra_split(vring.num, &state, &extra);


Nit: we can pass e.g &vq->split.desc_state here to avoid extra temp 
variable and assignment.

Other looks good.

Thanks


> +	if (err) {
> +		kfree(vq);
> +		return NULL;
> +	}
>   
> -	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
> -	if (!vq->split.desc_extra)
> -		goto err_extra;
> +	vq->split.desc_state = state;
> +	vq->split.desc_extra = extra;
>   
>   	/* Put everything in free lists. */
>   	vq->free_head = 0;
> -	memset(vq->split.desc_state, 0, vring.num *
> -			sizeof(struct vring_desc_state_split));
>   
>   	spin_lock(&vdev->vqs_list_lock);
>   	list_add_tail(&vq->vq.list, &vdev->vqs);
>   	spin_unlock(&vdev->vqs_list_lock);
>   	return &vq->vq;
> -
> -err_extra:
> -	kfree(vq->split.desc_state);
> -err_state:
> -	kfree(vq);
> -	return NULL;
>   }
>   EXPORT_SYMBOL_GPL(__vring_new_virtqueue);
>   

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 07/32] virtio_ring: split: extract the logic of alloc state and extra

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of creating desc_state, desc_extra, and subsequent
> patches will call it independently.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++++----------
>   1 file changed, 38 insertions(+), 15 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 72d5ae063fa0..6de67439cb57 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -198,6 +198,7 @@ struct vring_virtqueue {
>   #endif
>   };
>   
> +static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
>   
>   /*
>    * Helpers.
> @@ -915,6 +916,33 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> +static int vring_alloc_state_extra_split(u32 num,
> +					 struct vring_desc_state_split **desc_state,
> +					 struct vring_desc_extra **desc_extra)
> +{
> +	struct vring_desc_state_split *state;
> +	struct vring_desc_extra *extra;
> +
> +	state = kmalloc_array(num, sizeof(struct vring_desc_state_split), GFP_KERNEL);
> +	if (!state)
> +		goto err_state;
> +
> +	extra = vring_alloc_desc_extra(num);
> +	if (!extra)
> +		goto err_extra;
> +
> +	memset(state, 0, num * sizeof(struct vring_desc_state_split));
> +
> +	*desc_state = state;
> +	*desc_extra = extra;
> +	return 0;
> +
> +err_extra:
> +	kfree(state);
> +err_state:
> +	return -ENOMEM;
> +}
> +
>   static void *vring_alloc_queue_split(struct virtio_device *vdev,
>   				     dma_addr_t *dma_addr,
>   				     u32 *n,
> @@ -2196,7 +2224,10 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   					void (*callback)(struct virtqueue *),
>   					const char *name)
>   {
> +	struct vring_desc_state_split *state;
> +	struct vring_desc_extra *extra;
>   	struct vring_virtqueue *vq;
> +	int err;
>   
>   	if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED))
>   		return NULL;
> @@ -2246,30 +2277,22 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   					vq->split.avail_flags_shadow);
>   	}
>   
> -	vq->split.desc_state = kmalloc_array(vring.num,
> -			sizeof(struct vring_desc_state_split), GFP_KERNEL);
> -	if (!vq->split.desc_state)
> -		goto err_state;
> +	err = vring_alloc_state_extra_split(vring.num, &state, &extra);


Nit: we can pass e.g &vq->split.desc_state here to avoid extra temp 
variable and assignment.

Other looks good.

Thanks


> +	if (err) {
> +		kfree(vq);
> +		return NULL;
> +	}
>   
> -	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
> -	if (!vq->split.desc_extra)
> -		goto err_extra;
> +	vq->split.desc_state = state;
> +	vq->split.desc_extra = extra;
>   
>   	/* Put everything in free lists. */
>   	vq->free_head = 0;
> -	memset(vq->split.desc_state, 0, vring.num *
> -			sizeof(struct vring_desc_state_split));
>   
>   	spin_lock(&vdev->vqs_list_lock);
>   	list_add_tail(&vq->vq.list, &vdev->vqs);
>   	spin_unlock(&vdev->vqs_list_lock);
>   	return &vq->vq;
> -
> -err_extra:
> -	kfree(vq->split.desc_state);
> -err_state:
> -	kfree(vq);
> -	return NULL;
>   }
>   EXPORT_SYMBOL_GPL(__vring_new_virtqueue);
>   


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 08/32] virtio_ring: split: extract the logic of attach vring

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of attach vring, subsequent patches will call it
> separately.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 20 ++++++++++++++------
>   1 file changed, 14 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 6de67439cb57..083f2992ba0d 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -916,6 +916,19 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> +static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
> +					 struct vring vring,
> +					 struct vring_desc_state_split *desc_state,
> +					 struct vring_desc_extra *desc_extra)
> +{
> +	vq->split.vring = vring;
> +	vq->split.queue_dma_addr = 0;
> +	vq->split.queue_size_in_bytes = 0;


Any reason to add the above two assignment in attach? It seems belong to 
free or reset.

Thanks


> +
> +	vq->split.desc_state = desc_state;
> +	vq->split.desc_extra = desc_extra;
> +}
> +
>   static int vring_alloc_state_extra_split(u32 num,
>   					 struct vring_desc_state_split **desc_state,
>   					 struct vring_desc_extra **desc_extra)
> @@ -2262,10 +2275,6 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
>   		vq->weak_barriers = false;
>   
> -	vq->split.queue_dma_addr = 0;
> -	vq->split.queue_size_in_bytes = 0;
> -
> -	vq->split.vring = vring;
>   	vq->split.avail_flags_shadow = 0;
>   	vq->split.avail_idx_shadow = 0;
>   
> @@ -2283,8 +2292,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   		return NULL;
>   	}
>   
> -	vq->split.desc_state = state;
> -	vq->split.desc_extra = extra;
> +	vring_virtqueue_attach_split(vq, vring, state, extra);
>   
>   	/* Put everything in free lists. */
>   	vq->free_head = 0;

Re: [PATCH v9 08/32] virtio_ring: split: extract the logic of attach vring

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of attach vring, subsequent patches will call it
> separately.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 20 ++++++++++++++------
>   1 file changed, 14 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 6de67439cb57..083f2992ba0d 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -916,6 +916,19 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> +static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
> +					 struct vring vring,
> +					 struct vring_desc_state_split *desc_state,
> +					 struct vring_desc_extra *desc_extra)
> +{
> +	vq->split.vring = vring;
> +	vq->split.queue_dma_addr = 0;
> +	vq->split.queue_size_in_bytes = 0;


Any reason to add the above two assignment in attach? It seems belong to 
free or reset.

Thanks


> +
> +	vq->split.desc_state = desc_state;
> +	vq->split.desc_extra = desc_extra;
> +}
> +
>   static int vring_alloc_state_extra_split(u32 num,
>   					 struct vring_desc_state_split **desc_state,
>   					 struct vring_desc_extra **desc_extra)
> @@ -2262,10 +2275,6 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
>   		vq->weak_barriers = false;
>   
> -	vq->split.queue_dma_addr = 0;
> -	vq->split.queue_size_in_bytes = 0;
> -
> -	vq->split.vring = vring;
>   	vq->split.avail_flags_shadow = 0;
>   	vq->split.avail_idx_shadow = 0;
>   
> @@ -2283,8 +2292,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   		return NULL;
>   	}
>   
> -	vq->split.desc_state = state;
> -	vq->split.desc_extra = extra;
> +	vring_virtqueue_attach_split(vq, vring, state, extra);
>   
>   	/* Put everything in free lists. */
>   	vq->free_head = 0;

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 08/32] virtio_ring: split: extract the logic of attach vring

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of attach vring, subsequent patches will call it
> separately.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 20 ++++++++++++++------
>   1 file changed, 14 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 6de67439cb57..083f2992ba0d 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -916,6 +916,19 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> +static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
> +					 struct vring vring,
> +					 struct vring_desc_state_split *desc_state,
> +					 struct vring_desc_extra *desc_extra)
> +{
> +	vq->split.vring = vring;
> +	vq->split.queue_dma_addr = 0;
> +	vq->split.queue_size_in_bytes = 0;


Any reason to add the above two assignment in attach? It seems belong to 
free or reset.

Thanks


> +
> +	vq->split.desc_state = desc_state;
> +	vq->split.desc_extra = desc_extra;
> +}
> +
>   static int vring_alloc_state_extra_split(u32 num,
>   					 struct vring_desc_state_split **desc_state,
>   					 struct vring_desc_extra **desc_extra)
> @@ -2262,10 +2275,6 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
>   		vq->weak_barriers = false;
>   
> -	vq->split.queue_dma_addr = 0;
> -	vq->split.queue_size_in_bytes = 0;
> -
> -	vq->split.vring = vring;
>   	vq->split.avail_flags_shadow = 0;
>   	vq->split.avail_idx_shadow = 0;
>   
> @@ -2283,8 +2292,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   		return NULL;
>   	}
>   
> -	vq->split.desc_state = state;
> -	vq->split.desc_extra = extra;
> +	vring_virtqueue_attach_split(vq, vring, state, extra);
>   
>   	/* Put everything in free lists. */
>   	vq->free_head = 0;


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 09/32] virtio_ring: split: extract the logic of vq init

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of initializing vq, and subsequent patches will call
> it separately.
>
> The feature of this part is that it does not depend on the information
> passed by the upper layer and can be called repeatedly.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 68 ++++++++++++++++++++----------------
>   1 file changed, 38 insertions(+), 30 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 083f2992ba0d..874f878087a3 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -916,6 +916,43 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> +static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
> +				       struct virtio_device *vdev,
> +				       bool own_ring)
> +{
> +	vq->packed_ring = false;
> +	vq->vq.num_free = vq->split.vring.num;
> +	vq->we_own_ring = own_ring;
> +	vq->broken = false;
> +	vq->last_used_idx = 0;
> +	vq->event_triggered = false;
> +	vq->num_added = 0;
> +	vq->use_dma_api = vring_use_dma_api(vdev);
> +#ifdef DEBUG
> +	vq->in_use = false;
> +	vq->last_add_time_valid = false;
> +#endif
> +
> +	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
> +
> +	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> +		vq->weak_barriers = false;
> +
> +	vq->split.avail_flags_shadow = 0;
> +	vq->split.avail_idx_shadow = 0;
> +
> +	/* No callback?  Tell other side not to bother us. */
> +	if (!vq->vq.callback) {
> +		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
> +		if (!vq->event)
> +			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
> +					vq->split.avail_flags_shadow);
> +	}
> +
> +	/* Put everything in free lists. */
> +	vq->free_head = 0;


It's not clear what kind of initialization that we want to do here. E.g 
it mixes split specific setups with some general setups which is kind of 
duplication of vring_virtqueue_init_packed().

I wonder if it's better to only do split specific setups here and have a 
common helper to do the setup that is irrelevant to ring layout.

Thanks


> +}
> +
>   static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
>   					 struct vring vring,
>   					 struct vring_desc_state_split *desc_state,
> @@ -2249,42 +2286,15 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	if (!vq)
>   		return NULL;
>   
> -	vq->packed_ring = false;
>   	vq->vq.callback = callback;
>   	vq->vq.vdev = vdev;
>   	vq->vq.name = name;
> -	vq->vq.num_free = vring.num;
>   	vq->vq.index = index;
> -	vq->we_own_ring = false;
>   	vq->notify = notify;
>   	vq->weak_barriers = weak_barriers;
> -	vq->broken = false;
> -	vq->last_used_idx = 0;
> -	vq->event_triggered = false;
> -	vq->num_added = 0;
> -	vq->use_dma_api = vring_use_dma_api(vdev);
> -#ifdef DEBUG
> -	vq->in_use = false;
> -	vq->last_add_time_valid = false;
> -#endif
>   
>   	vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) &&
>   		!context;
> -	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
> -
> -	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> -		vq->weak_barriers = false;
> -
> -	vq->split.avail_flags_shadow = 0;
> -	vq->split.avail_idx_shadow = 0;
> -
> -	/* No callback?  Tell other side not to bother us. */
> -	if (!callback) {
> -		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
> -		if (!vq->event)
> -			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
> -					vq->split.avail_flags_shadow);
> -	}
>   
>   	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
>   	if (err) {
> @@ -2293,9 +2303,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	}
>   
>   	vring_virtqueue_attach_split(vq, vring, state, extra);
> -
> -	/* Put everything in free lists. */
> -	vq->free_head = 0;
> +	vring_virtqueue_init_split(vq, vdev, false);
>   
>   	spin_lock(&vdev->vqs_list_lock);
>   	list_add_tail(&vq->vq.list, &vdev->vqs);

Re: [PATCH v9 09/32] virtio_ring: split: extract the logic of vq init

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of initializing vq, and subsequent patches will call
> it separately.
>
> The feature of this part is that it does not depend on the information
> passed by the upper layer and can be called repeatedly.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 68 ++++++++++++++++++++----------------
>   1 file changed, 38 insertions(+), 30 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 083f2992ba0d..874f878087a3 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -916,6 +916,43 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> +static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
> +				       struct virtio_device *vdev,
> +				       bool own_ring)
> +{
> +	vq->packed_ring = false;
> +	vq->vq.num_free = vq->split.vring.num;
> +	vq->we_own_ring = own_ring;
> +	vq->broken = false;
> +	vq->last_used_idx = 0;
> +	vq->event_triggered = false;
> +	vq->num_added = 0;
> +	vq->use_dma_api = vring_use_dma_api(vdev);
> +#ifdef DEBUG
> +	vq->in_use = false;
> +	vq->last_add_time_valid = false;
> +#endif
> +
> +	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
> +
> +	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> +		vq->weak_barriers = false;
> +
> +	vq->split.avail_flags_shadow = 0;
> +	vq->split.avail_idx_shadow = 0;
> +
> +	/* No callback?  Tell other side not to bother us. */
> +	if (!vq->vq.callback) {
> +		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
> +		if (!vq->event)
> +			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
> +					vq->split.avail_flags_shadow);
> +	}
> +
> +	/* Put everything in free lists. */
> +	vq->free_head = 0;


It's not clear what kind of initialization that we want to do here. E.g 
it mixes split specific setups with some general setups which is kind of 
duplication of vring_virtqueue_init_packed().

I wonder if it's better to only do split specific setups here and have a 
common helper to do the setup that is irrelevant to ring layout.

Thanks


> +}
> +
>   static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
>   					 struct vring vring,
>   					 struct vring_desc_state_split *desc_state,
> @@ -2249,42 +2286,15 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	if (!vq)
>   		return NULL;
>   
> -	vq->packed_ring = false;
>   	vq->vq.callback = callback;
>   	vq->vq.vdev = vdev;
>   	vq->vq.name = name;
> -	vq->vq.num_free = vring.num;
>   	vq->vq.index = index;
> -	vq->we_own_ring = false;
>   	vq->notify = notify;
>   	vq->weak_barriers = weak_barriers;
> -	vq->broken = false;
> -	vq->last_used_idx = 0;
> -	vq->event_triggered = false;
> -	vq->num_added = 0;
> -	vq->use_dma_api = vring_use_dma_api(vdev);
> -#ifdef DEBUG
> -	vq->in_use = false;
> -	vq->last_add_time_valid = false;
> -#endif
>   
>   	vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) &&
>   		!context;
> -	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
> -
> -	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> -		vq->weak_barriers = false;
> -
> -	vq->split.avail_flags_shadow = 0;
> -	vq->split.avail_idx_shadow = 0;
> -
> -	/* No callback?  Tell other side not to bother us. */
> -	if (!callback) {
> -		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
> -		if (!vq->event)
> -			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
> -					vq->split.avail_flags_shadow);
> -	}
>   
>   	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
>   	if (err) {
> @@ -2293,9 +2303,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	}
>   
>   	vring_virtqueue_attach_split(vq, vring, state, extra);
> -
> -	/* Put everything in free lists. */
> -	vq->free_head = 0;
> +	vring_virtqueue_init_split(vq, vdev, false);
>   
>   	spin_lock(&vdev->vqs_list_lock);
>   	list_add_tail(&vq->vq.list, &vdev->vqs);

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 09/32] virtio_ring: split: extract the logic of vq init

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of initializing vq, and subsequent patches will call
> it separately.
>
> The feature of this part is that it does not depend on the information
> passed by the upper layer and can be called repeatedly.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 68 ++++++++++++++++++++----------------
>   1 file changed, 38 insertions(+), 30 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 083f2992ba0d..874f878087a3 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -916,6 +916,43 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
>   	return NULL;
>   }
>   
> +static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
> +				       struct virtio_device *vdev,
> +				       bool own_ring)
> +{
> +	vq->packed_ring = false;
> +	vq->vq.num_free = vq->split.vring.num;
> +	vq->we_own_ring = own_ring;
> +	vq->broken = false;
> +	vq->last_used_idx = 0;
> +	vq->event_triggered = false;
> +	vq->num_added = 0;
> +	vq->use_dma_api = vring_use_dma_api(vdev);
> +#ifdef DEBUG
> +	vq->in_use = false;
> +	vq->last_add_time_valid = false;
> +#endif
> +
> +	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
> +
> +	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> +		vq->weak_barriers = false;
> +
> +	vq->split.avail_flags_shadow = 0;
> +	vq->split.avail_idx_shadow = 0;
> +
> +	/* No callback?  Tell other side not to bother us. */
> +	if (!vq->vq.callback) {
> +		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
> +		if (!vq->event)
> +			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
> +					vq->split.avail_flags_shadow);
> +	}
> +
> +	/* Put everything in free lists. */
> +	vq->free_head = 0;


It's not clear what kind of initialization that we want to do here. E.g 
it mixes split specific setups with some general setups which is kind of 
duplication of vring_virtqueue_init_packed().

I wonder if it's better to only do split specific setups here and have a 
common helper to do the setup that is irrelevant to ring layout.

Thanks


> +}
> +
>   static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
>   					 struct vring vring,
>   					 struct vring_desc_state_split *desc_state,
> @@ -2249,42 +2286,15 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	if (!vq)
>   		return NULL;
>   
> -	vq->packed_ring = false;
>   	vq->vq.callback = callback;
>   	vq->vq.vdev = vdev;
>   	vq->vq.name = name;
> -	vq->vq.num_free = vring.num;
>   	vq->vq.index = index;
> -	vq->we_own_ring = false;
>   	vq->notify = notify;
>   	vq->weak_barriers = weak_barriers;
> -	vq->broken = false;
> -	vq->last_used_idx = 0;
> -	vq->event_triggered = false;
> -	vq->num_added = 0;
> -	vq->use_dma_api = vring_use_dma_api(vdev);
> -#ifdef DEBUG
> -	vq->in_use = false;
> -	vq->last_add_time_valid = false;
> -#endif
>   
>   	vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) &&
>   		!context;
> -	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
> -
> -	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> -		vq->weak_barriers = false;
> -
> -	vq->split.avail_flags_shadow = 0;
> -	vq->split.avail_idx_shadow = 0;
> -
> -	/* No callback?  Tell other side not to bother us. */
> -	if (!callback) {
> -		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
> -		if (!vq->event)
> -			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
> -					vq->split.avail_flags_shadow);
> -	}
>   
>   	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
>   	if (err) {
> @@ -2293,9 +2303,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	}
>   
>   	vring_virtqueue_attach_split(vq, vring, state, extra);
> -
> -	/* Put everything in free lists. */
> -	vq->free_head = 0;
> +	vring_virtqueue_init_split(vq, vdev, false);
>   
>   	spin_lock(&vdev->vqs_list_lock);
>   	list_add_tail(&vq->vq.list, &vdev->vqs);


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 10/32] virtio_ring: split: introduce virtqueue_reinit_split()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce a function to initialize vq without allocating new ring,
> desc_state, desc_extra.
>
> Subsequent patches will call this function after reset vq to
> reinitialize vq.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_ring.c | 19 +++++++++++++++++++
>   1 file changed, 19 insertions(+)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 874f878087a3..3dc6ace2ba7a 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -953,6 +953,25 @@ static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
>   	vq->free_head = 0;
>   }
>   
> +static void virtqueue_reinit_split(struct vring_virtqueue *vq)
> +{
> +	struct virtio_device *vdev = vq->vq.vdev;
> +	int size, i;
> +
> +	memset(vq->split.vring.desc, 0, vq->split.queue_size_in_bytes);
> +
> +	size = sizeof(struct vring_desc_state_split) * vq->split.vring.num;
> +	memset(vq->split.desc_state, 0, size);
> +
> +	size = sizeof(struct vring_desc_extra) * vq->split.vring.num;
> +	memset(vq->split.desc_extra, 0, size);
> +
> +	for (i = 0; i < vq->split.vring.num - 1; i++)
> +		vq->split.desc_extra[i].next = i + 1;
> +
> +	vring_virtqueue_init_split(vq, vdev, true);
> +}
> +
>   static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
>   					 struct vring vring,
>   					 struct vring_desc_state_split *desc_state,

Re: [PATCH v9 10/32] virtio_ring: split: introduce virtqueue_reinit_split()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce a function to initialize vq without allocating new ring,
> desc_state, desc_extra.
>
> Subsequent patches will call this function after reset vq to
> reinitialize vq.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_ring.c | 19 +++++++++++++++++++
>   1 file changed, 19 insertions(+)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 874f878087a3..3dc6ace2ba7a 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -953,6 +953,25 @@ static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
>   	vq->free_head = 0;
>   }
>   
> +static void virtqueue_reinit_split(struct vring_virtqueue *vq)
> +{
> +	struct virtio_device *vdev = vq->vq.vdev;
> +	int size, i;
> +
> +	memset(vq->split.vring.desc, 0, vq->split.queue_size_in_bytes);
> +
> +	size = sizeof(struct vring_desc_state_split) * vq->split.vring.num;
> +	memset(vq->split.desc_state, 0, size);
> +
> +	size = sizeof(struct vring_desc_extra) * vq->split.vring.num;
> +	memset(vq->split.desc_extra, 0, size);
> +
> +	for (i = 0; i < vq->split.vring.num - 1; i++)
> +		vq->split.desc_extra[i].next = i + 1;
> +
> +	vring_virtqueue_init_split(vq, vdev, true);
> +}
> +
>   static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
>   					 struct vring vring,
>   					 struct vring_desc_state_split *desc_state,

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 10/32] virtio_ring: split: introduce virtqueue_reinit_split()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce a function to initialize vq without allocating new ring,
> desc_state, desc_extra.
>
> Subsequent patches will call this function after reset vq to
> reinitialize vq.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_ring.c | 19 +++++++++++++++++++
>   1 file changed, 19 insertions(+)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 874f878087a3..3dc6ace2ba7a 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -953,6 +953,25 @@ static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
>   	vq->free_head = 0;
>   }
>   
> +static void virtqueue_reinit_split(struct vring_virtqueue *vq)
> +{
> +	struct virtio_device *vdev = vq->vq.vdev;
> +	int size, i;
> +
> +	memset(vq->split.vring.desc, 0, vq->split.queue_size_in_bytes);
> +
> +	size = sizeof(struct vring_desc_state_split) * vq->split.vring.num;
> +	memset(vq->split.desc_state, 0, size);
> +
> +	size = sizeof(struct vring_desc_extra) * vq->split.vring.num;
> +	memset(vq->split.desc_extra, 0, size);
> +
> +	for (i = 0; i < vq->split.vring.num - 1; i++)
> +		vq->split.desc_extra[i].next = i + 1;
> +
> +	vring_virtqueue_init_split(vq, vdev, true);
> +}
> +
>   static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
>   					 struct vring vring,
>   					 struct vring_desc_state_split *desc_state,


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 11/32] virtio_ring: split: introduce virtqueue_resize_split()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> virtio ring split supports resize.
>
> Only after the new vring is successfully allocated based on the new num,
> we will release the old vring. In any case, an error is returned,
> indicating that the vring still points to the old vring.
>
> In the case of an error, the caller must
> re-initialize(virtqueue_reinit_split()) the virtqueue to ensure that the
> vring can be used.
>
> In addition, vring_align, may_reduce_num are necessary for reallocating
> vring, so they are retained for creating vq.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 47 ++++++++++++++++++++++++++++++++++++
>   1 file changed, 47 insertions(+)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 3dc6ace2ba7a..33864134a744 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -139,6 +139,12 @@ struct vring_virtqueue {
>   			/* DMA address and size information */
>   			dma_addr_t queue_dma_addr;
>   			size_t queue_size_in_bytes;
> +
> +			/* The parameters for creating vrings are reserved for
> +			 * creating new vring.
> +			 */
> +			u32 vring_align;
> +			bool may_reduce_num;
>   		} split;
>   
>   		/* Available for packed ring */
> @@ -199,6 +205,7 @@ struct vring_virtqueue {
>   };
>   
>   static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
> +static void vring_free(struct virtqueue *_vq);
>   
>   /*
>    * Helpers.
> @@ -1088,6 +1095,8 @@ static struct virtqueue *vring_create_virtqueue_split(
>   		return NULL;
>   	}
>   
> +	to_vvq(vq)->split.vring_align = vring_align;
> +	to_vvq(vq)->split.may_reduce_num = may_reduce_num;


It looks to me the above should belong to patch 6.


>   	to_vvq(vq)->split.queue_dma_addr = dma_addr;
>   	to_vvq(vq)->split.queue_size_in_bytes = queue_size_in_bytes;
>   	to_vvq(vq)->we_own_ring = true;
> @@ -1095,6 +1104,44 @@ static struct virtqueue *vring_create_virtqueue_split(
>   	return vq;
>   }
>   
> +static int virtqueue_resize_split(struct virtqueue *_vq, u32 num)
> +{
> +	struct vring_virtqueue *vq = to_vvq(_vq);
> +	struct virtio_device *vdev = _vq->vdev;
> +	struct vring_desc_state_split *state;
> +	struct vring_desc_extra *extra;
> +	size_t queue_size_in_bytes;
> +	dma_addr_t dma_addr;
> +	struct vring vring;
> +	int err = -ENOMEM;
> +	void *queue;
> +
> +	queue = vring_alloc_queue_split(vdev, &dma_addr, &num,
> +					vq->split.vring_align,
> +					vq->weak_barriers,
> +					vq->split.may_reduce_num);
> +	if (!queue)
> +		return -ENOMEM;
> +
> +	queue_size_in_bytes = vring_size(num, vq->split.vring_align);
> +
> +	err = vring_alloc_state_extra_split(num, &state, &extra);
> +	if (err) {
> +		vring_free_queue(vdev, queue_size_in_bytes, queue, dma_addr);
> +		return -ENOMEM;
> +	}
> +
> +	vring_free(&vq->vq);
> +
> +	vring_init(&vring, num, queue, vq->split.vring_align);
> +	vring_virtqueue_attach_split(vq, vring, state, extra);
> +	vq->split.queue_dma_addr = dma_addr;
> +	vq->split.queue_size_in_bytes = queue_size_in_bytes;


I wonder if it's better to move the above assignments to 
vring_virtqueue_attach_split().

Other looks good.

Thanks


> +
> +	vring_virtqueue_init_split(vq, vdev, true);
> +	return 0;
> +}
> +
>   
>   /*
>    * Packed ring specific functions - *_packed().

Re: [PATCH v9 11/32] virtio_ring: split: introduce virtqueue_resize_split()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> virtio ring split supports resize.
>
> Only after the new vring is successfully allocated based on the new num,
> we will release the old vring. In any case, an error is returned,
> indicating that the vring still points to the old vring.
>
> In the case of an error, the caller must
> re-initialize(virtqueue_reinit_split()) the virtqueue to ensure that the
> vring can be used.
>
> In addition, vring_align, may_reduce_num are necessary for reallocating
> vring, so they are retained for creating vq.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 47 ++++++++++++++++++++++++++++++++++++
>   1 file changed, 47 insertions(+)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 3dc6ace2ba7a..33864134a744 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -139,6 +139,12 @@ struct vring_virtqueue {
>   			/* DMA address and size information */
>   			dma_addr_t queue_dma_addr;
>   			size_t queue_size_in_bytes;
> +
> +			/* The parameters for creating vrings are reserved for
> +			 * creating new vring.
> +			 */
> +			u32 vring_align;
> +			bool may_reduce_num;
>   		} split;
>   
>   		/* Available for packed ring */
> @@ -199,6 +205,7 @@ struct vring_virtqueue {
>   };
>   
>   static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
> +static void vring_free(struct virtqueue *_vq);
>   
>   /*
>    * Helpers.
> @@ -1088,6 +1095,8 @@ static struct virtqueue *vring_create_virtqueue_split(
>   		return NULL;
>   	}
>   
> +	to_vvq(vq)->split.vring_align = vring_align;
> +	to_vvq(vq)->split.may_reduce_num = may_reduce_num;


It looks to me the above should belong to patch 6.


>   	to_vvq(vq)->split.queue_dma_addr = dma_addr;
>   	to_vvq(vq)->split.queue_size_in_bytes = queue_size_in_bytes;
>   	to_vvq(vq)->we_own_ring = true;
> @@ -1095,6 +1104,44 @@ static struct virtqueue *vring_create_virtqueue_split(
>   	return vq;
>   }
>   
> +static int virtqueue_resize_split(struct virtqueue *_vq, u32 num)
> +{
> +	struct vring_virtqueue *vq = to_vvq(_vq);
> +	struct virtio_device *vdev = _vq->vdev;
> +	struct vring_desc_state_split *state;
> +	struct vring_desc_extra *extra;
> +	size_t queue_size_in_bytes;
> +	dma_addr_t dma_addr;
> +	struct vring vring;
> +	int err = -ENOMEM;
> +	void *queue;
> +
> +	queue = vring_alloc_queue_split(vdev, &dma_addr, &num,
> +					vq->split.vring_align,
> +					vq->weak_barriers,
> +					vq->split.may_reduce_num);
> +	if (!queue)
> +		return -ENOMEM;
> +
> +	queue_size_in_bytes = vring_size(num, vq->split.vring_align);
> +
> +	err = vring_alloc_state_extra_split(num, &state, &extra);
> +	if (err) {
> +		vring_free_queue(vdev, queue_size_in_bytes, queue, dma_addr);
> +		return -ENOMEM;
> +	}
> +
> +	vring_free(&vq->vq);
> +
> +	vring_init(&vring, num, queue, vq->split.vring_align);
> +	vring_virtqueue_attach_split(vq, vring, state, extra);
> +	vq->split.queue_dma_addr = dma_addr;
> +	vq->split.queue_size_in_bytes = queue_size_in_bytes;


I wonder if it's better to move the above assignments to 
vring_virtqueue_attach_split().

Other looks good.

Thanks


> +
> +	vring_virtqueue_init_split(vq, vdev, true);
> +	return 0;
> +}
> +
>   
>   /*
>    * Packed ring specific functions - *_packed().

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 11/32] virtio_ring: split: introduce virtqueue_resize_split()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> virtio ring split supports resize.
>
> Only after the new vring is successfully allocated based on the new num,
> we will release the old vring. In any case, an error is returned,
> indicating that the vring still points to the old vring.
>
> In the case of an error, the caller must
> re-initialize(virtqueue_reinit_split()) the virtqueue to ensure that the
> vring can be used.
>
> In addition, vring_align, may_reduce_num are necessary for reallocating
> vring, so they are retained for creating vq.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 47 ++++++++++++++++++++++++++++++++++++
>   1 file changed, 47 insertions(+)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 3dc6ace2ba7a..33864134a744 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -139,6 +139,12 @@ struct vring_virtqueue {
>   			/* DMA address and size information */
>   			dma_addr_t queue_dma_addr;
>   			size_t queue_size_in_bytes;
> +
> +			/* The parameters for creating vrings are reserved for
> +			 * creating new vring.
> +			 */
> +			u32 vring_align;
> +			bool may_reduce_num;
>   		} split;
>   
>   		/* Available for packed ring */
> @@ -199,6 +205,7 @@ struct vring_virtqueue {
>   };
>   
>   static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
> +static void vring_free(struct virtqueue *_vq);
>   
>   /*
>    * Helpers.
> @@ -1088,6 +1095,8 @@ static struct virtqueue *vring_create_virtqueue_split(
>   		return NULL;
>   	}
>   
> +	to_vvq(vq)->split.vring_align = vring_align;
> +	to_vvq(vq)->split.may_reduce_num = may_reduce_num;


It looks to me the above should belong to patch 6.


>   	to_vvq(vq)->split.queue_dma_addr = dma_addr;
>   	to_vvq(vq)->split.queue_size_in_bytes = queue_size_in_bytes;
>   	to_vvq(vq)->we_own_ring = true;
> @@ -1095,6 +1104,44 @@ static struct virtqueue *vring_create_virtqueue_split(
>   	return vq;
>   }
>   
> +static int virtqueue_resize_split(struct virtqueue *_vq, u32 num)
> +{
> +	struct vring_virtqueue *vq = to_vvq(_vq);
> +	struct virtio_device *vdev = _vq->vdev;
> +	struct vring_desc_state_split *state;
> +	struct vring_desc_extra *extra;
> +	size_t queue_size_in_bytes;
> +	dma_addr_t dma_addr;
> +	struct vring vring;
> +	int err = -ENOMEM;
> +	void *queue;
> +
> +	queue = vring_alloc_queue_split(vdev, &dma_addr, &num,
> +					vq->split.vring_align,
> +					vq->weak_barriers,
> +					vq->split.may_reduce_num);
> +	if (!queue)
> +		return -ENOMEM;
> +
> +	queue_size_in_bytes = vring_size(num, vq->split.vring_align);
> +
> +	err = vring_alloc_state_extra_split(num, &state, &extra);
> +	if (err) {
> +		vring_free_queue(vdev, queue_size_in_bytes, queue, dma_addr);
> +		return -ENOMEM;
> +	}
> +
> +	vring_free(&vq->vq);
> +
> +	vring_init(&vring, num, queue, vq->split.vring_align);
> +	vring_virtqueue_attach_split(vq, vring, state, extra);
> +	vq->split.queue_dma_addr = dma_addr;
> +	vq->split.queue_size_in_bytes = queue_size_in_bytes;


I wonder if it's better to move the above assignments to 
vring_virtqueue_attach_split().

Other looks good.

Thanks


> +
> +	vring_virtqueue_init_split(vq, vdev, true);
> +	return 0;
> +}
> +
>   
>   /*
>    * Packed ring specific functions - *_packed().


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 12/32] virtio_ring: packed: extract the logic of alloc queue

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of packed to create vring queue.
>
> For the convenience of passing parameters, add a structure
> vring_packed.
>
> This feature is required for subsequent virtuqueue reset vring.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 70 ++++++++++++++++++++++++++++--------
>   1 file changed, 56 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 33864134a744..ea451ae2aaef 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -1817,19 +1817,17 @@ static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
>   	return desc_extra;
>   }
>   
> -static struct virtqueue *vring_create_virtqueue_packed(
> -	unsigned int index,
> -	unsigned int num,
> -	unsigned int vring_align,
> -	struct virtio_device *vdev,
> -	bool weak_barriers,
> -	bool may_reduce_num,
> -	bool context,
> -	bool (*notify)(struct virtqueue *),
> -	void (*callback)(struct virtqueue *),
> -	const char *name)
> +static int vring_alloc_queue_packed(struct virtio_device *vdev,
> +				    u32 num,
> +				    struct vring_packed_desc **_ring,
> +				    struct vring_packed_desc_event **_driver,
> +				    struct vring_packed_desc_event **_device,
> +				    dma_addr_t *_ring_dma_addr,
> +				    dma_addr_t *_driver_event_dma_addr,
> +				    dma_addr_t *_device_event_dma_addr,
> +				    size_t *_ring_size_in_bytes,
> +				    size_t *_event_size_in_bytes)
>   {
> -	struct vring_virtqueue *vq;
>   	struct vring_packed_desc *ring;
>   	struct vring_packed_desc_event *driver, *device;
>   	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> @@ -1857,6 +1855,52 @@ static struct virtqueue *vring_create_virtqueue_packed(
>   	if (!device)
>   		goto err_device;
>   
> +	*_ring                   = ring;
> +	*_driver                 = driver;
> +	*_device                 = device;
> +	*_ring_dma_addr          = ring_dma_addr;
> +	*_driver_event_dma_addr  = driver_event_dma_addr;
> +	*_device_event_dma_addr  = device_event_dma_addr;
> +	*_ring_size_in_bytes     = ring_size_in_bytes;
> +	*_event_size_in_bytes    = event_size_in_bytes;


I wonder if we can simply factor out split and packed from struct 
vring_virtqueue:

struct vring_virtqueue {
     union {
         struct {} split;
         struct {} packed;
     };
};

to

struct vring_virtqueue_split {};
struct vring_virtqueue_packed {};

Then we can do things like:

vring_create_virtqueue_packed(struct virtio_device *vdev, u32 num, 
struct vring_virtqueue_packed *packed);

and

vring_vritqueue_attach_packed(struct vring_virtqueue *vq, struct 
vring_virtqueue_packed packed);

Thanks


> +
> +	return 0;
> +
> +err_device:
> +	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> +
> +err_driver:
> +	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> +
> +err_ring:
> +	return -ENOMEM;
> +}
> +
> +static struct virtqueue *vring_create_virtqueue_packed(
> +	unsigned int index,
> +	unsigned int num,
> +	unsigned int vring_align,
> +	struct virtio_device *vdev,
> +	bool weak_barriers,
> +	bool may_reduce_num,
> +	bool context,
> +	bool (*notify)(struct virtqueue *),
> +	void (*callback)(struct virtqueue *),
> +	const char *name)
> +{
> +	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> +	struct vring_packed_desc_event *driver, *device;
> +	size_t ring_size_in_bytes, event_size_in_bytes;
> +	struct vring_packed_desc *ring;
> +	struct vring_virtqueue *vq;
> +
> +	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
> +				     &ring_dma_addr, &driver_event_dma_addr,
> +				     &device_event_dma_addr,
> +				     &ring_size_in_bytes,
> +				     &event_size_in_bytes))
> +		goto err_ring;
> +
>   	vq = kmalloc(sizeof(*vq), GFP_KERNEL);
>   	if (!vq)
>   		goto err_vq;
> @@ -1939,9 +1983,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
>   	kfree(vq);
>   err_vq:
>   	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
> -err_device:
>   	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> -err_driver:
>   	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
>   err_ring:
>   	return NULL;

Re: [PATCH v9 12/32] virtio_ring: packed: extract the logic of alloc queue

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of packed to create vring queue.
>
> For the convenience of passing parameters, add a structure
> vring_packed.
>
> This feature is required for subsequent virtuqueue reset vring.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 70 ++++++++++++++++++++++++++++--------
>   1 file changed, 56 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 33864134a744..ea451ae2aaef 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -1817,19 +1817,17 @@ static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
>   	return desc_extra;
>   }
>   
> -static struct virtqueue *vring_create_virtqueue_packed(
> -	unsigned int index,
> -	unsigned int num,
> -	unsigned int vring_align,
> -	struct virtio_device *vdev,
> -	bool weak_barriers,
> -	bool may_reduce_num,
> -	bool context,
> -	bool (*notify)(struct virtqueue *),
> -	void (*callback)(struct virtqueue *),
> -	const char *name)
> +static int vring_alloc_queue_packed(struct virtio_device *vdev,
> +				    u32 num,
> +				    struct vring_packed_desc **_ring,
> +				    struct vring_packed_desc_event **_driver,
> +				    struct vring_packed_desc_event **_device,
> +				    dma_addr_t *_ring_dma_addr,
> +				    dma_addr_t *_driver_event_dma_addr,
> +				    dma_addr_t *_device_event_dma_addr,
> +				    size_t *_ring_size_in_bytes,
> +				    size_t *_event_size_in_bytes)
>   {
> -	struct vring_virtqueue *vq;
>   	struct vring_packed_desc *ring;
>   	struct vring_packed_desc_event *driver, *device;
>   	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> @@ -1857,6 +1855,52 @@ static struct virtqueue *vring_create_virtqueue_packed(
>   	if (!device)
>   		goto err_device;
>   
> +	*_ring                   = ring;
> +	*_driver                 = driver;
> +	*_device                 = device;
> +	*_ring_dma_addr          = ring_dma_addr;
> +	*_driver_event_dma_addr  = driver_event_dma_addr;
> +	*_device_event_dma_addr  = device_event_dma_addr;
> +	*_ring_size_in_bytes     = ring_size_in_bytes;
> +	*_event_size_in_bytes    = event_size_in_bytes;


I wonder if we can simply factor out split and packed from struct 
vring_virtqueue:

struct vring_virtqueue {
     union {
         struct {} split;
         struct {} packed;
     };
};

to

struct vring_virtqueue_split {};
struct vring_virtqueue_packed {};

Then we can do things like:

vring_create_virtqueue_packed(struct virtio_device *vdev, u32 num, 
struct vring_virtqueue_packed *packed);

and

vring_vritqueue_attach_packed(struct vring_virtqueue *vq, struct 
vring_virtqueue_packed packed);

Thanks


> +
> +	return 0;
> +
> +err_device:
> +	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> +
> +err_driver:
> +	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> +
> +err_ring:
> +	return -ENOMEM;
> +}
> +
> +static struct virtqueue *vring_create_virtqueue_packed(
> +	unsigned int index,
> +	unsigned int num,
> +	unsigned int vring_align,
> +	struct virtio_device *vdev,
> +	bool weak_barriers,
> +	bool may_reduce_num,
> +	bool context,
> +	bool (*notify)(struct virtqueue *),
> +	void (*callback)(struct virtqueue *),
> +	const char *name)
> +{
> +	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> +	struct vring_packed_desc_event *driver, *device;
> +	size_t ring_size_in_bytes, event_size_in_bytes;
> +	struct vring_packed_desc *ring;
> +	struct vring_virtqueue *vq;
> +
> +	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
> +				     &ring_dma_addr, &driver_event_dma_addr,
> +				     &device_event_dma_addr,
> +				     &ring_size_in_bytes,
> +				     &event_size_in_bytes))
> +		goto err_ring;
> +
>   	vq = kmalloc(sizeof(*vq), GFP_KERNEL);
>   	if (!vq)
>   		goto err_vq;
> @@ -1939,9 +1983,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
>   	kfree(vq);
>   err_vq:
>   	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
> -err_device:
>   	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> -err_driver:
>   	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
>   err_ring:
>   	return NULL;

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 12/32] virtio_ring: packed: extract the logic of alloc queue

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Separate the logic of packed to create vring queue.
>
> For the convenience of passing parameters, add a structure
> vring_packed.
>
> This feature is required for subsequent virtuqueue reset vring.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 70 ++++++++++++++++++++++++++++--------
>   1 file changed, 56 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 33864134a744..ea451ae2aaef 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -1817,19 +1817,17 @@ static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
>   	return desc_extra;
>   }
>   
> -static struct virtqueue *vring_create_virtqueue_packed(
> -	unsigned int index,
> -	unsigned int num,
> -	unsigned int vring_align,
> -	struct virtio_device *vdev,
> -	bool weak_barriers,
> -	bool may_reduce_num,
> -	bool context,
> -	bool (*notify)(struct virtqueue *),
> -	void (*callback)(struct virtqueue *),
> -	const char *name)
> +static int vring_alloc_queue_packed(struct virtio_device *vdev,
> +				    u32 num,
> +				    struct vring_packed_desc **_ring,
> +				    struct vring_packed_desc_event **_driver,
> +				    struct vring_packed_desc_event **_device,
> +				    dma_addr_t *_ring_dma_addr,
> +				    dma_addr_t *_driver_event_dma_addr,
> +				    dma_addr_t *_device_event_dma_addr,
> +				    size_t *_ring_size_in_bytes,
> +				    size_t *_event_size_in_bytes)
>   {
> -	struct vring_virtqueue *vq;
>   	struct vring_packed_desc *ring;
>   	struct vring_packed_desc_event *driver, *device;
>   	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> @@ -1857,6 +1855,52 @@ static struct virtqueue *vring_create_virtqueue_packed(
>   	if (!device)
>   		goto err_device;
>   
> +	*_ring                   = ring;
> +	*_driver                 = driver;
> +	*_device                 = device;
> +	*_ring_dma_addr          = ring_dma_addr;
> +	*_driver_event_dma_addr  = driver_event_dma_addr;
> +	*_device_event_dma_addr  = device_event_dma_addr;
> +	*_ring_size_in_bytes     = ring_size_in_bytes;
> +	*_event_size_in_bytes    = event_size_in_bytes;


I wonder if we can simply factor out split and packed from struct 
vring_virtqueue:

struct vring_virtqueue {
     union {
         struct {} split;
         struct {} packed;
     };
};

to

struct vring_virtqueue_split {};
struct vring_virtqueue_packed {};

Then we can do things like:

vring_create_virtqueue_packed(struct virtio_device *vdev, u32 num, 
struct vring_virtqueue_packed *packed);

and

vring_vritqueue_attach_packed(struct vring_virtqueue *vq, struct 
vring_virtqueue_packed packed);

Thanks


> +
> +	return 0;
> +
> +err_device:
> +	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> +
> +err_driver:
> +	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> +
> +err_ring:
> +	return -ENOMEM;
> +}
> +
> +static struct virtqueue *vring_create_virtqueue_packed(
> +	unsigned int index,
> +	unsigned int num,
> +	unsigned int vring_align,
> +	struct virtio_device *vdev,
> +	bool weak_barriers,
> +	bool may_reduce_num,
> +	bool context,
> +	bool (*notify)(struct virtqueue *),
> +	void (*callback)(struct virtqueue *),
> +	const char *name)
> +{
> +	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> +	struct vring_packed_desc_event *driver, *device;
> +	size_t ring_size_in_bytes, event_size_in_bytes;
> +	struct vring_packed_desc *ring;
> +	struct vring_virtqueue *vq;
> +
> +	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
> +				     &ring_dma_addr, &driver_event_dma_addr,
> +				     &device_event_dma_addr,
> +				     &ring_size_in_bytes,
> +				     &event_size_in_bytes))
> +		goto err_ring;
> +
>   	vq = kmalloc(sizeof(*vq), GFP_KERNEL);
>   	if (!vq)
>   		goto err_vq;
> @@ -1939,9 +1983,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
>   	kfree(vq);
>   err_vq:
>   	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
> -err_device:
>   	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> -err_driver:
>   	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
>   err_ring:
>   	return NULL;


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 18/32] virtio_ring: introduce virtqueue_resize()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce virtqueue_resize() to implement the resize of vring.
> Based on these, the driver can dynamically adjust the size of the vring.
> For example: ethtool -G.
>
> virtqueue_resize() implements resize based on the vq reset function. In
> case of failure to allocate a new vring, it will give up resize and use
> the original vring.
>
> During this process, if the re-enable reset vq fails, the vq can no
> longer be used. Although the probability of this situation is not high.
>
> The parameter recycle is used to recycle the buffer that is no longer
> used.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++++++++++
>   include/linux/virtio.h       |  3 ++
>   2 files changed, 72 insertions(+)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 06f66b15c86c..6250e19fc5bf 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2554,6 +2554,75 @@ struct virtqueue *vring_create_virtqueue(
>   }
>   EXPORT_SYMBOL_GPL(vring_create_virtqueue);
>   
> +/**
> + * virtqueue_resize - resize the vring of vq
> + * @_vq: the struct virtqueue we're talking about.
> + * @num: new ring num
> + * @recycle: callback for recycle the useless buffer
> + *
> + * When it is really necessary to create a new vring, it will set the current vq
> + * into the reset state. Then call the passed callback to recycle the buffer
> + * that is no longer used. Only after the new vring is successfully created, the
> + * old vring will be released.
> + *
> + * Caller must ensure we don't call this with other virtqueue operations
> + * at the same time (except where noted).
> + *
> + * Returns zero or a negative error.


Should we document that the virtqueue is kept unchanged (still 
available) on (specific) failure?


> + */
> +int virtqueue_resize(struct virtqueue *_vq, u32 num,
> +		     void (*recycle)(struct virtqueue *vq, void *buf))
> +{
> +	struct vring_virtqueue *vq = to_vvq(_vq);
> +	struct virtio_device *vdev = vq->vq.vdev;
> +	bool packed;
> +	void *buf;
> +	int err;
> +
> +	if (!vq->we_own_ring)
> +		return -EINVAL;
> +
> +	if (num > vq->vq.num_max)
> +		return -E2BIG;
> +
> +	if (!num)
> +		return -EINVAL;
> +
> +	packed = virtio_has_feature(vdev, VIRTIO_F_RING_PACKED) ? true : false;
> +
> +	if ((packed ? vq->packed.vring.num : vq->split.vring.num) == num)
> +		return 0;
> +
> +	if (!vdev->config->reset_vq)
> +		return -ENOENT;
> +
> +	if (!vdev->config->enable_reset_vq)
> +		return -ENOENT;
> +
> +	err = vdev->config->reset_vq(_vq);
> +	if (err)
> +		return err;
> +
> +	while ((buf = virtqueue_detach_unused_buf(_vq)) != NULL)
> +		recycle(_vq, buf);
> +
> +	if (packed) {
> +		err = virtqueue_resize_packed(_vq, num);
> +		if (err)
> +			virtqueue_reinit_packed(vq);


Calling reinit here seems a little bit odd, it looks more like a reset 
of the virtqueue. Consider we may re-use virtqueue reset for more 
purpose, I wonder if we need a helper like:

virtqueue_resize() {
     vdev->config->reset_vq(_vq);
     if (packed)
         virtqueue_reinit_packed(_vq)
     else
         virtqueue_reinit_split(_vq)
}

Thanks


> +	} else {
> +		err = virtqueue_resize_split(_vq, num);
> +		if (err)
> +			virtqueue_reinit_split(vq);
> +	}
> +
> +	if (vdev->config->enable_reset_vq(_vq))
> +		return -EBUSY;
> +
> +	return err;
> +}
> +EXPORT_SYMBOL_GPL(virtqueue_resize);
> +
>   /* Only available for split ring */
>   struct virtqueue *vring_new_virtqueue(unsigned int index,
>   				      unsigned int num,
> diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> index d59adc4be068..c86ff02e0ca0 100644
> --- a/include/linux/virtio.h
> +++ b/include/linux/virtio.h
> @@ -91,6 +91,9 @@ dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
>   dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
>   dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
>   
> +int virtqueue_resize(struct virtqueue *vq, u32 num,
> +		     void (*recycle)(struct virtqueue *vq, void *buf));
> +
>   /**
>    * virtio_device - representation of a device using virtio
>    * @index: unique position on the virtio bus

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 18/32] virtio_ring: introduce virtqueue_resize()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce virtqueue_resize() to implement the resize of vring.
> Based on these, the driver can dynamically adjust the size of the vring.
> For example: ethtool -G.
>
> virtqueue_resize() implements resize based on the vq reset function. In
> case of failure to allocate a new vring, it will give up resize and use
> the original vring.
>
> During this process, if the re-enable reset vq fails, the vq can no
> longer be used. Although the probability of this situation is not high.
>
> The parameter recycle is used to recycle the buffer that is no longer
> used.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++++++++++
>   include/linux/virtio.h       |  3 ++
>   2 files changed, 72 insertions(+)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 06f66b15c86c..6250e19fc5bf 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2554,6 +2554,75 @@ struct virtqueue *vring_create_virtqueue(
>   }
>   EXPORT_SYMBOL_GPL(vring_create_virtqueue);
>   
> +/**
> + * virtqueue_resize - resize the vring of vq
> + * @_vq: the struct virtqueue we're talking about.
> + * @num: new ring num
> + * @recycle: callback for recycle the useless buffer
> + *
> + * When it is really necessary to create a new vring, it will set the current vq
> + * into the reset state. Then call the passed callback to recycle the buffer
> + * that is no longer used. Only after the new vring is successfully created, the
> + * old vring will be released.
> + *
> + * Caller must ensure we don't call this with other virtqueue operations
> + * at the same time (except where noted).
> + *
> + * Returns zero or a negative error.


Should we document that the virtqueue is kept unchanged (still 
available) on (specific) failure?


> + */
> +int virtqueue_resize(struct virtqueue *_vq, u32 num,
> +		     void (*recycle)(struct virtqueue *vq, void *buf))
> +{
> +	struct vring_virtqueue *vq = to_vvq(_vq);
> +	struct virtio_device *vdev = vq->vq.vdev;
> +	bool packed;
> +	void *buf;
> +	int err;
> +
> +	if (!vq->we_own_ring)
> +		return -EINVAL;
> +
> +	if (num > vq->vq.num_max)
> +		return -E2BIG;
> +
> +	if (!num)
> +		return -EINVAL;
> +
> +	packed = virtio_has_feature(vdev, VIRTIO_F_RING_PACKED) ? true : false;
> +
> +	if ((packed ? vq->packed.vring.num : vq->split.vring.num) == num)
> +		return 0;
> +
> +	if (!vdev->config->reset_vq)
> +		return -ENOENT;
> +
> +	if (!vdev->config->enable_reset_vq)
> +		return -ENOENT;
> +
> +	err = vdev->config->reset_vq(_vq);
> +	if (err)
> +		return err;
> +
> +	while ((buf = virtqueue_detach_unused_buf(_vq)) != NULL)
> +		recycle(_vq, buf);
> +
> +	if (packed) {
> +		err = virtqueue_resize_packed(_vq, num);
> +		if (err)
> +			virtqueue_reinit_packed(vq);


Calling reinit here seems a little bit odd, it looks more like a reset 
of the virtqueue. Consider we may re-use virtqueue reset for more 
purpose, I wonder if we need a helper like:

virtqueue_resize() {
     vdev->config->reset_vq(_vq);
     if (packed)
         virtqueue_reinit_packed(_vq)
     else
         virtqueue_reinit_split(_vq)
}

Thanks


> +	} else {
> +		err = virtqueue_resize_split(_vq, num);
> +		if (err)
> +			virtqueue_reinit_split(vq);
> +	}
> +
> +	if (vdev->config->enable_reset_vq(_vq))
> +		return -EBUSY;
> +
> +	return err;
> +}
> +EXPORT_SYMBOL_GPL(virtqueue_resize);
> +
>   /* Only available for split ring */
>   struct virtqueue *vring_new_virtqueue(unsigned int index,
>   				      unsigned int num,
> diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> index d59adc4be068..c86ff02e0ca0 100644
> --- a/include/linux/virtio.h
> +++ b/include/linux/virtio.h
> @@ -91,6 +91,9 @@ dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
>   dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
>   dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
>   
> +int virtqueue_resize(struct virtqueue *vq, u32 num,
> +		     void (*recycle)(struct virtqueue *vq, void *buf));
> +
>   /**
>    * virtio_device - representation of a device using virtio
>    * @index: unique position on the virtio bus

Re: [PATCH v9 18/32] virtio_ring: introduce virtqueue_resize()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce virtqueue_resize() to implement the resize of vring.
> Based on these, the driver can dynamically adjust the size of the vring.
> For example: ethtool -G.
>
> virtqueue_resize() implements resize based on the vq reset function. In
> case of failure to allocate a new vring, it will give up resize and use
> the original vring.
>
> During this process, if the re-enable reset vq fails, the vq can no
> longer be used. Although the probability of this situation is not high.
>
> The parameter recycle is used to recycle the buffer that is no longer
> used.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++++++++++
>   include/linux/virtio.h       |  3 ++
>   2 files changed, 72 insertions(+)
>
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 06f66b15c86c..6250e19fc5bf 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2554,6 +2554,75 @@ struct virtqueue *vring_create_virtqueue(
>   }
>   EXPORT_SYMBOL_GPL(vring_create_virtqueue);
>   
> +/**
> + * virtqueue_resize - resize the vring of vq
> + * @_vq: the struct virtqueue we're talking about.
> + * @num: new ring num
> + * @recycle: callback for recycle the useless buffer
> + *
> + * When it is really necessary to create a new vring, it will set the current vq
> + * into the reset state. Then call the passed callback to recycle the buffer
> + * that is no longer used. Only after the new vring is successfully created, the
> + * old vring will be released.
> + *
> + * Caller must ensure we don't call this with other virtqueue operations
> + * at the same time (except where noted).
> + *
> + * Returns zero or a negative error.


Should we document that the virtqueue is kept unchanged (still 
available) on (specific) failure?


> + */
> +int virtqueue_resize(struct virtqueue *_vq, u32 num,
> +		     void (*recycle)(struct virtqueue *vq, void *buf))
> +{
> +	struct vring_virtqueue *vq = to_vvq(_vq);
> +	struct virtio_device *vdev = vq->vq.vdev;
> +	bool packed;
> +	void *buf;
> +	int err;
> +
> +	if (!vq->we_own_ring)
> +		return -EINVAL;
> +
> +	if (num > vq->vq.num_max)
> +		return -E2BIG;
> +
> +	if (!num)
> +		return -EINVAL;
> +
> +	packed = virtio_has_feature(vdev, VIRTIO_F_RING_PACKED) ? true : false;
> +
> +	if ((packed ? vq->packed.vring.num : vq->split.vring.num) == num)
> +		return 0;
> +
> +	if (!vdev->config->reset_vq)
> +		return -ENOENT;
> +
> +	if (!vdev->config->enable_reset_vq)
> +		return -ENOENT;
> +
> +	err = vdev->config->reset_vq(_vq);
> +	if (err)
> +		return err;
> +
> +	while ((buf = virtqueue_detach_unused_buf(_vq)) != NULL)
> +		recycle(_vq, buf);
> +
> +	if (packed) {
> +		err = virtqueue_resize_packed(_vq, num);
> +		if (err)
> +			virtqueue_reinit_packed(vq);


Calling reinit here seems a little bit odd, it looks more like a reset 
of the virtqueue. Consider we may re-use virtqueue reset for more 
purpose, I wonder if we need a helper like:

virtqueue_resize() {
     vdev->config->reset_vq(_vq);
     if (packed)
         virtqueue_reinit_packed(_vq)
     else
         virtqueue_reinit_split(_vq)
}

Thanks


> +	} else {
> +		err = virtqueue_resize_split(_vq, num);
> +		if (err)
> +			virtqueue_reinit_split(vq);
> +	}
> +
> +	if (vdev->config->enable_reset_vq(_vq))
> +		return -EBUSY;
> +
> +	return err;
> +}
> +EXPORT_SYMBOL_GPL(virtqueue_resize);
> +
>   /* Only available for split ring */
>   struct virtqueue *vring_new_virtqueue(unsigned int index,
>   				      unsigned int num,
> diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> index d59adc4be068..c86ff02e0ca0 100644
> --- a/include/linux/virtio.h
> +++ b/include/linux/virtio.h
> @@ -91,6 +91,9 @@ dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
>   dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
>   dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
>   
> +int virtqueue_resize(struct virtqueue *vq, u32 num,
> +		     void (*recycle)(struct virtqueue *vq, void *buf));
> +
>   /**
>    * virtio_device - representation of a device using virtio
>    * @index: unique position on the virtio bus


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 21/32] virtio_pci: queue_reset: update struct virtio_pci_common_cfg and option functions

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Add queue_reset in virtio_pci_common_cfg, and add related operation
> functions.
>
> For not breaks uABI, add a new struct virtio_pci_common_cfg_reset.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_pci_modern_dev.c | 36 ++++++++++++++++++++++++++
>   include/linux/virtio_pci_modern.h      |  2 ++
>   include/uapi/linux/virtio_pci.h        |  7 +++++
>   3 files changed, 45 insertions(+)
>
> diff --git a/drivers/virtio/virtio_pci_modern_dev.c b/drivers/virtio/virtio_pci_modern_dev.c
> index e8b3ff2b9fbc..8c74b00bc511 100644
> --- a/drivers/virtio/virtio_pci_modern_dev.c
> +++ b/drivers/virtio/virtio_pci_modern_dev.c
> @@ -3,6 +3,7 @@
>   #include <linux/virtio_pci_modern.h>
>   #include <linux/module.h>
>   #include <linux/pci.h>
> +#include <linux/delay.h>
>   
>   /*
>    * vp_modern_map_capability - map a part of virtio pci capability
> @@ -463,6 +464,41 @@ void vp_modern_set_status(struct virtio_pci_modern_device *mdev,
>   }
>   EXPORT_SYMBOL_GPL(vp_modern_set_status);
>   
> +/*
> + * vp_modern_get_queue_reset - get the queue reset status
> + * @mdev: the modern virtio-pci device
> + * @index: queue index
> + */
> +int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index)
> +{
> +	struct virtio_pci_common_cfg_reset __iomem *cfg;
> +
> +	cfg = (struct virtio_pci_common_cfg_reset __iomem *)mdev->common;
> +
> +	vp_iowrite16(index, &cfg->cfg.queue_select);
> +	return vp_ioread16(&cfg->queue_reset);
> +}
> +EXPORT_SYMBOL_GPL(vp_modern_get_queue_reset);
> +
> +/*
> + * vp_modern_set_queue_reset - reset the queue
> + * @mdev: the modern virtio-pci device
> + * @index: queue index
> + */
> +void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index)
> +{
> +	struct virtio_pci_common_cfg_reset __iomem *cfg;
> +
> +	cfg = (struct virtio_pci_common_cfg_reset __iomem *)mdev->common;
> +
> +	vp_iowrite16(index, &cfg->cfg.queue_select);
> +	vp_iowrite16(1, &cfg->queue_reset);
> +
> +	while (vp_ioread16(&cfg->queue_reset) != 1)
> +		msleep(1);
> +}
> +EXPORT_SYMBOL_GPL(vp_modern_set_queue_reset);
> +
>   /*
>    * vp_modern_queue_vector - set the MSIX vector for a specific virtqueue
>    * @mdev: the modern virtio-pci device
> diff --git a/include/linux/virtio_pci_modern.h b/include/linux/virtio_pci_modern.h
> index eb2bd9b4077d..cc4154dd7b28 100644
> --- a/include/linux/virtio_pci_modern.h
> +++ b/include/linux/virtio_pci_modern.h
> @@ -106,4 +106,6 @@ void __iomem * vp_modern_map_vq_notify(struct virtio_pci_modern_device *mdev,
>   				       u16 index, resource_size_t *pa);
>   int vp_modern_probe(struct virtio_pci_modern_device *mdev);
>   void vp_modern_remove(struct virtio_pci_modern_device *mdev);
> +int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index);
> +void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index);
>   #endif
> diff --git a/include/uapi/linux/virtio_pci.h b/include/uapi/linux/virtio_pci.h
> index 22bec9bd0dfc..d9462efd6ce8 100644
> --- a/include/uapi/linux/virtio_pci.h
> +++ b/include/uapi/linux/virtio_pci.h
> @@ -173,6 +173,13 @@ struct virtio_pci_common_cfg_notify {
>   	__le16 padding;
>   };
>   
> +struct virtio_pci_common_cfg_reset {
> +	struct virtio_pci_common_cfg cfg;
> +
> +	__le16 queue_notify_data;	/* read-write */
> +	__le16 queue_reset;		/* read-write */
> +};


I prefer to use a separate patch for the uAPI change.

Other looks good.

Thanks


> +
>   /* Fields in VIRTIO_PCI_CAP_PCI_CFG: */
>   struct virtio_pci_cfg_cap {
>   	struct virtio_pci_cap cap;

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 21/32] virtio_pci: queue_reset: update struct virtio_pci_common_cfg and option functions

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Add queue_reset in virtio_pci_common_cfg, and add related operation
> functions.
>
> For not breaks uABI, add a new struct virtio_pci_common_cfg_reset.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_pci_modern_dev.c | 36 ++++++++++++++++++++++++++
>   include/linux/virtio_pci_modern.h      |  2 ++
>   include/uapi/linux/virtio_pci.h        |  7 +++++
>   3 files changed, 45 insertions(+)
>
> diff --git a/drivers/virtio/virtio_pci_modern_dev.c b/drivers/virtio/virtio_pci_modern_dev.c
> index e8b3ff2b9fbc..8c74b00bc511 100644
> --- a/drivers/virtio/virtio_pci_modern_dev.c
> +++ b/drivers/virtio/virtio_pci_modern_dev.c
> @@ -3,6 +3,7 @@
>   #include <linux/virtio_pci_modern.h>
>   #include <linux/module.h>
>   #include <linux/pci.h>
> +#include <linux/delay.h>
>   
>   /*
>    * vp_modern_map_capability - map a part of virtio pci capability
> @@ -463,6 +464,41 @@ void vp_modern_set_status(struct virtio_pci_modern_device *mdev,
>   }
>   EXPORT_SYMBOL_GPL(vp_modern_set_status);
>   
> +/*
> + * vp_modern_get_queue_reset - get the queue reset status
> + * @mdev: the modern virtio-pci device
> + * @index: queue index
> + */
> +int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index)
> +{
> +	struct virtio_pci_common_cfg_reset __iomem *cfg;
> +
> +	cfg = (struct virtio_pci_common_cfg_reset __iomem *)mdev->common;
> +
> +	vp_iowrite16(index, &cfg->cfg.queue_select);
> +	return vp_ioread16(&cfg->queue_reset);
> +}
> +EXPORT_SYMBOL_GPL(vp_modern_get_queue_reset);
> +
> +/*
> + * vp_modern_set_queue_reset - reset the queue
> + * @mdev: the modern virtio-pci device
> + * @index: queue index
> + */
> +void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index)
> +{
> +	struct virtio_pci_common_cfg_reset __iomem *cfg;
> +
> +	cfg = (struct virtio_pci_common_cfg_reset __iomem *)mdev->common;
> +
> +	vp_iowrite16(index, &cfg->cfg.queue_select);
> +	vp_iowrite16(1, &cfg->queue_reset);
> +
> +	while (vp_ioread16(&cfg->queue_reset) != 1)
> +		msleep(1);
> +}
> +EXPORT_SYMBOL_GPL(vp_modern_set_queue_reset);
> +
>   /*
>    * vp_modern_queue_vector - set the MSIX vector for a specific virtqueue
>    * @mdev: the modern virtio-pci device
> diff --git a/include/linux/virtio_pci_modern.h b/include/linux/virtio_pci_modern.h
> index eb2bd9b4077d..cc4154dd7b28 100644
> --- a/include/linux/virtio_pci_modern.h
> +++ b/include/linux/virtio_pci_modern.h
> @@ -106,4 +106,6 @@ void __iomem * vp_modern_map_vq_notify(struct virtio_pci_modern_device *mdev,
>   				       u16 index, resource_size_t *pa);
>   int vp_modern_probe(struct virtio_pci_modern_device *mdev);
>   void vp_modern_remove(struct virtio_pci_modern_device *mdev);
> +int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index);
> +void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index);
>   #endif
> diff --git a/include/uapi/linux/virtio_pci.h b/include/uapi/linux/virtio_pci.h
> index 22bec9bd0dfc..d9462efd6ce8 100644
> --- a/include/uapi/linux/virtio_pci.h
> +++ b/include/uapi/linux/virtio_pci.h
> @@ -173,6 +173,13 @@ struct virtio_pci_common_cfg_notify {
>   	__le16 padding;
>   };
>   
> +struct virtio_pci_common_cfg_reset {
> +	struct virtio_pci_common_cfg cfg;
> +
> +	__le16 queue_notify_data;	/* read-write */
> +	__le16 queue_reset;		/* read-write */
> +};


I prefer to use a separate patch for the uAPI change.

Other looks good.

Thanks


> +
>   /* Fields in VIRTIO_PCI_CAP_PCI_CFG: */
>   struct virtio_pci_cfg_cap {
>   	struct virtio_pci_cap cap;

Re: [PATCH v9 21/32] virtio_pci: queue_reset: update struct virtio_pci_common_cfg and option functions

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Add queue_reset in virtio_pci_common_cfg, and add related operation
> functions.
>
> For not breaks uABI, add a new struct virtio_pci_common_cfg_reset.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_pci_modern_dev.c | 36 ++++++++++++++++++++++++++
>   include/linux/virtio_pci_modern.h      |  2 ++
>   include/uapi/linux/virtio_pci.h        |  7 +++++
>   3 files changed, 45 insertions(+)
>
> diff --git a/drivers/virtio/virtio_pci_modern_dev.c b/drivers/virtio/virtio_pci_modern_dev.c
> index e8b3ff2b9fbc..8c74b00bc511 100644
> --- a/drivers/virtio/virtio_pci_modern_dev.c
> +++ b/drivers/virtio/virtio_pci_modern_dev.c
> @@ -3,6 +3,7 @@
>   #include <linux/virtio_pci_modern.h>
>   #include <linux/module.h>
>   #include <linux/pci.h>
> +#include <linux/delay.h>
>   
>   /*
>    * vp_modern_map_capability - map a part of virtio pci capability
> @@ -463,6 +464,41 @@ void vp_modern_set_status(struct virtio_pci_modern_device *mdev,
>   }
>   EXPORT_SYMBOL_GPL(vp_modern_set_status);
>   
> +/*
> + * vp_modern_get_queue_reset - get the queue reset status
> + * @mdev: the modern virtio-pci device
> + * @index: queue index
> + */
> +int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index)
> +{
> +	struct virtio_pci_common_cfg_reset __iomem *cfg;
> +
> +	cfg = (struct virtio_pci_common_cfg_reset __iomem *)mdev->common;
> +
> +	vp_iowrite16(index, &cfg->cfg.queue_select);
> +	return vp_ioread16(&cfg->queue_reset);
> +}
> +EXPORT_SYMBOL_GPL(vp_modern_get_queue_reset);
> +
> +/*
> + * vp_modern_set_queue_reset - reset the queue
> + * @mdev: the modern virtio-pci device
> + * @index: queue index
> + */
> +void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index)
> +{
> +	struct virtio_pci_common_cfg_reset __iomem *cfg;
> +
> +	cfg = (struct virtio_pci_common_cfg_reset __iomem *)mdev->common;
> +
> +	vp_iowrite16(index, &cfg->cfg.queue_select);
> +	vp_iowrite16(1, &cfg->queue_reset);
> +
> +	while (vp_ioread16(&cfg->queue_reset) != 1)
> +		msleep(1);
> +}
> +EXPORT_SYMBOL_GPL(vp_modern_set_queue_reset);
> +
>   /*
>    * vp_modern_queue_vector - set the MSIX vector for a specific virtqueue
>    * @mdev: the modern virtio-pci device
> diff --git a/include/linux/virtio_pci_modern.h b/include/linux/virtio_pci_modern.h
> index eb2bd9b4077d..cc4154dd7b28 100644
> --- a/include/linux/virtio_pci_modern.h
> +++ b/include/linux/virtio_pci_modern.h
> @@ -106,4 +106,6 @@ void __iomem * vp_modern_map_vq_notify(struct virtio_pci_modern_device *mdev,
>   				       u16 index, resource_size_t *pa);
>   int vp_modern_probe(struct virtio_pci_modern_device *mdev);
>   void vp_modern_remove(struct virtio_pci_modern_device *mdev);
> +int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index);
> +void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index);
>   #endif
> diff --git a/include/uapi/linux/virtio_pci.h b/include/uapi/linux/virtio_pci.h
> index 22bec9bd0dfc..d9462efd6ce8 100644
> --- a/include/uapi/linux/virtio_pci.h
> +++ b/include/uapi/linux/virtio_pci.h
> @@ -173,6 +173,13 @@ struct virtio_pci_common_cfg_notify {
>   	__le16 padding;
>   };
>   
> +struct virtio_pci_common_cfg_reset {
> +	struct virtio_pci_common_cfg cfg;
> +
> +	__le16 queue_notify_data;	/* read-write */
> +	__le16 queue_reset;		/* read-write */
> +};


I prefer to use a separate patch for the uAPI change.

Other looks good.

Thanks


> +
>   /* Fields in VIRTIO_PCI_CAP_PCI_CFG: */
>   struct virtio_pci_cfg_cap {
>   	struct virtio_pci_cap cap;


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 22/32] virtio_pci: queue_reset: extract the logic of active vq for modern pci

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce vp_active_vq() to configure vring to backend after vq attach
> vring. And configure vq vector if necessary.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_pci_modern.c | 46 ++++++++++++++++++------------
>   1 file changed, 28 insertions(+), 18 deletions(-)
>
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 86d301f272b8..49a4493732cf 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -176,6 +176,29 @@ static void vp_reset(struct virtio_device *vdev)
>   	vp_disable_cbs(vdev);
>   }
>   
> +static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> +{
> +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> +	unsigned long index;
> +
> +	index = vq->index;
> +
> +	/* activate the queue */
> +	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> +	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> +				virtqueue_get_avail_addr(vq),
> +				virtqueue_get_used_addr(vq));
> +
> +	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> +		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> +		if (msix_vec == VIRTIO_MSI_NO_VECTOR)
> +			return -EBUSY;
> +	}
> +
> +	return 0;
> +}
> +
>   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
>   {
>   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> @@ -220,32 +243,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   
>   	vq->num_max = num;
>   
> -	/* activate the queue */
> -	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> -	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> -				virtqueue_get_avail_addr(vq),
> -				virtqueue_get_used_addr(vq));
> +	err = vp_active_vq(vq, msix_vec);
> +	if (err)
> +		goto err;
>   
>   	vq->priv = (void __force *)vp_modern_map_vq_notify(mdev, index, NULL);
>   	if (!vq->priv) {
>   		err = -ENOMEM;
> -		goto err_map_notify;
> -	}
> -
> -	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> -		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> -		if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
> -			err = -EBUSY;
> -			goto err_assign_vector;
> -		}
> +		goto err;
>   	}
>   
>   	return vq;
>   
> -err_assign_vector:
> -	if (!mdev->notify_base)
> -		pci_iounmap(mdev->pci_dev, (void __iomem __force *)vq->priv);


We need keep this or anything I missed?

Thanks


> -err_map_notify:
> +err:
>   	vring_del_virtqueue(vq);
>   	return ERR_PTR(err);
>   }

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 22/32] virtio_pci: queue_reset: extract the logic of active vq for modern pci

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce vp_active_vq() to configure vring to backend after vq attach
> vring. And configure vq vector if necessary.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_pci_modern.c | 46 ++++++++++++++++++------------
>   1 file changed, 28 insertions(+), 18 deletions(-)
>
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 86d301f272b8..49a4493732cf 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -176,6 +176,29 @@ static void vp_reset(struct virtio_device *vdev)
>   	vp_disable_cbs(vdev);
>   }
>   
> +static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> +{
> +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> +	unsigned long index;
> +
> +	index = vq->index;
> +
> +	/* activate the queue */
> +	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> +	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> +				virtqueue_get_avail_addr(vq),
> +				virtqueue_get_used_addr(vq));
> +
> +	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> +		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> +		if (msix_vec == VIRTIO_MSI_NO_VECTOR)
> +			return -EBUSY;
> +	}
> +
> +	return 0;
> +}
> +
>   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
>   {
>   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> @@ -220,32 +243,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   
>   	vq->num_max = num;
>   
> -	/* activate the queue */
> -	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> -	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> -				virtqueue_get_avail_addr(vq),
> -				virtqueue_get_used_addr(vq));
> +	err = vp_active_vq(vq, msix_vec);
> +	if (err)
> +		goto err;
>   
>   	vq->priv = (void __force *)vp_modern_map_vq_notify(mdev, index, NULL);
>   	if (!vq->priv) {
>   		err = -ENOMEM;
> -		goto err_map_notify;
> -	}
> -
> -	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> -		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> -		if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
> -			err = -EBUSY;
> -			goto err_assign_vector;
> -		}
> +		goto err;
>   	}
>   
>   	return vq;
>   
> -err_assign_vector:
> -	if (!mdev->notify_base)
> -		pci_iounmap(mdev->pci_dev, (void __iomem __force *)vq->priv);


We need keep this or anything I missed?

Thanks


> -err_map_notify:
> +err:
>   	vring_del_virtqueue(vq);
>   	return ERR_PTR(err);
>   }

Re: [PATCH v9 22/32] virtio_pci: queue_reset: extract the logic of active vq for modern pci

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce vp_active_vq() to configure vring to backend after vq attach
> vring. And configure vq vector if necessary.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_pci_modern.c | 46 ++++++++++++++++++------------
>   1 file changed, 28 insertions(+), 18 deletions(-)
>
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 86d301f272b8..49a4493732cf 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -176,6 +176,29 @@ static void vp_reset(struct virtio_device *vdev)
>   	vp_disable_cbs(vdev);
>   }
>   
> +static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> +{
> +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> +	unsigned long index;
> +
> +	index = vq->index;
> +
> +	/* activate the queue */
> +	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> +	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> +				virtqueue_get_avail_addr(vq),
> +				virtqueue_get_used_addr(vq));
> +
> +	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> +		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> +		if (msix_vec == VIRTIO_MSI_NO_VECTOR)
> +			return -EBUSY;
> +	}
> +
> +	return 0;
> +}
> +
>   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
>   {
>   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> @@ -220,32 +243,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   
>   	vq->num_max = num;
>   
> -	/* activate the queue */
> -	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> -	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> -				virtqueue_get_avail_addr(vq),
> -				virtqueue_get_used_addr(vq));
> +	err = vp_active_vq(vq, msix_vec);
> +	if (err)
> +		goto err;
>   
>   	vq->priv = (void __force *)vp_modern_map_vq_notify(mdev, index, NULL);
>   	if (!vq->priv) {
>   		err = -ENOMEM;
> -		goto err_map_notify;
> -	}
> -
> -	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> -		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> -		if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
> -			err = -EBUSY;
> -			goto err_assign_vector;
> -		}
> +		goto err;
>   	}
>   
>   	return vq;
>   
> -err_assign_vector:
> -	if (!mdev->notify_base)
> -		pci_iounmap(mdev->pci_dev, (void __iomem __force *)vq->priv);


We need keep this or anything I missed?

Thanks


> -err_map_notify:
> +err:
>   	vring_del_virtqueue(vq);
>   	return ERR_PTR(err);
>   }


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> This patch implements virtio pci support for QUEUE RESET.
>
> Performing reset on a queue is divided into these steps:
>
>   1. notify the device to reset the queue
>   2. recycle the buffer submitted
>   3. reset the vring (may re-alloc)
>   4. mmap vring to device, and enable the queue
>
> This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
> pci scenario.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_pci_common.c |  8 +--
>   drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
>   drivers/virtio/virtio_ring.c       |  2 +
>   include/linux/virtio.h             |  1 +
>   4 files changed, 92 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> index fdbde1db5ec5..863d3a8a0956 100644
> --- a/drivers/virtio/virtio_pci_common.c
> +++ b/drivers/virtio/virtio_pci_common.c
> @@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
>   	struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
>   	unsigned long flags;
>   
> -	spin_lock_irqsave(&vp_dev->lock, flags);
> -	list_del(&info->node);
> -	spin_unlock_irqrestore(&vp_dev->lock, flags);
> +	if (!vq->reset) {


On which condition that we may hit this path?


> +		spin_lock_irqsave(&vp_dev->lock, flags);
> +		list_del(&info->node);
> +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> +	}
>   
>   	vp_dev->del_vq(info);
>   	kfree(info);
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 49a4493732cf..cb5d38f1c9c8 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
>   	if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
>   			pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
>   		__virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
> +
> +	if (features & BIT_ULL(VIRTIO_F_RING_RESET))
> +		__virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
>   }
>   
>   /* virtio config->finalize_features() implementation */
> @@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
>   	return 0;
>   }
>   
> +static int vp_modern_reset_vq(struct virtqueue *vq)
> +{
> +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> +	struct virtio_pci_vq_info *info;
> +	unsigned long flags;
> +
> +	if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
> +		return -ENOENT;
> +
> +	vp_modern_set_queue_reset(mdev, vq->index);
> +
> +	info = vp_dev->vqs[vq->index];
> +
> +	/* delete vq from irq handler */
> +	spin_lock_irqsave(&vp_dev->lock, flags);
> +	list_del(&info->node);
> +	spin_unlock_irqrestore(&vp_dev->lock, flags);
> +
> +	INIT_LIST_HEAD(&info->node);
> +
> +	/* For the case where vq has an exclusive irq, to prevent the irq from
> +	 * being received again and the pending irq, call disable_irq().
> +	 *
> +	 * In the scenario based on shared interrupts, vq will be searched from
> +	 * the queue virtqueues. Since the previous list_del() has been deleted
> +	 * from the queue, it is impossible for vq to be called in this case.
> +	 * There is no need to close the corresponding interrupt.
> +	 */
> +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> +		disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));


See the previous discussion and the revert of the first try to harden 
the interrupt. We probably can't use disable_irq() since it conflicts 
with the affinity managed IRQ that is used by some drivers.

We need to use synchonize_irq() and per virtqueue flag instead. As 
mentioned in previous patches, this could be done on top of my rework on 
the IRQ hardening .


> +
> +	vq->reset = true;
> +
> +	return 0;
> +}
> +
> +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> +{
> +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> +	struct virtio_pci_vq_info *info;
> +	unsigned long flags, index;
> +	int err;
> +
> +	if (!vq->reset)
> +		return -EBUSY;
> +
> +	index = vq->index;
> +	info = vp_dev->vqs[index];
> +
> +	/* check queue reset status */
> +	if (vp_modern_get_queue_reset(mdev, index) != 1)
> +		return -EBUSY;
> +
> +	err = vp_active_vq(vq, info->msix_vector);
> +	if (err)
> +		return err;
> +
> +	if (vq->callback) {
> +		spin_lock_irqsave(&vp_dev->lock, flags);
> +		list_add(&info->node, &vp_dev->virtqueues);
> +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> +	} else {
> +		INIT_LIST_HEAD(&info->node);
> +	}
> +
> +	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> +
> +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> +		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));


We had the same issue as disable_irq().

Thanks


> +
> +	vq->reset = false;
> +
> +	return 0;
> +}
> +
>   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
>   {
>   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> @@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
>   	.set_vq_affinity = vp_set_vq_affinity,
>   	.get_vq_affinity = vp_get_vq_affinity,
>   	.get_shm_region  = vp_get_shm_region,
> +	.reset_vq	 = vp_modern_reset_vq,
> +	.enable_reset_vq = vp_modern_enable_reset_vq,
>   };
>   
>   static const struct virtio_config_ops virtio_pci_config_ops = {
> @@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
>   	.set_vq_affinity = vp_set_vq_affinity,
>   	.get_vq_affinity = vp_get_vq_affinity,
>   	.get_shm_region  = vp_get_shm_region,
> +	.reset_vq	 = vp_modern_reset_vq,
> +	.enable_reset_vq = vp_modern_enable_reset_vq,
>   };
>   
>   /* the PCI probing function */
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 6250e19fc5bf..91937e21edca 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
>   	vq->vq.vdev = vdev;
>   	vq->vq.name = name;
>   	vq->vq.index = index;
> +	vq->vq.reset = false;
>   	vq->notify = notify;
>   	vq->weak_barriers = weak_barriers;
>   
> @@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	vq->vq.vdev = vdev;
>   	vq->vq.name = name;
>   	vq->vq.index = index;
> +	vq->vq.reset = false;
>   	vq->notify = notify;
>   	vq->weak_barriers = weak_barriers;
>   
> diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> index c86ff02e0ca0..33ab003c5100 100644
> --- a/include/linux/virtio.h
> +++ b/include/linux/virtio.h
> @@ -33,6 +33,7 @@ struct virtqueue {
>   	unsigned int num_free;
>   	unsigned int num_max;
>   	void *priv;
> +	bool reset;
>   };
>   
>   int virtqueue_add_outbuf(struct virtqueue *vq,

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> This patch implements virtio pci support for QUEUE RESET.
>
> Performing reset on a queue is divided into these steps:
>
>   1. notify the device to reset the queue
>   2. recycle the buffer submitted
>   3. reset the vring (may re-alloc)
>   4. mmap vring to device, and enable the queue
>
> This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
> pci scenario.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_pci_common.c |  8 +--
>   drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
>   drivers/virtio/virtio_ring.c       |  2 +
>   include/linux/virtio.h             |  1 +
>   4 files changed, 92 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> index fdbde1db5ec5..863d3a8a0956 100644
> --- a/drivers/virtio/virtio_pci_common.c
> +++ b/drivers/virtio/virtio_pci_common.c
> @@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
>   	struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
>   	unsigned long flags;
>   
> -	spin_lock_irqsave(&vp_dev->lock, flags);
> -	list_del(&info->node);
> -	spin_unlock_irqrestore(&vp_dev->lock, flags);
> +	if (!vq->reset) {


On which condition that we may hit this path?


> +		spin_lock_irqsave(&vp_dev->lock, flags);
> +		list_del(&info->node);
> +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> +	}
>   
>   	vp_dev->del_vq(info);
>   	kfree(info);
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 49a4493732cf..cb5d38f1c9c8 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
>   	if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
>   			pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
>   		__virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
> +
> +	if (features & BIT_ULL(VIRTIO_F_RING_RESET))
> +		__virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
>   }
>   
>   /* virtio config->finalize_features() implementation */
> @@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
>   	return 0;
>   }
>   
> +static int vp_modern_reset_vq(struct virtqueue *vq)
> +{
> +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> +	struct virtio_pci_vq_info *info;
> +	unsigned long flags;
> +
> +	if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
> +		return -ENOENT;
> +
> +	vp_modern_set_queue_reset(mdev, vq->index);
> +
> +	info = vp_dev->vqs[vq->index];
> +
> +	/* delete vq from irq handler */
> +	spin_lock_irqsave(&vp_dev->lock, flags);
> +	list_del(&info->node);
> +	spin_unlock_irqrestore(&vp_dev->lock, flags);
> +
> +	INIT_LIST_HEAD(&info->node);
> +
> +	/* For the case where vq has an exclusive irq, to prevent the irq from
> +	 * being received again and the pending irq, call disable_irq().
> +	 *
> +	 * In the scenario based on shared interrupts, vq will be searched from
> +	 * the queue virtqueues. Since the previous list_del() has been deleted
> +	 * from the queue, it is impossible for vq to be called in this case.
> +	 * There is no need to close the corresponding interrupt.
> +	 */
> +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> +		disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));


See the previous discussion and the revert of the first try to harden 
the interrupt. We probably can't use disable_irq() since it conflicts 
with the affinity managed IRQ that is used by some drivers.

We need to use synchonize_irq() and per virtqueue flag instead. As 
mentioned in previous patches, this could be done on top of my rework on 
the IRQ hardening .


> +
> +	vq->reset = true;
> +
> +	return 0;
> +}
> +
> +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> +{
> +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> +	struct virtio_pci_vq_info *info;
> +	unsigned long flags, index;
> +	int err;
> +
> +	if (!vq->reset)
> +		return -EBUSY;
> +
> +	index = vq->index;
> +	info = vp_dev->vqs[index];
> +
> +	/* check queue reset status */
> +	if (vp_modern_get_queue_reset(mdev, index) != 1)
> +		return -EBUSY;
> +
> +	err = vp_active_vq(vq, info->msix_vector);
> +	if (err)
> +		return err;
> +
> +	if (vq->callback) {
> +		spin_lock_irqsave(&vp_dev->lock, flags);
> +		list_add(&info->node, &vp_dev->virtqueues);
> +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> +	} else {
> +		INIT_LIST_HEAD(&info->node);
> +	}
> +
> +	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> +
> +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> +		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));


We had the same issue as disable_irq().

Thanks


> +
> +	vq->reset = false;
> +
> +	return 0;
> +}
> +
>   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
>   {
>   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> @@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
>   	.set_vq_affinity = vp_set_vq_affinity,
>   	.get_vq_affinity = vp_get_vq_affinity,
>   	.get_shm_region  = vp_get_shm_region,
> +	.reset_vq	 = vp_modern_reset_vq,
> +	.enable_reset_vq = vp_modern_enable_reset_vq,
>   };
>   
>   static const struct virtio_config_ops virtio_pci_config_ops = {
> @@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
>   	.set_vq_affinity = vp_set_vq_affinity,
>   	.get_vq_affinity = vp_get_vq_affinity,
>   	.get_shm_region  = vp_get_shm_region,
> +	.reset_vq	 = vp_modern_reset_vq,
> +	.enable_reset_vq = vp_modern_enable_reset_vq,
>   };
>   
>   /* the PCI probing function */
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 6250e19fc5bf..91937e21edca 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
>   	vq->vq.vdev = vdev;
>   	vq->vq.name = name;
>   	vq->vq.index = index;
> +	vq->vq.reset = false;
>   	vq->notify = notify;
>   	vq->weak_barriers = weak_barriers;
>   
> @@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	vq->vq.vdev = vdev;
>   	vq->vq.name = name;
>   	vq->vq.index = index;
> +	vq->vq.reset = false;
>   	vq->notify = notify;
>   	vq->weak_barriers = weak_barriers;
>   
> diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> index c86ff02e0ca0..33ab003c5100 100644
> --- a/include/linux/virtio.h
> +++ b/include/linux/virtio.h
> @@ -33,6 +33,7 @@ struct virtqueue {
>   	unsigned int num_free;
>   	unsigned int num_max;
>   	void *priv;
> +	bool reset;
>   };
>   
>   int virtqueue_add_outbuf(struct virtqueue *vq,

Re: [PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> This patch implements virtio pci support for QUEUE RESET.
>
> Performing reset on a queue is divided into these steps:
>
>   1. notify the device to reset the queue
>   2. recycle the buffer submitted
>   3. reset the vring (may re-alloc)
>   4. mmap vring to device, and enable the queue
>
> This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
> pci scenario.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/virtio/virtio_pci_common.c |  8 +--
>   drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
>   drivers/virtio/virtio_ring.c       |  2 +
>   include/linux/virtio.h             |  1 +
>   4 files changed, 92 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> index fdbde1db5ec5..863d3a8a0956 100644
> --- a/drivers/virtio/virtio_pci_common.c
> +++ b/drivers/virtio/virtio_pci_common.c
> @@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
>   	struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
>   	unsigned long flags;
>   
> -	spin_lock_irqsave(&vp_dev->lock, flags);
> -	list_del(&info->node);
> -	spin_unlock_irqrestore(&vp_dev->lock, flags);
> +	if (!vq->reset) {


On which condition that we may hit this path?


> +		spin_lock_irqsave(&vp_dev->lock, flags);
> +		list_del(&info->node);
> +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> +	}
>   
>   	vp_dev->del_vq(info);
>   	kfree(info);
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 49a4493732cf..cb5d38f1c9c8 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
>   	if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
>   			pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
>   		__virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
> +
> +	if (features & BIT_ULL(VIRTIO_F_RING_RESET))
> +		__virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
>   }
>   
>   /* virtio config->finalize_features() implementation */
> @@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
>   	return 0;
>   }
>   
> +static int vp_modern_reset_vq(struct virtqueue *vq)
> +{
> +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> +	struct virtio_pci_vq_info *info;
> +	unsigned long flags;
> +
> +	if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
> +		return -ENOENT;
> +
> +	vp_modern_set_queue_reset(mdev, vq->index);
> +
> +	info = vp_dev->vqs[vq->index];
> +
> +	/* delete vq from irq handler */
> +	spin_lock_irqsave(&vp_dev->lock, flags);
> +	list_del(&info->node);
> +	spin_unlock_irqrestore(&vp_dev->lock, flags);
> +
> +	INIT_LIST_HEAD(&info->node);
> +
> +	/* For the case where vq has an exclusive irq, to prevent the irq from
> +	 * being received again and the pending irq, call disable_irq().
> +	 *
> +	 * In the scenario based on shared interrupts, vq will be searched from
> +	 * the queue virtqueues. Since the previous list_del() has been deleted
> +	 * from the queue, it is impossible for vq to be called in this case.
> +	 * There is no need to close the corresponding interrupt.
> +	 */
> +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> +		disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));


See the previous discussion and the revert of the first try to harden 
the interrupt. We probably can't use disable_irq() since it conflicts 
with the affinity managed IRQ that is used by some drivers.

We need to use synchonize_irq() and per virtqueue flag instead. As 
mentioned in previous patches, this could be done on top of my rework on 
the IRQ hardening .


> +
> +	vq->reset = true;
> +
> +	return 0;
> +}
> +
> +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> +{
> +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> +	struct virtio_pci_vq_info *info;
> +	unsigned long flags, index;
> +	int err;
> +
> +	if (!vq->reset)
> +		return -EBUSY;
> +
> +	index = vq->index;
> +	info = vp_dev->vqs[index];
> +
> +	/* check queue reset status */
> +	if (vp_modern_get_queue_reset(mdev, index) != 1)
> +		return -EBUSY;
> +
> +	err = vp_active_vq(vq, info->msix_vector);
> +	if (err)
> +		return err;
> +
> +	if (vq->callback) {
> +		spin_lock_irqsave(&vp_dev->lock, flags);
> +		list_add(&info->node, &vp_dev->virtqueues);
> +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> +	} else {
> +		INIT_LIST_HEAD(&info->node);
> +	}
> +
> +	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> +
> +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> +		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));


We had the same issue as disable_irq().

Thanks


> +
> +	vq->reset = false;
> +
> +	return 0;
> +}
> +
>   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
>   {
>   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> @@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
>   	.set_vq_affinity = vp_set_vq_affinity,
>   	.get_vq_affinity = vp_get_vq_affinity,
>   	.get_shm_region  = vp_get_shm_region,
> +	.reset_vq	 = vp_modern_reset_vq,
> +	.enable_reset_vq = vp_modern_enable_reset_vq,
>   };
>   
>   static const struct virtio_config_ops virtio_pci_config_ops = {
> @@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
>   	.set_vq_affinity = vp_set_vq_affinity,
>   	.get_vq_affinity = vp_get_vq_affinity,
>   	.get_shm_region  = vp_get_shm_region,
> +	.reset_vq	 = vp_modern_reset_vq,
> +	.enable_reset_vq = vp_modern_enable_reset_vq,
>   };
>   
>   /* the PCI probing function */
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 6250e19fc5bf..91937e21edca 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
>   	vq->vq.vdev = vdev;
>   	vq->vq.name = name;
>   	vq->vq.index = index;
> +	vq->vq.reset = false;
>   	vq->notify = notify;
>   	vq->weak_barriers = weak_barriers;
>   
> @@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
>   	vq->vq.vdev = vdev;
>   	vq->vq.name = name;
>   	vq->vq.index = index;
> +	vq->vq.reset = false;
>   	vq->notify = notify;
>   	vq->weak_barriers = weak_barriers;
>   
> diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> index c86ff02e0ca0..33ab003c5100 100644
> --- a/include/linux/virtio.h
> +++ b/include/linux/virtio.h
> @@ -33,6 +33,7 @@ struct virtqueue {
>   	unsigned int num_free;
>   	unsigned int num_max;
>   	void *priv;
> +	bool reset;
>   };
>   
>   int virtqueue_add_outbuf(struct virtqueue *vq,


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 24/32] virtio: find_vqs() add arg sizes

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> find_vqs() adds a new parameter sizes to specify the size of each vq
> vring.
>
> 0 means use the maximum size supported by the backend.


Does this mean driver still need to prepare a array of 0 or it can 
simply pass NULL to find_vqs()?

Thanks


>
> In the split scenario, the meaning of size is the largest size, because
> it may be limited by memory, the virtio core will try a smaller size.
> And the size is power of 2.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> Acked-by: Hans de Goede <hdegoede@redhat.com>
> Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>   arch/um/drivers/virtio_uml.c             |  2 +-
>   drivers/platform/mellanox/mlxbf-tmfifo.c |  1 +
>   drivers/remoteproc/remoteproc_virtio.c   |  1 +
>   drivers/s390/virtio/virtio_ccw.c         |  1 +
>   drivers/virtio/virtio_mmio.c             |  1 +
>   drivers/virtio/virtio_pci_common.c       |  2 +-
>   drivers/virtio/virtio_pci_common.h       |  2 +-
>   drivers/virtio/virtio_pci_modern.c       |  7 +++++--
>   drivers/virtio/virtio_vdpa.c             |  1 +
>   include/linux/virtio_config.h            | 14 +++++++++-----
>   10 files changed, 22 insertions(+), 10 deletions(-)
>
> diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
> index 904993d15a85..6af98d130840 100644
> --- a/arch/um/drivers/virtio_uml.c
> +++ b/arch/um/drivers/virtio_uml.c
> @@ -998,7 +998,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
>   
>   static int vu_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		       struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		       const char * const names[], const bool *ctx,
> +		       const char * const names[], u32 sizes[], const bool *ctx,
>   		       struct irq_affinity *desc)
>   {
>   	struct virtio_uml_device *vu_dev = to_virtio_uml_device(vdev);
> diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
> index 1ae3c56b66b0..8be13d416f48 100644
> --- a/drivers/platform/mellanox/mlxbf-tmfifo.c
> +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
> @@ -928,6 +928,7 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
>   					struct virtqueue *vqs[],
>   					vq_callback_t *callbacks[],
>   					const char * const names[],
> +					u32 sizes[],
>   					const bool *ctx,
>   					struct irq_affinity *desc)
>   {
> diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
> index 7611755d0ae2..baad31c9da45 100644
> --- a/drivers/remoteproc/remoteproc_virtio.c
> +++ b/drivers/remoteproc/remoteproc_virtio.c
> @@ -158,6 +158,7 @@ static int rproc_virtio_find_vqs(struct virtio_device *vdev, unsigned int nvqs,
>   				 struct virtqueue *vqs[],
>   				 vq_callback_t *callbacks[],
>   				 const char * const names[],
> +				 u32 sizes[],
>   				 const bool * ctx,
>   				 struct irq_affinity *desc)
>   {
> diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
> index 468da60b56c5..f0c814a54e78 100644
> --- a/drivers/s390/virtio/virtio_ccw.c
> +++ b/drivers/s390/virtio/virtio_ccw.c
> @@ -634,6 +634,7 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   			       struct virtqueue *vqs[],
>   			       vq_callback_t *callbacks[],
>   			       const char * const names[],
> +			       u32 sizes[],
>   			       const bool *ctx,
>   			       struct irq_affinity *desc)
>   {
> diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> index a41abc8051b9..9d5a674bdeec 100644
> --- a/drivers/virtio/virtio_mmio.c
> +++ b/drivers/virtio/virtio_mmio.c
> @@ -461,6 +461,7 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		       struct virtqueue *vqs[],
>   		       vq_callback_t *callbacks[],
>   		       const char * const names[],
> +		       u32 sizes[],
>   		       const bool *ctx,
>   		       struct irq_affinity *desc)
>   {
> diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> index 863d3a8a0956..826ea2e35d54 100644
> --- a/drivers/virtio/virtio_pci_common.c
> +++ b/drivers/virtio/virtio_pci_common.c
> @@ -427,7 +427,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
>   /* the config->find_vqs() implementation */
>   int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		const char * const names[], const bool *ctx,
> +		const char * const names[], u32 sizes[], const bool *ctx,
>   		struct irq_affinity *desc)
>   {
>   	int err;
> diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
> index 23f6c5c678d5..859eed559e10 100644
> --- a/drivers/virtio/virtio_pci_common.h
> +++ b/drivers/virtio/virtio_pci_common.h
> @@ -113,7 +113,7 @@ void vp_del_vqs(struct virtio_device *vdev);
>   /* the config->find_vqs() implementation */
>   int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		const char * const names[], const bool *ctx,
> +		const char * const names[], u32 sizes[], const bool *ctx,
>   		struct irq_affinity *desc);
>   const char *vp_bus_name(struct virtio_device *vdev);
>   
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index cb5d38f1c9c8..3b35e5056165 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -343,12 +343,15 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   static int vp_modern_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   			      struct virtqueue *vqs[],
>   			      vq_callback_t *callbacks[],
> -			      const char * const names[], const bool *ctx,
> +			      const char * const names[],
> +			      u32 sizes[],
> +			      const bool *ctx,
>   			      struct irq_affinity *desc)
>   {
>   	struct virtio_pci_device *vp_dev = to_vp_device(vdev);
>   	struct virtqueue *vq;
> -	int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, ctx, desc);
> +	int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, sizes, ctx,
> +			     desc);
>   
>   	if (rc)
>   		return rc;
> diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
> index 39e4c08eb0f2..ac86d3a6633c 100644
> --- a/drivers/virtio/virtio_vdpa.c
> +++ b/drivers/virtio/virtio_vdpa.c
> @@ -269,6 +269,7 @@ static int virtio_vdpa_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   				struct virtqueue *vqs[],
>   				vq_callback_t *callbacks[],
>   				const char * const names[],
> +				u32 sizes[],
>   				const bool *ctx,
>   				struct irq_affinity *desc)
>   {
> diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> index d4adcd0e1c57..0f7def7ddfd2 100644
> --- a/include/linux/virtio_config.h
> +++ b/include/linux/virtio_config.h
> @@ -57,6 +57,7 @@ struct virtio_shm_region {
>    *		include a NULL entry for vqs that do not need a callback
>    *	names: array of virtqueue names (mainly for debugging)
>    *		include a NULL entry for vqs unused by driver
> + *	sizes: array of virtqueue sizes
>    *	Returns 0 on success or error status
>    * @del_vqs: free virtqueues found by find_vqs().
>    * @get_features: get the array of feature bits for this device.
> @@ -98,7 +99,9 @@ struct virtio_config_ops {
>   	void (*reset)(struct virtio_device *vdev);
>   	int (*find_vqs)(struct virtio_device *, unsigned nvqs,
>   			struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -			const char * const names[], const bool *ctx,
> +			const char * const names[],
> +			u32 sizes[],
> +			const bool *ctx,
>   			struct irq_affinity *desc);
>   	void (*del_vqs)(struct virtio_device *);
>   	u64 (*get_features)(struct virtio_device *vdev);
> @@ -206,7 +209,7 @@ struct virtqueue *virtio_find_single_vq(struct virtio_device *vdev,
>   	const char *names[] = { n };
>   	struct virtqueue *vq;
>   	int err = vdev->config->find_vqs(vdev, 1, &vq, callbacks, names, NULL,
> -					 NULL);
> +					 NULL, NULL);
>   	if (err < 0)
>   		return ERR_PTR(err);
>   	return vq;
> @@ -218,7 +221,8 @@ int virtio_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   			const char * const names[],
>   			struct irq_affinity *desc)
>   {
> -	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL, desc);
> +	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL,
> +				      NULL, desc);
>   }
>   
>   static inline
> @@ -227,8 +231,8 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
>   			const char * const names[], const bool *ctx,
>   			struct irq_affinity *desc)
>   {
> -	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, ctx,
> -				      desc);
> +	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL,
> +				      ctx, desc);
>   }
>   
>   /**

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 24/32] virtio: find_vqs() add arg sizes

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> find_vqs() adds a new parameter sizes to specify the size of each vq
> vring.
>
> 0 means use the maximum size supported by the backend.


Does this mean driver still need to prepare a array of 0 or it can 
simply pass NULL to find_vqs()?

Thanks


>
> In the split scenario, the meaning of size is the largest size, because
> it may be limited by memory, the virtio core will try a smaller size.
> And the size is power of 2.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> Acked-by: Hans de Goede <hdegoede@redhat.com>
> Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>   arch/um/drivers/virtio_uml.c             |  2 +-
>   drivers/platform/mellanox/mlxbf-tmfifo.c |  1 +
>   drivers/remoteproc/remoteproc_virtio.c   |  1 +
>   drivers/s390/virtio/virtio_ccw.c         |  1 +
>   drivers/virtio/virtio_mmio.c             |  1 +
>   drivers/virtio/virtio_pci_common.c       |  2 +-
>   drivers/virtio/virtio_pci_common.h       |  2 +-
>   drivers/virtio/virtio_pci_modern.c       |  7 +++++--
>   drivers/virtio/virtio_vdpa.c             |  1 +
>   include/linux/virtio_config.h            | 14 +++++++++-----
>   10 files changed, 22 insertions(+), 10 deletions(-)
>
> diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
> index 904993d15a85..6af98d130840 100644
> --- a/arch/um/drivers/virtio_uml.c
> +++ b/arch/um/drivers/virtio_uml.c
> @@ -998,7 +998,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
>   
>   static int vu_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		       struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		       const char * const names[], const bool *ctx,
> +		       const char * const names[], u32 sizes[], const bool *ctx,
>   		       struct irq_affinity *desc)
>   {
>   	struct virtio_uml_device *vu_dev = to_virtio_uml_device(vdev);
> diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
> index 1ae3c56b66b0..8be13d416f48 100644
> --- a/drivers/platform/mellanox/mlxbf-tmfifo.c
> +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
> @@ -928,6 +928,7 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
>   					struct virtqueue *vqs[],
>   					vq_callback_t *callbacks[],
>   					const char * const names[],
> +					u32 sizes[],
>   					const bool *ctx,
>   					struct irq_affinity *desc)
>   {
> diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
> index 7611755d0ae2..baad31c9da45 100644
> --- a/drivers/remoteproc/remoteproc_virtio.c
> +++ b/drivers/remoteproc/remoteproc_virtio.c
> @@ -158,6 +158,7 @@ static int rproc_virtio_find_vqs(struct virtio_device *vdev, unsigned int nvqs,
>   				 struct virtqueue *vqs[],
>   				 vq_callback_t *callbacks[],
>   				 const char * const names[],
> +				 u32 sizes[],
>   				 const bool * ctx,
>   				 struct irq_affinity *desc)
>   {
> diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
> index 468da60b56c5..f0c814a54e78 100644
> --- a/drivers/s390/virtio/virtio_ccw.c
> +++ b/drivers/s390/virtio/virtio_ccw.c
> @@ -634,6 +634,7 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   			       struct virtqueue *vqs[],
>   			       vq_callback_t *callbacks[],
>   			       const char * const names[],
> +			       u32 sizes[],
>   			       const bool *ctx,
>   			       struct irq_affinity *desc)
>   {
> diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> index a41abc8051b9..9d5a674bdeec 100644
> --- a/drivers/virtio/virtio_mmio.c
> +++ b/drivers/virtio/virtio_mmio.c
> @@ -461,6 +461,7 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		       struct virtqueue *vqs[],
>   		       vq_callback_t *callbacks[],
>   		       const char * const names[],
> +		       u32 sizes[],
>   		       const bool *ctx,
>   		       struct irq_affinity *desc)
>   {
> diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> index 863d3a8a0956..826ea2e35d54 100644
> --- a/drivers/virtio/virtio_pci_common.c
> +++ b/drivers/virtio/virtio_pci_common.c
> @@ -427,7 +427,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
>   /* the config->find_vqs() implementation */
>   int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		const char * const names[], const bool *ctx,
> +		const char * const names[], u32 sizes[], const bool *ctx,
>   		struct irq_affinity *desc)
>   {
>   	int err;
> diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
> index 23f6c5c678d5..859eed559e10 100644
> --- a/drivers/virtio/virtio_pci_common.h
> +++ b/drivers/virtio/virtio_pci_common.h
> @@ -113,7 +113,7 @@ void vp_del_vqs(struct virtio_device *vdev);
>   /* the config->find_vqs() implementation */
>   int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		const char * const names[], const bool *ctx,
> +		const char * const names[], u32 sizes[], const bool *ctx,
>   		struct irq_affinity *desc);
>   const char *vp_bus_name(struct virtio_device *vdev);
>   
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index cb5d38f1c9c8..3b35e5056165 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -343,12 +343,15 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   static int vp_modern_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   			      struct virtqueue *vqs[],
>   			      vq_callback_t *callbacks[],
> -			      const char * const names[], const bool *ctx,
> +			      const char * const names[],
> +			      u32 sizes[],
> +			      const bool *ctx,
>   			      struct irq_affinity *desc)
>   {
>   	struct virtio_pci_device *vp_dev = to_vp_device(vdev);
>   	struct virtqueue *vq;
> -	int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, ctx, desc);
> +	int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, sizes, ctx,
> +			     desc);
>   
>   	if (rc)
>   		return rc;
> diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
> index 39e4c08eb0f2..ac86d3a6633c 100644
> --- a/drivers/virtio/virtio_vdpa.c
> +++ b/drivers/virtio/virtio_vdpa.c
> @@ -269,6 +269,7 @@ static int virtio_vdpa_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   				struct virtqueue *vqs[],
>   				vq_callback_t *callbacks[],
>   				const char * const names[],
> +				u32 sizes[],
>   				const bool *ctx,
>   				struct irq_affinity *desc)
>   {
> diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> index d4adcd0e1c57..0f7def7ddfd2 100644
> --- a/include/linux/virtio_config.h
> +++ b/include/linux/virtio_config.h
> @@ -57,6 +57,7 @@ struct virtio_shm_region {
>    *		include a NULL entry for vqs that do not need a callback
>    *	names: array of virtqueue names (mainly for debugging)
>    *		include a NULL entry for vqs unused by driver
> + *	sizes: array of virtqueue sizes
>    *	Returns 0 on success or error status
>    * @del_vqs: free virtqueues found by find_vqs().
>    * @get_features: get the array of feature bits for this device.
> @@ -98,7 +99,9 @@ struct virtio_config_ops {
>   	void (*reset)(struct virtio_device *vdev);
>   	int (*find_vqs)(struct virtio_device *, unsigned nvqs,
>   			struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -			const char * const names[], const bool *ctx,
> +			const char * const names[],
> +			u32 sizes[],
> +			const bool *ctx,
>   			struct irq_affinity *desc);
>   	void (*del_vqs)(struct virtio_device *);
>   	u64 (*get_features)(struct virtio_device *vdev);
> @@ -206,7 +209,7 @@ struct virtqueue *virtio_find_single_vq(struct virtio_device *vdev,
>   	const char *names[] = { n };
>   	struct virtqueue *vq;
>   	int err = vdev->config->find_vqs(vdev, 1, &vq, callbacks, names, NULL,
> -					 NULL);
> +					 NULL, NULL);
>   	if (err < 0)
>   		return ERR_PTR(err);
>   	return vq;
> @@ -218,7 +221,8 @@ int virtio_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   			const char * const names[],
>   			struct irq_affinity *desc)
>   {
> -	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL, desc);
> +	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL,
> +				      NULL, desc);
>   }
>   
>   static inline
> @@ -227,8 +231,8 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
>   			const char * const names[], const bool *ctx,
>   			struct irq_affinity *desc)
>   {
> -	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, ctx,
> -				      desc);
> +	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL,
> +				      ctx, desc);
>   }
>   
>   /**

Re: [PATCH v9 24/32] virtio: find_vqs() add arg sizes

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> find_vqs() adds a new parameter sizes to specify the size of each vq
> vring.
>
> 0 means use the maximum size supported by the backend.


Does this mean driver still need to prepare a array of 0 or it can 
simply pass NULL to find_vqs()?

Thanks


>
> In the split scenario, the meaning of size is the largest size, because
> it may be limited by memory, the virtio core will try a smaller size.
> And the size is power of 2.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> Acked-by: Hans de Goede <hdegoede@redhat.com>
> Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>   arch/um/drivers/virtio_uml.c             |  2 +-
>   drivers/platform/mellanox/mlxbf-tmfifo.c |  1 +
>   drivers/remoteproc/remoteproc_virtio.c   |  1 +
>   drivers/s390/virtio/virtio_ccw.c         |  1 +
>   drivers/virtio/virtio_mmio.c             |  1 +
>   drivers/virtio/virtio_pci_common.c       |  2 +-
>   drivers/virtio/virtio_pci_common.h       |  2 +-
>   drivers/virtio/virtio_pci_modern.c       |  7 +++++--
>   drivers/virtio/virtio_vdpa.c             |  1 +
>   include/linux/virtio_config.h            | 14 +++++++++-----
>   10 files changed, 22 insertions(+), 10 deletions(-)
>
> diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
> index 904993d15a85..6af98d130840 100644
> --- a/arch/um/drivers/virtio_uml.c
> +++ b/arch/um/drivers/virtio_uml.c
> @@ -998,7 +998,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
>   
>   static int vu_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		       struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		       const char * const names[], const bool *ctx,
> +		       const char * const names[], u32 sizes[], const bool *ctx,
>   		       struct irq_affinity *desc)
>   {
>   	struct virtio_uml_device *vu_dev = to_virtio_uml_device(vdev);
> diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
> index 1ae3c56b66b0..8be13d416f48 100644
> --- a/drivers/platform/mellanox/mlxbf-tmfifo.c
> +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
> @@ -928,6 +928,7 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
>   					struct virtqueue *vqs[],
>   					vq_callback_t *callbacks[],
>   					const char * const names[],
> +					u32 sizes[],
>   					const bool *ctx,
>   					struct irq_affinity *desc)
>   {
> diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
> index 7611755d0ae2..baad31c9da45 100644
> --- a/drivers/remoteproc/remoteproc_virtio.c
> +++ b/drivers/remoteproc/remoteproc_virtio.c
> @@ -158,6 +158,7 @@ static int rproc_virtio_find_vqs(struct virtio_device *vdev, unsigned int nvqs,
>   				 struct virtqueue *vqs[],
>   				 vq_callback_t *callbacks[],
>   				 const char * const names[],
> +				 u32 sizes[],
>   				 const bool * ctx,
>   				 struct irq_affinity *desc)
>   {
> diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
> index 468da60b56c5..f0c814a54e78 100644
> --- a/drivers/s390/virtio/virtio_ccw.c
> +++ b/drivers/s390/virtio/virtio_ccw.c
> @@ -634,6 +634,7 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   			       struct virtqueue *vqs[],
>   			       vq_callback_t *callbacks[],
>   			       const char * const names[],
> +			       u32 sizes[],
>   			       const bool *ctx,
>   			       struct irq_affinity *desc)
>   {
> diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> index a41abc8051b9..9d5a674bdeec 100644
> --- a/drivers/virtio/virtio_mmio.c
> +++ b/drivers/virtio/virtio_mmio.c
> @@ -461,6 +461,7 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		       struct virtqueue *vqs[],
>   		       vq_callback_t *callbacks[],
>   		       const char * const names[],
> +		       u32 sizes[],
>   		       const bool *ctx,
>   		       struct irq_affinity *desc)
>   {
> diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> index 863d3a8a0956..826ea2e35d54 100644
> --- a/drivers/virtio/virtio_pci_common.c
> +++ b/drivers/virtio/virtio_pci_common.c
> @@ -427,7 +427,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
>   /* the config->find_vqs() implementation */
>   int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		const char * const names[], const bool *ctx,
> +		const char * const names[], u32 sizes[], const bool *ctx,
>   		struct irq_affinity *desc)
>   {
>   	int err;
> diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
> index 23f6c5c678d5..859eed559e10 100644
> --- a/drivers/virtio/virtio_pci_common.h
> +++ b/drivers/virtio/virtio_pci_common.h
> @@ -113,7 +113,7 @@ void vp_del_vqs(struct virtio_device *vdev);
>   /* the config->find_vqs() implementation */
>   int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		const char * const names[], const bool *ctx,
> +		const char * const names[], u32 sizes[], const bool *ctx,
>   		struct irq_affinity *desc);
>   const char *vp_bus_name(struct virtio_device *vdev);
>   
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index cb5d38f1c9c8..3b35e5056165 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -343,12 +343,15 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   static int vp_modern_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   			      struct virtqueue *vqs[],
>   			      vq_callback_t *callbacks[],
> -			      const char * const names[], const bool *ctx,
> +			      const char * const names[],
> +			      u32 sizes[],
> +			      const bool *ctx,
>   			      struct irq_affinity *desc)
>   {
>   	struct virtio_pci_device *vp_dev = to_vp_device(vdev);
>   	struct virtqueue *vq;
> -	int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, ctx, desc);
> +	int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, sizes, ctx,
> +			     desc);
>   
>   	if (rc)
>   		return rc;
> diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
> index 39e4c08eb0f2..ac86d3a6633c 100644
> --- a/drivers/virtio/virtio_vdpa.c
> +++ b/drivers/virtio/virtio_vdpa.c
> @@ -269,6 +269,7 @@ static int virtio_vdpa_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   				struct virtqueue *vqs[],
>   				vq_callback_t *callbacks[],
>   				const char * const names[],
> +				u32 sizes[],
>   				const bool *ctx,
>   				struct irq_affinity *desc)
>   {
> diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> index d4adcd0e1c57..0f7def7ddfd2 100644
> --- a/include/linux/virtio_config.h
> +++ b/include/linux/virtio_config.h
> @@ -57,6 +57,7 @@ struct virtio_shm_region {
>    *		include a NULL entry for vqs that do not need a callback
>    *	names: array of virtqueue names (mainly for debugging)
>    *		include a NULL entry for vqs unused by driver
> + *	sizes: array of virtqueue sizes
>    *	Returns 0 on success or error status
>    * @del_vqs: free virtqueues found by find_vqs().
>    * @get_features: get the array of feature bits for this device.
> @@ -98,7 +99,9 @@ struct virtio_config_ops {
>   	void (*reset)(struct virtio_device *vdev);
>   	int (*find_vqs)(struct virtio_device *, unsigned nvqs,
>   			struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -			const char * const names[], const bool *ctx,
> +			const char * const names[],
> +			u32 sizes[],
> +			const bool *ctx,
>   			struct irq_affinity *desc);
>   	void (*del_vqs)(struct virtio_device *);
>   	u64 (*get_features)(struct virtio_device *vdev);
> @@ -206,7 +209,7 @@ struct virtqueue *virtio_find_single_vq(struct virtio_device *vdev,
>   	const char *names[] = { n };
>   	struct virtqueue *vq;
>   	int err = vdev->config->find_vqs(vdev, 1, &vq, callbacks, names, NULL,
> -					 NULL);
> +					 NULL, NULL);
>   	if (err < 0)
>   		return ERR_PTR(err);
>   	return vq;
> @@ -218,7 +221,8 @@ int virtio_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   			const char * const names[],
>   			struct irq_affinity *desc)
>   {
> -	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL, desc);
> +	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL,
> +				      NULL, desc);
>   }
>   
>   static inline
> @@ -227,8 +231,8 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
>   			const char * const names[], const bool *ctx,
>   			struct irq_affinity *desc)
>   {
> -	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, ctx,
> -				      desc);
> +	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL,
> +				      ctx, desc);
>   }
>   
>   /**


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 25/32] virtio_pci: support the arg sizes of find_vqs()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Virtio PCI supports new parameter sizes of find_vqs().
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_pci_common.c | 18 ++++++++++--------
>   drivers/virtio/virtio_pci_common.h |  1 +
>   drivers/virtio/virtio_pci_legacy.c |  6 +++++-
>   drivers/virtio/virtio_pci_modern.c | 10 +++++++---
>   4 files changed, 23 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> index 826ea2e35d54..23976c61583f 100644
> --- a/drivers/virtio/virtio_pci_common.c
> +++ b/drivers/virtio/virtio_pci_common.c
> @@ -208,6 +208,7 @@ static int vp_request_msix_vectors(struct virtio_device *vdev, int nvectors,
>   static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index,
>   				     void (*callback)(struct virtqueue *vq),
>   				     const char *name,
> +				     u32 size,
>   				     bool ctx,
>   				     u16 msix_vec)
>   {
> @@ -220,7 +221,7 @@ static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index,
>   	if (!info)
>   		return ERR_PTR(-ENOMEM);
>   
> -	vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, ctx,
> +	vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, size, ctx,
>   			      msix_vec);
>   	if (IS_ERR(vq))
>   		goto out_info;
> @@ -314,7 +315,7 @@ void vp_del_vqs(struct virtio_device *vdev)
>   
>   static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
>   		struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		const char * const names[], bool per_vq_vectors,
> +		const char * const names[], u32 sizes[], bool per_vq_vectors,
>   		const bool *ctx,
>   		struct irq_affinity *desc)
>   {
> @@ -357,8 +358,8 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
>   		else
>   			msix_vec = VP_MSIX_VQ_VECTOR;
>   		vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
> -				     ctx ? ctx[i] : false,
> -				     msix_vec);
> +				     sizes ? sizes[i] : 0,
> +				     ctx ? ctx[i] : false, msix_vec);
>   		if (IS_ERR(vqs[i])) {
>   			err = PTR_ERR(vqs[i]);
>   			goto error_find;
> @@ -388,7 +389,7 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
>   
>   static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
>   		struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		const char * const names[], const bool *ctx)
> +		const char * const names[], u32 sizes[], const bool *ctx)
>   {
>   	struct virtio_pci_device *vp_dev = to_vp_device(vdev);
>   	int i, err, queue_idx = 0;
> @@ -410,6 +411,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
>   			continue;
>   		}
>   		vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
> +				     sizes ? sizes[i] : 0,
>   				     ctx ? ctx[i] : false,
>   				     VIRTIO_MSI_NO_VECTOR);
>   		if (IS_ERR(vqs[i])) {
> @@ -433,15 +435,15 @@ int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   	int err;
>   
>   	/* Try MSI-X with one vector per queue. */
> -	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, true, ctx, desc);
> +	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, sizes, true, ctx, desc);
>   	if (!err)
>   		return 0;
>   	/* Fallback: MSI-X with one vector for config, one shared for queues. */
> -	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, false, ctx, desc);
> +	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, sizes, false, ctx, desc);
>   	if (!err)
>   		return 0;
>   	/* Finally fall back to regular interrupts. */
> -	return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, ctx);
> +	return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, sizes, ctx);
>   }
>   
>   const char *vp_bus_name(struct virtio_device *vdev)
> diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
> index 859eed559e10..fbf5a6d4b164 100644
> --- a/drivers/virtio/virtio_pci_common.h
> +++ b/drivers/virtio/virtio_pci_common.h
> @@ -81,6 +81,7 @@ struct virtio_pci_device {
>   				      unsigned idx,
>   				      void (*callback)(struct virtqueue *vq),
>   				      const char *name,
> +				      u32 size,
>   				      bool ctx,
>   				      u16 msix_vec);
>   	void (*del_vq)(struct virtio_pci_vq_info *info);
> diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
> index b68934fe6b5d..2c4ade5fb420 100644
> --- a/drivers/virtio/virtio_pci_legacy.c
> +++ b/drivers/virtio/virtio_pci_legacy.c
> @@ -112,6 +112,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   				  unsigned index,
>   				  void (*callback)(struct virtqueue *vq),
>   				  const char *name,
> +				  u32 size,
>   				  bool ctx,
>   				  u16 msix_vec)
>   {
> @@ -125,10 +126,13 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   	if (!num || vp_legacy_get_queue_enable(&vp_dev->ldev, index))
>   		return ERR_PTR(-ENOENT);
>   
> +	if (!size || size > num)
> +		size = num;
> +
>   	info->msix_vector = msix_vec;
>   
>   	/* create the vring */
> -	vq = vring_create_virtqueue(index, num,
> +	vq = vring_create_virtqueue(index, size,
>   				    VIRTIO_PCI_VRING_ALIGN, &vp_dev->vdev,
>   				    true, false, ctx,
>   				    vp_notify, callback, name);
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 3b35e5056165..a17c47d4435a 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -289,6 +289,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   				  unsigned index,
>   				  void (*callback)(struct virtqueue *vq),
>   				  const char *name,
> +				  u32 size,
>   				  bool ctx,
>   				  u16 msix_vec)
>   {
> @@ -306,15 +307,18 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   	if (!num || vp_modern_get_queue_enable(mdev, index))
>   		return ERR_PTR(-ENOENT);
>   
> -	if (num & (num - 1)) {
> -		dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", num);
> +	if (!size || size > num)
> +		size = num;
> +
> +	if (size & (size - 1)) {
> +		dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", size);
>   		return ERR_PTR(-EINVAL);
>   	}
>   
>   	info->msix_vector = msix_vec;
>   
>   	/* create the vring */
> -	vq = vring_create_virtqueue(index, num,
> +	vq = vring_create_virtqueue(index, size,
>   				    SMP_CACHE_BYTES, &vp_dev->vdev,
>   				    true, true, ctx,
>   				    vp_notify, callback, name);

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 25/32] virtio_pci: support the arg sizes of find_vqs()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Virtio PCI supports new parameter sizes of find_vqs().
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_pci_common.c | 18 ++++++++++--------
>   drivers/virtio/virtio_pci_common.h |  1 +
>   drivers/virtio/virtio_pci_legacy.c |  6 +++++-
>   drivers/virtio/virtio_pci_modern.c | 10 +++++++---
>   4 files changed, 23 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> index 826ea2e35d54..23976c61583f 100644
> --- a/drivers/virtio/virtio_pci_common.c
> +++ b/drivers/virtio/virtio_pci_common.c
> @@ -208,6 +208,7 @@ static int vp_request_msix_vectors(struct virtio_device *vdev, int nvectors,
>   static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index,
>   				     void (*callback)(struct virtqueue *vq),
>   				     const char *name,
> +				     u32 size,
>   				     bool ctx,
>   				     u16 msix_vec)
>   {
> @@ -220,7 +221,7 @@ static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index,
>   	if (!info)
>   		return ERR_PTR(-ENOMEM);
>   
> -	vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, ctx,
> +	vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, size, ctx,
>   			      msix_vec);
>   	if (IS_ERR(vq))
>   		goto out_info;
> @@ -314,7 +315,7 @@ void vp_del_vqs(struct virtio_device *vdev)
>   
>   static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
>   		struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		const char * const names[], bool per_vq_vectors,
> +		const char * const names[], u32 sizes[], bool per_vq_vectors,
>   		const bool *ctx,
>   		struct irq_affinity *desc)
>   {
> @@ -357,8 +358,8 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
>   		else
>   			msix_vec = VP_MSIX_VQ_VECTOR;
>   		vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
> -				     ctx ? ctx[i] : false,
> -				     msix_vec);
> +				     sizes ? sizes[i] : 0,
> +				     ctx ? ctx[i] : false, msix_vec);
>   		if (IS_ERR(vqs[i])) {
>   			err = PTR_ERR(vqs[i]);
>   			goto error_find;
> @@ -388,7 +389,7 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
>   
>   static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
>   		struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		const char * const names[], const bool *ctx)
> +		const char * const names[], u32 sizes[], const bool *ctx)
>   {
>   	struct virtio_pci_device *vp_dev = to_vp_device(vdev);
>   	int i, err, queue_idx = 0;
> @@ -410,6 +411,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
>   			continue;
>   		}
>   		vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
> +				     sizes ? sizes[i] : 0,
>   				     ctx ? ctx[i] : false,
>   				     VIRTIO_MSI_NO_VECTOR);
>   		if (IS_ERR(vqs[i])) {
> @@ -433,15 +435,15 @@ int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   	int err;
>   
>   	/* Try MSI-X with one vector per queue. */
> -	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, true, ctx, desc);
> +	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, sizes, true, ctx, desc);
>   	if (!err)
>   		return 0;
>   	/* Fallback: MSI-X with one vector for config, one shared for queues. */
> -	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, false, ctx, desc);
> +	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, sizes, false, ctx, desc);
>   	if (!err)
>   		return 0;
>   	/* Finally fall back to regular interrupts. */
> -	return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, ctx);
> +	return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, sizes, ctx);
>   }
>   
>   const char *vp_bus_name(struct virtio_device *vdev)
> diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
> index 859eed559e10..fbf5a6d4b164 100644
> --- a/drivers/virtio/virtio_pci_common.h
> +++ b/drivers/virtio/virtio_pci_common.h
> @@ -81,6 +81,7 @@ struct virtio_pci_device {
>   				      unsigned idx,
>   				      void (*callback)(struct virtqueue *vq),
>   				      const char *name,
> +				      u32 size,
>   				      bool ctx,
>   				      u16 msix_vec);
>   	void (*del_vq)(struct virtio_pci_vq_info *info);
> diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
> index b68934fe6b5d..2c4ade5fb420 100644
> --- a/drivers/virtio/virtio_pci_legacy.c
> +++ b/drivers/virtio/virtio_pci_legacy.c
> @@ -112,6 +112,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   				  unsigned index,
>   				  void (*callback)(struct virtqueue *vq),
>   				  const char *name,
> +				  u32 size,
>   				  bool ctx,
>   				  u16 msix_vec)
>   {
> @@ -125,10 +126,13 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   	if (!num || vp_legacy_get_queue_enable(&vp_dev->ldev, index))
>   		return ERR_PTR(-ENOENT);
>   
> +	if (!size || size > num)
> +		size = num;
> +
>   	info->msix_vector = msix_vec;
>   
>   	/* create the vring */
> -	vq = vring_create_virtqueue(index, num,
> +	vq = vring_create_virtqueue(index, size,
>   				    VIRTIO_PCI_VRING_ALIGN, &vp_dev->vdev,
>   				    true, false, ctx,
>   				    vp_notify, callback, name);
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 3b35e5056165..a17c47d4435a 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -289,6 +289,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   				  unsigned index,
>   				  void (*callback)(struct virtqueue *vq),
>   				  const char *name,
> +				  u32 size,
>   				  bool ctx,
>   				  u16 msix_vec)
>   {
> @@ -306,15 +307,18 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   	if (!num || vp_modern_get_queue_enable(mdev, index))
>   		return ERR_PTR(-ENOENT);
>   
> -	if (num & (num - 1)) {
> -		dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", num);
> +	if (!size || size > num)
> +		size = num;
> +
> +	if (size & (size - 1)) {
> +		dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", size);
>   		return ERR_PTR(-EINVAL);
>   	}
>   
>   	info->msix_vector = msix_vec;
>   
>   	/* create the vring */
> -	vq = vring_create_virtqueue(index, num,
> +	vq = vring_create_virtqueue(index, size,
>   				    SMP_CACHE_BYTES, &vp_dev->vdev,
>   				    true, true, ctx,
>   				    vp_notify, callback, name);

Re: [PATCH v9 25/32] virtio_pci: support the arg sizes of find_vqs()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Virtio PCI supports new parameter sizes of find_vqs().
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/virtio/virtio_pci_common.c | 18 ++++++++++--------
>   drivers/virtio/virtio_pci_common.h |  1 +
>   drivers/virtio/virtio_pci_legacy.c |  6 +++++-
>   drivers/virtio/virtio_pci_modern.c | 10 +++++++---
>   4 files changed, 23 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> index 826ea2e35d54..23976c61583f 100644
> --- a/drivers/virtio/virtio_pci_common.c
> +++ b/drivers/virtio/virtio_pci_common.c
> @@ -208,6 +208,7 @@ static int vp_request_msix_vectors(struct virtio_device *vdev, int nvectors,
>   static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index,
>   				     void (*callback)(struct virtqueue *vq),
>   				     const char *name,
> +				     u32 size,
>   				     bool ctx,
>   				     u16 msix_vec)
>   {
> @@ -220,7 +221,7 @@ static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index,
>   	if (!info)
>   		return ERR_PTR(-ENOMEM);
>   
> -	vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, ctx,
> +	vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, size, ctx,
>   			      msix_vec);
>   	if (IS_ERR(vq))
>   		goto out_info;
> @@ -314,7 +315,7 @@ void vp_del_vqs(struct virtio_device *vdev)
>   
>   static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
>   		struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		const char * const names[], bool per_vq_vectors,
> +		const char * const names[], u32 sizes[], bool per_vq_vectors,
>   		const bool *ctx,
>   		struct irq_affinity *desc)
>   {
> @@ -357,8 +358,8 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
>   		else
>   			msix_vec = VP_MSIX_VQ_VECTOR;
>   		vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
> -				     ctx ? ctx[i] : false,
> -				     msix_vec);
> +				     sizes ? sizes[i] : 0,
> +				     ctx ? ctx[i] : false, msix_vec);
>   		if (IS_ERR(vqs[i])) {
>   			err = PTR_ERR(vqs[i]);
>   			goto error_find;
> @@ -388,7 +389,7 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs,
>   
>   static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
>   		struct virtqueue *vqs[], vq_callback_t *callbacks[],
> -		const char * const names[], const bool *ctx)
> +		const char * const names[], u32 sizes[], const bool *ctx)
>   {
>   	struct virtio_pci_device *vp_dev = to_vp_device(vdev);
>   	int i, err, queue_idx = 0;
> @@ -410,6 +411,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs,
>   			continue;
>   		}
>   		vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
> +				     sizes ? sizes[i] : 0,
>   				     ctx ? ctx[i] : false,
>   				     VIRTIO_MSI_NO_VECTOR);
>   		if (IS_ERR(vqs[i])) {
> @@ -433,15 +435,15 @@ int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   	int err;
>   
>   	/* Try MSI-X with one vector per queue. */
> -	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, true, ctx, desc);
> +	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, sizes, true, ctx, desc);
>   	if (!err)
>   		return 0;
>   	/* Fallback: MSI-X with one vector for config, one shared for queues. */
> -	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, false, ctx, desc);
> +	err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, sizes, false, ctx, desc);
>   	if (!err)
>   		return 0;
>   	/* Finally fall back to regular interrupts. */
> -	return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, ctx);
> +	return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, sizes, ctx);
>   }
>   
>   const char *vp_bus_name(struct virtio_device *vdev)
> diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
> index 859eed559e10..fbf5a6d4b164 100644
> --- a/drivers/virtio/virtio_pci_common.h
> +++ b/drivers/virtio/virtio_pci_common.h
> @@ -81,6 +81,7 @@ struct virtio_pci_device {
>   				      unsigned idx,
>   				      void (*callback)(struct virtqueue *vq),
>   				      const char *name,
> +				      u32 size,
>   				      bool ctx,
>   				      u16 msix_vec);
>   	void (*del_vq)(struct virtio_pci_vq_info *info);
> diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
> index b68934fe6b5d..2c4ade5fb420 100644
> --- a/drivers/virtio/virtio_pci_legacy.c
> +++ b/drivers/virtio/virtio_pci_legacy.c
> @@ -112,6 +112,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   				  unsigned index,
>   				  void (*callback)(struct virtqueue *vq),
>   				  const char *name,
> +				  u32 size,
>   				  bool ctx,
>   				  u16 msix_vec)
>   {
> @@ -125,10 +126,13 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   	if (!num || vp_legacy_get_queue_enable(&vp_dev->ldev, index))
>   		return ERR_PTR(-ENOENT);
>   
> +	if (!size || size > num)
> +		size = num;
> +
>   	info->msix_vector = msix_vec;
>   
>   	/* create the vring */
> -	vq = vring_create_virtqueue(index, num,
> +	vq = vring_create_virtqueue(index, size,
>   				    VIRTIO_PCI_VRING_ALIGN, &vp_dev->vdev,
>   				    true, false, ctx,
>   				    vp_notify, callback, name);
> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 3b35e5056165..a17c47d4435a 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -289,6 +289,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   				  unsigned index,
>   				  void (*callback)(struct virtqueue *vq),
>   				  const char *name,
> +				  u32 size,
>   				  bool ctx,
>   				  u16 msix_vec)
>   {
> @@ -306,15 +307,18 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>   	if (!num || vp_modern_get_queue_enable(mdev, index))
>   		return ERR_PTR(-ENOENT);
>   
> -	if (num & (num - 1)) {
> -		dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", num);
> +	if (!size || size > num)
> +		size = num;
> +
> +	if (size & (size - 1)) {
> +		dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", size);
>   		return ERR_PTR(-EINVAL);
>   	}
>   
>   	info->msix_vector = msix_vec;
>   
>   	/* create the vring */
> -	vq = vring_create_virtqueue(index, num,
> +	vq = vring_create_virtqueue(index, size,
>   				    SMP_CACHE_BYTES, &vp_dev->vdev,
>   				    true, true, ctx,
>   				    vp_notify, callback, name);


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 26/32] virtio_mmio: support the arg sizes of find_vqs()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Virtio MMIO support the new parameter sizes of find_vqs().
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---


Acked-by: Jason Wang <jasowang@redhat.com>


>   drivers/virtio/virtio_mmio.c | 8 ++++++--
>   1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> index 9d5a674bdeec..51cf51764a92 100644
> --- a/drivers/virtio/virtio_mmio.c
> +++ b/drivers/virtio/virtio_mmio.c
> @@ -347,7 +347,7 @@ static void vm_del_vqs(struct virtio_device *vdev)
>   
>   static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
>   				  void (*callback)(struct virtqueue *vq),
> -				  const char *name, bool ctx)
> +				  const char *name, u32 size, bool ctx)
>   {
>   	struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
>   	struct virtio_mmio_vq_info *info;
> @@ -382,8 +382,11 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
>   		goto error_new_virtqueue;
>   	}
>   
> +	if (!size || size > num)
> +		size = num;
> +
>   	/* Create the vring */
> -	vq = vring_create_virtqueue(index, num, VIRTIO_MMIO_VRING_ALIGN, vdev,
> +	vq = vring_create_virtqueue(index, size, VIRTIO_MMIO_VRING_ALIGN, vdev,
>   				 true, true, ctx, vm_notify, callback, name);
>   	if (!vq) {
>   		err = -ENOMEM;
> @@ -484,6 +487,7 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		}
>   
>   		vqs[i] = vm_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
> +				     sizes ? sizes[i] : 0,
>   				     ctx ? ctx[i] : false);
>   		if (IS_ERR(vqs[i])) {
>   			vm_del_vqs(vdev);

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 26/32] virtio_mmio: support the arg sizes of find_vqs()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Virtio MMIO support the new parameter sizes of find_vqs().
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---


Acked-by: Jason Wang <jasowang@redhat.com>


>   drivers/virtio/virtio_mmio.c | 8 ++++++--
>   1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> index 9d5a674bdeec..51cf51764a92 100644
> --- a/drivers/virtio/virtio_mmio.c
> +++ b/drivers/virtio/virtio_mmio.c
> @@ -347,7 +347,7 @@ static void vm_del_vqs(struct virtio_device *vdev)
>   
>   static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
>   				  void (*callback)(struct virtqueue *vq),
> -				  const char *name, bool ctx)
> +				  const char *name, u32 size, bool ctx)
>   {
>   	struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
>   	struct virtio_mmio_vq_info *info;
> @@ -382,8 +382,11 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
>   		goto error_new_virtqueue;
>   	}
>   
> +	if (!size || size > num)
> +		size = num;
> +
>   	/* Create the vring */
> -	vq = vring_create_virtqueue(index, num, VIRTIO_MMIO_VRING_ALIGN, vdev,
> +	vq = vring_create_virtqueue(index, size, VIRTIO_MMIO_VRING_ALIGN, vdev,
>   				 true, true, ctx, vm_notify, callback, name);
>   	if (!vq) {
>   		err = -ENOMEM;
> @@ -484,6 +487,7 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		}
>   
>   		vqs[i] = vm_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
> +				     sizes ? sizes[i] : 0,
>   				     ctx ? ctx[i] : false);
>   		if (IS_ERR(vqs[i])) {
>   			vm_del_vqs(vdev);

Re: [PATCH v9 26/32] virtio_mmio: support the arg sizes of find_vqs()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Virtio MMIO support the new parameter sizes of find_vqs().
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---


Acked-by: Jason Wang <jasowang@redhat.com>


>   drivers/virtio/virtio_mmio.c | 8 ++++++--
>   1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> index 9d5a674bdeec..51cf51764a92 100644
> --- a/drivers/virtio/virtio_mmio.c
> +++ b/drivers/virtio/virtio_mmio.c
> @@ -347,7 +347,7 @@ static void vm_del_vqs(struct virtio_device *vdev)
>   
>   static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
>   				  void (*callback)(struct virtqueue *vq),
> -				  const char *name, bool ctx)
> +				  const char *name, u32 size, bool ctx)
>   {
>   	struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
>   	struct virtio_mmio_vq_info *info;
> @@ -382,8 +382,11 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
>   		goto error_new_virtqueue;
>   	}
>   
> +	if (!size || size > num)
> +		size = num;
> +
>   	/* Create the vring */
> -	vq = vring_create_virtqueue(index, num, VIRTIO_MMIO_VRING_ALIGN, vdev,
> +	vq = vring_create_virtqueue(index, size, VIRTIO_MMIO_VRING_ALIGN, vdev,
>   				 true, true, ctx, vm_notify, callback, name);
>   	if (!vq) {
>   		err = -ENOMEM;
> @@ -484,6 +487,7 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned nvqs,
>   		}
>   
>   		vqs[i] = vm_setup_vq(vdev, queue_idx++, callbacks[i], names[i],
> +				     sizes ? sizes[i] : 0,
>   				     ctx ? ctx[i] : false);
>   		if (IS_ERR(vqs[i])) {
>   			vm_del_vqs(vdev);


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 27/32] virtio: add helper virtio_find_vqs_ctx_size()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce helper virtio_find_vqs_ctx_size() to call find_vqs and specify
> the maximum size of each vq ring.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   include/linux/virtio_config.h | 12 ++++++++++++
>   1 file changed, 12 insertions(+)
>
> diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> index 0f7def7ddfd2..22e29c926946 100644
> --- a/include/linux/virtio_config.h
> +++ b/include/linux/virtio_config.h
> @@ -235,6 +235,18 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
>   				      ctx, desc);
>   }
>   
> +static inline
> +int virtio_find_vqs_ctx_size(struct virtio_device *vdev, u32 nvqs,
> +				 struct virtqueue *vqs[],
> +				 vq_callback_t *callbacks[],
> +				 const char * const names[],
> +				 u32 sizes[],
> +				 const bool *ctx, struct irq_affinity *desc)
> +{
> +	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, sizes,
> +				      ctx, desc);
> +}
> +
>   /**
>    * virtio_device_ready - enable vq use in probe function
>    * @vdev: the device

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 27/32] virtio: add helper virtio_find_vqs_ctx_size()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce helper virtio_find_vqs_ctx_size() to call find_vqs and specify
> the maximum size of each vq ring.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   include/linux/virtio_config.h | 12 ++++++++++++
>   1 file changed, 12 insertions(+)
>
> diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> index 0f7def7ddfd2..22e29c926946 100644
> --- a/include/linux/virtio_config.h
> +++ b/include/linux/virtio_config.h
> @@ -235,6 +235,18 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
>   				      ctx, desc);
>   }
>   
> +static inline
> +int virtio_find_vqs_ctx_size(struct virtio_device *vdev, u32 nvqs,
> +				 struct virtqueue *vqs[],
> +				 vq_callback_t *callbacks[],
> +				 const char * const names[],
> +				 u32 sizes[],
> +				 const bool *ctx, struct irq_affinity *desc)
> +{
> +	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, sizes,
> +				      ctx, desc);
> +}
> +
>   /**
>    * virtio_device_ready - enable vq use in probe function
>    * @vdev: the device

Re: [PATCH v9 27/32] virtio: add helper virtio_find_vqs_ctx_size()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Introduce helper virtio_find_vqs_ctx_size() to call find_vqs and specify
> the maximum size of each vq ring.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   include/linux/virtio_config.h | 12 ++++++++++++
>   1 file changed, 12 insertions(+)
>
> diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> index 0f7def7ddfd2..22e29c926946 100644
> --- a/include/linux/virtio_config.h
> +++ b/include/linux/virtio_config.h
> @@ -235,6 +235,18 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
>   				      ctx, desc);
>   }
>   
> +static inline
> +int virtio_find_vqs_ctx_size(struct virtio_device *vdev, u32 nvqs,
> +				 struct virtqueue *vqs[],
> +				 vq_callback_t *callbacks[],
> +				 const char * const names[],
> +				 u32 sizes[],
> +				 const bool *ctx, struct irq_affinity *desc)
> +{
> +	return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, sizes,
> +				      ctx, desc);
> +}
> +
>   /**
>    * virtio_device_ready - enable vq use in probe function
>    * @vdev: the device


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 28/32] virtio_net: set the default max ring size by find_vqs()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Use virtio_find_vqs_ctx_size() to specify the maximum ring size of tx,
> rx at the same time.
>
>                           | rx/tx ring size
> -------------------------------------------
> speed == UNKNOWN or < 10G| 1024
> speed < 40G              | 4096
> speed >= 40G             | 8192
>
> Call virtnet_update_settings() once before calling init_vqs() to update
> speed.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---


Acked-by: Jason Wang <jasowang@redhat.com>


>   drivers/net/virtio_net.c | 42 ++++++++++++++++++++++++++++++++++++----
>   1 file changed, 38 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index a801ea40908f..dad497a47b3a 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -2861,6 +2861,29 @@ static unsigned int mergeable_min_buf_len(struct virtnet_info *vi, struct virtqu
>   		   (unsigned int)GOOD_PACKET_LEN);
>   }
>   
> +static void virtnet_config_sizes(struct virtnet_info *vi, u32 *sizes)
> +{
> +	u32 i, rx_size, tx_size;
> +
> +	if (vi->speed == SPEED_UNKNOWN || vi->speed < SPEED_10000) {
> +		rx_size = 1024;
> +		tx_size = 1024;
> +
> +	} else if (vi->speed < SPEED_40000) {
> +		rx_size = 1024 * 4;
> +		tx_size = 1024 * 4;
> +
> +	} else {
> +		rx_size = 1024 * 8;
> +		tx_size = 1024 * 8;
> +	}
> +
> +	for (i = 0; i < vi->max_queue_pairs; i++) {
> +		sizes[rxq2vq(i)] = rx_size;
> +		sizes[txq2vq(i)] = tx_size;
> +	}
> +}
> +
>   static int virtnet_find_vqs(struct virtnet_info *vi)
>   {
>   	vq_callback_t **callbacks;
> @@ -2868,6 +2891,7 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>   	int ret = -ENOMEM;
>   	int i, total_vqs;
>   	const char **names;
> +	u32 *sizes;
>   	bool *ctx;
>   
>   	/* We expect 1 RX virtqueue followed by 1 TX virtqueue, followed by
> @@ -2895,10 +2919,15 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>   		ctx = NULL;
>   	}
>   
> +	sizes = kmalloc_array(total_vqs, sizeof(*sizes), GFP_KERNEL);
> +	if (!sizes)
> +		goto err_sizes;
> +
>   	/* Parameters for control virtqueue, if any */
>   	if (vi->has_cvq) {
>   		callbacks[total_vqs - 1] = NULL;
>   		names[total_vqs - 1] = "control";
> +		sizes[total_vqs - 1] = 64;
>   	}
>   
>   	/* Allocate/initialize parameters for send/receive virtqueues */
> @@ -2913,8 +2942,10 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>   			ctx[rxq2vq(i)] = true;
>   	}
>   
> -	ret = virtio_find_vqs_ctx(vi->vdev, total_vqs, vqs, callbacks,
> -				  names, ctx, NULL);
> +	virtnet_config_sizes(vi, sizes);
> +
> +	ret = virtio_find_vqs_ctx_size(vi->vdev, total_vqs, vqs, callbacks,
> +				       names, sizes, ctx, NULL);
>   	if (ret)
>   		goto err_find;
>   
> @@ -2934,6 +2965,8 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>   
>   
>   err_find:
> +	kfree(sizes);
> +err_sizes:
>   	kfree(ctx);
>   err_ctx:
>   	kfree(names);
> @@ -3252,6 +3285,9 @@ static int virtnet_probe(struct virtio_device *vdev)
>   		vi->curr_queue_pairs = num_online_cpus();
>   	vi->max_queue_pairs = max_queue_pairs;
>   
> +	virtnet_init_settings(dev);
> +	virtnet_update_settings(vi);
> +
>   	/* Allocate/initialize the rx/tx queues, and invoke find_vqs */
>   	err = init_vqs(vi);
>   	if (err)
> @@ -3264,8 +3300,6 @@ static int virtnet_probe(struct virtio_device *vdev)
>   	netif_set_real_num_tx_queues(dev, vi->curr_queue_pairs);
>   	netif_set_real_num_rx_queues(dev, vi->curr_queue_pairs);
>   
> -	virtnet_init_settings(dev);
> -
>   	if (virtio_has_feature(vdev, VIRTIO_NET_F_STANDBY)) {
>   		vi->failover = net_failover_create(vi->dev);
>   		if (IS_ERR(vi->failover)) {

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 28/32] virtio_net: set the default max ring size by find_vqs()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Use virtio_find_vqs_ctx_size() to specify the maximum ring size of tx,
> rx at the same time.
>
>                           | rx/tx ring size
> -------------------------------------------
> speed == UNKNOWN or < 10G| 1024
> speed < 40G              | 4096
> speed >= 40G             | 8192
>
> Call virtnet_update_settings() once before calling init_vqs() to update
> speed.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---


Acked-by: Jason Wang <jasowang@redhat.com>


>   drivers/net/virtio_net.c | 42 ++++++++++++++++++++++++++++++++++++----
>   1 file changed, 38 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index a801ea40908f..dad497a47b3a 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -2861,6 +2861,29 @@ static unsigned int mergeable_min_buf_len(struct virtnet_info *vi, struct virtqu
>   		   (unsigned int)GOOD_PACKET_LEN);
>   }
>   
> +static void virtnet_config_sizes(struct virtnet_info *vi, u32 *sizes)
> +{
> +	u32 i, rx_size, tx_size;
> +
> +	if (vi->speed == SPEED_UNKNOWN || vi->speed < SPEED_10000) {
> +		rx_size = 1024;
> +		tx_size = 1024;
> +
> +	} else if (vi->speed < SPEED_40000) {
> +		rx_size = 1024 * 4;
> +		tx_size = 1024 * 4;
> +
> +	} else {
> +		rx_size = 1024 * 8;
> +		tx_size = 1024 * 8;
> +	}
> +
> +	for (i = 0; i < vi->max_queue_pairs; i++) {
> +		sizes[rxq2vq(i)] = rx_size;
> +		sizes[txq2vq(i)] = tx_size;
> +	}
> +}
> +
>   static int virtnet_find_vqs(struct virtnet_info *vi)
>   {
>   	vq_callback_t **callbacks;
> @@ -2868,6 +2891,7 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>   	int ret = -ENOMEM;
>   	int i, total_vqs;
>   	const char **names;
> +	u32 *sizes;
>   	bool *ctx;
>   
>   	/* We expect 1 RX virtqueue followed by 1 TX virtqueue, followed by
> @@ -2895,10 +2919,15 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>   		ctx = NULL;
>   	}
>   
> +	sizes = kmalloc_array(total_vqs, sizeof(*sizes), GFP_KERNEL);
> +	if (!sizes)
> +		goto err_sizes;
> +
>   	/* Parameters for control virtqueue, if any */
>   	if (vi->has_cvq) {
>   		callbacks[total_vqs - 1] = NULL;
>   		names[total_vqs - 1] = "control";
> +		sizes[total_vqs - 1] = 64;
>   	}
>   
>   	/* Allocate/initialize parameters for send/receive virtqueues */
> @@ -2913,8 +2942,10 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>   			ctx[rxq2vq(i)] = true;
>   	}
>   
> -	ret = virtio_find_vqs_ctx(vi->vdev, total_vqs, vqs, callbacks,
> -				  names, ctx, NULL);
> +	virtnet_config_sizes(vi, sizes);
> +
> +	ret = virtio_find_vqs_ctx_size(vi->vdev, total_vqs, vqs, callbacks,
> +				       names, sizes, ctx, NULL);
>   	if (ret)
>   		goto err_find;
>   
> @@ -2934,6 +2965,8 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>   
>   
>   err_find:
> +	kfree(sizes);
> +err_sizes:
>   	kfree(ctx);
>   err_ctx:
>   	kfree(names);
> @@ -3252,6 +3285,9 @@ static int virtnet_probe(struct virtio_device *vdev)
>   		vi->curr_queue_pairs = num_online_cpus();
>   	vi->max_queue_pairs = max_queue_pairs;
>   
> +	virtnet_init_settings(dev);
> +	virtnet_update_settings(vi);
> +
>   	/* Allocate/initialize the rx/tx queues, and invoke find_vqs */
>   	err = init_vqs(vi);
>   	if (err)
> @@ -3264,8 +3300,6 @@ static int virtnet_probe(struct virtio_device *vdev)
>   	netif_set_real_num_tx_queues(dev, vi->curr_queue_pairs);
>   	netif_set_real_num_rx_queues(dev, vi->curr_queue_pairs);
>   
> -	virtnet_init_settings(dev);
> -
>   	if (virtio_has_feature(vdev, VIRTIO_NET_F_STANDBY)) {
>   		vi->failover = net_failover_create(vi->dev);
>   		if (IS_ERR(vi->failover)) {

Re: [PATCH v9 28/32] virtio_net: set the default max ring size by find_vqs()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Use virtio_find_vqs_ctx_size() to specify the maximum ring size of tx,
> rx at the same time.
>
>                           | rx/tx ring size
> -------------------------------------------
> speed == UNKNOWN or < 10G| 1024
> speed < 40G              | 4096
> speed >= 40G             | 8192
>
> Call virtnet_update_settings() once before calling init_vqs() to update
> speed.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---


Acked-by: Jason Wang <jasowang@redhat.com>


>   drivers/net/virtio_net.c | 42 ++++++++++++++++++++++++++++++++++++----
>   1 file changed, 38 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index a801ea40908f..dad497a47b3a 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -2861,6 +2861,29 @@ static unsigned int mergeable_min_buf_len(struct virtnet_info *vi, struct virtqu
>   		   (unsigned int)GOOD_PACKET_LEN);
>   }
>   
> +static void virtnet_config_sizes(struct virtnet_info *vi, u32 *sizes)
> +{
> +	u32 i, rx_size, tx_size;
> +
> +	if (vi->speed == SPEED_UNKNOWN || vi->speed < SPEED_10000) {
> +		rx_size = 1024;
> +		tx_size = 1024;
> +
> +	} else if (vi->speed < SPEED_40000) {
> +		rx_size = 1024 * 4;
> +		tx_size = 1024 * 4;
> +
> +	} else {
> +		rx_size = 1024 * 8;
> +		tx_size = 1024 * 8;
> +	}
> +
> +	for (i = 0; i < vi->max_queue_pairs; i++) {
> +		sizes[rxq2vq(i)] = rx_size;
> +		sizes[txq2vq(i)] = tx_size;
> +	}
> +}
> +
>   static int virtnet_find_vqs(struct virtnet_info *vi)
>   {
>   	vq_callback_t **callbacks;
> @@ -2868,6 +2891,7 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>   	int ret = -ENOMEM;
>   	int i, total_vqs;
>   	const char **names;
> +	u32 *sizes;
>   	bool *ctx;
>   
>   	/* We expect 1 RX virtqueue followed by 1 TX virtqueue, followed by
> @@ -2895,10 +2919,15 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>   		ctx = NULL;
>   	}
>   
> +	sizes = kmalloc_array(total_vqs, sizeof(*sizes), GFP_KERNEL);
> +	if (!sizes)
> +		goto err_sizes;
> +
>   	/* Parameters for control virtqueue, if any */
>   	if (vi->has_cvq) {
>   		callbacks[total_vqs - 1] = NULL;
>   		names[total_vqs - 1] = "control";
> +		sizes[total_vqs - 1] = 64;
>   	}
>   
>   	/* Allocate/initialize parameters for send/receive virtqueues */
> @@ -2913,8 +2942,10 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>   			ctx[rxq2vq(i)] = true;
>   	}
>   
> -	ret = virtio_find_vqs_ctx(vi->vdev, total_vqs, vqs, callbacks,
> -				  names, ctx, NULL);
> +	virtnet_config_sizes(vi, sizes);
> +
> +	ret = virtio_find_vqs_ctx_size(vi->vdev, total_vqs, vqs, callbacks,
> +				       names, sizes, ctx, NULL);
>   	if (ret)
>   		goto err_find;
>   
> @@ -2934,6 +2965,8 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>   
>   
>   err_find:
> +	kfree(sizes);
> +err_sizes:
>   	kfree(ctx);
>   err_ctx:
>   	kfree(names);
> @@ -3252,6 +3285,9 @@ static int virtnet_probe(struct virtio_device *vdev)
>   		vi->curr_queue_pairs = num_online_cpus();
>   	vi->max_queue_pairs = max_queue_pairs;
>   
> +	virtnet_init_settings(dev);
> +	virtnet_update_settings(vi);
> +
>   	/* Allocate/initialize the rx/tx queues, and invoke find_vqs */
>   	err = init_vqs(vi);
>   	if (err)
> @@ -3264,8 +3300,6 @@ static int virtnet_probe(struct virtio_device *vdev)
>   	netif_set_real_num_tx_queues(dev, vi->curr_queue_pairs);
>   	netif_set_real_num_rx_queues(dev, vi->curr_queue_pairs);
>   
> -	virtnet_init_settings(dev);
> -
>   	if (virtio_has_feature(vdev, VIRTIO_NET_F_STANDBY)) {
>   		vi->failover = net_failover_create(vi->dev);
>   		if (IS_ERR(vi->failover)) {


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 29/32] virtio_net: get ringparam by virtqueue_get_vring_max_size()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Use virtqueue_get_vring_max_size() in virtnet_get_ringparam() to set
> tx,rx_max_pending.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---


Acked-by: Jason Wang <jasowang@redhat.com>


>   drivers/net/virtio_net.c | 8 ++++----
>   1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index dad497a47b3a..96d96c666c8c 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -2177,10 +2177,10 @@ static void virtnet_get_ringparam(struct net_device *dev,
>   {
>   	struct virtnet_info *vi = netdev_priv(dev);
>   
> -	ring->rx_max_pending = virtqueue_get_vring_size(vi->rq[0].vq);
> -	ring->tx_max_pending = virtqueue_get_vring_size(vi->sq[0].vq);
> -	ring->rx_pending = ring->rx_max_pending;
> -	ring->tx_pending = ring->tx_max_pending;
> +	ring->rx_max_pending = virtqueue_get_vring_max_size(vi->rq[0].vq);
> +	ring->tx_max_pending = virtqueue_get_vring_max_size(vi->sq[0].vq);
> +	ring->rx_pending = virtqueue_get_vring_size(vi->rq[0].vq);
> +	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
>   }
>   
>   

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 29/32] virtio_net: get ringparam by virtqueue_get_vring_max_size()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Use virtqueue_get_vring_max_size() in virtnet_get_ringparam() to set
> tx,rx_max_pending.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---


Acked-by: Jason Wang <jasowang@redhat.com>


>   drivers/net/virtio_net.c | 8 ++++----
>   1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index dad497a47b3a..96d96c666c8c 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -2177,10 +2177,10 @@ static void virtnet_get_ringparam(struct net_device *dev,
>   {
>   	struct virtnet_info *vi = netdev_priv(dev);
>   
> -	ring->rx_max_pending = virtqueue_get_vring_size(vi->rq[0].vq);
> -	ring->tx_max_pending = virtqueue_get_vring_size(vi->sq[0].vq);
> -	ring->rx_pending = ring->rx_max_pending;
> -	ring->tx_pending = ring->tx_max_pending;
> +	ring->rx_max_pending = virtqueue_get_vring_max_size(vi->rq[0].vq);
> +	ring->tx_max_pending = virtqueue_get_vring_max_size(vi->sq[0].vq);
> +	ring->rx_pending = virtqueue_get_vring_size(vi->rq[0].vq);
> +	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
>   }
>   
>   

Re: [PATCH v9 29/32] virtio_net: get ringparam by virtqueue_get_vring_max_size()

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Use virtqueue_get_vring_max_size() in virtnet_get_ringparam() to set
> tx,rx_max_pending.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---


Acked-by: Jason Wang <jasowang@redhat.com>


>   drivers/net/virtio_net.c | 8 ++++----
>   1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index dad497a47b3a..96d96c666c8c 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -2177,10 +2177,10 @@ static void virtnet_get_ringparam(struct net_device *dev,
>   {
>   	struct virtnet_info *vi = netdev_priv(dev);
>   
> -	ring->rx_max_pending = virtqueue_get_vring_size(vi->rq[0].vq);
> -	ring->tx_max_pending = virtqueue_get_vring_size(vi->sq[0].vq);
> -	ring->rx_pending = ring->rx_max_pending;
> -	ring->tx_pending = ring->tx_max_pending;
> +	ring->rx_max_pending = virtqueue_get_vring_max_size(vi->rq[0].vq);
> +	ring->tx_max_pending = virtqueue_get_vring_max_size(vi->sq[0].vq);
> +	ring->rx_pending = virtqueue_get_vring_size(vi->rq[0].vq);
> +	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
>   }
>   
>   


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 01/32] virtio: add helper virtqueue_get_vring_max_size()

[Xuan Zhuo]

On Tue, 12 Apr 2022 10:41:03 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Record the maximum queue num supported by the device.
> >
> > virtio-net can display the maximum (supported by hardware) ring size in
> > ethtool -g eth0.
> >
> > When the subsequent patch implements vring reset, it can judge whether
> > the ring size passed by the driver is legal based on this.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   arch/um/drivers/virtio_uml.c             |  1 +
> >   drivers/platform/mellanox/mlxbf-tmfifo.c |  2 ++
> >   drivers/remoteproc/remoteproc_virtio.c   |  2 ++
> >   drivers/s390/virtio/virtio_ccw.c         |  3 +++
> >   drivers/virtio/virtio_mmio.c             |  2 ++
> >   drivers/virtio/virtio_pci_legacy.c       |  2 ++
> >   drivers/virtio/virtio_pci_modern.c       |  2 ++
> >   drivers/virtio/virtio_ring.c             | 14 ++++++++++++++
> >   drivers/virtio/virtio_vdpa.c             |  2 ++
> >   include/linux/virtio.h                   |  2 ++
> >   10 files changed, 32 insertions(+)
> >
> > diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
> > index ba562d68dc04..904993d15a85 100644
> > --- a/arch/um/drivers/virtio_uml.c
> > +++ b/arch/um/drivers/virtio_uml.c
> > @@ -945,6 +945,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
> >   		goto error_create;
> >   	}
> >   	vq->priv = info;
> > +	vq->num_max = num;
> >   	num = virtqueue_get_vring_size(vq);
> >
> >   	if (vu_dev->protocol_features &
> > diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
> > index 38800e86ed8a..1ae3c56b66b0 100644
> > --- a/drivers/platform/mellanox/mlxbf-tmfifo.c
> > +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
> > @@ -959,6 +959,8 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
> >   			goto error;
> >   		}
> >
> > +		vq->num_max = vring->num;
> > +
> >   		vqs[i] = vq;
> >   		vring->vq = vq;
> >   		vq->priv = vring;
> > diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
> > index 70ab496d0431..7611755d0ae2 100644
> > --- a/drivers/remoteproc/remoteproc_virtio.c
> > +++ b/drivers/remoteproc/remoteproc_virtio.c
> > @@ -125,6 +125,8 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev,
> >   		return ERR_PTR(-ENOMEM);
> >   	}
> >
> > +	vq->num_max = len;
>
>
> I wonder if this is correct.
>
> It looks to me len is counted in bytes:
>
> /**
>   * struct rproc_vring - remoteproc vring state
>   * @va: virtual address
>   * @len: length, in bytes
>   * @da: device address
>   * @align: vring alignment
>   * @notifyid: rproc-specific unique vring index
>   * @rvdev: remote vdev
>   * @vq: the virtqueue of this vring
>   */
> struct rproc_vring {
>          void *va;
>          int len;
>          u32 da;
>          u32 align;
>          int notifyid;
>          struct rproc_vdev *rvdev;
>          struct virtqueue *vq;
> };
>

I think this comment is incorrect because here len is passed as num to
vring_new_virtqueue().

There is also this usage:

	/* actual size of vring (in bytes) */
	size = PAGE_ALIGN(vring_size(rvring->len, rvring->align));


And this value comes from here:

	static int
	rproc_parse_vring(struct rproc_vdev *rvdev, struct fw_rsc_vdev *rsc, int i)
	{
		struct rproc *rproc = rvdev->rproc;
		struct device *dev = &rproc->dev;
		struct fw_rsc_vdev_vring *vring = &rsc->vring[i];
		struct rproc_vring *rvring = &rvdev->vring[i];

		dev_dbg(dev, "vdev rsc: vring%d: da 0x%x, qsz %d, align %d\n",
			i, vring->da, vring->num, vring->align);

		/* verify queue size and vring alignment are sane */
		if (!vring->num || !vring->align) {
			dev_err(dev, "invalid qsz (%d) or alignment (%d)\n",
				vring->num, vring->align);
			return -EINVAL;
		}

       >	rvring->len = vring->num;
		rvring->align = vring->align;
		rvring->rvdev = rvdev;

		return 0;
	}

/**
 * struct fw_rsc_vdev_vring - vring descriptor entry
 * @da: device address
 * @align: the alignment between the consumer and producer parts of the vring
 * @num: num of buffers supported by this vring (must be power of two)
 * @notifyid: a unique rproc-wide notify index for this vring. This notify
 * index is used when kicking a remote processor, to let it know that this
 * vring is triggered.
 * @pa: physical address
 *
 * This descriptor is not a resource entry by itself; it is part of the
 * vdev resource type (see below).
 *
 * Note that @da should either contain the device address where
 * the remote processor is expecting the vring, or indicate that
 * dynamically allocation of the vring's device address is supported.
 */
struct fw_rsc_vdev_vring {
	u32 da;
	u32 align;
	u32 num;
	u32 notifyid;
	u32 pa;
} __packed;

So I think the 'len' here may have changed its meaning in a version update.

Thanks.

>
> Other looks good.
>
> Thanks
>
>
> > +
> >   	rvring->vq = vq;
> >   	vq->priv = rvring;
> >
> > diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
> > index d35e7a3f7067..468da60b56c5 100644
> > --- a/drivers/s390/virtio/virtio_ccw.c
> > +++ b/drivers/s390/virtio/virtio_ccw.c
> > @@ -529,6 +529,9 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,
> >   		err = -ENOMEM;
> >   		goto out_err;
> >   	}
> > +
> > +	vq->num_max = info->num;
> > +
> >   	/* it may have been reduced */
> >   	info->num = virtqueue_get_vring_size(vq);
> >
> > diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> > index 56128b9c46eb..a41abc8051b9 100644
> > --- a/drivers/virtio/virtio_mmio.c
> > +++ b/drivers/virtio/virtio_mmio.c
> > @@ -390,6 +390,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
> >   		goto error_new_virtqueue;
> >   	}
> >
> > +	vq->num_max = num;
> > +
> >   	/* Activate the queue */
> >   	writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
> >   	if (vm_dev->version == 1) {
> > diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
> > index 34141b9abe27..b68934fe6b5d 100644
> > --- a/drivers/virtio/virtio_pci_legacy.c
> > +++ b/drivers/virtio/virtio_pci_legacy.c
> > @@ -135,6 +135,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> >   	if (!vq)
> >   		return ERR_PTR(-ENOMEM);
> >
> > +	vq->num_max = num;
> > +
> >   	q_pfn = virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT;
> >   	if (q_pfn >> 32) {
> >   		dev_err(&vp_dev->pci_dev->dev,
> > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > index 5455bc041fb6..86d301f272b8 100644
> > --- a/drivers/virtio/virtio_pci_modern.c
> > +++ b/drivers/virtio/virtio_pci_modern.c
> > @@ -218,6 +218,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> >   	if (!vq)
> >   		return ERR_PTR(-ENOMEM);
> >
> > +	vq->num_max = num;
> > +
> >   	/* activate the queue */
> >   	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> >   	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 962f1477b1fa..b87130c8f312 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2371,6 +2371,20 @@ void vring_transport_features(struct virtio_device *vdev)
> >   }
> >   EXPORT_SYMBOL_GPL(vring_transport_features);
> >
> > +/**
> > + * virtqueue_get_vring_max_size - return the max size of the virtqueue's vring
> > + * @_vq: the struct virtqueue containing the vring of interest.
> > + *
> > + * Returns the max size of the vring.
> > + *
> > + * Unlike other operations, this need not be serialized.
> > + */
> > +unsigned int virtqueue_get_vring_max_size(struct virtqueue *_vq)
> > +{
> > +	return _vq->num_max;
> > +}
> > +EXPORT_SYMBOL_GPL(virtqueue_get_vring_max_size);
> > +
> >   /**
> >    * virtqueue_get_vring_size - return the size of the virtqueue's vring
> >    * @_vq: the struct virtqueue containing the vring of interest.
> > diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
> > index 7767a7f0119b..39e4c08eb0f2 100644
> > --- a/drivers/virtio/virtio_vdpa.c
> > +++ b/drivers/virtio/virtio_vdpa.c
> > @@ -183,6 +183,8 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
> >   		goto error_new_virtqueue;
> >   	}
> >
> > +	vq->num_max = max_num;
> > +
> >   	/* Setup virtqueue callback */
> >   	cb.callback = virtio_vdpa_virtqueue_cb;
> >   	cb.private = info;
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index 72292a62cd90..d59adc4be068 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -31,6 +31,7 @@ struct virtqueue {
> >   	struct virtio_device *vdev;
> >   	unsigned int index;
> >   	unsigned int num_free;
> > +	unsigned int num_max;
> >   	void *priv;
> >   };
> >
> > @@ -80,6 +81,7 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *vq);
> >
> >   void *virtqueue_detach_unused_buf(struct virtqueue *vq);
> >
> > +unsigned int virtqueue_get_vring_max_size(struct virtqueue *vq);
> >   unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
> >
> >   bool virtqueue_is_broken(struct virtqueue *vq);
>

Re: [PATCH v9 01/32] virtio: add helper virtqueue_get_vring_max_size()

[Xuan Zhuo]

On Tue, 12 Apr 2022 10:41:03 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Record the maximum queue num supported by the device.
> >
> > virtio-net can display the maximum (supported by hardware) ring size in
> > ethtool -g eth0.
> >
> > When the subsequent patch implements vring reset, it can judge whether
> > the ring size passed by the driver is legal based on this.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   arch/um/drivers/virtio_uml.c             |  1 +
> >   drivers/platform/mellanox/mlxbf-tmfifo.c |  2 ++
> >   drivers/remoteproc/remoteproc_virtio.c   |  2 ++
> >   drivers/s390/virtio/virtio_ccw.c         |  3 +++
> >   drivers/virtio/virtio_mmio.c             |  2 ++
> >   drivers/virtio/virtio_pci_legacy.c       |  2 ++
> >   drivers/virtio/virtio_pci_modern.c       |  2 ++
> >   drivers/virtio/virtio_ring.c             | 14 ++++++++++++++
> >   drivers/virtio/virtio_vdpa.c             |  2 ++
> >   include/linux/virtio.h                   |  2 ++
> >   10 files changed, 32 insertions(+)
> >
> > diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
> > index ba562d68dc04..904993d15a85 100644
> > --- a/arch/um/drivers/virtio_uml.c
> > +++ b/arch/um/drivers/virtio_uml.c
> > @@ -945,6 +945,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
> >   		goto error_create;
> >   	}
> >   	vq->priv = info;
> > +	vq->num_max = num;
> >   	num = virtqueue_get_vring_size(vq);
> >
> >   	if (vu_dev->protocol_features &
> > diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
> > index 38800e86ed8a..1ae3c56b66b0 100644
> > --- a/drivers/platform/mellanox/mlxbf-tmfifo.c
> > +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
> > @@ -959,6 +959,8 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
> >   			goto error;
> >   		}
> >
> > +		vq->num_max = vring->num;
> > +
> >   		vqs[i] = vq;
> >   		vring->vq = vq;
> >   		vq->priv = vring;
> > diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
> > index 70ab496d0431..7611755d0ae2 100644
> > --- a/drivers/remoteproc/remoteproc_virtio.c
> > +++ b/drivers/remoteproc/remoteproc_virtio.c
> > @@ -125,6 +125,8 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev,
> >   		return ERR_PTR(-ENOMEM);
> >   	}
> >
> > +	vq->num_max = len;
>
>
> I wonder if this is correct.
>
> It looks to me len is counted in bytes:
>
> /**
>   * struct rproc_vring - remoteproc vring state
>   * @va: virtual address
>   * @len: length, in bytes
>   * @da: device address
>   * @align: vring alignment
>   * @notifyid: rproc-specific unique vring index
>   * @rvdev: remote vdev
>   * @vq: the virtqueue of this vring
>   */
> struct rproc_vring {
>          void *va;
>          int len;
>          u32 da;
>          u32 align;
>          int notifyid;
>          struct rproc_vdev *rvdev;
>          struct virtqueue *vq;
> };
>

I think this comment is incorrect because here len is passed as num to
vring_new_virtqueue().

There is also this usage:

	/* actual size of vring (in bytes) */
	size = PAGE_ALIGN(vring_size(rvring->len, rvring->align));


And this value comes from here:

	static int
	rproc_parse_vring(struct rproc_vdev *rvdev, struct fw_rsc_vdev *rsc, int i)
	{
		struct rproc *rproc = rvdev->rproc;
		struct device *dev = &rproc->dev;
		struct fw_rsc_vdev_vring *vring = &rsc->vring[i];
		struct rproc_vring *rvring = &rvdev->vring[i];

		dev_dbg(dev, "vdev rsc: vring%d: da 0x%x, qsz %d, align %d\n",
			i, vring->da, vring->num, vring->align);

		/* verify queue size and vring alignment are sane */
		if (!vring->num || !vring->align) {
			dev_err(dev, "invalid qsz (%d) or alignment (%d)\n",
				vring->num, vring->align);
			return -EINVAL;
		}

       >	rvring->len = vring->num;
		rvring->align = vring->align;
		rvring->rvdev = rvdev;

		return 0;
	}

/**
 * struct fw_rsc_vdev_vring - vring descriptor entry
 * @da: device address
 * @align: the alignment between the consumer and producer parts of the vring
 * @num: num of buffers supported by this vring (must be power of two)
 * @notifyid: a unique rproc-wide notify index for this vring. This notify
 * index is used when kicking a remote processor, to let it know that this
 * vring is triggered.
 * @pa: physical address
 *
 * This descriptor is not a resource entry by itself; it is part of the
 * vdev resource type (see below).
 *
 * Note that @da should either contain the device address where
 * the remote processor is expecting the vring, or indicate that
 * dynamically allocation of the vring's device address is supported.
 */
struct fw_rsc_vdev_vring {
	u32 da;
	u32 align;
	u32 num;
	u32 notifyid;
	u32 pa;
} __packed;

So I think the 'len' here may have changed its meaning in a version update.

Thanks.

>
> Other looks good.
>
> Thanks
>
>
> > +
> >   	rvring->vq = vq;
> >   	vq->priv = rvring;
> >
> > diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
> > index d35e7a3f7067..468da60b56c5 100644
> > --- a/drivers/s390/virtio/virtio_ccw.c
> > +++ b/drivers/s390/virtio/virtio_ccw.c
> > @@ -529,6 +529,9 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,
> >   		err = -ENOMEM;
> >   		goto out_err;
> >   	}
> > +
> > +	vq->num_max = info->num;
> > +
> >   	/* it may have been reduced */
> >   	info->num = virtqueue_get_vring_size(vq);
> >
> > diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> > index 56128b9c46eb..a41abc8051b9 100644
> > --- a/drivers/virtio/virtio_mmio.c
> > +++ b/drivers/virtio/virtio_mmio.c
> > @@ -390,6 +390,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
> >   		goto error_new_virtqueue;
> >   	}
> >
> > +	vq->num_max = num;
> > +
> >   	/* Activate the queue */
> >   	writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
> >   	if (vm_dev->version == 1) {
> > diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
> > index 34141b9abe27..b68934fe6b5d 100644
> > --- a/drivers/virtio/virtio_pci_legacy.c
> > +++ b/drivers/virtio/virtio_pci_legacy.c
> > @@ -135,6 +135,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> >   	if (!vq)
> >   		return ERR_PTR(-ENOMEM);
> >
> > +	vq->num_max = num;
> > +
> >   	q_pfn = virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT;
> >   	if (q_pfn >> 32) {
> >   		dev_err(&vp_dev->pci_dev->dev,
> > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > index 5455bc041fb6..86d301f272b8 100644
> > --- a/drivers/virtio/virtio_pci_modern.c
> > +++ b/drivers/virtio/virtio_pci_modern.c
> > @@ -218,6 +218,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> >   	if (!vq)
> >   		return ERR_PTR(-ENOMEM);
> >
> > +	vq->num_max = num;
> > +
> >   	/* activate the queue */
> >   	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> >   	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 962f1477b1fa..b87130c8f312 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2371,6 +2371,20 @@ void vring_transport_features(struct virtio_device *vdev)
> >   }
> >   EXPORT_SYMBOL_GPL(vring_transport_features);
> >
> > +/**
> > + * virtqueue_get_vring_max_size - return the max size of the virtqueue's vring
> > + * @_vq: the struct virtqueue containing the vring of interest.
> > + *
> > + * Returns the max size of the vring.
> > + *
> > + * Unlike other operations, this need not be serialized.
> > + */
> > +unsigned int virtqueue_get_vring_max_size(struct virtqueue *_vq)
> > +{
> > +	return _vq->num_max;
> > +}
> > +EXPORT_SYMBOL_GPL(virtqueue_get_vring_max_size);
> > +
> >   /**
> >    * virtqueue_get_vring_size - return the size of the virtqueue's vring
> >    * @_vq: the struct virtqueue containing the vring of interest.
> > diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
> > index 7767a7f0119b..39e4c08eb0f2 100644
> > --- a/drivers/virtio/virtio_vdpa.c
> > +++ b/drivers/virtio/virtio_vdpa.c
> > @@ -183,6 +183,8 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
> >   		goto error_new_virtqueue;
> >   	}
> >
> > +	vq->num_max = max_num;
> > +
> >   	/* Setup virtqueue callback */
> >   	cb.callback = virtio_vdpa_virtqueue_cb;
> >   	cb.private = info;
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index 72292a62cd90..d59adc4be068 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -31,6 +31,7 @@ struct virtqueue {
> >   	struct virtio_device *vdev;
> >   	unsigned int index;
> >   	unsigned int num_free;
> > +	unsigned int num_max;
> >   	void *priv;
> >   };
> >
> > @@ -80,6 +81,7 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *vq);
> >
> >   void *virtqueue_detach_unused_buf(struct virtqueue *vq);
> >
> > +unsigned int virtqueue_get_vring_max_size(struct virtqueue *vq);
> >   unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
> >
> >   bool virtqueue_is_broken(struct virtqueue *vq);
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 01/32] virtio: add helper virtqueue_get_vring_max_size()

[Xuan Zhuo]

On Tue, 12 Apr 2022 10:41:03 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Record the maximum queue num supported by the device.
> >
> > virtio-net can display the maximum (supported by hardware) ring size in
> > ethtool -g eth0.
> >
> > When the subsequent patch implements vring reset, it can judge whether
> > the ring size passed by the driver is legal based on this.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   arch/um/drivers/virtio_uml.c             |  1 +
> >   drivers/platform/mellanox/mlxbf-tmfifo.c |  2 ++
> >   drivers/remoteproc/remoteproc_virtio.c   |  2 ++
> >   drivers/s390/virtio/virtio_ccw.c         |  3 +++
> >   drivers/virtio/virtio_mmio.c             |  2 ++
> >   drivers/virtio/virtio_pci_legacy.c       |  2 ++
> >   drivers/virtio/virtio_pci_modern.c       |  2 ++
> >   drivers/virtio/virtio_ring.c             | 14 ++++++++++++++
> >   drivers/virtio/virtio_vdpa.c             |  2 ++
> >   include/linux/virtio.h                   |  2 ++
> >   10 files changed, 32 insertions(+)
> >
> > diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
> > index ba562d68dc04..904993d15a85 100644
> > --- a/arch/um/drivers/virtio_uml.c
> > +++ b/arch/um/drivers/virtio_uml.c
> > @@ -945,6 +945,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
> >   		goto error_create;
> >   	}
> >   	vq->priv = info;
> > +	vq->num_max = num;
> >   	num = virtqueue_get_vring_size(vq);
> >
> >   	if (vu_dev->protocol_features &
> > diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
> > index 38800e86ed8a..1ae3c56b66b0 100644
> > --- a/drivers/platform/mellanox/mlxbf-tmfifo.c
> > +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
> > @@ -959,6 +959,8 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
> >   			goto error;
> >   		}
> >
> > +		vq->num_max = vring->num;
> > +
> >   		vqs[i] = vq;
> >   		vring->vq = vq;
> >   		vq->priv = vring;
> > diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
> > index 70ab496d0431..7611755d0ae2 100644
> > --- a/drivers/remoteproc/remoteproc_virtio.c
> > +++ b/drivers/remoteproc/remoteproc_virtio.c
> > @@ -125,6 +125,8 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev,
> >   		return ERR_PTR(-ENOMEM);
> >   	}
> >
> > +	vq->num_max = len;
>
>
> I wonder if this is correct.
>
> It looks to me len is counted in bytes:
>
> /**
>   * struct rproc_vring - remoteproc vring state
>   * @va: virtual address
>   * @len: length, in bytes
>   * @da: device address
>   * @align: vring alignment
>   * @notifyid: rproc-specific unique vring index
>   * @rvdev: remote vdev
>   * @vq: the virtqueue of this vring
>   */
> struct rproc_vring {
>          void *va;
>          int len;
>          u32 da;
>          u32 align;
>          int notifyid;
>          struct rproc_vdev *rvdev;
>          struct virtqueue *vq;
> };
>

I think this comment is incorrect because here len is passed as num to
vring_new_virtqueue().

There is also this usage:

	/* actual size of vring (in bytes) */
	size = PAGE_ALIGN(vring_size(rvring->len, rvring->align));


And this value comes from here:

	static int
	rproc_parse_vring(struct rproc_vdev *rvdev, struct fw_rsc_vdev *rsc, int i)
	{
		struct rproc *rproc = rvdev->rproc;
		struct device *dev = &rproc->dev;
		struct fw_rsc_vdev_vring *vring = &rsc->vring[i];
		struct rproc_vring *rvring = &rvdev->vring[i];

		dev_dbg(dev, "vdev rsc: vring%d: da 0x%x, qsz %d, align %d\n",
			i, vring->da, vring->num, vring->align);

		/* verify queue size and vring alignment are sane */
		if (!vring->num || !vring->align) {
			dev_err(dev, "invalid qsz (%d) or alignment (%d)\n",
				vring->num, vring->align);
			return -EINVAL;
		}

       >	rvring->len = vring->num;
		rvring->align = vring->align;
		rvring->rvdev = rvdev;

		return 0;
	}

/**
 * struct fw_rsc_vdev_vring - vring descriptor entry
 * @da: device address
 * @align: the alignment between the consumer and producer parts of the vring
 * @num: num of buffers supported by this vring (must be power of two)
 * @notifyid: a unique rproc-wide notify index for this vring. This notify
 * index is used when kicking a remote processor, to let it know that this
 * vring is triggered.
 * @pa: physical address
 *
 * This descriptor is not a resource entry by itself; it is part of the
 * vdev resource type (see below).
 *
 * Note that @da should either contain the device address where
 * the remote processor is expecting the vring, or indicate that
 * dynamically allocation of the vring's device address is supported.
 */
struct fw_rsc_vdev_vring {
	u32 da;
	u32 align;
	u32 num;
	u32 notifyid;
	u32 pa;
} __packed;

So I think the 'len' here may have changed its meaning in a version update.

Thanks.

>
> Other looks good.
>
> Thanks
>
>
> > +
> >   	rvring->vq = vq;
> >   	vq->priv = rvring;
> >
> > diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
> > index d35e7a3f7067..468da60b56c5 100644
> > --- a/drivers/s390/virtio/virtio_ccw.c
> > +++ b/drivers/s390/virtio/virtio_ccw.c
> > @@ -529,6 +529,9 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,
> >   		err = -ENOMEM;
> >   		goto out_err;
> >   	}
> > +
> > +	vq->num_max = info->num;
> > +
> >   	/* it may have been reduced */
> >   	info->num = virtqueue_get_vring_size(vq);
> >
> > diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> > index 56128b9c46eb..a41abc8051b9 100644
> > --- a/drivers/virtio/virtio_mmio.c
> > +++ b/drivers/virtio/virtio_mmio.c
> > @@ -390,6 +390,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
> >   		goto error_new_virtqueue;
> >   	}
> >
> > +	vq->num_max = num;
> > +
> >   	/* Activate the queue */
> >   	writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
> >   	if (vm_dev->version == 1) {
> > diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
> > index 34141b9abe27..b68934fe6b5d 100644
> > --- a/drivers/virtio/virtio_pci_legacy.c
> > +++ b/drivers/virtio/virtio_pci_legacy.c
> > @@ -135,6 +135,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> >   	if (!vq)
> >   		return ERR_PTR(-ENOMEM);
> >
> > +	vq->num_max = num;
> > +
> >   	q_pfn = virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT;
> >   	if (q_pfn >> 32) {
> >   		dev_err(&vp_dev->pci_dev->dev,
> > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > index 5455bc041fb6..86d301f272b8 100644
> > --- a/drivers/virtio/virtio_pci_modern.c
> > +++ b/drivers/virtio/virtio_pci_modern.c
> > @@ -218,6 +218,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> >   	if (!vq)
> >   		return ERR_PTR(-ENOMEM);
> >
> > +	vq->num_max = num;
> > +
> >   	/* activate the queue */
> >   	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> >   	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 962f1477b1fa..b87130c8f312 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2371,6 +2371,20 @@ void vring_transport_features(struct virtio_device *vdev)
> >   }
> >   EXPORT_SYMBOL_GPL(vring_transport_features);
> >
> > +/**
> > + * virtqueue_get_vring_max_size - return the max size of the virtqueue's vring
> > + * @_vq: the struct virtqueue containing the vring of interest.
> > + *
> > + * Returns the max size of the vring.
> > + *
> > + * Unlike other operations, this need not be serialized.
> > + */
> > +unsigned int virtqueue_get_vring_max_size(struct virtqueue *_vq)
> > +{
> > +	return _vq->num_max;
> > +}
> > +EXPORT_SYMBOL_GPL(virtqueue_get_vring_max_size);
> > +
> >   /**
> >    * virtqueue_get_vring_size - return the size of the virtqueue's vring
> >    * @_vq: the struct virtqueue containing the vring of interest.
> > diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
> > index 7767a7f0119b..39e4c08eb0f2 100644
> > --- a/drivers/virtio/virtio_vdpa.c
> > +++ b/drivers/virtio/virtio_vdpa.c
> > @@ -183,6 +183,8 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
> >   		goto error_new_virtqueue;
> >   	}
> >
> > +	vq->num_max = max_num;
> > +
> >   	/* Setup virtqueue callback */
> >   	cb.callback = virtio_vdpa_virtqueue_cb;
> >   	cb.private = info;
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index 72292a62cd90..d59adc4be068 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -31,6 +31,7 @@ struct virtqueue {
> >   	struct virtio_device *vdev;
> >   	unsigned int index;
> >   	unsigned int num_free;
> > +	unsigned int num_max;
> >   	void *priv;
> >   };
> >
> > @@ -80,6 +81,7 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *vq);
> >
> >   void *virtqueue_detach_unused_buf(struct virtqueue *vq);
> >
> > +unsigned int virtqueue_get_vring_max_size(struct virtqueue *vq);
> >   unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
> >
> >   bool virtqueue_is_broken(struct virtqueue *vq);
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Xuan Zhuo]

On Tue, 12 Apr 2022 15:07:58 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > This patch implements virtio pci support for QUEUE RESET.
> >
> > Performing reset on a queue is divided into these steps:
> >
> >   1. notify the device to reset the queue
> >   2. recycle the buffer submitted
> >   3. reset the vring (may re-alloc)
> >   4. mmap vring to device, and enable the queue
> >
> > This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
> > pci scenario.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_pci_common.c |  8 +--
> >   drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
> >   drivers/virtio/virtio_ring.c       |  2 +
> >   include/linux/virtio.h             |  1 +
> >   4 files changed, 92 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> > index fdbde1db5ec5..863d3a8a0956 100644
> > --- a/drivers/virtio/virtio_pci_common.c
> > +++ b/drivers/virtio/virtio_pci_common.c
> > @@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
> >   	struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
> >   	unsigned long flags;
> >
> > -	spin_lock_irqsave(&vp_dev->lock, flags);
> > -	list_del(&info->node);
> > -	spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	if (!vq->reset) {
>
>
> On which condition that we may hit this path?
>
>
> > +		spin_lock_irqsave(&vp_dev->lock, flags);
> > +		list_del(&info->node);
> > +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	}
> >
> >   	vp_dev->del_vq(info);
> >   	kfree(info);
> > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > index 49a4493732cf..cb5d38f1c9c8 100644
> > --- a/drivers/virtio/virtio_pci_modern.c
> > +++ b/drivers/virtio/virtio_pci_modern.c
> > @@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
> >   	if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
> >   			pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
> >   		__virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
> > +
> > +	if (features & BIT_ULL(VIRTIO_F_RING_RESET))
> > +		__virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
> >   }
> >
> >   /* virtio config->finalize_features() implementation */
> > @@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> >   	return 0;
> >   }
> >
> > +static int vp_modern_reset_vq(struct virtqueue *vq)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	struct virtio_pci_vq_info *info;
> > +	unsigned long flags;
> > +
> > +	if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
> > +		return -ENOENT;
> > +
> > +	vp_modern_set_queue_reset(mdev, vq->index);
> > +
> > +	info = vp_dev->vqs[vq->index];
> > +
> > +	/* delete vq from irq handler */
> > +	spin_lock_irqsave(&vp_dev->lock, flags);
> > +	list_del(&info->node);
> > +	spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +
> > +	INIT_LIST_HEAD(&info->node);
> > +
> > +	/* For the case where vq has an exclusive irq, to prevent the irq from
> > +	 * being received again and the pending irq, call disable_irq().
> > +	 *
> > +	 * In the scenario based on shared interrupts, vq will be searched from
> > +	 * the queue virtqueues. Since the previous list_del() has been deleted
> > +	 * from the queue, it is impossible for vq to be called in this case.
> > +	 * There is no need to close the corresponding interrupt.
> > +	 */
> > +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +		disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>
>
> See the previous discussion and the revert of the first try to harden
> the interrupt. We probably can't use disable_irq() since it conflicts
> with the affinity managed IRQ that is used by some drivers.
>
> We need to use synchonize_irq() and per virtqueue flag instead. As
> mentioned in previous patches, this could be done on top of my rework on
> the IRQ hardening .

OK, the next version will contain hardened features by per virtqueue flag.

Thanks.

>
>
> > +
> > +	vq->reset = true;
> > +
> > +	return 0;
> > +}
> > +
> > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	struct virtio_pci_vq_info *info;
> > +	unsigned long flags, index;
> > +	int err;
> > +
> > +	if (!vq->reset)
> > +		return -EBUSY;
> > +
> > +	index = vq->index;
> > +	info = vp_dev->vqs[index];
> > +
> > +	/* check queue reset status */
> > +	if (vp_modern_get_queue_reset(mdev, index) != 1)
> > +		return -EBUSY;
> > +
> > +	err = vp_active_vq(vq, info->msix_vector);
> > +	if (err)
> > +		return err;
> > +
> > +	if (vq->callback) {
> > +		spin_lock_irqsave(&vp_dev->lock, flags);
> > +		list_add(&info->node, &vp_dev->virtqueues);
> > +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	} else {
> > +		INIT_LIST_HEAD(&info->node);
> > +	}
> > +
> > +	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > +
> > +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>
>
> We had the same issue as disable_irq().
>
> Thanks
>
>
> > +
> > +	vq->reset = false;
> > +
> > +	return 0;
> > +}
> > +
> >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> >   {
> >   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> > @@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
> >   	.set_vq_affinity = vp_set_vq_affinity,
> >   	.get_vq_affinity = vp_get_vq_affinity,
> >   	.get_shm_region  = vp_get_shm_region,
> > +	.reset_vq	 = vp_modern_reset_vq,
> > +	.enable_reset_vq = vp_modern_enable_reset_vq,
> >   };
> >
> >   static const struct virtio_config_ops virtio_pci_config_ops = {
> > @@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
> >   	.set_vq_affinity = vp_set_vq_affinity,
> >   	.get_vq_affinity = vp_get_vq_affinity,
> >   	.get_shm_region  = vp_get_shm_region,
> > +	.reset_vq	 = vp_modern_reset_vq,
> > +	.enable_reset_vq = vp_modern_enable_reset_vq,
> >   };
> >
> >   /* the PCI probing function */
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 6250e19fc5bf..91937e21edca 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> >   	vq->vq.index = index;
> > +	vq->vq.reset = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> >
> > @@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> >   	vq->vq.index = index;
> > +	vq->vq.reset = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> >
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index c86ff02e0ca0..33ab003c5100 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -33,6 +33,7 @@ struct virtqueue {
> >   	unsigned int num_free;
> >   	unsigned int num_max;
> >   	void *priv;
> > +	bool reset;
> >   };
> >
> >   int virtqueue_add_outbuf(struct virtqueue *vq,
>

Re: [PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Xuan Zhuo]

On Tue, 12 Apr 2022 15:07:58 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > This patch implements virtio pci support for QUEUE RESET.
> >
> > Performing reset on a queue is divided into these steps:
> >
> >   1. notify the device to reset the queue
> >   2. recycle the buffer submitted
> >   3. reset the vring (may re-alloc)
> >   4. mmap vring to device, and enable the queue
> >
> > This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
> > pci scenario.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_pci_common.c |  8 +--
> >   drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
> >   drivers/virtio/virtio_ring.c       |  2 +
> >   include/linux/virtio.h             |  1 +
> >   4 files changed, 92 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> > index fdbde1db5ec5..863d3a8a0956 100644
> > --- a/drivers/virtio/virtio_pci_common.c
> > +++ b/drivers/virtio/virtio_pci_common.c
> > @@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
> >   	struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
> >   	unsigned long flags;
> >
> > -	spin_lock_irqsave(&vp_dev->lock, flags);
> > -	list_del(&info->node);
> > -	spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	if (!vq->reset) {
>
>
> On which condition that we may hit this path?
>
>
> > +		spin_lock_irqsave(&vp_dev->lock, flags);
> > +		list_del(&info->node);
> > +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	}
> >
> >   	vp_dev->del_vq(info);
> >   	kfree(info);
> > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > index 49a4493732cf..cb5d38f1c9c8 100644
> > --- a/drivers/virtio/virtio_pci_modern.c
> > +++ b/drivers/virtio/virtio_pci_modern.c
> > @@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
> >   	if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
> >   			pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
> >   		__virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
> > +
> > +	if (features & BIT_ULL(VIRTIO_F_RING_RESET))
> > +		__virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
> >   }
> >
> >   /* virtio config->finalize_features() implementation */
> > @@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> >   	return 0;
> >   }
> >
> > +static int vp_modern_reset_vq(struct virtqueue *vq)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	struct virtio_pci_vq_info *info;
> > +	unsigned long flags;
> > +
> > +	if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
> > +		return -ENOENT;
> > +
> > +	vp_modern_set_queue_reset(mdev, vq->index);
> > +
> > +	info = vp_dev->vqs[vq->index];
> > +
> > +	/* delete vq from irq handler */
> > +	spin_lock_irqsave(&vp_dev->lock, flags);
> > +	list_del(&info->node);
> > +	spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +
> > +	INIT_LIST_HEAD(&info->node);
> > +
> > +	/* For the case where vq has an exclusive irq, to prevent the irq from
> > +	 * being received again and the pending irq, call disable_irq().
> > +	 *
> > +	 * In the scenario based on shared interrupts, vq will be searched from
> > +	 * the queue virtqueues. Since the previous list_del() has been deleted
> > +	 * from the queue, it is impossible for vq to be called in this case.
> > +	 * There is no need to close the corresponding interrupt.
> > +	 */
> > +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +		disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>
>
> See the previous discussion and the revert of the first try to harden
> the interrupt. We probably can't use disable_irq() since it conflicts
> with the affinity managed IRQ that is used by some drivers.
>
> We need to use synchonize_irq() and per virtqueue flag instead. As
> mentioned in previous patches, this could be done on top of my rework on
> the IRQ hardening .

OK, the next version will contain hardened features by per virtqueue flag.

Thanks.

>
>
> > +
> > +	vq->reset = true;
> > +
> > +	return 0;
> > +}
> > +
> > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	struct virtio_pci_vq_info *info;
> > +	unsigned long flags, index;
> > +	int err;
> > +
> > +	if (!vq->reset)
> > +		return -EBUSY;
> > +
> > +	index = vq->index;
> > +	info = vp_dev->vqs[index];
> > +
> > +	/* check queue reset status */
> > +	if (vp_modern_get_queue_reset(mdev, index) != 1)
> > +		return -EBUSY;
> > +
> > +	err = vp_active_vq(vq, info->msix_vector);
> > +	if (err)
> > +		return err;
> > +
> > +	if (vq->callback) {
> > +		spin_lock_irqsave(&vp_dev->lock, flags);
> > +		list_add(&info->node, &vp_dev->virtqueues);
> > +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	} else {
> > +		INIT_LIST_HEAD(&info->node);
> > +	}
> > +
> > +	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > +
> > +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>
>
> We had the same issue as disable_irq().
>
> Thanks
>
>
> > +
> > +	vq->reset = false;
> > +
> > +	return 0;
> > +}
> > +
> >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> >   {
> >   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> > @@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
> >   	.set_vq_affinity = vp_set_vq_affinity,
> >   	.get_vq_affinity = vp_get_vq_affinity,
> >   	.get_shm_region  = vp_get_shm_region,
> > +	.reset_vq	 = vp_modern_reset_vq,
> > +	.enable_reset_vq = vp_modern_enable_reset_vq,
> >   };
> >
> >   static const struct virtio_config_ops virtio_pci_config_ops = {
> > @@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
> >   	.set_vq_affinity = vp_set_vq_affinity,
> >   	.get_vq_affinity = vp_get_vq_affinity,
> >   	.get_shm_region  = vp_get_shm_region,
> > +	.reset_vq	 = vp_modern_reset_vq,
> > +	.enable_reset_vq = vp_modern_enable_reset_vq,
> >   };
> >
> >   /* the PCI probing function */
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 6250e19fc5bf..91937e21edca 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> >   	vq->vq.index = index;
> > +	vq->vq.reset = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> >
> > @@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> >   	vq->vq.index = index;
> > +	vq->vq.reset = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> >
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index c86ff02e0ca0..33ab003c5100 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -33,6 +33,7 @@ struct virtqueue {
> >   	unsigned int num_free;
> >   	unsigned int num_max;
> >   	void *priv;
> > +	bool reset;
> >   };
> >
> >   int virtqueue_add_outbuf(struct virtqueue *vq,
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Xuan Zhuo]

On Tue, 12 Apr 2022 15:07:58 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > This patch implements virtio pci support for QUEUE RESET.
> >
> > Performing reset on a queue is divided into these steps:
> >
> >   1. notify the device to reset the queue
> >   2. recycle the buffer submitted
> >   3. reset the vring (may re-alloc)
> >   4. mmap vring to device, and enable the queue
> >
> > This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
> > pci scenario.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_pci_common.c |  8 +--
> >   drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
> >   drivers/virtio/virtio_ring.c       |  2 +
> >   include/linux/virtio.h             |  1 +
> >   4 files changed, 92 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> > index fdbde1db5ec5..863d3a8a0956 100644
> > --- a/drivers/virtio/virtio_pci_common.c
> > +++ b/drivers/virtio/virtio_pci_common.c
> > @@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
> >   	struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
> >   	unsigned long flags;
> >
> > -	spin_lock_irqsave(&vp_dev->lock, flags);
> > -	list_del(&info->node);
> > -	spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	if (!vq->reset) {
>
>
> On which condition that we may hit this path?
>
>
> > +		spin_lock_irqsave(&vp_dev->lock, flags);
> > +		list_del(&info->node);
> > +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	}
> >
> >   	vp_dev->del_vq(info);
> >   	kfree(info);
> > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > index 49a4493732cf..cb5d38f1c9c8 100644
> > --- a/drivers/virtio/virtio_pci_modern.c
> > +++ b/drivers/virtio/virtio_pci_modern.c
> > @@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
> >   	if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
> >   			pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
> >   		__virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
> > +
> > +	if (features & BIT_ULL(VIRTIO_F_RING_RESET))
> > +		__virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
> >   }
> >
> >   /* virtio config->finalize_features() implementation */
> > @@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> >   	return 0;
> >   }
> >
> > +static int vp_modern_reset_vq(struct virtqueue *vq)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	struct virtio_pci_vq_info *info;
> > +	unsigned long flags;
> > +
> > +	if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
> > +		return -ENOENT;
> > +
> > +	vp_modern_set_queue_reset(mdev, vq->index);
> > +
> > +	info = vp_dev->vqs[vq->index];
> > +
> > +	/* delete vq from irq handler */
> > +	spin_lock_irqsave(&vp_dev->lock, flags);
> > +	list_del(&info->node);
> > +	spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +
> > +	INIT_LIST_HEAD(&info->node);
> > +
> > +	/* For the case where vq has an exclusive irq, to prevent the irq from
> > +	 * being received again and the pending irq, call disable_irq().
> > +	 *
> > +	 * In the scenario based on shared interrupts, vq will be searched from
> > +	 * the queue virtqueues. Since the previous list_del() has been deleted
> > +	 * from the queue, it is impossible for vq to be called in this case.
> > +	 * There is no need to close the corresponding interrupt.
> > +	 */
> > +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +		disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>
>
> See the previous discussion and the revert of the first try to harden
> the interrupt. We probably can't use disable_irq() since it conflicts
> with the affinity managed IRQ that is used by some drivers.
>
> We need to use synchonize_irq() and per virtqueue flag instead. As
> mentioned in previous patches, this could be done on top of my rework on
> the IRQ hardening .

OK, the next version will contain hardened features by per virtqueue flag.

Thanks.

>
>
> > +
> > +	vq->reset = true;
> > +
> > +	return 0;
> > +}
> > +
> > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	struct virtio_pci_vq_info *info;
> > +	unsigned long flags, index;
> > +	int err;
> > +
> > +	if (!vq->reset)
> > +		return -EBUSY;
> > +
> > +	index = vq->index;
> > +	info = vp_dev->vqs[index];
> > +
> > +	/* check queue reset status */
> > +	if (vp_modern_get_queue_reset(mdev, index) != 1)
> > +		return -EBUSY;
> > +
> > +	err = vp_active_vq(vq, info->msix_vector);
> > +	if (err)
> > +		return err;
> > +
> > +	if (vq->callback) {
> > +		spin_lock_irqsave(&vp_dev->lock, flags);
> > +		list_add(&info->node, &vp_dev->virtqueues);
> > +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	} else {
> > +		INIT_LIST_HEAD(&info->node);
> > +	}
> > +
> > +	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > +
> > +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>
>
> We had the same issue as disable_irq().
>
> Thanks
>
>
> > +
> > +	vq->reset = false;
> > +
> > +	return 0;
> > +}
> > +
> >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> >   {
> >   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> > @@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
> >   	.set_vq_affinity = vp_set_vq_affinity,
> >   	.get_vq_affinity = vp_get_vq_affinity,
> >   	.get_shm_region  = vp_get_shm_region,
> > +	.reset_vq	 = vp_modern_reset_vq,
> > +	.enable_reset_vq = vp_modern_enable_reset_vq,
> >   };
> >
> >   static const struct virtio_config_ops virtio_pci_config_ops = {
> > @@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
> >   	.set_vq_affinity = vp_set_vq_affinity,
> >   	.get_vq_affinity = vp_get_vq_affinity,
> >   	.get_shm_region  = vp_get_shm_region,
> > +	.reset_vq	 = vp_modern_reset_vq,
> > +	.enable_reset_vq = vp_modern_enable_reset_vq,
> >   };
> >
> >   /* the PCI probing function */
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 6250e19fc5bf..91937e21edca 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> >   	vq->vq.index = index;
> > +	vq->vq.reset = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> >
> > @@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> >   	vq->vq.index = index;
> > +	vq->vq.reset = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> >
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index c86ff02e0ca0..33ab003c5100 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -33,6 +33,7 @@ struct virtqueue {
> >   	unsigned int num_free;
> >   	unsigned int num_max;
> >   	void *priv;
> > +	bool reset;
> >   };
> >
> >   int virtqueue_add_outbuf(struct virtqueue *vq,
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 12/32] virtio_ring: packed: extract the logic of alloc queue

[Xuan Zhuo]

On Tue, 12 Apr 2022 14:28:24 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of packed to create vring queue.
> >
> > For the convenience of passing parameters, add a structure
> > vring_packed.
> >
> > This feature is required for subsequent virtuqueue reset vring.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 70 ++++++++++++++++++++++++++++--------
> >   1 file changed, 56 insertions(+), 14 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 33864134a744..ea451ae2aaef 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -1817,19 +1817,17 @@ static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
> >   	return desc_extra;
> >   }
> >
> > -static struct virtqueue *vring_create_virtqueue_packed(
> > -	unsigned int index,
> > -	unsigned int num,
> > -	unsigned int vring_align,
> > -	struct virtio_device *vdev,
> > -	bool weak_barriers,
> > -	bool may_reduce_num,
> > -	bool context,
> > -	bool (*notify)(struct virtqueue *),
> > -	void (*callback)(struct virtqueue *),
> > -	const char *name)
> > +static int vring_alloc_queue_packed(struct virtio_device *vdev,
> > +				    u32 num,
> > +				    struct vring_packed_desc **_ring,
> > +				    struct vring_packed_desc_event **_driver,
> > +				    struct vring_packed_desc_event **_device,
> > +				    dma_addr_t *_ring_dma_addr,
> > +				    dma_addr_t *_driver_event_dma_addr,
> > +				    dma_addr_t *_device_event_dma_addr,
> > +				    size_t *_ring_size_in_bytes,
> > +				    size_t *_event_size_in_bytes)
> >   {
> > -	struct vring_virtqueue *vq;
> >   	struct vring_packed_desc *ring;
> >   	struct vring_packed_desc_event *driver, *device;
> >   	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> > @@ -1857,6 +1855,52 @@ static struct virtqueue *vring_create_virtqueue_packed(
> >   	if (!device)
> >   		goto err_device;
> >
> > +	*_ring                   = ring;
> > +	*_driver                 = driver;
> > +	*_device                 = device;
> > +	*_ring_dma_addr          = ring_dma_addr;
> > +	*_driver_event_dma_addr  = driver_event_dma_addr;
> > +	*_device_event_dma_addr  = device_event_dma_addr;
> > +	*_ring_size_in_bytes     = ring_size_in_bytes;
> > +	*_event_size_in_bytes    = event_size_in_bytes;
>
>
> I wonder if we can simply factor out split and packed from struct
> vring_virtqueue:
>
> struct vring_virtqueue {
>      union {
>          struct {} split;
>          struct {} packed;
>      };
> };
>
> to
>
> struct vring_virtqueue_split {};
> struct vring_virtqueue_packed {};
>
> Then we can do things like:
>
> vring_create_virtqueue_packed(struct virtio_device *vdev, u32 num,
> struct vring_virtqueue_packed *packed);
>
> and
>
> vring_vritqueue_attach_packed(struct vring_virtqueue *vq, struct
> vring_virtqueue_packed packed);

This idea is very similar to my previous idea, just without introducing a new
structure.

I'd be more than happy to revise this.

Thanks.


>
> Thanks
>
>
> > +
> > +	return 0;
> > +
> > +err_device:
> > +	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> > +
> > +err_driver:
> > +	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> > +
> > +err_ring:
> > +	return -ENOMEM;
> > +}
> > +
> > +static struct virtqueue *vring_create_virtqueue_packed(
> > +	unsigned int index,
> > +	unsigned int num,
> > +	unsigned int vring_align,
> > +	struct virtio_device *vdev,
> > +	bool weak_barriers,
> > +	bool may_reduce_num,
> > +	bool context,
> > +	bool (*notify)(struct virtqueue *),
> > +	void (*callback)(struct virtqueue *),
> > +	const char *name)
> > +{
> > +	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> > +	struct vring_packed_desc_event *driver, *device;
> > +	size_t ring_size_in_bytes, event_size_in_bytes;
> > +	struct vring_packed_desc *ring;
> > +	struct vring_virtqueue *vq;
> > +
> > +	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
> > +				     &ring_dma_addr, &driver_event_dma_addr,
> > +				     &device_event_dma_addr,
> > +				     &ring_size_in_bytes,
> > +				     &event_size_in_bytes))
> > +		goto err_ring;
> > +
> >   	vq = kmalloc(sizeof(*vq), GFP_KERNEL);
> >   	if (!vq)
> >   		goto err_vq;
> > @@ -1939,9 +1983,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> >   	kfree(vq);
> >   err_vq:
> >   	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
> > -err_device:
> >   	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> > -err_driver:
> >   	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> >   err_ring:
> >   	return NULL;
>

Re: [PATCH v9 12/32] virtio_ring: packed: extract the logic of alloc queue

[Xuan Zhuo]

On Tue, 12 Apr 2022 14:28:24 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of packed to create vring queue.
> >
> > For the convenience of passing parameters, add a structure
> > vring_packed.
> >
> > This feature is required for subsequent virtuqueue reset vring.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 70 ++++++++++++++++++++++++++++--------
> >   1 file changed, 56 insertions(+), 14 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 33864134a744..ea451ae2aaef 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -1817,19 +1817,17 @@ static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
> >   	return desc_extra;
> >   }
> >
> > -static struct virtqueue *vring_create_virtqueue_packed(
> > -	unsigned int index,
> > -	unsigned int num,
> > -	unsigned int vring_align,
> > -	struct virtio_device *vdev,
> > -	bool weak_barriers,
> > -	bool may_reduce_num,
> > -	bool context,
> > -	bool (*notify)(struct virtqueue *),
> > -	void (*callback)(struct virtqueue *),
> > -	const char *name)
> > +static int vring_alloc_queue_packed(struct virtio_device *vdev,
> > +				    u32 num,
> > +				    struct vring_packed_desc **_ring,
> > +				    struct vring_packed_desc_event **_driver,
> > +				    struct vring_packed_desc_event **_device,
> > +				    dma_addr_t *_ring_dma_addr,
> > +				    dma_addr_t *_driver_event_dma_addr,
> > +				    dma_addr_t *_device_event_dma_addr,
> > +				    size_t *_ring_size_in_bytes,
> > +				    size_t *_event_size_in_bytes)
> >   {
> > -	struct vring_virtqueue *vq;
> >   	struct vring_packed_desc *ring;
> >   	struct vring_packed_desc_event *driver, *device;
> >   	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> > @@ -1857,6 +1855,52 @@ static struct virtqueue *vring_create_virtqueue_packed(
> >   	if (!device)
> >   		goto err_device;
> >
> > +	*_ring                   = ring;
> > +	*_driver                 = driver;
> > +	*_device                 = device;
> > +	*_ring_dma_addr          = ring_dma_addr;
> > +	*_driver_event_dma_addr  = driver_event_dma_addr;
> > +	*_device_event_dma_addr  = device_event_dma_addr;
> > +	*_ring_size_in_bytes     = ring_size_in_bytes;
> > +	*_event_size_in_bytes    = event_size_in_bytes;
>
>
> I wonder if we can simply factor out split and packed from struct
> vring_virtqueue:
>
> struct vring_virtqueue {
>      union {
>          struct {} split;
>          struct {} packed;
>      };
> };
>
> to
>
> struct vring_virtqueue_split {};
> struct vring_virtqueue_packed {};
>
> Then we can do things like:
>
> vring_create_virtqueue_packed(struct virtio_device *vdev, u32 num,
> struct vring_virtqueue_packed *packed);
>
> and
>
> vring_vritqueue_attach_packed(struct vring_virtqueue *vq, struct
> vring_virtqueue_packed packed);

This idea is very similar to my previous idea, just without introducing a new
structure.

I'd be more than happy to revise this.

Thanks.


>
> Thanks
>
>
> > +
> > +	return 0;
> > +
> > +err_device:
> > +	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> > +
> > +err_driver:
> > +	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> > +
> > +err_ring:
> > +	return -ENOMEM;
> > +}
> > +
> > +static struct virtqueue *vring_create_virtqueue_packed(
> > +	unsigned int index,
> > +	unsigned int num,
> > +	unsigned int vring_align,
> > +	struct virtio_device *vdev,
> > +	bool weak_barriers,
> > +	bool may_reduce_num,
> > +	bool context,
> > +	bool (*notify)(struct virtqueue *),
> > +	void (*callback)(struct virtqueue *),
> > +	const char *name)
> > +{
> > +	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> > +	struct vring_packed_desc_event *driver, *device;
> > +	size_t ring_size_in_bytes, event_size_in_bytes;
> > +	struct vring_packed_desc *ring;
> > +	struct vring_virtqueue *vq;
> > +
> > +	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
> > +				     &ring_dma_addr, &driver_event_dma_addr,
> > +				     &device_event_dma_addr,
> > +				     &ring_size_in_bytes,
> > +				     &event_size_in_bytes))
> > +		goto err_ring;
> > +
> >   	vq = kmalloc(sizeof(*vq), GFP_KERNEL);
> >   	if (!vq)
> >   		goto err_vq;
> > @@ -1939,9 +1983,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> >   	kfree(vq);
> >   err_vq:
> >   	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
> > -err_device:
> >   	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> > -err_driver:
> >   	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> >   err_ring:
> >   	return NULL;
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 12/32] virtio_ring: packed: extract the logic of alloc queue

[Xuan Zhuo]

On Tue, 12 Apr 2022 14:28:24 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of packed to create vring queue.
> >
> > For the convenience of passing parameters, add a structure
> > vring_packed.
> >
> > This feature is required for subsequent virtuqueue reset vring.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 70 ++++++++++++++++++++++++++++--------
> >   1 file changed, 56 insertions(+), 14 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 33864134a744..ea451ae2aaef 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -1817,19 +1817,17 @@ static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
> >   	return desc_extra;
> >   }
> >
> > -static struct virtqueue *vring_create_virtqueue_packed(
> > -	unsigned int index,
> > -	unsigned int num,
> > -	unsigned int vring_align,
> > -	struct virtio_device *vdev,
> > -	bool weak_barriers,
> > -	bool may_reduce_num,
> > -	bool context,
> > -	bool (*notify)(struct virtqueue *),
> > -	void (*callback)(struct virtqueue *),
> > -	const char *name)
> > +static int vring_alloc_queue_packed(struct virtio_device *vdev,
> > +				    u32 num,
> > +				    struct vring_packed_desc **_ring,
> > +				    struct vring_packed_desc_event **_driver,
> > +				    struct vring_packed_desc_event **_device,
> > +				    dma_addr_t *_ring_dma_addr,
> > +				    dma_addr_t *_driver_event_dma_addr,
> > +				    dma_addr_t *_device_event_dma_addr,
> > +				    size_t *_ring_size_in_bytes,
> > +				    size_t *_event_size_in_bytes)
> >   {
> > -	struct vring_virtqueue *vq;
> >   	struct vring_packed_desc *ring;
> >   	struct vring_packed_desc_event *driver, *device;
> >   	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> > @@ -1857,6 +1855,52 @@ static struct virtqueue *vring_create_virtqueue_packed(
> >   	if (!device)
> >   		goto err_device;
> >
> > +	*_ring                   = ring;
> > +	*_driver                 = driver;
> > +	*_device                 = device;
> > +	*_ring_dma_addr          = ring_dma_addr;
> > +	*_driver_event_dma_addr  = driver_event_dma_addr;
> > +	*_device_event_dma_addr  = device_event_dma_addr;
> > +	*_ring_size_in_bytes     = ring_size_in_bytes;
> > +	*_event_size_in_bytes    = event_size_in_bytes;
>
>
> I wonder if we can simply factor out split and packed from struct
> vring_virtqueue:
>
> struct vring_virtqueue {
>      union {
>          struct {} split;
>          struct {} packed;
>      };
> };
>
> to
>
> struct vring_virtqueue_split {};
> struct vring_virtqueue_packed {};
>
> Then we can do things like:
>
> vring_create_virtqueue_packed(struct virtio_device *vdev, u32 num,
> struct vring_virtqueue_packed *packed);
>
> and
>
> vring_vritqueue_attach_packed(struct vring_virtqueue *vq, struct
> vring_virtqueue_packed packed);

This idea is very similar to my previous idea, just without introducing a new
structure.

I'd be more than happy to revise this.

Thanks.


>
> Thanks
>
>
> > +
> > +	return 0;
> > +
> > +err_device:
> > +	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> > +
> > +err_driver:
> > +	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> > +
> > +err_ring:
> > +	return -ENOMEM;
> > +}
> > +
> > +static struct virtqueue *vring_create_virtqueue_packed(
> > +	unsigned int index,
> > +	unsigned int num,
> > +	unsigned int vring_align,
> > +	struct virtio_device *vdev,
> > +	bool weak_barriers,
> > +	bool may_reduce_num,
> > +	bool context,
> > +	bool (*notify)(struct virtqueue *),
> > +	void (*callback)(struct virtqueue *),
> > +	const char *name)
> > +{
> > +	dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> > +	struct vring_packed_desc_event *driver, *device;
> > +	size_t ring_size_in_bytes, event_size_in_bytes;
> > +	struct vring_packed_desc *ring;
> > +	struct vring_virtqueue *vq;
> > +
> > +	if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
> > +				     &ring_dma_addr, &driver_event_dma_addr,
> > +				     &device_event_dma_addr,
> > +				     &ring_size_in_bytes,
> > +				     &event_size_in_bytes))
> > +		goto err_ring;
> > +
> >   	vq = kmalloc(sizeof(*vq), GFP_KERNEL);
> >   	if (!vq)
> >   		goto err_vq;
> > @@ -1939,9 +1983,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> >   	kfree(vq);
> >   err_vq:
> >   	vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
> > -err_device:
> >   	vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> > -err_driver:
> >   	vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> >   err_ring:
> >   	return NULL;
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 11/32] virtio_ring: split: introduce virtqueue_resize_split()

[Xuan Zhuo]

On Tue, 12 Apr 2022 13:53:44 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > virtio ring split supports resize.
> >
> > Only after the new vring is successfully allocated based on the new num,
> > we will release the old vring. In any case, an error is returned,
> > indicating that the vring still points to the old vring.
> >
> > In the case of an error, the caller must
> > re-initialize(virtqueue_reinit_split()) the virtqueue to ensure that the
> > vring can be used.
> >
> > In addition, vring_align, may_reduce_num are necessary for reallocating
> > vring, so they are retained for creating vq.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 47 ++++++++++++++++++++++++++++++++++++
> >   1 file changed, 47 insertions(+)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 3dc6ace2ba7a..33864134a744 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -139,6 +139,12 @@ struct vring_virtqueue {
> >   			/* DMA address and size information */
> >   			dma_addr_t queue_dma_addr;
> >   			size_t queue_size_in_bytes;
> > +
> > +			/* The parameters for creating vrings are reserved for
> > +			 * creating new vring.
> > +			 */
> > +			u32 vring_align;
> > +			bool may_reduce_num;
> >   		} split;
> >
> >   		/* Available for packed ring */
> > @@ -199,6 +205,7 @@ struct vring_virtqueue {
> >   };
> >
> >   static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
> > +static void vring_free(struct virtqueue *_vq);
> >
> >   /*
> >    * Helpers.
> > @@ -1088,6 +1095,8 @@ static struct virtqueue *vring_create_virtqueue_split(
> >   		return NULL;
> >   	}
> >
> > +	to_vvq(vq)->split.vring_align = vring_align;
> > +	to_vvq(vq)->split.may_reduce_num = may_reduce_num;
>
>
> It looks to me the above should belong to patch 6.

patch 6 just extracts a function, no logical modification.

to_vvq(vq)->split.may_reduce_num is newly added, so I don't think it should be
merged into patch 6.

>
>
> >   	to_vvq(vq)->split.queue_dma_addr = dma_addr;
> >   	to_vvq(vq)->split.queue_size_in_bytes = queue_size_in_bytes;
> >   	to_vvq(vq)->we_own_ring = true;
> > @@ -1095,6 +1104,44 @@ static struct virtqueue *vring_create_virtqueue_split(
> >   	return vq;
> >   }
> >
> > +static int virtqueue_resize_split(struct virtqueue *_vq, u32 num)
> > +{
> > +	struct vring_virtqueue *vq = to_vvq(_vq);
> > +	struct virtio_device *vdev = _vq->vdev;
> > +	struct vring_desc_state_split *state;
> > +	struct vring_desc_extra *extra;
> > +	size_t queue_size_in_bytes;
> > +	dma_addr_t dma_addr;
> > +	struct vring vring;
> > +	int err = -ENOMEM;
> > +	void *queue;
> > +
> > +	queue = vring_alloc_queue_split(vdev, &dma_addr, &num,
> > +					vq->split.vring_align,
> > +					vq->weak_barriers,
> > +					vq->split.may_reduce_num);
> > +	if (!queue)
> > +		return -ENOMEM;
> > +
> > +	queue_size_in_bytes = vring_size(num, vq->split.vring_align);
> > +
> > +	err = vring_alloc_state_extra_split(num, &state, &extra);
> > +	if (err) {
> > +		vring_free_queue(vdev, queue_size_in_bytes, queue, dma_addr);
> > +		return -ENOMEM;
> > +	}
> > +
> > +	vring_free(&vq->vq);
> > +
> > +	vring_init(&vring, num, queue, vq->split.vring_align);
> > +	vring_virtqueue_attach_split(vq, vring, state, extra);
> > +	vq->split.queue_dma_addr = dma_addr;
> > +	vq->split.queue_size_in_bytes = queue_size_in_bytes;
>
>
> I wonder if it's better to move the above assignments to
> vring_virtqueue_attach_split().

I also think so, the reason for not doing this is that there is no dma_addr and
queue_size_in_bytes when vring_virtqueue_attach_split is called in
__vring_new_virtqueue.

As discussed in patch 12, we can pass the struct struct vring_virtqueue_split to
vring_virtqueue_attach_split(). This is much more convenient.

Thanks.

>
> Other looks good.
>
> Thanks
>
>
> > +
> > +	vring_virtqueue_init_split(vq, vdev, true);
> > +	return 0;
> > +}
> > +
> >
> >   /*
> >    * Packed ring specific functions - *_packed().
>

Re: [PATCH v9 11/32] virtio_ring: split: introduce virtqueue_resize_split()

[Xuan Zhuo]

On Tue, 12 Apr 2022 13:53:44 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > virtio ring split supports resize.
> >
> > Only after the new vring is successfully allocated based on the new num,
> > we will release the old vring. In any case, an error is returned,
> > indicating that the vring still points to the old vring.
> >
> > In the case of an error, the caller must
> > re-initialize(virtqueue_reinit_split()) the virtqueue to ensure that the
> > vring can be used.
> >
> > In addition, vring_align, may_reduce_num are necessary for reallocating
> > vring, so they are retained for creating vq.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 47 ++++++++++++++++++++++++++++++++++++
> >   1 file changed, 47 insertions(+)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 3dc6ace2ba7a..33864134a744 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -139,6 +139,12 @@ struct vring_virtqueue {
> >   			/* DMA address and size information */
> >   			dma_addr_t queue_dma_addr;
> >   			size_t queue_size_in_bytes;
> > +
> > +			/* The parameters for creating vrings are reserved for
> > +			 * creating new vring.
> > +			 */
> > +			u32 vring_align;
> > +			bool may_reduce_num;
> >   		} split;
> >
> >   		/* Available for packed ring */
> > @@ -199,6 +205,7 @@ struct vring_virtqueue {
> >   };
> >
> >   static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
> > +static void vring_free(struct virtqueue *_vq);
> >
> >   /*
> >    * Helpers.
> > @@ -1088,6 +1095,8 @@ static struct virtqueue *vring_create_virtqueue_split(
> >   		return NULL;
> >   	}
> >
> > +	to_vvq(vq)->split.vring_align = vring_align;
> > +	to_vvq(vq)->split.may_reduce_num = may_reduce_num;
>
>
> It looks to me the above should belong to patch 6.

patch 6 just extracts a function, no logical modification.

to_vvq(vq)->split.may_reduce_num is newly added, so I don't think it should be
merged into patch 6.

>
>
> >   	to_vvq(vq)->split.queue_dma_addr = dma_addr;
> >   	to_vvq(vq)->split.queue_size_in_bytes = queue_size_in_bytes;
> >   	to_vvq(vq)->we_own_ring = true;
> > @@ -1095,6 +1104,44 @@ static struct virtqueue *vring_create_virtqueue_split(
> >   	return vq;
> >   }
> >
> > +static int virtqueue_resize_split(struct virtqueue *_vq, u32 num)
> > +{
> > +	struct vring_virtqueue *vq = to_vvq(_vq);
> > +	struct virtio_device *vdev = _vq->vdev;
> > +	struct vring_desc_state_split *state;
> > +	struct vring_desc_extra *extra;
> > +	size_t queue_size_in_bytes;
> > +	dma_addr_t dma_addr;
> > +	struct vring vring;
> > +	int err = -ENOMEM;
> > +	void *queue;
> > +
> > +	queue = vring_alloc_queue_split(vdev, &dma_addr, &num,
> > +					vq->split.vring_align,
> > +					vq->weak_barriers,
> > +					vq->split.may_reduce_num);
> > +	if (!queue)
> > +		return -ENOMEM;
> > +
> > +	queue_size_in_bytes = vring_size(num, vq->split.vring_align);
> > +
> > +	err = vring_alloc_state_extra_split(num, &state, &extra);
> > +	if (err) {
> > +		vring_free_queue(vdev, queue_size_in_bytes, queue, dma_addr);
> > +		return -ENOMEM;
> > +	}
> > +
> > +	vring_free(&vq->vq);
> > +
> > +	vring_init(&vring, num, queue, vq->split.vring_align);
> > +	vring_virtqueue_attach_split(vq, vring, state, extra);
> > +	vq->split.queue_dma_addr = dma_addr;
> > +	vq->split.queue_size_in_bytes = queue_size_in_bytes;
>
>
> I wonder if it's better to move the above assignments to
> vring_virtqueue_attach_split().

I also think so, the reason for not doing this is that there is no dma_addr and
queue_size_in_bytes when vring_virtqueue_attach_split is called in
__vring_new_virtqueue.

As discussed in patch 12, we can pass the struct struct vring_virtqueue_split to
vring_virtqueue_attach_split(). This is much more convenient.

Thanks.

>
> Other looks good.
>
> Thanks
>
>
> > +
> > +	vring_virtqueue_init_split(vq, vdev, true);
> > +	return 0;
> > +}
> > +
> >
> >   /*
> >    * Packed ring specific functions - *_packed().
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 11/32] virtio_ring: split: introduce virtqueue_resize_split()

[Xuan Zhuo]

On Tue, 12 Apr 2022 13:53:44 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > virtio ring split supports resize.
> >
> > Only after the new vring is successfully allocated based on the new num,
> > we will release the old vring. In any case, an error is returned,
> > indicating that the vring still points to the old vring.
> >
> > In the case of an error, the caller must
> > re-initialize(virtqueue_reinit_split()) the virtqueue to ensure that the
> > vring can be used.
> >
> > In addition, vring_align, may_reduce_num are necessary for reallocating
> > vring, so they are retained for creating vq.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 47 ++++++++++++++++++++++++++++++++++++
> >   1 file changed, 47 insertions(+)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 3dc6ace2ba7a..33864134a744 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -139,6 +139,12 @@ struct vring_virtqueue {
> >   			/* DMA address and size information */
> >   			dma_addr_t queue_dma_addr;
> >   			size_t queue_size_in_bytes;
> > +
> > +			/* The parameters for creating vrings are reserved for
> > +			 * creating new vring.
> > +			 */
> > +			u32 vring_align;
> > +			bool may_reduce_num;
> >   		} split;
> >
> >   		/* Available for packed ring */
> > @@ -199,6 +205,7 @@ struct vring_virtqueue {
> >   };
> >
> >   static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
> > +static void vring_free(struct virtqueue *_vq);
> >
> >   /*
> >    * Helpers.
> > @@ -1088,6 +1095,8 @@ static struct virtqueue *vring_create_virtqueue_split(
> >   		return NULL;
> >   	}
> >
> > +	to_vvq(vq)->split.vring_align = vring_align;
> > +	to_vvq(vq)->split.may_reduce_num = may_reduce_num;
>
>
> It looks to me the above should belong to patch 6.

patch 6 just extracts a function, no logical modification.

to_vvq(vq)->split.may_reduce_num is newly added, so I don't think it should be
merged into patch 6.

>
>
> >   	to_vvq(vq)->split.queue_dma_addr = dma_addr;
> >   	to_vvq(vq)->split.queue_size_in_bytes = queue_size_in_bytes;
> >   	to_vvq(vq)->we_own_ring = true;
> > @@ -1095,6 +1104,44 @@ static struct virtqueue *vring_create_virtqueue_split(
> >   	return vq;
> >   }
> >
> > +static int virtqueue_resize_split(struct virtqueue *_vq, u32 num)
> > +{
> > +	struct vring_virtqueue *vq = to_vvq(_vq);
> > +	struct virtio_device *vdev = _vq->vdev;
> > +	struct vring_desc_state_split *state;
> > +	struct vring_desc_extra *extra;
> > +	size_t queue_size_in_bytes;
> > +	dma_addr_t dma_addr;
> > +	struct vring vring;
> > +	int err = -ENOMEM;
> > +	void *queue;
> > +
> > +	queue = vring_alloc_queue_split(vdev, &dma_addr, &num,
> > +					vq->split.vring_align,
> > +					vq->weak_barriers,
> > +					vq->split.may_reduce_num);
> > +	if (!queue)
> > +		return -ENOMEM;
> > +
> > +	queue_size_in_bytes = vring_size(num, vq->split.vring_align);
> > +
> > +	err = vring_alloc_state_extra_split(num, &state, &extra);
> > +	if (err) {
> > +		vring_free_queue(vdev, queue_size_in_bytes, queue, dma_addr);
> > +		return -ENOMEM;
> > +	}
> > +
> > +	vring_free(&vq->vq);
> > +
> > +	vring_init(&vring, num, queue, vq->split.vring_align);
> > +	vring_virtqueue_attach_split(vq, vring, state, extra);
> > +	vq->split.queue_dma_addr = dma_addr;
> > +	vq->split.queue_size_in_bytes = queue_size_in_bytes;
>
>
> I wonder if it's better to move the above assignments to
> vring_virtqueue_attach_split().

I also think so, the reason for not doing this is that there is no dma_addr and
queue_size_in_bytes when vring_virtqueue_attach_split is called in
__vring_new_virtqueue.

As discussed in patch 12, we can pass the struct struct vring_virtqueue_split to
vring_virtqueue_attach_split(). This is much more convenient.

Thanks.

>
> Other looks good.
>
> Thanks
>
>
> > +
> > +	vring_virtqueue_init_split(vq, vdev, true);
> > +	return 0;
> > +}
> > +
> >
> >   /*
> >    * Packed ring specific functions - *_packed().
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 08/32] virtio_ring: split: extract the logic of attach vring

[Xuan Zhuo]

On Tue, 12 Apr 2022 11:31:08 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of attach vring, subsequent patches will call it
> > separately.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 20 ++++++++++++++------
> >   1 file changed, 14 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 6de67439cb57..083f2992ba0d 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -916,6 +916,19 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
> >   	return NULL;
> >   }
> >
> > +static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
> > +					 struct vring vring,
> > +					 struct vring_desc_state_split *desc_state,
> > +					 struct vring_desc_extra *desc_extra)
> > +{
> > +	vq->split.vring = vring;
> > +	vq->split.queue_dma_addr = 0;
> > +	vq->split.queue_size_in_bytes = 0;
>
>
> Any reason to add the above two assignment in attach? It seems belong to
> free or reset.

As discussed in patch 11, since there is no dma_addr in __vring_new_virtqueue(),
the corresponding vq->split.queue_dma_addr cannot be set, so the purpose here
is just to initialize it.

In the next version, struct vring_virtqueue_split will be passed to
vring_virtqueue_attach_split() to make the logic here look more reasonable.

Thanks.


>
> Thanks
>
>
> > +
> > +	vq->split.desc_state = desc_state;
> > +	vq->split.desc_extra = desc_extra;
> > +}
> > +
> >   static int vring_alloc_state_extra_split(u32 num,
> >   					 struct vring_desc_state_split **desc_state,
> >   					 struct vring_desc_extra **desc_extra)
> > @@ -2262,10 +2275,6 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> >   		vq->weak_barriers = false;
> >
> > -	vq->split.queue_dma_addr = 0;
> > -	vq->split.queue_size_in_bytes = 0;
> > -
> > -	vq->split.vring = vring;
> >   	vq->split.avail_flags_shadow = 0;
> >   	vq->split.avail_idx_shadow = 0;
> >
> > @@ -2283,8 +2292,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   		return NULL;
> >   	}
> >
> > -	vq->split.desc_state = state;
> > -	vq->split.desc_extra = extra;
> > +	vring_virtqueue_attach_split(vq, vring, state, extra);
> >
> >   	/* Put everything in free lists. */
> >   	vq->free_head = 0;
>

Re: [PATCH v9 08/32] virtio_ring: split: extract the logic of attach vring

[Xuan Zhuo]

On Tue, 12 Apr 2022 11:31:08 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of attach vring, subsequent patches will call it
> > separately.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 20 ++++++++++++++------
> >   1 file changed, 14 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 6de67439cb57..083f2992ba0d 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -916,6 +916,19 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
> >   	return NULL;
> >   }
> >
> > +static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
> > +					 struct vring vring,
> > +					 struct vring_desc_state_split *desc_state,
> > +					 struct vring_desc_extra *desc_extra)
> > +{
> > +	vq->split.vring = vring;
> > +	vq->split.queue_dma_addr = 0;
> > +	vq->split.queue_size_in_bytes = 0;
>
>
> Any reason to add the above two assignment in attach? It seems belong to
> free or reset.

As discussed in patch 11, since there is no dma_addr in __vring_new_virtqueue(),
the corresponding vq->split.queue_dma_addr cannot be set, so the purpose here
is just to initialize it.

In the next version, struct vring_virtqueue_split will be passed to
vring_virtqueue_attach_split() to make the logic here look more reasonable.

Thanks.


>
> Thanks
>
>
> > +
> > +	vq->split.desc_state = desc_state;
> > +	vq->split.desc_extra = desc_extra;
> > +}
> > +
> >   static int vring_alloc_state_extra_split(u32 num,
> >   					 struct vring_desc_state_split **desc_state,
> >   					 struct vring_desc_extra **desc_extra)
> > @@ -2262,10 +2275,6 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> >   		vq->weak_barriers = false;
> >
> > -	vq->split.queue_dma_addr = 0;
> > -	vq->split.queue_size_in_bytes = 0;
> > -
> > -	vq->split.vring = vring;
> >   	vq->split.avail_flags_shadow = 0;
> >   	vq->split.avail_idx_shadow = 0;
> >
> > @@ -2283,8 +2292,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   		return NULL;
> >   	}
> >
> > -	vq->split.desc_state = state;
> > -	vq->split.desc_extra = extra;
> > +	vring_virtqueue_attach_split(vq, vring, state, extra);
> >
> >   	/* Put everything in free lists. */
> >   	vq->free_head = 0;
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 08/32] virtio_ring: split: extract the logic of attach vring

[Xuan Zhuo]

On Tue, 12 Apr 2022 11:31:08 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of attach vring, subsequent patches will call it
> > separately.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 20 ++++++++++++++------
> >   1 file changed, 14 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 6de67439cb57..083f2992ba0d 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -916,6 +916,19 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
> >   	return NULL;
> >   }
> >
> > +static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
> > +					 struct vring vring,
> > +					 struct vring_desc_state_split *desc_state,
> > +					 struct vring_desc_extra *desc_extra)
> > +{
> > +	vq->split.vring = vring;
> > +	vq->split.queue_dma_addr = 0;
> > +	vq->split.queue_size_in_bytes = 0;
>
>
> Any reason to add the above two assignment in attach? It seems belong to
> free or reset.

As discussed in patch 11, since there is no dma_addr in __vring_new_virtqueue(),
the corresponding vq->split.queue_dma_addr cannot be set, so the purpose here
is just to initialize it.

In the next version, struct vring_virtqueue_split will be passed to
vring_virtqueue_attach_split() to make the logic here look more reasonable.

Thanks.


>
> Thanks
>
>
> > +
> > +	vq->split.desc_state = desc_state;
> > +	vq->split.desc_extra = desc_extra;
> > +}
> > +
> >   static int vring_alloc_state_extra_split(u32 num,
> >   					 struct vring_desc_state_split **desc_state,
> >   					 struct vring_desc_extra **desc_extra)
> > @@ -2262,10 +2275,6 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> >   		vq->weak_barriers = false;
> >
> > -	vq->split.queue_dma_addr = 0;
> > -	vq->split.queue_size_in_bytes = 0;
> > -
> > -	vq->split.vring = vring;
> >   	vq->split.avail_flags_shadow = 0;
> >   	vq->split.avail_idx_shadow = 0;
> >
> > @@ -2283,8 +2292,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   		return NULL;
> >   	}
> >
> > -	vq->split.desc_state = state;
> > -	vq->split.desc_extra = extra;
> > +	vring_virtqueue_attach_split(vq, vring, state, extra);
> >
> >   	/* Put everything in free lists. */
> >   	vq->free_head = 0;
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 07/32] virtio_ring: split: extract the logic of alloc state and extra

[Xuan Zhuo]

On Tue, 12 Apr 2022 11:26:49 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of creating desc_state, desc_extra, and subsequent
> > patches will call it independently.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++++----------
> >   1 file changed, 38 insertions(+), 15 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 72d5ae063fa0..6de67439cb57 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -198,6 +198,7 @@ struct vring_virtqueue {
> >   #endif
> >   };
> >
> > +static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
> >
> >   /*
> >    * Helpers.
> > @@ -915,6 +916,33 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
> >   	return NULL;
> >   }
> >
> > +static int vring_alloc_state_extra_split(u32 num,
> > +					 struct vring_desc_state_split **desc_state,
> > +					 struct vring_desc_extra **desc_extra)
> > +{
> > +	struct vring_desc_state_split *state;
> > +	struct vring_desc_extra *extra;
> > +
> > +	state = kmalloc_array(num, sizeof(struct vring_desc_state_split), GFP_KERNEL);
> > +	if (!state)
> > +		goto err_state;
> > +
> > +	extra = vring_alloc_desc_extra(num);
> > +	if (!extra)
> > +		goto err_extra;
> > +
> > +	memset(state, 0, num * sizeof(struct vring_desc_state_split));
> > +
> > +	*desc_state = state;
> > +	*desc_extra = extra;
> > +	return 0;
> > +
> > +err_extra:
> > +	kfree(state);
> > +err_state:
> > +	return -ENOMEM;
> > +}
> > +
> >   static void *vring_alloc_queue_split(struct virtio_device *vdev,
> >   				     dma_addr_t *dma_addr,
> >   				     u32 *n,
> > @@ -2196,7 +2224,10 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   					void (*callback)(struct virtqueue *),
> >   					const char *name)
> >   {
> > +	struct vring_desc_state_split *state;
> > +	struct vring_desc_extra *extra;
> >   	struct vring_virtqueue *vq;
> > +	int err;
> >
> >   	if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED))
> >   		return NULL;
> > @@ -2246,30 +2277,22 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   					vq->split.avail_flags_shadow);
> >   	}
> >
> > -	vq->split.desc_state = kmalloc_array(vring.num,
> > -			sizeof(struct vring_desc_state_split), GFP_KERNEL);
> > -	if (!vq->split.desc_state)
> > -		goto err_state;
> > +	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
>
>
> Nit: we can pass e.g &vq->split.desc_state here to avoid extra temp
> variable and assignment.

The reason for not doing this is that when we implement resize later, when we
call vring_alloc_state_extra_split(), we want to keep the old desc_state, and
desc_extra because we want to release them.

As discussed in patch 11, 12, struct vring_virtqueue_split will optimize this
logic.

Thanks.

>
> Other looks good.
>
> Thanks
>
>
> > +	if (err) {
> > +		kfree(vq);
> > +		return NULL;
> > +	}
> >
> > -	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
> > -	if (!vq->split.desc_extra)
> > -		goto err_extra;
> > +	vq->split.desc_state = state;
> > +	vq->split.desc_extra = extra;
> >
> >   	/* Put everything in free lists. */
> >   	vq->free_head = 0;
> > -	memset(vq->split.desc_state, 0, vring.num *
> > -			sizeof(struct vring_desc_state_split));
> >
> >   	spin_lock(&vdev->vqs_list_lock);
> >   	list_add_tail(&vq->vq.list, &vdev->vqs);
> >   	spin_unlock(&vdev->vqs_list_lock);
> >   	return &vq->vq;
> > -
> > -err_extra:
> > -	kfree(vq->split.desc_state);
> > -err_state:
> > -	kfree(vq);
> > -	return NULL;
> >   }
> >   EXPORT_SYMBOL_GPL(__vring_new_virtqueue);
> >
>

Re: [PATCH v9 07/32] virtio_ring: split: extract the logic of alloc state and extra

[Xuan Zhuo]

On Tue, 12 Apr 2022 11:26:49 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of creating desc_state, desc_extra, and subsequent
> > patches will call it independently.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++++----------
> >   1 file changed, 38 insertions(+), 15 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 72d5ae063fa0..6de67439cb57 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -198,6 +198,7 @@ struct vring_virtqueue {
> >   #endif
> >   };
> >
> > +static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
> >
> >   /*
> >    * Helpers.
> > @@ -915,6 +916,33 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
> >   	return NULL;
> >   }
> >
> > +static int vring_alloc_state_extra_split(u32 num,
> > +					 struct vring_desc_state_split **desc_state,
> > +					 struct vring_desc_extra **desc_extra)
> > +{
> > +	struct vring_desc_state_split *state;
> > +	struct vring_desc_extra *extra;
> > +
> > +	state = kmalloc_array(num, sizeof(struct vring_desc_state_split), GFP_KERNEL);
> > +	if (!state)
> > +		goto err_state;
> > +
> > +	extra = vring_alloc_desc_extra(num);
> > +	if (!extra)
> > +		goto err_extra;
> > +
> > +	memset(state, 0, num * sizeof(struct vring_desc_state_split));
> > +
> > +	*desc_state = state;
> > +	*desc_extra = extra;
> > +	return 0;
> > +
> > +err_extra:
> > +	kfree(state);
> > +err_state:
> > +	return -ENOMEM;
> > +}
> > +
> >   static void *vring_alloc_queue_split(struct virtio_device *vdev,
> >   				     dma_addr_t *dma_addr,
> >   				     u32 *n,
> > @@ -2196,7 +2224,10 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   					void (*callback)(struct virtqueue *),
> >   					const char *name)
> >   {
> > +	struct vring_desc_state_split *state;
> > +	struct vring_desc_extra *extra;
> >   	struct vring_virtqueue *vq;
> > +	int err;
> >
> >   	if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED))
> >   		return NULL;
> > @@ -2246,30 +2277,22 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   					vq->split.avail_flags_shadow);
> >   	}
> >
> > -	vq->split.desc_state = kmalloc_array(vring.num,
> > -			sizeof(struct vring_desc_state_split), GFP_KERNEL);
> > -	if (!vq->split.desc_state)
> > -		goto err_state;
> > +	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
>
>
> Nit: we can pass e.g &vq->split.desc_state here to avoid extra temp
> variable and assignment.

The reason for not doing this is that when we implement resize later, when we
call vring_alloc_state_extra_split(), we want to keep the old desc_state, and
desc_extra because we want to release them.

As discussed in patch 11, 12, struct vring_virtqueue_split will optimize this
logic.

Thanks.

>
> Other looks good.
>
> Thanks
>
>
> > +	if (err) {
> > +		kfree(vq);
> > +		return NULL;
> > +	}
> >
> > -	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
> > -	if (!vq->split.desc_extra)
> > -		goto err_extra;
> > +	vq->split.desc_state = state;
> > +	vq->split.desc_extra = extra;
> >
> >   	/* Put everything in free lists. */
> >   	vq->free_head = 0;
> > -	memset(vq->split.desc_state, 0, vring.num *
> > -			sizeof(struct vring_desc_state_split));
> >
> >   	spin_lock(&vdev->vqs_list_lock);
> >   	list_add_tail(&vq->vq.list, &vdev->vqs);
> >   	spin_unlock(&vdev->vqs_list_lock);
> >   	return &vq->vq;
> > -
> > -err_extra:
> > -	kfree(vq->split.desc_state);
> > -err_state:
> > -	kfree(vq);
> > -	return NULL;
> >   }
> >   EXPORT_SYMBOL_GPL(__vring_new_virtqueue);
> >
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 07/32] virtio_ring: split: extract the logic of alloc state and extra

[Xuan Zhuo]

On Tue, 12 Apr 2022 11:26:49 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of creating desc_state, desc_extra, and subsequent
> > patches will call it independently.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++++----------
> >   1 file changed, 38 insertions(+), 15 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 72d5ae063fa0..6de67439cb57 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -198,6 +198,7 @@ struct vring_virtqueue {
> >   #endif
> >   };
> >
> > +static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
> >
> >   /*
> >    * Helpers.
> > @@ -915,6 +916,33 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
> >   	return NULL;
> >   }
> >
> > +static int vring_alloc_state_extra_split(u32 num,
> > +					 struct vring_desc_state_split **desc_state,
> > +					 struct vring_desc_extra **desc_extra)
> > +{
> > +	struct vring_desc_state_split *state;
> > +	struct vring_desc_extra *extra;
> > +
> > +	state = kmalloc_array(num, sizeof(struct vring_desc_state_split), GFP_KERNEL);
> > +	if (!state)
> > +		goto err_state;
> > +
> > +	extra = vring_alloc_desc_extra(num);
> > +	if (!extra)
> > +		goto err_extra;
> > +
> > +	memset(state, 0, num * sizeof(struct vring_desc_state_split));
> > +
> > +	*desc_state = state;
> > +	*desc_extra = extra;
> > +	return 0;
> > +
> > +err_extra:
> > +	kfree(state);
> > +err_state:
> > +	return -ENOMEM;
> > +}
> > +
> >   static void *vring_alloc_queue_split(struct virtio_device *vdev,
> >   				     dma_addr_t *dma_addr,
> >   				     u32 *n,
> > @@ -2196,7 +2224,10 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   					void (*callback)(struct virtqueue *),
> >   					const char *name)
> >   {
> > +	struct vring_desc_state_split *state;
> > +	struct vring_desc_extra *extra;
> >   	struct vring_virtqueue *vq;
> > +	int err;
> >
> >   	if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED))
> >   		return NULL;
> > @@ -2246,30 +2277,22 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   					vq->split.avail_flags_shadow);
> >   	}
> >
> > -	vq->split.desc_state = kmalloc_array(vring.num,
> > -			sizeof(struct vring_desc_state_split), GFP_KERNEL);
> > -	if (!vq->split.desc_state)
> > -		goto err_state;
> > +	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
>
>
> Nit: we can pass e.g &vq->split.desc_state here to avoid extra temp
> variable and assignment.

The reason for not doing this is that when we implement resize later, when we
call vring_alloc_state_extra_split(), we want to keep the old desc_state, and
desc_extra because we want to release them.

As discussed in patch 11, 12, struct vring_virtqueue_split will optimize this
logic.

Thanks.

>
> Other looks good.
>
> Thanks
>
>
> > +	if (err) {
> > +		kfree(vq);
> > +		return NULL;
> > +	}
> >
> > -	vq->split.desc_extra = vring_alloc_desc_extra(vring.num);
> > -	if (!vq->split.desc_extra)
> > -		goto err_extra;
> > +	vq->split.desc_state = state;
> > +	vq->split.desc_extra = extra;
> >
> >   	/* Put everything in free lists. */
> >   	vq->free_head = 0;
> > -	memset(vq->split.desc_state, 0, vring.num *
> > -			sizeof(struct vring_desc_state_split));
> >
> >   	spin_lock(&vdev->vqs_list_lock);
> >   	list_add_tail(&vq->vq.list, &vdev->vqs);
> >   	spin_unlock(&vdev->vqs_list_lock);
> >   	return &vq->vq;
> > -
> > -err_extra:
> > -	kfree(vq->split.desc_state);
> > -err_state:
> > -	kfree(vq);
> > -	return NULL;
> >   }
> >   EXPORT_SYMBOL_GPL(__vring_new_virtqueue);
> >
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 06/32] virtio_ring: split: extract the logic of alloc queue

[Xuan Zhuo]

On Tue, 12 Apr 2022 11:22:33 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of split to create vring queue.
> >
> > This feature is required for subsequent virtuqueue reset vring.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++------------
> >   1 file changed, 36 insertions(+), 17 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 33fddfb907a6..72d5ae063fa0 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -915,23 +915,15 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
> >   	return NULL;
> >   }
> >
> > -static struct virtqueue *vring_create_virtqueue_split(
> > -	unsigned int index,
> > -	unsigned int num,
> > -	unsigned int vring_align,
> > -	struct virtio_device *vdev,
> > -	bool weak_barriers,
> > -	bool may_reduce_num,
> > -	bool context,
> > -	bool (*notify)(struct virtqueue *),
> > -	void (*callback)(struct virtqueue *),
> > -	const char *name)
> > +static void *vring_alloc_queue_split(struct virtio_device *vdev,
> > +				     dma_addr_t *dma_addr,
> > +				     u32 *n,
> > +				     unsigned int vring_align,
> > +				     bool weak_barriers,
>
>
> This is not used in this function.

The next version will fix it.

Thanks.


>
> Thanks
>
>
> > +				     bool may_reduce_num)
> >   {
> > -	struct virtqueue *vq;
> >   	void *queue = NULL;
> > -	dma_addr_t dma_addr;
> > -	size_t queue_size_in_bytes;
> > -	struct vring vring;
> > +	u32 num = *n;
> >
> >   	/* We assume num is a power of 2. */
> >   	if (num & (num - 1)) {
> > @@ -942,7 +934,7 @@ static struct virtqueue *vring_create_virtqueue_split(
> >   	/* TODO: allocate each queue chunk individually */
> >   	for (; num && vring_size(num, vring_align) > PAGE_SIZE; num /= 2) {
> >   		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
> > -					  &dma_addr,
> > +					  dma_addr,
> >   					  GFP_KERNEL|__GFP_NOWARN|__GFP_ZERO);
> >   		if (queue)
> >   			break;
> > @@ -956,11 +948,38 @@ static struct virtqueue *vring_create_virtqueue_split(
> >   	if (!queue) {
> >   		/* Try to get a single page. You are my only hope! */
> >   		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
> > -					  &dma_addr, GFP_KERNEL|__GFP_ZERO);
> > +					  dma_addr, GFP_KERNEL|__GFP_ZERO);
> >   	}
> >   	if (!queue)
> >   		return NULL;
> >
> > +	*n = num;
> > +	return queue;
> > +}
> > +
> > +static struct virtqueue *vring_create_virtqueue_split(
> > +	unsigned int index,
> > +	unsigned int num,
> > +	unsigned int vring_align,
> > +	struct virtio_device *vdev,
> > +	bool weak_barriers,
> > +	bool may_reduce_num,
> > +	bool context,
> > +	bool (*notify)(struct virtqueue *),
> > +	void (*callback)(struct virtqueue *),
> > +	const char *name)
> > +{
> > +	size_t queue_size_in_bytes;
> > +	struct virtqueue *vq;
> > +	dma_addr_t dma_addr;
> > +	struct vring vring;
> > +	void *queue;
> > +
> > +	queue = vring_alloc_queue_split(vdev, &dma_addr, &num, vring_align,
> > +					weak_barriers, may_reduce_num);
> > +	if (!queue)
> > +		return NULL;
> > +
> >   	queue_size_in_bytes = vring_size(num, vring_align);
> >   	vring_init(&vring, num, queue, vring_align);
> >
>

Re: [PATCH v9 06/32] virtio_ring: split: extract the logic of alloc queue

[Xuan Zhuo]

On Tue, 12 Apr 2022 11:22:33 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of split to create vring queue.
> >
> > This feature is required for subsequent virtuqueue reset vring.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++------------
> >   1 file changed, 36 insertions(+), 17 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 33fddfb907a6..72d5ae063fa0 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -915,23 +915,15 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
> >   	return NULL;
> >   }
> >
> > -static struct virtqueue *vring_create_virtqueue_split(
> > -	unsigned int index,
> > -	unsigned int num,
> > -	unsigned int vring_align,
> > -	struct virtio_device *vdev,
> > -	bool weak_barriers,
> > -	bool may_reduce_num,
> > -	bool context,
> > -	bool (*notify)(struct virtqueue *),
> > -	void (*callback)(struct virtqueue *),
> > -	const char *name)
> > +static void *vring_alloc_queue_split(struct virtio_device *vdev,
> > +				     dma_addr_t *dma_addr,
> > +				     u32 *n,
> > +				     unsigned int vring_align,
> > +				     bool weak_barriers,
>
>
> This is not used in this function.

The next version will fix it.

Thanks.


>
> Thanks
>
>
> > +				     bool may_reduce_num)
> >   {
> > -	struct virtqueue *vq;
> >   	void *queue = NULL;
> > -	dma_addr_t dma_addr;
> > -	size_t queue_size_in_bytes;
> > -	struct vring vring;
> > +	u32 num = *n;
> >
> >   	/* We assume num is a power of 2. */
> >   	if (num & (num - 1)) {
> > @@ -942,7 +934,7 @@ static struct virtqueue *vring_create_virtqueue_split(
> >   	/* TODO: allocate each queue chunk individually */
> >   	for (; num && vring_size(num, vring_align) > PAGE_SIZE; num /= 2) {
> >   		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
> > -					  &dma_addr,
> > +					  dma_addr,
> >   					  GFP_KERNEL|__GFP_NOWARN|__GFP_ZERO);
> >   		if (queue)
> >   			break;
> > @@ -956,11 +948,38 @@ static struct virtqueue *vring_create_virtqueue_split(
> >   	if (!queue) {
> >   		/* Try to get a single page. You are my only hope! */
> >   		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
> > -					  &dma_addr, GFP_KERNEL|__GFP_ZERO);
> > +					  dma_addr, GFP_KERNEL|__GFP_ZERO);
> >   	}
> >   	if (!queue)
> >   		return NULL;
> >
> > +	*n = num;
> > +	return queue;
> > +}
> > +
> > +static struct virtqueue *vring_create_virtqueue_split(
> > +	unsigned int index,
> > +	unsigned int num,
> > +	unsigned int vring_align,
> > +	struct virtio_device *vdev,
> > +	bool weak_barriers,
> > +	bool may_reduce_num,
> > +	bool context,
> > +	bool (*notify)(struct virtqueue *),
> > +	void (*callback)(struct virtqueue *),
> > +	const char *name)
> > +{
> > +	size_t queue_size_in_bytes;
> > +	struct virtqueue *vq;
> > +	dma_addr_t dma_addr;
> > +	struct vring vring;
> > +	void *queue;
> > +
> > +	queue = vring_alloc_queue_split(vdev, &dma_addr, &num, vring_align,
> > +					weak_barriers, may_reduce_num);
> > +	if (!queue)
> > +		return NULL;
> > +
> >   	queue_size_in_bytes = vring_size(num, vring_align);
> >   	vring_init(&vring, num, queue, vring_align);
> >
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 06/32] virtio_ring: split: extract the logic of alloc queue

[Xuan Zhuo]

On Tue, 12 Apr 2022 11:22:33 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of split to create vring queue.
> >
> > This feature is required for subsequent virtuqueue reset vring.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 53 ++++++++++++++++++++++++------------
> >   1 file changed, 36 insertions(+), 17 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 33fddfb907a6..72d5ae063fa0 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -915,23 +915,15 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
> >   	return NULL;
> >   }
> >
> > -static struct virtqueue *vring_create_virtqueue_split(
> > -	unsigned int index,
> > -	unsigned int num,
> > -	unsigned int vring_align,
> > -	struct virtio_device *vdev,
> > -	bool weak_barriers,
> > -	bool may_reduce_num,
> > -	bool context,
> > -	bool (*notify)(struct virtqueue *),
> > -	void (*callback)(struct virtqueue *),
> > -	const char *name)
> > +static void *vring_alloc_queue_split(struct virtio_device *vdev,
> > +				     dma_addr_t *dma_addr,
> > +				     u32 *n,
> > +				     unsigned int vring_align,
> > +				     bool weak_barriers,
>
>
> This is not used in this function.

The next version will fix it.

Thanks.


>
> Thanks
>
>
> > +				     bool may_reduce_num)
> >   {
> > -	struct virtqueue *vq;
> >   	void *queue = NULL;
> > -	dma_addr_t dma_addr;
> > -	size_t queue_size_in_bytes;
> > -	struct vring vring;
> > +	u32 num = *n;
> >
> >   	/* We assume num is a power of 2. */
> >   	if (num & (num - 1)) {
> > @@ -942,7 +934,7 @@ static struct virtqueue *vring_create_virtqueue_split(
> >   	/* TODO: allocate each queue chunk individually */
> >   	for (; num && vring_size(num, vring_align) > PAGE_SIZE; num /= 2) {
> >   		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
> > -					  &dma_addr,
> > +					  dma_addr,
> >   					  GFP_KERNEL|__GFP_NOWARN|__GFP_ZERO);
> >   		if (queue)
> >   			break;
> > @@ -956,11 +948,38 @@ static struct virtqueue *vring_create_virtqueue_split(
> >   	if (!queue) {
> >   		/* Try to get a single page. You are my only hope! */
> >   		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
> > -					  &dma_addr, GFP_KERNEL|__GFP_ZERO);
> > +					  dma_addr, GFP_KERNEL|__GFP_ZERO);
> >   	}
> >   	if (!queue)
> >   		return NULL;
> >
> > +	*n = num;
> > +	return queue;
> > +}
> > +
> > +static struct virtqueue *vring_create_virtqueue_split(
> > +	unsigned int index,
> > +	unsigned int num,
> > +	unsigned int vring_align,
> > +	struct virtio_device *vdev,
> > +	bool weak_barriers,
> > +	bool may_reduce_num,
> > +	bool context,
> > +	bool (*notify)(struct virtqueue *),
> > +	void (*callback)(struct virtqueue *),
> > +	const char *name)
> > +{
> > +	size_t queue_size_in_bytes;
> > +	struct virtqueue *vq;
> > +	dma_addr_t dma_addr;
> > +	struct vring vring;
> > +	void *queue;
> > +
> > +	queue = vring_alloc_queue_split(vdev, &dma_addr, &num, vring_align,
> > +					weak_barriers, may_reduce_num);
> > +	if (!queue)
> > +		return NULL;
> > +
> >   	queue_size_in_bytes = vring_size(num, vring_align);
> >   	vring_init(&vring, num, queue, vring_align);
> >
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 09/32] virtio_ring: split: extract the logic of vq init

[Xuan Zhuo]

On Tue, 12 Apr 2022 11:42:25 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of initializing vq, and subsequent patches will call
> > it separately.
> >
> > The feature of this part is that it does not depend on the information
> > passed by the upper layer and can be called repeatedly.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 68 ++++++++++++++++++++----------------
> >   1 file changed, 38 insertions(+), 30 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 083f2992ba0d..874f878087a3 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -916,6 +916,43 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
> >   	return NULL;
> >   }
> >
> > +static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
> > +				       struct virtio_device *vdev,
> > +				       bool own_ring)
> > +{
> > +	vq->packed_ring = false;
> > +	vq->vq.num_free = vq->split.vring.num;
> > +	vq->we_own_ring = own_ring;
> > +	vq->broken = false;
> > +	vq->last_used_idx = 0;
> > +	vq->event_triggered = false;
> > +	vq->num_added = 0;
> > +	vq->use_dma_api = vring_use_dma_api(vdev);
> > +#ifdef DEBUG
> > +	vq->in_use = false;
> > +	vq->last_add_time_valid = false;
> > +#endif
> > +
> > +	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
> > +
> > +	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> > +		vq->weak_barriers = false;
> > +
> > +	vq->split.avail_flags_shadow = 0;
> > +	vq->split.avail_idx_shadow = 0;
> > +
> > +	/* No callback?  Tell other side not to bother us. */
> > +	if (!vq->vq.callback) {
> > +		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
> > +		if (!vq->event)
> > +			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
> > +					vq->split.avail_flags_shadow);
> > +	}
> > +
> > +	/* Put everything in free lists. */
> > +	vq->free_head = 0;
>
>
> It's not clear what kind of initialization that we want to do here. E.g
> it mixes split specific setups with some general setups which is kind of
> duplication of vring_virtqueue_init_packed().
>
> I wonder if it's better to only do split specific setups here and have a
> common helper to do the setup that is irrelevant to ring layout.

Yes, you are right, I didn't notice this situation before.

Thanks.

>
> Thanks
>
>
> > +}
> > +
> >   static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
> >   					 struct vring vring,
> >   					 struct vring_desc_state_split *desc_state,
> > @@ -2249,42 +2286,15 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	if (!vq)
> >   		return NULL;
> >
> > -	vq->packed_ring = false;
> >   	vq->vq.callback = callback;
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> > -	vq->vq.num_free = vring.num;
> >   	vq->vq.index = index;
> > -	vq->we_own_ring = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> > -	vq->broken = false;
> > -	vq->last_used_idx = 0;
> > -	vq->event_triggered = false;
> > -	vq->num_added = 0;
> > -	vq->use_dma_api = vring_use_dma_api(vdev);
> > -#ifdef DEBUG
> > -	vq->in_use = false;
> > -	vq->last_add_time_valid = false;
> > -#endif
> >
> >   	vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) &&
> >   		!context;
> > -	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
> > -
> > -	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> > -		vq->weak_barriers = false;
> > -
> > -	vq->split.avail_flags_shadow = 0;
> > -	vq->split.avail_idx_shadow = 0;
> > -
> > -	/* No callback?  Tell other side not to bother us. */
> > -	if (!callback) {
> > -		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
> > -		if (!vq->event)
> > -			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
> > -					vq->split.avail_flags_shadow);
> > -	}
> >
> >   	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
> >   	if (err) {
> > @@ -2293,9 +2303,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	}
> >
> >   	vring_virtqueue_attach_split(vq, vring, state, extra);
> > -
> > -	/* Put everything in free lists. */
> > -	vq->free_head = 0;
> > +	vring_virtqueue_init_split(vq, vdev, false);
> >
> >   	spin_lock(&vdev->vqs_list_lock);
> >   	list_add_tail(&vq->vq.list, &vdev->vqs);
>

Re: [PATCH v9 09/32] virtio_ring: split: extract the logic of vq init

[Xuan Zhuo]

On Tue, 12 Apr 2022 11:42:25 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of initializing vq, and subsequent patches will call
> > it separately.
> >
> > The feature of this part is that it does not depend on the information
> > passed by the upper layer and can be called repeatedly.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 68 ++++++++++++++++++++----------------
> >   1 file changed, 38 insertions(+), 30 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 083f2992ba0d..874f878087a3 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -916,6 +916,43 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
> >   	return NULL;
> >   }
> >
> > +static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
> > +				       struct virtio_device *vdev,
> > +				       bool own_ring)
> > +{
> > +	vq->packed_ring = false;
> > +	vq->vq.num_free = vq->split.vring.num;
> > +	vq->we_own_ring = own_ring;
> > +	vq->broken = false;
> > +	vq->last_used_idx = 0;
> > +	vq->event_triggered = false;
> > +	vq->num_added = 0;
> > +	vq->use_dma_api = vring_use_dma_api(vdev);
> > +#ifdef DEBUG
> > +	vq->in_use = false;
> > +	vq->last_add_time_valid = false;
> > +#endif
> > +
> > +	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
> > +
> > +	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> > +		vq->weak_barriers = false;
> > +
> > +	vq->split.avail_flags_shadow = 0;
> > +	vq->split.avail_idx_shadow = 0;
> > +
> > +	/* No callback?  Tell other side not to bother us. */
> > +	if (!vq->vq.callback) {
> > +		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
> > +		if (!vq->event)
> > +			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
> > +					vq->split.avail_flags_shadow);
> > +	}
> > +
> > +	/* Put everything in free lists. */
> > +	vq->free_head = 0;
>
>
> It's not clear what kind of initialization that we want to do here. E.g
> it mixes split specific setups with some general setups which is kind of
> duplication of vring_virtqueue_init_packed().
>
> I wonder if it's better to only do split specific setups here and have a
> common helper to do the setup that is irrelevant to ring layout.

Yes, you are right, I didn't notice this situation before.

Thanks.

>
> Thanks
>
>
> > +}
> > +
> >   static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
> >   					 struct vring vring,
> >   					 struct vring_desc_state_split *desc_state,
> > @@ -2249,42 +2286,15 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	if (!vq)
> >   		return NULL;
> >
> > -	vq->packed_ring = false;
> >   	vq->vq.callback = callback;
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> > -	vq->vq.num_free = vring.num;
> >   	vq->vq.index = index;
> > -	vq->we_own_ring = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> > -	vq->broken = false;
> > -	vq->last_used_idx = 0;
> > -	vq->event_triggered = false;
> > -	vq->num_added = 0;
> > -	vq->use_dma_api = vring_use_dma_api(vdev);
> > -#ifdef DEBUG
> > -	vq->in_use = false;
> > -	vq->last_add_time_valid = false;
> > -#endif
> >
> >   	vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) &&
> >   		!context;
> > -	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
> > -
> > -	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> > -		vq->weak_barriers = false;
> > -
> > -	vq->split.avail_flags_shadow = 0;
> > -	vq->split.avail_idx_shadow = 0;
> > -
> > -	/* No callback?  Tell other side not to bother us. */
> > -	if (!callback) {
> > -		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
> > -		if (!vq->event)
> > -			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
> > -					vq->split.avail_flags_shadow);
> > -	}
> >
> >   	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
> >   	if (err) {
> > @@ -2293,9 +2303,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	}
> >
> >   	vring_virtqueue_attach_split(vq, vring, state, extra);
> > -
> > -	/* Put everything in free lists. */
> > -	vq->free_head = 0;
> > +	vring_virtqueue_init_split(vq, vdev, false);
> >
> >   	spin_lock(&vdev->vqs_list_lock);
> >   	list_add_tail(&vq->vq.list, &vdev->vqs);
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 09/32] virtio_ring: split: extract the logic of vq init

[Xuan Zhuo]

On Tue, 12 Apr 2022 11:42:25 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Separate the logic of initializing vq, and subsequent patches will call
> > it separately.
> >
> > The feature of this part is that it does not depend on the information
> > passed by the upper layer and can be called repeatedly.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 68 ++++++++++++++++++++----------------
> >   1 file changed, 38 insertions(+), 30 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 083f2992ba0d..874f878087a3 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -916,6 +916,43 @@ static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq)
> >   	return NULL;
> >   }
> >
> > +static void vring_virtqueue_init_split(struct vring_virtqueue *vq,
> > +				       struct virtio_device *vdev,
> > +				       bool own_ring)
> > +{
> > +	vq->packed_ring = false;
> > +	vq->vq.num_free = vq->split.vring.num;
> > +	vq->we_own_ring = own_ring;
> > +	vq->broken = false;
> > +	vq->last_used_idx = 0;
> > +	vq->event_triggered = false;
> > +	vq->num_added = 0;
> > +	vq->use_dma_api = vring_use_dma_api(vdev);
> > +#ifdef DEBUG
> > +	vq->in_use = false;
> > +	vq->last_add_time_valid = false;
> > +#endif
> > +
> > +	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
> > +
> > +	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> > +		vq->weak_barriers = false;
> > +
> > +	vq->split.avail_flags_shadow = 0;
> > +	vq->split.avail_idx_shadow = 0;
> > +
> > +	/* No callback?  Tell other side not to bother us. */
> > +	if (!vq->vq.callback) {
> > +		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
> > +		if (!vq->event)
> > +			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
> > +					vq->split.avail_flags_shadow);
> > +	}
> > +
> > +	/* Put everything in free lists. */
> > +	vq->free_head = 0;
>
>
> It's not clear what kind of initialization that we want to do here. E.g
> it mixes split specific setups with some general setups which is kind of
> duplication of vring_virtqueue_init_packed().
>
> I wonder if it's better to only do split specific setups here and have a
> common helper to do the setup that is irrelevant to ring layout.

Yes, you are right, I didn't notice this situation before.

Thanks.

>
> Thanks
>
>
> > +}
> > +
> >   static void vring_virtqueue_attach_split(struct vring_virtqueue *vq,
> >   					 struct vring vring,
> >   					 struct vring_desc_state_split *desc_state,
> > @@ -2249,42 +2286,15 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	if (!vq)
> >   		return NULL;
> >
> > -	vq->packed_ring = false;
> >   	vq->vq.callback = callback;
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> > -	vq->vq.num_free = vring.num;
> >   	vq->vq.index = index;
> > -	vq->we_own_ring = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> > -	vq->broken = false;
> > -	vq->last_used_idx = 0;
> > -	vq->event_triggered = false;
> > -	vq->num_added = 0;
> > -	vq->use_dma_api = vring_use_dma_api(vdev);
> > -#ifdef DEBUG
> > -	vq->in_use = false;
> > -	vq->last_add_time_valid = false;
> > -#endif
> >
> >   	vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) &&
> >   		!context;
> > -	vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX);
> > -
> > -	if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
> > -		vq->weak_barriers = false;
> > -
> > -	vq->split.avail_flags_shadow = 0;
> > -	vq->split.avail_idx_shadow = 0;
> > -
> > -	/* No callback?  Tell other side not to bother us. */
> > -	if (!callback) {
> > -		vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
> > -		if (!vq->event)
> > -			vq->split.vring.avail->flags = cpu_to_virtio16(vdev,
> > -					vq->split.avail_flags_shadow);
> > -	}
> >
> >   	err = vring_alloc_state_extra_split(vring.num, &state, &extra);
> >   	if (err) {
> > @@ -2293,9 +2303,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	}
> >
> >   	vring_virtqueue_attach_split(vq, vring, state, extra);
> > -
> > -	/* Put everything in free lists. */
> > -	vq->free_head = 0;
> > +	vring_virtqueue_init_split(vq, vdev, false);
> >
> >   	spin_lock(&vdev->vqs_list_lock);
> >   	list_add_tail(&vq->vq.list, &vdev->vqs);
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> This patch implements the resize function of the rx, tx queues.
> Based on this function, it is possible to modify the ring num of the
> queue.
>
> There may be an exception during the resize process, the resize may
> fail, or the vq can no longer be used. Either way, we must execute
> napi_enable(). Because napi_disable is similar to a lock, napi_enable
> must be called after calling napi_disable.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 81 insertions(+)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index b8bf00525177..ba6859f305f7 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
>   	char padding[4];
>   };
>   
> +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> +
>   static bool is_xdp_frame(void *ptr)
>   {
>   	return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
>   {
>   	napi_enable(napi);
>   
> +	/* Check if vq is in reset state. The normal reset/resize process will
> +	 * be protected by napi. However, the protection of napi is only enabled
> +	 * during the operation, and the protection of napi will end after the
> +	 * operation is completed. If re-enable fails during the process, vq
> +	 * will remain unavailable with reset state.
> +	 */
> +	if (vq->reset)
> +		return;


I don't get when could we hit this condition.


> +
>   	/* If all buffers were filled by other side before we napi_enabled, we
>   	 * won't get another interrupt, so process any outstanding packets now.
>   	 * Call local_bh_enable after to trigger softIRQ processing.
> @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
>   		struct receive_queue *rq = &vi->rq[i];
>   
>   		napi_disable(&rq->napi);
> +
> +		/* Check if vq is in reset state. See more in
> +		 * virtnet_napi_enable()
> +		 */
> +		if (rq->vq->reset) {
> +			virtnet_napi_enable(rq->vq, &rq->napi);
> +			continue;
> +		}


Can we do something similar in virtnet_close() by canceling the work?


> +
>   		still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
>   		virtnet_napi_enable(rq->vq, &rq->napi);
>   
> @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
>   	if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
>   		return;
>   
> +	/* Check if vq is in reset state. See more in virtnet_napi_enable() */
> +	if (sq->vq->reset)
> +		return;


We've disabled TX napi, any chance we can still hit this?


> +
>   	if (__netif_tx_trylock(txq)) {
>   		do {
>   			virtqueue_disable_cb(sq->vq);
> @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
>   	return NETDEV_TX_OK;
>   }
>   
> +static int virtnet_rx_resize(struct virtnet_info *vi,
> +			     struct receive_queue *rq, u32 ring_num)
> +{
> +	int err;
> +
> +	napi_disable(&rq->napi);
> +
> +	err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> +	if (err)
> +		goto err;
> +
> +	if (!try_fill_recv(vi, rq, GFP_KERNEL))
> +		schedule_delayed_work(&vi->refill, 0);
> +
> +	virtnet_napi_enable(rq->vq, &rq->napi);
> +	return 0;
> +
> +err:
> +	netdev_err(vi->dev,
> +		   "reset rx reset vq fail: rx queue index: %td err: %d\n",
> +		   rq - vi->rq, err);
> +	virtnet_napi_enable(rq->vq, &rq->napi);
> +	return err;
> +}
> +
> +static int virtnet_tx_resize(struct virtnet_info *vi,
> +			     struct send_queue *sq, u32 ring_num)
> +{
> +	struct netdev_queue *txq;
> +	int err, qindex;
> +
> +	qindex = sq - vi->sq;
> +
> +	virtnet_napi_tx_disable(&sq->napi);
> +
> +	txq = netdev_get_tx_queue(vi->dev, qindex);
> +	__netif_tx_lock_bh(txq);
> +	netif_stop_subqueue(vi->dev, qindex);
> +	__netif_tx_unlock_bh(txq);
> +
> +	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> +	if (err)
> +		goto err;
> +
> +	netif_start_subqueue(vi->dev, qindex);
> +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> +	return 0;
> +
> +err:


I guess we can still start the queue in this case? (Since we don't 
change the queue if resize fails).


> +	netdev_err(vi->dev,
> +		   "reset tx reset vq fail: tx queue index: %td err: %d\n",
> +		   sq - vi->sq, err);
> +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> +	return err;
> +}
> +
>   /*
>    * Send command via the control virtqueue and check status.  Commands
>    * supported by the hypervisor, as indicated by feature bits, should

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> This patch implements the resize function of the rx, tx queues.
> Based on this function, it is possible to modify the ring num of the
> queue.
>
> There may be an exception during the resize process, the resize may
> fail, or the vq can no longer be used. Either way, we must execute
> napi_enable(). Because napi_disable is similar to a lock, napi_enable
> must be called after calling napi_disable.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 81 insertions(+)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index b8bf00525177..ba6859f305f7 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
>   	char padding[4];
>   };
>   
> +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> +
>   static bool is_xdp_frame(void *ptr)
>   {
>   	return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
>   {
>   	napi_enable(napi);
>   
> +	/* Check if vq is in reset state. The normal reset/resize process will
> +	 * be protected by napi. However, the protection of napi is only enabled
> +	 * during the operation, and the protection of napi will end after the
> +	 * operation is completed. If re-enable fails during the process, vq
> +	 * will remain unavailable with reset state.
> +	 */
> +	if (vq->reset)
> +		return;


I don't get when could we hit this condition.


> +
>   	/* If all buffers were filled by other side before we napi_enabled, we
>   	 * won't get another interrupt, so process any outstanding packets now.
>   	 * Call local_bh_enable after to trigger softIRQ processing.
> @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
>   		struct receive_queue *rq = &vi->rq[i];
>   
>   		napi_disable(&rq->napi);
> +
> +		/* Check if vq is in reset state. See more in
> +		 * virtnet_napi_enable()
> +		 */
> +		if (rq->vq->reset) {
> +			virtnet_napi_enable(rq->vq, &rq->napi);
> +			continue;
> +		}


Can we do something similar in virtnet_close() by canceling the work?


> +
>   		still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
>   		virtnet_napi_enable(rq->vq, &rq->napi);
>   
> @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
>   	if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
>   		return;
>   
> +	/* Check if vq is in reset state. See more in virtnet_napi_enable() */
> +	if (sq->vq->reset)
> +		return;


We've disabled TX napi, any chance we can still hit this?


> +
>   	if (__netif_tx_trylock(txq)) {
>   		do {
>   			virtqueue_disable_cb(sq->vq);
> @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
>   	return NETDEV_TX_OK;
>   }
>   
> +static int virtnet_rx_resize(struct virtnet_info *vi,
> +			     struct receive_queue *rq, u32 ring_num)
> +{
> +	int err;
> +
> +	napi_disable(&rq->napi);
> +
> +	err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> +	if (err)
> +		goto err;
> +
> +	if (!try_fill_recv(vi, rq, GFP_KERNEL))
> +		schedule_delayed_work(&vi->refill, 0);
> +
> +	virtnet_napi_enable(rq->vq, &rq->napi);
> +	return 0;
> +
> +err:
> +	netdev_err(vi->dev,
> +		   "reset rx reset vq fail: rx queue index: %td err: %d\n",
> +		   rq - vi->rq, err);
> +	virtnet_napi_enable(rq->vq, &rq->napi);
> +	return err;
> +}
> +
> +static int virtnet_tx_resize(struct virtnet_info *vi,
> +			     struct send_queue *sq, u32 ring_num)
> +{
> +	struct netdev_queue *txq;
> +	int err, qindex;
> +
> +	qindex = sq - vi->sq;
> +
> +	virtnet_napi_tx_disable(&sq->napi);
> +
> +	txq = netdev_get_tx_queue(vi->dev, qindex);
> +	__netif_tx_lock_bh(txq);
> +	netif_stop_subqueue(vi->dev, qindex);
> +	__netif_tx_unlock_bh(txq);
> +
> +	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> +	if (err)
> +		goto err;
> +
> +	netif_start_subqueue(vi->dev, qindex);
> +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> +	return 0;
> +
> +err:


I guess we can still start the queue in this case? (Since we don't 
change the queue if resize fails).


> +	netdev_err(vi->dev,
> +		   "reset tx reset vq fail: tx queue index: %td err: %d\n",
> +		   sq - vi->sq, err);
> +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> +	return err;
> +}
> +
>   /*
>    * Send command via the control virtqueue and check status.  Commands
>    * supported by the hypervisor, as indicated by feature bits, should

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> This patch implements the resize function of the rx, tx queues.
> Based on this function, it is possible to modify the ring num of the
> queue.
>
> There may be an exception during the resize process, the resize may
> fail, or the vq can no longer be used. Either way, we must execute
> napi_enable(). Because napi_disable is similar to a lock, napi_enable
> must be called after calling napi_disable.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
>   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 81 insertions(+)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index b8bf00525177..ba6859f305f7 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
>   	char padding[4];
>   };
>   
> +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> +
>   static bool is_xdp_frame(void *ptr)
>   {
>   	return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
>   {
>   	napi_enable(napi);
>   
> +	/* Check if vq is in reset state. The normal reset/resize process will
> +	 * be protected by napi. However, the protection of napi is only enabled
> +	 * during the operation, and the protection of napi will end after the
> +	 * operation is completed. If re-enable fails during the process, vq
> +	 * will remain unavailable with reset state.
> +	 */
> +	if (vq->reset)
> +		return;


I don't get when could we hit this condition.


> +
>   	/* If all buffers were filled by other side before we napi_enabled, we
>   	 * won't get another interrupt, so process any outstanding packets now.
>   	 * Call local_bh_enable after to trigger softIRQ processing.
> @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
>   		struct receive_queue *rq = &vi->rq[i];
>   
>   		napi_disable(&rq->napi);
> +
> +		/* Check if vq is in reset state. See more in
> +		 * virtnet_napi_enable()
> +		 */
> +		if (rq->vq->reset) {
> +			virtnet_napi_enable(rq->vq, &rq->napi);
> +			continue;
> +		}


Can we do something similar in virtnet_close() by canceling the work?


> +
>   		still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
>   		virtnet_napi_enable(rq->vq, &rq->napi);
>   
> @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
>   	if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
>   		return;
>   
> +	/* Check if vq is in reset state. See more in virtnet_napi_enable() */
> +	if (sq->vq->reset)
> +		return;


We've disabled TX napi, any chance we can still hit this?


> +
>   	if (__netif_tx_trylock(txq)) {
>   		do {
>   			virtqueue_disable_cb(sq->vq);
> @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
>   	return NETDEV_TX_OK;
>   }
>   
> +static int virtnet_rx_resize(struct virtnet_info *vi,
> +			     struct receive_queue *rq, u32 ring_num)
> +{
> +	int err;
> +
> +	napi_disable(&rq->napi);
> +
> +	err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> +	if (err)
> +		goto err;
> +
> +	if (!try_fill_recv(vi, rq, GFP_KERNEL))
> +		schedule_delayed_work(&vi->refill, 0);
> +
> +	virtnet_napi_enable(rq->vq, &rq->napi);
> +	return 0;
> +
> +err:
> +	netdev_err(vi->dev,
> +		   "reset rx reset vq fail: rx queue index: %td err: %d\n",
> +		   rq - vi->rq, err);
> +	virtnet_napi_enable(rq->vq, &rq->napi);
> +	return err;
> +}
> +
> +static int virtnet_tx_resize(struct virtnet_info *vi,
> +			     struct send_queue *sq, u32 ring_num)
> +{
> +	struct netdev_queue *txq;
> +	int err, qindex;
> +
> +	qindex = sq - vi->sq;
> +
> +	virtnet_napi_tx_disable(&sq->napi);
> +
> +	txq = netdev_get_tx_queue(vi->dev, qindex);
> +	__netif_tx_lock_bh(txq);
> +	netif_stop_subqueue(vi->dev, qindex);
> +	__netif_tx_unlock_bh(txq);
> +
> +	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> +	if (err)
> +		goto err;
> +
> +	netif_start_subqueue(vi->dev, qindex);
> +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> +	return 0;
> +
> +err:


I guess we can still start the queue in this case? (Since we don't 
change the queue if resize fails).


> +	netdev_err(vi->dev,
> +		   "reset tx reset vq fail: tx queue index: %td err: %d\n",
> +		   sq - vi->sq, err);
> +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> +	return err;
> +}
> +
>   /*
>    * Send command via the control virtqueue and check status.  Commands
>    * supported by the hypervisor, as indicated by feature bits, should


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 32/32] virtio_net: support set_ringparam

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Support set_ringparam based on virtio queue reset.
>
> Users can use ethtool -G eth0 <ring_num> to modify the ring size of
> virtio-net.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>

(One thing that I see is that, when resize fails, the param reported via 
get_ringparam might be wrong, this is a corner case but might worth to 
fix in the future).


> ---
>   drivers/net/virtio_net.c | 47 ++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 47 insertions(+)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index ba6859f305f7..37e4e27f1e4e 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -2264,6 +2264,52 @@ static void virtnet_get_ringparam(struct net_device *dev,
>   	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
>   }
>   
> +static int virtnet_set_ringparam(struct net_device *dev,
> +				 struct ethtool_ringparam *ring,
> +				 struct kernel_ethtool_ringparam *kernel_ring,
> +				 struct netlink_ext_ack *extack)
> +{
> +	struct virtnet_info *vi = netdev_priv(dev);
> +	u32 rx_pending, tx_pending;
> +	struct receive_queue *rq;
> +	struct send_queue *sq;
> +	int i, err;
> +
> +	if (ring->rx_mini_pending || ring->rx_jumbo_pending)
> +		return -EINVAL;
> +
> +	rx_pending = virtqueue_get_vring_size(vi->rq[0].vq);
> +	tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
> +
> +	if (ring->rx_pending == rx_pending &&
> +	    ring->tx_pending == tx_pending)
> +		return 0;
> +
> +	if (ring->rx_pending > virtqueue_get_vring_max_size(vi->rq[0].vq))
> +		return -EINVAL;
> +
> +	if (ring->tx_pending > virtqueue_get_vring_max_size(vi->sq[0].vq))
> +		return -EINVAL;
> +
> +	for (i = 0; i < vi->max_queue_pairs; i++) {
> +		rq = vi->rq + i;
> +		sq = vi->sq + i;
> +
> +		if (ring->tx_pending != tx_pending) {
> +			err = virtnet_tx_resize(vi, sq, ring->tx_pending);
> +			if (err)
> +				return err;
> +		}
> +
> +		if (ring->rx_pending != rx_pending) {
> +			err = virtnet_rx_resize(vi, rq, ring->rx_pending);
> +			if (err)
> +				return err;
> +		}
> +	}
> +
> +	return 0;
> +}
>   
>   static void virtnet_get_drvinfo(struct net_device *dev,
>   				struct ethtool_drvinfo *info)
> @@ -2497,6 +2543,7 @@ static const struct ethtool_ops virtnet_ethtool_ops = {
>   	.get_drvinfo = virtnet_get_drvinfo,
>   	.get_link = ethtool_op_get_link,
>   	.get_ringparam = virtnet_get_ringparam,
> +	.set_ringparam = virtnet_set_ringparam,
>   	.get_strings = virtnet_get_strings,
>   	.get_sset_count = virtnet_get_sset_count,
>   	.get_ethtool_stats = virtnet_get_ethtool_stats,

Re: [PATCH v9 32/32] virtio_net: support set_ringparam

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Support set_ringparam based on virtio queue reset.
>
> Users can use ethtool -G eth0 <ring_num> to modify the ring size of
> virtio-net.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>

(One thing that I see is that, when resize fails, the param reported via 
get_ringparam might be wrong, this is a corner case but might worth to 
fix in the future).


> ---
>   drivers/net/virtio_net.c | 47 ++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 47 insertions(+)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index ba6859f305f7..37e4e27f1e4e 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -2264,6 +2264,52 @@ static void virtnet_get_ringparam(struct net_device *dev,
>   	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
>   }
>   
> +static int virtnet_set_ringparam(struct net_device *dev,
> +				 struct ethtool_ringparam *ring,
> +				 struct kernel_ethtool_ringparam *kernel_ring,
> +				 struct netlink_ext_ack *extack)
> +{
> +	struct virtnet_info *vi = netdev_priv(dev);
> +	u32 rx_pending, tx_pending;
> +	struct receive_queue *rq;
> +	struct send_queue *sq;
> +	int i, err;
> +
> +	if (ring->rx_mini_pending || ring->rx_jumbo_pending)
> +		return -EINVAL;
> +
> +	rx_pending = virtqueue_get_vring_size(vi->rq[0].vq);
> +	tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
> +
> +	if (ring->rx_pending == rx_pending &&
> +	    ring->tx_pending == tx_pending)
> +		return 0;
> +
> +	if (ring->rx_pending > virtqueue_get_vring_max_size(vi->rq[0].vq))
> +		return -EINVAL;
> +
> +	if (ring->tx_pending > virtqueue_get_vring_max_size(vi->sq[0].vq))
> +		return -EINVAL;
> +
> +	for (i = 0; i < vi->max_queue_pairs; i++) {
> +		rq = vi->rq + i;
> +		sq = vi->sq + i;
> +
> +		if (ring->tx_pending != tx_pending) {
> +			err = virtnet_tx_resize(vi, sq, ring->tx_pending);
> +			if (err)
> +				return err;
> +		}
> +
> +		if (ring->rx_pending != rx_pending) {
> +			err = virtnet_rx_resize(vi, rq, ring->rx_pending);
> +			if (err)
> +				return err;
> +		}
> +	}
> +
> +	return 0;
> +}
>   
>   static void virtnet_get_drvinfo(struct net_device *dev,
>   				struct ethtool_drvinfo *info)
> @@ -2497,6 +2543,7 @@ static const struct ethtool_ops virtnet_ethtool_ops = {
>   	.get_drvinfo = virtnet_get_drvinfo,
>   	.get_link = ethtool_op_get_link,
>   	.get_ringparam = virtnet_get_ringparam,
> +	.set_ringparam = virtnet_set_ringparam,
>   	.get_strings = virtnet_get_strings,
>   	.get_sset_count = virtnet_get_sset_count,
>   	.get_ethtool_stats = virtnet_get_ethtool_stats,

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 32/32] virtio_net: support set_ringparam

[Jason Wang]

在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> Support set_ringparam based on virtio queue reset.
>
> Users can use ethtool -G eth0 <ring_num> to modify the ring size of
> virtio-net.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


Acked-by: Jason Wang <jasowang@redhat.com>

(One thing that I see is that, when resize fails, the param reported via 
get_ringparam might be wrong, this is a corner case but might worth to 
fix in the future).


> ---
>   drivers/net/virtio_net.c | 47 ++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 47 insertions(+)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index ba6859f305f7..37e4e27f1e4e 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -2264,6 +2264,52 @@ static void virtnet_get_ringparam(struct net_device *dev,
>   	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
>   }
>   
> +static int virtnet_set_ringparam(struct net_device *dev,
> +				 struct ethtool_ringparam *ring,
> +				 struct kernel_ethtool_ringparam *kernel_ring,
> +				 struct netlink_ext_ack *extack)
> +{
> +	struct virtnet_info *vi = netdev_priv(dev);
> +	u32 rx_pending, tx_pending;
> +	struct receive_queue *rq;
> +	struct send_queue *sq;
> +	int i, err;
> +
> +	if (ring->rx_mini_pending || ring->rx_jumbo_pending)
> +		return -EINVAL;
> +
> +	rx_pending = virtqueue_get_vring_size(vi->rq[0].vq);
> +	tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
> +
> +	if (ring->rx_pending == rx_pending &&
> +	    ring->tx_pending == tx_pending)
> +		return 0;
> +
> +	if (ring->rx_pending > virtqueue_get_vring_max_size(vi->rq[0].vq))
> +		return -EINVAL;
> +
> +	if (ring->tx_pending > virtqueue_get_vring_max_size(vi->sq[0].vq))
> +		return -EINVAL;
> +
> +	for (i = 0; i < vi->max_queue_pairs; i++) {
> +		rq = vi->rq + i;
> +		sq = vi->sq + i;
> +
> +		if (ring->tx_pending != tx_pending) {
> +			err = virtnet_tx_resize(vi, sq, ring->tx_pending);
> +			if (err)
> +				return err;
> +		}
> +
> +		if (ring->rx_pending != rx_pending) {
> +			err = virtnet_rx_resize(vi, rq, ring->rx_pending);
> +			if (err)
> +				return err;
> +		}
> +	}
> +
> +	return 0;
> +}
>   
>   static void virtnet_get_drvinfo(struct net_device *dev,
>   				struct ethtool_drvinfo *info)
> @@ -2497,6 +2543,7 @@ static const struct ethtool_ops virtnet_ethtool_ops = {
>   	.get_drvinfo = virtnet_get_drvinfo,
>   	.get_link = ethtool_op_get_link,
>   	.get_ringparam = virtnet_get_ringparam,
> +	.set_ringparam = virtnet_set_ringparam,
>   	.get_strings = virtnet_get_strings,
>   	.get_sset_count = virtnet_get_sset_count,
>   	.get_ethtool_stats = virtnet_get_ethtool_stats,


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > This patch implements the resize function of the rx, tx queues.
> > Based on this function, it is possible to modify the ring num of the
> > queue.
> >
> > There may be an exception during the resize process, the resize may
> > fail, or the vq can no longer be used. Either way, we must execute
> > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > must be called after calling napi_disable.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 81 insertions(+)
> >
> > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > index b8bf00525177..ba6859f305f7 100644
> > --- a/drivers/net/virtio_net.c
> > +++ b/drivers/net/virtio_net.c
> > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> >   	char padding[4];
> >   };
> >
> > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > +
> >   static bool is_xdp_frame(void *ptr)
> >   {
> >   	return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> >   {
> >   	napi_enable(napi);
> >
> > +	/* Check if vq is in reset state. The normal reset/resize process will
> > +	 * be protected by napi. However, the protection of napi is only enabled
> > +	 * during the operation, and the protection of napi will end after the
> > +	 * operation is completed. If re-enable fails during the process, vq
> > +	 * will remain unavailable with reset state.
> > +	 */
> > +	if (vq->reset)
> > +		return;
>
>
> I don't get when could we hit this condition.


In patch 23, the code to implement re-enable vq is as follows:

+static int vp_modern_enable_reset_vq(struct virtqueue *vq)
+{
+	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
+	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
+	struct virtio_pci_vq_info *info;
+	unsigned long flags, index;
+	int err;
+
+	if (!vq->reset)
+		return -EBUSY;
+
+	index = vq->index;
+	info = vp_dev->vqs[index];
+
+	/* check queue reset status */
+	if (vp_modern_get_queue_reset(mdev, index) != 1)
+		return -EBUSY;
+
+	err = vp_active_vq(vq, info->msix_vector);
+	if (err)
+		return err;
+
+	if (vq->callback) {
+		spin_lock_irqsave(&vp_dev->lock, flags);
+		list_add(&info->node, &vp_dev->virtqueues);
+		spin_unlock_irqrestore(&vp_dev->lock, flags);
+	} else {
+		INIT_LIST_HEAD(&info->node);
+	}
+
+	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
+
+	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
+		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
+
+	vq->reset = false;
+
+	return 0;
+}


There are three situations where an error will be returned. These are the
situations I want to handle.

But I'm rethinking the question, and I feel like you're right, although the
hardware setup may fail. We can no longer sync with the hardware. But using it
as a normal vq doesn't have any problems.

>
>
> > +
> >   	/* If all buffers were filled by other side before we napi_enabled, we
> >   	 * won't get another interrupt, so process any outstanding packets now.
> >   	 * Call local_bh_enable after to trigger softIRQ processing.
> > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> >   		struct receive_queue *rq = &vi->rq[i];
> >
> >   		napi_disable(&rq->napi);
> > +
> > +		/* Check if vq is in reset state. See more in
> > +		 * virtnet_napi_enable()
> > +		 */
> > +		if (rq->vq->reset) {
> > +			virtnet_napi_enable(rq->vq, &rq->napi);
> > +			continue;
> > +		}
>
>
> Can we do something similar in virtnet_close() by canceling the work?

I think there is no need to cancel the work here, because napi_disable will wait
for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
vq, this logic can be removed.


>
>
> > +
> >   		still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> >   		virtnet_napi_enable(rq->vq, &rq->napi);
> >
> > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> >   	if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> >   		return;
> >
> > +	/* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > +	if (sq->vq->reset)
> > +		return;
>
>
> We've disabled TX napi, any chance we can still hit this?

Same as above.

>
>
> > +
> >   	if (__netif_tx_trylock(txq)) {
> >   		do {
> >   			virtqueue_disable_cb(sq->vq);
> > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> >   	return NETDEV_TX_OK;
> >   }
> >
> > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > +			     struct receive_queue *rq, u32 ring_num)
> > +{
> > +	int err;
> > +
> > +	napi_disable(&rq->napi);
> > +
> > +	err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > +	if (err)
> > +		goto err;
> > +
> > +	if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > +		schedule_delayed_work(&vi->refill, 0);
> > +
> > +	virtnet_napi_enable(rq->vq, &rq->napi);
> > +	return 0;
> > +
> > +err:
> > +	netdev_err(vi->dev,
> > +		   "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > +		   rq - vi->rq, err);
> > +	virtnet_napi_enable(rq->vq, &rq->napi);
> > +	return err;
> > +}
> > +
> > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > +			     struct send_queue *sq, u32 ring_num)
> > +{
> > +	struct netdev_queue *txq;
> > +	int err, qindex;
> > +
> > +	qindex = sq - vi->sq;
> > +
> > +	virtnet_napi_tx_disable(&sq->napi);
> > +
> > +	txq = netdev_get_tx_queue(vi->dev, qindex);
> > +	__netif_tx_lock_bh(txq);
> > +	netif_stop_subqueue(vi->dev, qindex);
> > +	__netif_tx_unlock_bh(txq);
> > +
> > +	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > +	if (err)
> > +		goto err;
> > +
> > +	netif_start_subqueue(vi->dev, qindex);
> > +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > +	return 0;
> > +
> > +err:
>
>
> I guess we can still start the queue in this case? (Since we don't
> change the queue if resize fails).

Yes, you are right.

Thanks.

>
>
> > +	netdev_err(vi->dev,
> > +		   "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > +		   sq - vi->sq, err);
> > +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > +	return err;
> > +}
> > +
> >   /*
> >    * Send command via the control virtqueue and check status.  Commands
> >    * supported by the hypervisor, as indicated by feature bits, should
>

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > This patch implements the resize function of the rx, tx queues.
> > Based on this function, it is possible to modify the ring num of the
> > queue.
> >
> > There may be an exception during the resize process, the resize may
> > fail, or the vq can no longer be used. Either way, we must execute
> > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > must be called after calling napi_disable.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 81 insertions(+)
> >
> > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > index b8bf00525177..ba6859f305f7 100644
> > --- a/drivers/net/virtio_net.c
> > +++ b/drivers/net/virtio_net.c
> > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> >   	char padding[4];
> >   };
> >
> > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > +
> >   static bool is_xdp_frame(void *ptr)
> >   {
> >   	return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> >   {
> >   	napi_enable(napi);
> >
> > +	/* Check if vq is in reset state. The normal reset/resize process will
> > +	 * be protected by napi. However, the protection of napi is only enabled
> > +	 * during the operation, and the protection of napi will end after the
> > +	 * operation is completed. If re-enable fails during the process, vq
> > +	 * will remain unavailable with reset state.
> > +	 */
> > +	if (vq->reset)
> > +		return;
>
>
> I don't get when could we hit this condition.


In patch 23, the code to implement re-enable vq is as follows:

+static int vp_modern_enable_reset_vq(struct virtqueue *vq)
+{
+	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
+	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
+	struct virtio_pci_vq_info *info;
+	unsigned long flags, index;
+	int err;
+
+	if (!vq->reset)
+		return -EBUSY;
+
+	index = vq->index;
+	info = vp_dev->vqs[index];
+
+	/* check queue reset status */
+	if (vp_modern_get_queue_reset(mdev, index) != 1)
+		return -EBUSY;
+
+	err = vp_active_vq(vq, info->msix_vector);
+	if (err)
+		return err;
+
+	if (vq->callback) {
+		spin_lock_irqsave(&vp_dev->lock, flags);
+		list_add(&info->node, &vp_dev->virtqueues);
+		spin_unlock_irqrestore(&vp_dev->lock, flags);
+	} else {
+		INIT_LIST_HEAD(&info->node);
+	}
+
+	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
+
+	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
+		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
+
+	vq->reset = false;
+
+	return 0;
+}


There are three situations where an error will be returned. These are the
situations I want to handle.

But I'm rethinking the question, and I feel like you're right, although the
hardware setup may fail. We can no longer sync with the hardware. But using it
as a normal vq doesn't have any problems.

>
>
> > +
> >   	/* If all buffers were filled by other side before we napi_enabled, we
> >   	 * won't get another interrupt, so process any outstanding packets now.
> >   	 * Call local_bh_enable after to trigger softIRQ processing.
> > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> >   		struct receive_queue *rq = &vi->rq[i];
> >
> >   		napi_disable(&rq->napi);
> > +
> > +		/* Check if vq is in reset state. See more in
> > +		 * virtnet_napi_enable()
> > +		 */
> > +		if (rq->vq->reset) {
> > +			virtnet_napi_enable(rq->vq, &rq->napi);
> > +			continue;
> > +		}
>
>
> Can we do something similar in virtnet_close() by canceling the work?

I think there is no need to cancel the work here, because napi_disable will wait
for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
vq, this logic can be removed.


>
>
> > +
> >   		still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> >   		virtnet_napi_enable(rq->vq, &rq->napi);
> >
> > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> >   	if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> >   		return;
> >
> > +	/* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > +	if (sq->vq->reset)
> > +		return;
>
>
> We've disabled TX napi, any chance we can still hit this?

Same as above.

>
>
> > +
> >   	if (__netif_tx_trylock(txq)) {
> >   		do {
> >   			virtqueue_disable_cb(sq->vq);
> > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> >   	return NETDEV_TX_OK;
> >   }
> >
> > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > +			     struct receive_queue *rq, u32 ring_num)
> > +{
> > +	int err;
> > +
> > +	napi_disable(&rq->napi);
> > +
> > +	err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > +	if (err)
> > +		goto err;
> > +
> > +	if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > +		schedule_delayed_work(&vi->refill, 0);
> > +
> > +	virtnet_napi_enable(rq->vq, &rq->napi);
> > +	return 0;
> > +
> > +err:
> > +	netdev_err(vi->dev,
> > +		   "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > +		   rq - vi->rq, err);
> > +	virtnet_napi_enable(rq->vq, &rq->napi);
> > +	return err;
> > +}
> > +
> > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > +			     struct send_queue *sq, u32 ring_num)
> > +{
> > +	struct netdev_queue *txq;
> > +	int err, qindex;
> > +
> > +	qindex = sq - vi->sq;
> > +
> > +	virtnet_napi_tx_disable(&sq->napi);
> > +
> > +	txq = netdev_get_tx_queue(vi->dev, qindex);
> > +	__netif_tx_lock_bh(txq);
> > +	netif_stop_subqueue(vi->dev, qindex);
> > +	__netif_tx_unlock_bh(txq);
> > +
> > +	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > +	if (err)
> > +		goto err;
> > +
> > +	netif_start_subqueue(vi->dev, qindex);
> > +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > +	return 0;
> > +
> > +err:
>
>
> I guess we can still start the queue in this case? (Since we don't
> change the queue if resize fails).

Yes, you are right.

Thanks.

>
>
> > +	netdev_err(vi->dev,
> > +		   "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > +		   sq - vi->sq, err);
> > +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > +	return err;
> > +}
> > +
> >   /*
> >    * Send command via the control virtqueue and check status.  Commands
> >    * supported by the hypervisor, as indicated by feature bits, should
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > This patch implements the resize function of the rx, tx queues.
> > Based on this function, it is possible to modify the ring num of the
> > queue.
> >
> > There may be an exception during the resize process, the resize may
> > fail, or the vq can no longer be used. Either way, we must execute
> > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > must be called after calling napi_disable.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 81 insertions(+)
> >
> > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > index b8bf00525177..ba6859f305f7 100644
> > --- a/drivers/net/virtio_net.c
> > +++ b/drivers/net/virtio_net.c
> > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> >   	char padding[4];
> >   };
> >
> > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > +
> >   static bool is_xdp_frame(void *ptr)
> >   {
> >   	return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> >   {
> >   	napi_enable(napi);
> >
> > +	/* Check if vq is in reset state. The normal reset/resize process will
> > +	 * be protected by napi. However, the protection of napi is only enabled
> > +	 * during the operation, and the protection of napi will end after the
> > +	 * operation is completed. If re-enable fails during the process, vq
> > +	 * will remain unavailable with reset state.
> > +	 */
> > +	if (vq->reset)
> > +		return;
>
>
> I don't get when could we hit this condition.


In patch 23, the code to implement re-enable vq is as follows:

+static int vp_modern_enable_reset_vq(struct virtqueue *vq)
+{
+	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
+	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
+	struct virtio_pci_vq_info *info;
+	unsigned long flags, index;
+	int err;
+
+	if (!vq->reset)
+		return -EBUSY;
+
+	index = vq->index;
+	info = vp_dev->vqs[index];
+
+	/* check queue reset status */
+	if (vp_modern_get_queue_reset(mdev, index) != 1)
+		return -EBUSY;
+
+	err = vp_active_vq(vq, info->msix_vector);
+	if (err)
+		return err;
+
+	if (vq->callback) {
+		spin_lock_irqsave(&vp_dev->lock, flags);
+		list_add(&info->node, &vp_dev->virtqueues);
+		spin_unlock_irqrestore(&vp_dev->lock, flags);
+	} else {
+		INIT_LIST_HEAD(&info->node);
+	}
+
+	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
+
+	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
+		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
+
+	vq->reset = false;
+
+	return 0;
+}


There are three situations where an error will be returned. These are the
situations I want to handle.

But I'm rethinking the question, and I feel like you're right, although the
hardware setup may fail. We can no longer sync with the hardware. But using it
as a normal vq doesn't have any problems.

>
>
> > +
> >   	/* If all buffers were filled by other side before we napi_enabled, we
> >   	 * won't get another interrupt, so process any outstanding packets now.
> >   	 * Call local_bh_enable after to trigger softIRQ processing.
> > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> >   		struct receive_queue *rq = &vi->rq[i];
> >
> >   		napi_disable(&rq->napi);
> > +
> > +		/* Check if vq is in reset state. See more in
> > +		 * virtnet_napi_enable()
> > +		 */
> > +		if (rq->vq->reset) {
> > +			virtnet_napi_enable(rq->vq, &rq->napi);
> > +			continue;
> > +		}
>
>
> Can we do something similar in virtnet_close() by canceling the work?

I think there is no need to cancel the work here, because napi_disable will wait
for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
vq, this logic can be removed.


>
>
> > +
> >   		still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> >   		virtnet_napi_enable(rq->vq, &rq->napi);
> >
> > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> >   	if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> >   		return;
> >
> > +	/* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > +	if (sq->vq->reset)
> > +		return;
>
>
> We've disabled TX napi, any chance we can still hit this?

Same as above.

>
>
> > +
> >   	if (__netif_tx_trylock(txq)) {
> >   		do {
> >   			virtqueue_disable_cb(sq->vq);
> > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> >   	return NETDEV_TX_OK;
> >   }
> >
> > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > +			     struct receive_queue *rq, u32 ring_num)
> > +{
> > +	int err;
> > +
> > +	napi_disable(&rq->napi);
> > +
> > +	err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > +	if (err)
> > +		goto err;
> > +
> > +	if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > +		schedule_delayed_work(&vi->refill, 0);
> > +
> > +	virtnet_napi_enable(rq->vq, &rq->napi);
> > +	return 0;
> > +
> > +err:
> > +	netdev_err(vi->dev,
> > +		   "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > +		   rq - vi->rq, err);
> > +	virtnet_napi_enable(rq->vq, &rq->napi);
> > +	return err;
> > +}
> > +
> > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > +			     struct send_queue *sq, u32 ring_num)
> > +{
> > +	struct netdev_queue *txq;
> > +	int err, qindex;
> > +
> > +	qindex = sq - vi->sq;
> > +
> > +	virtnet_napi_tx_disable(&sq->napi);
> > +
> > +	txq = netdev_get_tx_queue(vi->dev, qindex);
> > +	__netif_tx_lock_bh(txq);
> > +	netif_stop_subqueue(vi->dev, qindex);
> > +	__netif_tx_unlock_bh(txq);
> > +
> > +	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > +	if (err)
> > +		goto err;
> > +
> > +	netif_start_subqueue(vi->dev, qindex);
> > +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > +	return 0;
> > +
> > +err:
>
>
> I guess we can still start the queue in this case? (Since we don't
> change the queue if resize fails).

Yes, you are right.

Thanks.

>
>
> > +	netdev_err(vi->dev,
> > +		   "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > +		   sq - vi->sq, err);
> > +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > +	return err;
> > +}
> > +
> >   /*
> >    * Send command via the control virtqueue and check status.  Commands
> >    * supported by the hypervisor, as indicated by feature bits, should
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Xuan Zhuo]

On Tue, 12 Apr 2022 15:07:58 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > This patch implements virtio pci support for QUEUE RESET.
> >
> > Performing reset on a queue is divided into these steps:
> >
> >   1. notify the device to reset the queue
> >   2. recycle the buffer submitted
> >   3. reset the vring (may re-alloc)
> >   4. mmap vring to device, and enable the queue
> >
> > This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
> > pci scenario.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_pci_common.c |  8 +--
> >   drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
> >   drivers/virtio/virtio_ring.c       |  2 +
> >   include/linux/virtio.h             |  1 +
> >   4 files changed, 92 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> > index fdbde1db5ec5..863d3a8a0956 100644
> > --- a/drivers/virtio/virtio_pci_common.c
> > +++ b/drivers/virtio/virtio_pci_common.c
> > @@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
> >   	struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
> >   	unsigned long flags;
> >
> > -	spin_lock_irqsave(&vp_dev->lock, flags);
> > -	list_del(&info->node);
> > -	spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	if (!vq->reset) {
>
>
> On which condition that we may hit this path?

As discussed in patch 31, it may fail when renable vq.

Thanks.

>
>
> > +		spin_lock_irqsave(&vp_dev->lock, flags);
> > +		list_del(&info->node);
> > +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	}
> >
> >   	vp_dev->del_vq(info);
> >   	kfree(info);
> > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > index 49a4493732cf..cb5d38f1c9c8 100644
> > --- a/drivers/virtio/virtio_pci_modern.c
> > +++ b/drivers/virtio/virtio_pci_modern.c
> > @@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
> >   	if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
> >   			pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
> >   		__virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
> > +
> > +	if (features & BIT_ULL(VIRTIO_F_RING_RESET))
> > +		__virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
> >   }
> >
> >   /* virtio config->finalize_features() implementation */
> > @@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> >   	return 0;
> >   }
> >
> > +static int vp_modern_reset_vq(struct virtqueue *vq)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	struct virtio_pci_vq_info *info;
> > +	unsigned long flags;
> > +
> > +	if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
> > +		return -ENOENT;
> > +
> > +	vp_modern_set_queue_reset(mdev, vq->index);
> > +
> > +	info = vp_dev->vqs[vq->index];
> > +
> > +	/* delete vq from irq handler */
> > +	spin_lock_irqsave(&vp_dev->lock, flags);
> > +	list_del(&info->node);
> > +	spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +
> > +	INIT_LIST_HEAD(&info->node);
> > +
> > +	/* For the case where vq has an exclusive irq, to prevent the irq from
> > +	 * being received again and the pending irq, call disable_irq().
> > +	 *
> > +	 * In the scenario based on shared interrupts, vq will be searched from
> > +	 * the queue virtqueues. Since the previous list_del() has been deleted
> > +	 * from the queue, it is impossible for vq to be called in this case.
> > +	 * There is no need to close the corresponding interrupt.
> > +	 */
> > +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +		disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>
>
> See the previous discussion and the revert of the first try to harden
> the interrupt. We probably can't use disable_irq() since it conflicts
> with the affinity managed IRQ that is used by some drivers.
>
> We need to use synchonize_irq() and per virtqueue flag instead. As
> mentioned in previous patches, this could be done on top of my rework on
> the IRQ hardening .
>
>
> > +
> > +	vq->reset = true;
> > +
> > +	return 0;
> > +}
> > +
> > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	struct virtio_pci_vq_info *info;
> > +	unsigned long flags, index;
> > +	int err;
> > +
> > +	if (!vq->reset)
> > +		return -EBUSY;
> > +
> > +	index = vq->index;
> > +	info = vp_dev->vqs[index];
> > +
> > +	/* check queue reset status */
> > +	if (vp_modern_get_queue_reset(mdev, index) != 1)
> > +		return -EBUSY;
> > +
> > +	err = vp_active_vq(vq, info->msix_vector);
> > +	if (err)
> > +		return err;
> > +
> > +	if (vq->callback) {
> > +		spin_lock_irqsave(&vp_dev->lock, flags);
> > +		list_add(&info->node, &vp_dev->virtqueues);
> > +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	} else {
> > +		INIT_LIST_HEAD(&info->node);
> > +	}
> > +
> > +	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > +
> > +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>
>
> We had the same issue as disable_irq().
>
> Thanks
>
>
> > +
> > +	vq->reset = false;
> > +
> > +	return 0;
> > +}
> > +
> >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> >   {
> >   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> > @@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
> >   	.set_vq_affinity = vp_set_vq_affinity,
> >   	.get_vq_affinity = vp_get_vq_affinity,
> >   	.get_shm_region  = vp_get_shm_region,
> > +	.reset_vq	 = vp_modern_reset_vq,
> > +	.enable_reset_vq = vp_modern_enable_reset_vq,
> >   };
> >
> >   static const struct virtio_config_ops virtio_pci_config_ops = {
> > @@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
> >   	.set_vq_affinity = vp_set_vq_affinity,
> >   	.get_vq_affinity = vp_get_vq_affinity,
> >   	.get_shm_region  = vp_get_shm_region,
> > +	.reset_vq	 = vp_modern_reset_vq,
> > +	.enable_reset_vq = vp_modern_enable_reset_vq,
> >   };
> >
> >   /* the PCI probing function */
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 6250e19fc5bf..91937e21edca 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> >   	vq->vq.index = index;
> > +	vq->vq.reset = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> >
> > @@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> >   	vq->vq.index = index;
> > +	vq->vq.reset = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> >
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index c86ff02e0ca0..33ab003c5100 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -33,6 +33,7 @@ struct virtqueue {
> >   	unsigned int num_free;
> >   	unsigned int num_max;
> >   	void *priv;
> > +	bool reset;
> >   };
> >
> >   int virtqueue_add_outbuf(struct virtqueue *vq,
>

Re: [PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Xuan Zhuo]

On Tue, 12 Apr 2022 15:07:58 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > This patch implements virtio pci support for QUEUE RESET.
> >
> > Performing reset on a queue is divided into these steps:
> >
> >   1. notify the device to reset the queue
> >   2. recycle the buffer submitted
> >   3. reset the vring (may re-alloc)
> >   4. mmap vring to device, and enable the queue
> >
> > This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
> > pci scenario.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_pci_common.c |  8 +--
> >   drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
> >   drivers/virtio/virtio_ring.c       |  2 +
> >   include/linux/virtio.h             |  1 +
> >   4 files changed, 92 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> > index fdbde1db5ec5..863d3a8a0956 100644
> > --- a/drivers/virtio/virtio_pci_common.c
> > +++ b/drivers/virtio/virtio_pci_common.c
> > @@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
> >   	struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
> >   	unsigned long flags;
> >
> > -	spin_lock_irqsave(&vp_dev->lock, flags);
> > -	list_del(&info->node);
> > -	spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	if (!vq->reset) {
>
>
> On which condition that we may hit this path?

As discussed in patch 31, it may fail when renable vq.

Thanks.

>
>
> > +		spin_lock_irqsave(&vp_dev->lock, flags);
> > +		list_del(&info->node);
> > +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	}
> >
> >   	vp_dev->del_vq(info);
> >   	kfree(info);
> > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > index 49a4493732cf..cb5d38f1c9c8 100644
> > --- a/drivers/virtio/virtio_pci_modern.c
> > +++ b/drivers/virtio/virtio_pci_modern.c
> > @@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
> >   	if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
> >   			pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
> >   		__virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
> > +
> > +	if (features & BIT_ULL(VIRTIO_F_RING_RESET))
> > +		__virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
> >   }
> >
> >   /* virtio config->finalize_features() implementation */
> > @@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> >   	return 0;
> >   }
> >
> > +static int vp_modern_reset_vq(struct virtqueue *vq)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	struct virtio_pci_vq_info *info;
> > +	unsigned long flags;
> > +
> > +	if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
> > +		return -ENOENT;
> > +
> > +	vp_modern_set_queue_reset(mdev, vq->index);
> > +
> > +	info = vp_dev->vqs[vq->index];
> > +
> > +	/* delete vq from irq handler */
> > +	spin_lock_irqsave(&vp_dev->lock, flags);
> > +	list_del(&info->node);
> > +	spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +
> > +	INIT_LIST_HEAD(&info->node);
> > +
> > +	/* For the case where vq has an exclusive irq, to prevent the irq from
> > +	 * being received again and the pending irq, call disable_irq().
> > +	 *
> > +	 * In the scenario based on shared interrupts, vq will be searched from
> > +	 * the queue virtqueues. Since the previous list_del() has been deleted
> > +	 * from the queue, it is impossible for vq to be called in this case.
> > +	 * There is no need to close the corresponding interrupt.
> > +	 */
> > +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +		disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>
>
> See the previous discussion and the revert of the first try to harden
> the interrupt. We probably can't use disable_irq() since it conflicts
> with the affinity managed IRQ that is used by some drivers.
>
> We need to use synchonize_irq() and per virtqueue flag instead. As
> mentioned in previous patches, this could be done on top of my rework on
> the IRQ hardening .
>
>
> > +
> > +	vq->reset = true;
> > +
> > +	return 0;
> > +}
> > +
> > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	struct virtio_pci_vq_info *info;
> > +	unsigned long flags, index;
> > +	int err;
> > +
> > +	if (!vq->reset)
> > +		return -EBUSY;
> > +
> > +	index = vq->index;
> > +	info = vp_dev->vqs[index];
> > +
> > +	/* check queue reset status */
> > +	if (vp_modern_get_queue_reset(mdev, index) != 1)
> > +		return -EBUSY;
> > +
> > +	err = vp_active_vq(vq, info->msix_vector);
> > +	if (err)
> > +		return err;
> > +
> > +	if (vq->callback) {
> > +		spin_lock_irqsave(&vp_dev->lock, flags);
> > +		list_add(&info->node, &vp_dev->virtqueues);
> > +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	} else {
> > +		INIT_LIST_HEAD(&info->node);
> > +	}
> > +
> > +	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > +
> > +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>
>
> We had the same issue as disable_irq().
>
> Thanks
>
>
> > +
> > +	vq->reset = false;
> > +
> > +	return 0;
> > +}
> > +
> >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> >   {
> >   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> > @@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
> >   	.set_vq_affinity = vp_set_vq_affinity,
> >   	.get_vq_affinity = vp_get_vq_affinity,
> >   	.get_shm_region  = vp_get_shm_region,
> > +	.reset_vq	 = vp_modern_reset_vq,
> > +	.enable_reset_vq = vp_modern_enable_reset_vq,
> >   };
> >
> >   static const struct virtio_config_ops virtio_pci_config_ops = {
> > @@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
> >   	.set_vq_affinity = vp_set_vq_affinity,
> >   	.get_vq_affinity = vp_get_vq_affinity,
> >   	.get_shm_region  = vp_get_shm_region,
> > +	.reset_vq	 = vp_modern_reset_vq,
> > +	.enable_reset_vq = vp_modern_enable_reset_vq,
> >   };
> >
> >   /* the PCI probing function */
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 6250e19fc5bf..91937e21edca 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> >   	vq->vq.index = index;
> > +	vq->vq.reset = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> >
> > @@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> >   	vq->vq.index = index;
> > +	vq->vq.reset = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> >
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index c86ff02e0ca0..33ab003c5100 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -33,6 +33,7 @@ struct virtqueue {
> >   	unsigned int num_free;
> >   	unsigned int num_max;
> >   	void *priv;
> > +	bool reset;
> >   };
> >
> >   int virtqueue_add_outbuf(struct virtqueue *vq,
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Xuan Zhuo]

On Tue, 12 Apr 2022 15:07:58 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > This patch implements virtio pci support for QUEUE RESET.
> >
> > Performing reset on a queue is divided into these steps:
> >
> >   1. notify the device to reset the queue
> >   2. recycle the buffer submitted
> >   3. reset the vring (may re-alloc)
> >   4. mmap vring to device, and enable the queue
> >
> > This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
> > pci scenario.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_pci_common.c |  8 +--
> >   drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
> >   drivers/virtio/virtio_ring.c       |  2 +
> >   include/linux/virtio.h             |  1 +
> >   4 files changed, 92 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> > index fdbde1db5ec5..863d3a8a0956 100644
> > --- a/drivers/virtio/virtio_pci_common.c
> > +++ b/drivers/virtio/virtio_pci_common.c
> > @@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
> >   	struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
> >   	unsigned long flags;
> >
> > -	spin_lock_irqsave(&vp_dev->lock, flags);
> > -	list_del(&info->node);
> > -	spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	if (!vq->reset) {
>
>
> On which condition that we may hit this path?

As discussed in patch 31, it may fail when renable vq.

Thanks.

>
>
> > +		spin_lock_irqsave(&vp_dev->lock, flags);
> > +		list_del(&info->node);
> > +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	}
> >
> >   	vp_dev->del_vq(info);
> >   	kfree(info);
> > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > index 49a4493732cf..cb5d38f1c9c8 100644
> > --- a/drivers/virtio/virtio_pci_modern.c
> > +++ b/drivers/virtio/virtio_pci_modern.c
> > @@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
> >   	if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
> >   			pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
> >   		__virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
> > +
> > +	if (features & BIT_ULL(VIRTIO_F_RING_RESET))
> > +		__virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
> >   }
> >
> >   /* virtio config->finalize_features() implementation */
> > @@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> >   	return 0;
> >   }
> >
> > +static int vp_modern_reset_vq(struct virtqueue *vq)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	struct virtio_pci_vq_info *info;
> > +	unsigned long flags;
> > +
> > +	if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
> > +		return -ENOENT;
> > +
> > +	vp_modern_set_queue_reset(mdev, vq->index);
> > +
> > +	info = vp_dev->vqs[vq->index];
> > +
> > +	/* delete vq from irq handler */
> > +	spin_lock_irqsave(&vp_dev->lock, flags);
> > +	list_del(&info->node);
> > +	spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +
> > +	INIT_LIST_HEAD(&info->node);
> > +
> > +	/* For the case where vq has an exclusive irq, to prevent the irq from
> > +	 * being received again and the pending irq, call disable_irq().
> > +	 *
> > +	 * In the scenario based on shared interrupts, vq will be searched from
> > +	 * the queue virtqueues. Since the previous list_del() has been deleted
> > +	 * from the queue, it is impossible for vq to be called in this case.
> > +	 * There is no need to close the corresponding interrupt.
> > +	 */
> > +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +		disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>
>
> See the previous discussion and the revert of the first try to harden
> the interrupt. We probably can't use disable_irq() since it conflicts
> with the affinity managed IRQ that is used by some drivers.
>
> We need to use synchonize_irq() and per virtqueue flag instead. As
> mentioned in previous patches, this could be done on top of my rework on
> the IRQ hardening .
>
>
> > +
> > +	vq->reset = true;
> > +
> > +	return 0;
> > +}
> > +
> > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	struct virtio_pci_vq_info *info;
> > +	unsigned long flags, index;
> > +	int err;
> > +
> > +	if (!vq->reset)
> > +		return -EBUSY;
> > +
> > +	index = vq->index;
> > +	info = vp_dev->vqs[index];
> > +
> > +	/* check queue reset status */
> > +	if (vp_modern_get_queue_reset(mdev, index) != 1)
> > +		return -EBUSY;
> > +
> > +	err = vp_active_vq(vq, info->msix_vector);
> > +	if (err)
> > +		return err;
> > +
> > +	if (vq->callback) {
> > +		spin_lock_irqsave(&vp_dev->lock, flags);
> > +		list_add(&info->node, &vp_dev->virtqueues);
> > +		spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +	} else {
> > +		INIT_LIST_HEAD(&info->node);
> > +	}
> > +
> > +	vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > +
> > +	if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +		enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>
>
> We had the same issue as disable_irq().
>
> Thanks
>
>
> > +
> > +	vq->reset = false;
> > +
> > +	return 0;
> > +}
> > +
> >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> >   {
> >   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> > @@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
> >   	.set_vq_affinity = vp_set_vq_affinity,
> >   	.get_vq_affinity = vp_get_vq_affinity,
> >   	.get_shm_region  = vp_get_shm_region,
> > +	.reset_vq	 = vp_modern_reset_vq,
> > +	.enable_reset_vq = vp_modern_enable_reset_vq,
> >   };
> >
> >   static const struct virtio_config_ops virtio_pci_config_ops = {
> > @@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
> >   	.set_vq_affinity = vp_set_vq_affinity,
> >   	.get_vq_affinity = vp_get_vq_affinity,
> >   	.get_shm_region  = vp_get_shm_region,
> > +	.reset_vq	 = vp_modern_reset_vq,
> > +	.enable_reset_vq = vp_modern_enable_reset_vq,
> >   };
> >
> >   /* the PCI probing function */
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 6250e19fc5bf..91937e21edca 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> >   	vq->vq.index = index;
> > +	vq->vq.reset = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> >
> > @@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> >   	vq->vq.vdev = vdev;
> >   	vq->vq.name = name;
> >   	vq->vq.index = index;
> > +	vq->vq.reset = false;
> >   	vq->notify = notify;
> >   	vq->weak_barriers = weak_barriers;
> >
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index c86ff02e0ca0..33ab003c5100 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -33,6 +33,7 @@ struct virtqueue {
> >   	unsigned int num_free;
> >   	unsigned int num_max;
> >   	void *priv;
> > +	bool reset;
> >   };
> >
> >   int virtqueue_add_outbuf(struct virtqueue *vq,
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 18/32] virtio_ring: introduce virtqueue_resize()

[Xuan Zhuo]

On Tue, 12 Apr 2022 14:41:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Introduce virtqueue_resize() to implement the resize of vring.
> > Based on these, the driver can dynamically adjust the size of the vring.
> > For example: ethtool -G.
> >
> > virtqueue_resize() implements resize based on the vq reset function. In
> > case of failure to allocate a new vring, it will give up resize and use
> > the original vring.
> >
> > During this process, if the re-enable reset vq fails, the vq can no
> > longer be used. Although the probability of this situation is not high.
> >
> > The parameter recycle is used to recycle the buffer that is no longer
> > used.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++++++++++
> >   include/linux/virtio.h       |  3 ++
> >   2 files changed, 72 insertions(+)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 06f66b15c86c..6250e19fc5bf 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2554,6 +2554,75 @@ struct virtqueue *vring_create_virtqueue(
> >   }
> >   EXPORT_SYMBOL_GPL(vring_create_virtqueue);
> >
> > +/**
> > + * virtqueue_resize - resize the vring of vq
> > + * @_vq: the struct virtqueue we're talking about.
> > + * @num: new ring num
> > + * @recycle: callback for recycle the useless buffer
> > + *
> > + * When it is really necessary to create a new vring, it will set the current vq
> > + * into the reset state. Then call the passed callback to recycle the buffer
> > + * that is no longer used. Only after the new vring is successfully created, the
> > + * old vring will be released.
> > + *
> > + * Caller must ensure we don't call this with other virtqueue operations
> > + * at the same time (except where noted).
> > + *
> > + * Returns zero or a negative error.
>
>
> Should we document that the virtqueue is kept unchanged (still
> available) on (specific) failure?
>

OK.

>
> > + */
> > +int virtqueue_resize(struct virtqueue *_vq, u32 num,
> > +		     void (*recycle)(struct virtqueue *vq, void *buf))
> > +{
> > +	struct vring_virtqueue *vq = to_vvq(_vq);
> > +	struct virtio_device *vdev = vq->vq.vdev;
> > +	bool packed;
> > +	void *buf;
> > +	int err;
> > +
> > +	if (!vq->we_own_ring)
> > +		return -EINVAL;
> > +
> > +	if (num > vq->vq.num_max)
> > +		return -E2BIG;
> > +
> > +	if (!num)
> > +		return -EINVAL;
> > +
> > +	packed = virtio_has_feature(vdev, VIRTIO_F_RING_PACKED) ? true : false;
> > +
> > +	if ((packed ? vq->packed.vring.num : vq->split.vring.num) == num)
> > +		return 0;
> > +
> > +	if (!vdev->config->reset_vq)
> > +		return -ENOENT;
> > +
> > +	if (!vdev->config->enable_reset_vq)
> > +		return -ENOENT;
> > +
> > +	err = vdev->config->reset_vq(_vq);
> > +	if (err)
> > +		return err;
> > +
> > +	while ((buf = virtqueue_detach_unused_buf(_vq)) != NULL)
> > +		recycle(_vq, buf);
> > +
> > +	if (packed) {
> > +		err = virtqueue_resize_packed(_vq, num);
> > +		if (err)
> > +			virtqueue_reinit_packed(vq);
>
>
> Calling reinit here seems a little bit odd, it looks more like a reset
> of the virtqueue. Consider we may re-use virtqueue reset for more
> purpose, I wonder if we need a helper like:
>
> virtqueue_resize() {
>      vdev->config->reset_vq(_vq);
>      if (packed)
>          virtqueue_reinit_packed(_vq)
>      else
>          virtqueue_reinit_split(_vq)
> }

Yes, currently we are implementing resize. This is used to implement
set_ringparam()

Later, when we implement virtio_net + AF_XDP, what we want is reset not resize,
so we need to implement a helper:

 virtqueue_reset() {
      vdev->config->reset_vq(_vq);
      if (packed)
          virtqueue_reinit_packed(_vq)
      else
          virtqueue_reinit_split(_vq)
 }

So I use virtqueue_reinit_* as a separate function not only to deal with the
case of resize failure, but also to consider the subsequent implementation of
virtqueue_reset()

Thanks.


>
> Thanks
>
>
> > +	} else {
> > +		err = virtqueue_resize_split(_vq, num);
> > +		if (err)
> > +			virtqueue_reinit_split(vq);
> > +	}
> > +
> > +	if (vdev->config->enable_reset_vq(_vq))
> > +		return -EBUSY;
> > +
> > +	return err;
> > +}
> > +EXPORT_SYMBOL_GPL(virtqueue_resize);
> > +
> >   /* Only available for split ring */
> >   struct virtqueue *vring_new_virtqueue(unsigned int index,
> >   				      unsigned int num,
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index d59adc4be068..c86ff02e0ca0 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -91,6 +91,9 @@ dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
> >   dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
> >   dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
> >
> > +int virtqueue_resize(struct virtqueue *vq, u32 num,
> > +		     void (*recycle)(struct virtqueue *vq, void *buf));
> > +
> >   /**
> >    * virtio_device - representation of a device using virtio
> >    * @index: unique position on the virtio bus
>

Re: [PATCH v9 18/32] virtio_ring: introduce virtqueue_resize()

[Xuan Zhuo]

On Tue, 12 Apr 2022 14:41:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Introduce virtqueue_resize() to implement the resize of vring.
> > Based on these, the driver can dynamically adjust the size of the vring.
> > For example: ethtool -G.
> >
> > virtqueue_resize() implements resize based on the vq reset function. In
> > case of failure to allocate a new vring, it will give up resize and use
> > the original vring.
> >
> > During this process, if the re-enable reset vq fails, the vq can no
> > longer be used. Although the probability of this situation is not high.
> >
> > The parameter recycle is used to recycle the buffer that is no longer
> > used.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++++++++++
> >   include/linux/virtio.h       |  3 ++
> >   2 files changed, 72 insertions(+)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 06f66b15c86c..6250e19fc5bf 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2554,6 +2554,75 @@ struct virtqueue *vring_create_virtqueue(
> >   }
> >   EXPORT_SYMBOL_GPL(vring_create_virtqueue);
> >
> > +/**
> > + * virtqueue_resize - resize the vring of vq
> > + * @_vq: the struct virtqueue we're talking about.
> > + * @num: new ring num
> > + * @recycle: callback for recycle the useless buffer
> > + *
> > + * When it is really necessary to create a new vring, it will set the current vq
> > + * into the reset state. Then call the passed callback to recycle the buffer
> > + * that is no longer used. Only after the new vring is successfully created, the
> > + * old vring will be released.
> > + *
> > + * Caller must ensure we don't call this with other virtqueue operations
> > + * at the same time (except where noted).
> > + *
> > + * Returns zero or a negative error.
>
>
> Should we document that the virtqueue is kept unchanged (still
> available) on (specific) failure?
>

OK.

>
> > + */
> > +int virtqueue_resize(struct virtqueue *_vq, u32 num,
> > +		     void (*recycle)(struct virtqueue *vq, void *buf))
> > +{
> > +	struct vring_virtqueue *vq = to_vvq(_vq);
> > +	struct virtio_device *vdev = vq->vq.vdev;
> > +	bool packed;
> > +	void *buf;
> > +	int err;
> > +
> > +	if (!vq->we_own_ring)
> > +		return -EINVAL;
> > +
> > +	if (num > vq->vq.num_max)
> > +		return -E2BIG;
> > +
> > +	if (!num)
> > +		return -EINVAL;
> > +
> > +	packed = virtio_has_feature(vdev, VIRTIO_F_RING_PACKED) ? true : false;
> > +
> > +	if ((packed ? vq->packed.vring.num : vq->split.vring.num) == num)
> > +		return 0;
> > +
> > +	if (!vdev->config->reset_vq)
> > +		return -ENOENT;
> > +
> > +	if (!vdev->config->enable_reset_vq)
> > +		return -ENOENT;
> > +
> > +	err = vdev->config->reset_vq(_vq);
> > +	if (err)
> > +		return err;
> > +
> > +	while ((buf = virtqueue_detach_unused_buf(_vq)) != NULL)
> > +		recycle(_vq, buf);
> > +
> > +	if (packed) {
> > +		err = virtqueue_resize_packed(_vq, num);
> > +		if (err)
> > +			virtqueue_reinit_packed(vq);
>
>
> Calling reinit here seems a little bit odd, it looks more like a reset
> of the virtqueue. Consider we may re-use virtqueue reset for more
> purpose, I wonder if we need a helper like:
>
> virtqueue_resize() {
>      vdev->config->reset_vq(_vq);
>      if (packed)
>          virtqueue_reinit_packed(_vq)
>      else
>          virtqueue_reinit_split(_vq)
> }

Yes, currently we are implementing resize. This is used to implement
set_ringparam()

Later, when we implement virtio_net + AF_XDP, what we want is reset not resize,
so we need to implement a helper:

 virtqueue_reset() {
      vdev->config->reset_vq(_vq);
      if (packed)
          virtqueue_reinit_packed(_vq)
      else
          virtqueue_reinit_split(_vq)
 }

So I use virtqueue_reinit_* as a separate function not only to deal with the
case of resize failure, but also to consider the subsequent implementation of
virtqueue_reset()

Thanks.


>
> Thanks
>
>
> > +	} else {
> > +		err = virtqueue_resize_split(_vq, num);
> > +		if (err)
> > +			virtqueue_reinit_split(vq);
> > +	}
> > +
> > +	if (vdev->config->enable_reset_vq(_vq))
> > +		return -EBUSY;
> > +
> > +	return err;
> > +}
> > +EXPORT_SYMBOL_GPL(virtqueue_resize);
> > +
> >   /* Only available for split ring */
> >   struct virtqueue *vring_new_virtqueue(unsigned int index,
> >   				      unsigned int num,
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index d59adc4be068..c86ff02e0ca0 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -91,6 +91,9 @@ dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
> >   dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
> >   dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
> >
> > +int virtqueue_resize(struct virtqueue *vq, u32 num,
> > +		     void (*recycle)(struct virtqueue *vq, void *buf));
> > +
> >   /**
> >    * virtio_device - representation of a device using virtio
> >    * @index: unique position on the virtio bus
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 18/32] virtio_ring: introduce virtqueue_resize()

[Xuan Zhuo]

On Tue, 12 Apr 2022 14:41:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Introduce virtqueue_resize() to implement the resize of vring.
> > Based on these, the driver can dynamically adjust the size of the vring.
> > For example: ethtool -G.
> >
> > virtqueue_resize() implements resize based on the vq reset function. In
> > case of failure to allocate a new vring, it will give up resize and use
> > the original vring.
> >
> > During this process, if the re-enable reset vq fails, the vq can no
> > longer be used. Although the probability of this situation is not high.
> >
> > The parameter recycle is used to recycle the buffer that is no longer
> > used.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++++++++++
> >   include/linux/virtio.h       |  3 ++
> >   2 files changed, 72 insertions(+)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 06f66b15c86c..6250e19fc5bf 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2554,6 +2554,75 @@ struct virtqueue *vring_create_virtqueue(
> >   }
> >   EXPORT_SYMBOL_GPL(vring_create_virtqueue);
> >
> > +/**
> > + * virtqueue_resize - resize the vring of vq
> > + * @_vq: the struct virtqueue we're talking about.
> > + * @num: new ring num
> > + * @recycle: callback for recycle the useless buffer
> > + *
> > + * When it is really necessary to create a new vring, it will set the current vq
> > + * into the reset state. Then call the passed callback to recycle the buffer
> > + * that is no longer used. Only after the new vring is successfully created, the
> > + * old vring will be released.
> > + *
> > + * Caller must ensure we don't call this with other virtqueue operations
> > + * at the same time (except where noted).
> > + *
> > + * Returns zero or a negative error.
>
>
> Should we document that the virtqueue is kept unchanged (still
> available) on (specific) failure?
>

OK.

>
> > + */
> > +int virtqueue_resize(struct virtqueue *_vq, u32 num,
> > +		     void (*recycle)(struct virtqueue *vq, void *buf))
> > +{
> > +	struct vring_virtqueue *vq = to_vvq(_vq);
> > +	struct virtio_device *vdev = vq->vq.vdev;
> > +	bool packed;
> > +	void *buf;
> > +	int err;
> > +
> > +	if (!vq->we_own_ring)
> > +		return -EINVAL;
> > +
> > +	if (num > vq->vq.num_max)
> > +		return -E2BIG;
> > +
> > +	if (!num)
> > +		return -EINVAL;
> > +
> > +	packed = virtio_has_feature(vdev, VIRTIO_F_RING_PACKED) ? true : false;
> > +
> > +	if ((packed ? vq->packed.vring.num : vq->split.vring.num) == num)
> > +		return 0;
> > +
> > +	if (!vdev->config->reset_vq)
> > +		return -ENOENT;
> > +
> > +	if (!vdev->config->enable_reset_vq)
> > +		return -ENOENT;
> > +
> > +	err = vdev->config->reset_vq(_vq);
> > +	if (err)
> > +		return err;
> > +
> > +	while ((buf = virtqueue_detach_unused_buf(_vq)) != NULL)
> > +		recycle(_vq, buf);
> > +
> > +	if (packed) {
> > +		err = virtqueue_resize_packed(_vq, num);
> > +		if (err)
> > +			virtqueue_reinit_packed(vq);
>
>
> Calling reinit here seems a little bit odd, it looks more like a reset
> of the virtqueue. Consider we may re-use virtqueue reset for more
> purpose, I wonder if we need a helper like:
>
> virtqueue_resize() {
>      vdev->config->reset_vq(_vq);
>      if (packed)
>          virtqueue_reinit_packed(_vq)
>      else
>          virtqueue_reinit_split(_vq)
> }

Yes, currently we are implementing resize. This is used to implement
set_ringparam()

Later, when we implement virtio_net + AF_XDP, what we want is reset not resize,
so we need to implement a helper:

 virtqueue_reset() {
      vdev->config->reset_vq(_vq);
      if (packed)
          virtqueue_reinit_packed(_vq)
      else
          virtqueue_reinit_split(_vq)
 }

So I use virtqueue_reinit_* as a separate function not only to deal with the
case of resize failure, but also to consider the subsequent implementation of
virtqueue_reset()

Thanks.


>
> Thanks
>
>
> > +	} else {
> > +		err = virtqueue_resize_split(_vq, num);
> > +		if (err)
> > +			virtqueue_reinit_split(vq);
> > +	}
> > +
> > +	if (vdev->config->enable_reset_vq(_vq))
> > +		return -EBUSY;
> > +
> > +	return err;
> > +}
> > +EXPORT_SYMBOL_GPL(virtqueue_resize);
> > +
> >   /* Only available for split ring */
> >   struct virtqueue *vring_new_virtqueue(unsigned int index,
> >   				      unsigned int num,
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index d59adc4be068..c86ff02e0ca0 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -91,6 +91,9 @@ dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
> >   dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
> >   dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
> >
> > +int virtqueue_resize(struct virtqueue *vq, u32 num,
> > +		     void (*recycle)(struct virtqueue *vq, void *buf));
> > +
> >   /**
> >    * virtio_device - representation of a device using virtio
> >    * @index: unique position on the virtio bus
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 18/32] virtio_ring: introduce virtqueue_resize()

[Xuan Zhuo]

On Tue, 12 Apr 2022 14:41:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Introduce virtqueue_resize() to implement the resize of vring.
> > Based on these, the driver can dynamically adjust the size of the vring.
> > For example: ethtool -G.
> >
> > virtqueue_resize() implements resize based on the vq reset function. In
> > case of failure to allocate a new vring, it will give up resize and use
> > the original vring.
> >
> > During this process, if the re-enable reset vq fails, the vq can no
> > longer be used. Although the probability of this situation is not high.
> >
> > The parameter recycle is used to recycle the buffer that is no longer
> > used.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++++++++++
> >   include/linux/virtio.h       |  3 ++
> >   2 files changed, 72 insertions(+)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 06f66b15c86c..6250e19fc5bf 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2554,6 +2554,75 @@ struct virtqueue *vring_create_virtqueue(
> >   }
> >   EXPORT_SYMBOL_GPL(vring_create_virtqueue);
> >
> > +/**
> > + * virtqueue_resize - resize the vring of vq
> > + * @_vq: the struct virtqueue we're talking about.
> > + * @num: new ring num
> > + * @recycle: callback for recycle the useless buffer
> > + *
> > + * When it is really necessary to create a new vring, it will set the current vq
> > + * into the reset state. Then call the passed callback to recycle the buffer
> > + * that is no longer used. Only after the new vring is successfully created, the
> > + * old vring will be released.
> > + *
> > + * Caller must ensure we don't call this with other virtqueue operations
> > + * at the same time (except where noted).
> > + *
> > + * Returns zero or a negative error.
>
>
> Should we document that the virtqueue is kept unchanged (still
> available) on (specific) failure?
>
>
> > + */
> > +int virtqueue_resize(struct virtqueue *_vq, u32 num,
> > +		     void (*recycle)(struct virtqueue *vq, void *buf))
> > +{
> > +	struct vring_virtqueue *vq = to_vvq(_vq);
> > +	struct virtio_device *vdev = vq->vq.vdev;
> > +	bool packed;
> > +	void *buf;
> > +	int err;
> > +
> > +	if (!vq->we_own_ring)
> > +		return -EINVAL;
> > +
> > +	if (num > vq->vq.num_max)
> > +		return -E2BIG;
> > +
> > +	if (!num)
> > +		return -EINVAL;
> > +
> > +	packed = virtio_has_feature(vdev, VIRTIO_F_RING_PACKED) ? true : false;
> > +
> > +	if ((packed ? vq->packed.vring.num : vq->split.vring.num) == num)
> > +		return 0;
> > +
> > +	if (!vdev->config->reset_vq)
> > +		return -ENOENT;
> > +
> > +	if (!vdev->config->enable_reset_vq)
> > +		return -ENOENT;
> > +
> > +	err = vdev->config->reset_vq(_vq);
> > +	if (err)
> > +		return err;
> > +
> > +	while ((buf = virtqueue_detach_unused_buf(_vq)) != NULL)
> > +		recycle(_vq, buf);
> > +
> > +	if (packed) {
> > +		err = virtqueue_resize_packed(_vq, num);
> > +		if (err)
> > +			virtqueue_reinit_packed(vq);
>
>
> Calling reinit here seems a little bit odd, it looks more like a reset


I also feel that this is a bit odd, I will put virtqueue_reinit_* into
virtqueue_resize_*.

Thanks.

> of the virtqueue. Consider we may re-use virtqueue reset for more
> purpose, I wonder if we need a helper like:
>
> virtqueue_resize() {
>      vdev->config->reset_vq(_vq);
>      if (packed)
>          virtqueue_reinit_packed(_vq)
>      else
>          virtqueue_reinit_split(_vq)
> }
>
> Thanks
>
>
> > +	} else {
> > +		err = virtqueue_resize_split(_vq, num);
> > +		if (err)
> > +			virtqueue_reinit_split(vq);
> > +	}
> > +
> > +	if (vdev->config->enable_reset_vq(_vq))
> > +		return -EBUSY;
> > +
> > +	return err;
> > +}
> > +EXPORT_SYMBOL_GPL(virtqueue_resize);
> > +
> >   /* Only available for split ring */
> >   struct virtqueue *vring_new_virtqueue(unsigned int index,
> >   				      unsigned int num,
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index d59adc4be068..c86ff02e0ca0 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -91,6 +91,9 @@ dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
> >   dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
> >   dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
> >
> > +int virtqueue_resize(struct virtqueue *vq, u32 num,
> > +		     void (*recycle)(struct virtqueue *vq, void *buf));
> > +
> >   /**
> >    * virtio_device - representation of a device using virtio
> >    * @index: unique position on the virtio bus
>

Re: [PATCH v9 18/32] virtio_ring: introduce virtqueue_resize()

[Xuan Zhuo]

On Tue, 12 Apr 2022 14:41:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Introduce virtqueue_resize() to implement the resize of vring.
> > Based on these, the driver can dynamically adjust the size of the vring.
> > For example: ethtool -G.
> >
> > virtqueue_resize() implements resize based on the vq reset function. In
> > case of failure to allocate a new vring, it will give up resize and use
> > the original vring.
> >
> > During this process, if the re-enable reset vq fails, the vq can no
> > longer be used. Although the probability of this situation is not high.
> >
> > The parameter recycle is used to recycle the buffer that is no longer
> > used.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++++++++++
> >   include/linux/virtio.h       |  3 ++
> >   2 files changed, 72 insertions(+)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 06f66b15c86c..6250e19fc5bf 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2554,6 +2554,75 @@ struct virtqueue *vring_create_virtqueue(
> >   }
> >   EXPORT_SYMBOL_GPL(vring_create_virtqueue);
> >
> > +/**
> > + * virtqueue_resize - resize the vring of vq
> > + * @_vq: the struct virtqueue we're talking about.
> > + * @num: new ring num
> > + * @recycle: callback for recycle the useless buffer
> > + *
> > + * When it is really necessary to create a new vring, it will set the current vq
> > + * into the reset state. Then call the passed callback to recycle the buffer
> > + * that is no longer used. Only after the new vring is successfully created, the
> > + * old vring will be released.
> > + *
> > + * Caller must ensure we don't call this with other virtqueue operations
> > + * at the same time (except where noted).
> > + *
> > + * Returns zero or a negative error.
>
>
> Should we document that the virtqueue is kept unchanged (still
> available) on (specific) failure?
>
>
> > + */
> > +int virtqueue_resize(struct virtqueue *_vq, u32 num,
> > +		     void (*recycle)(struct virtqueue *vq, void *buf))
> > +{
> > +	struct vring_virtqueue *vq = to_vvq(_vq);
> > +	struct virtio_device *vdev = vq->vq.vdev;
> > +	bool packed;
> > +	void *buf;
> > +	int err;
> > +
> > +	if (!vq->we_own_ring)
> > +		return -EINVAL;
> > +
> > +	if (num > vq->vq.num_max)
> > +		return -E2BIG;
> > +
> > +	if (!num)
> > +		return -EINVAL;
> > +
> > +	packed = virtio_has_feature(vdev, VIRTIO_F_RING_PACKED) ? true : false;
> > +
> > +	if ((packed ? vq->packed.vring.num : vq->split.vring.num) == num)
> > +		return 0;
> > +
> > +	if (!vdev->config->reset_vq)
> > +		return -ENOENT;
> > +
> > +	if (!vdev->config->enable_reset_vq)
> > +		return -ENOENT;
> > +
> > +	err = vdev->config->reset_vq(_vq);
> > +	if (err)
> > +		return err;
> > +
> > +	while ((buf = virtqueue_detach_unused_buf(_vq)) != NULL)
> > +		recycle(_vq, buf);
> > +
> > +	if (packed) {
> > +		err = virtqueue_resize_packed(_vq, num);
> > +		if (err)
> > +			virtqueue_reinit_packed(vq);
>
>
> Calling reinit here seems a little bit odd, it looks more like a reset


I also feel that this is a bit odd, I will put virtqueue_reinit_* into
virtqueue_resize_*.

Thanks.

> of the virtqueue. Consider we may re-use virtqueue reset for more
> purpose, I wonder if we need a helper like:
>
> virtqueue_resize() {
>      vdev->config->reset_vq(_vq);
>      if (packed)
>          virtqueue_reinit_packed(_vq)
>      else
>          virtqueue_reinit_split(_vq)
> }
>
> Thanks
>
>
> > +	} else {
> > +		err = virtqueue_resize_split(_vq, num);
> > +		if (err)
> > +			virtqueue_reinit_split(vq);
> > +	}
> > +
> > +	if (vdev->config->enable_reset_vq(_vq))
> > +		return -EBUSY;
> > +
> > +	return err;
> > +}
> > +EXPORT_SYMBOL_GPL(virtqueue_resize);
> > +
> >   /* Only available for split ring */
> >   struct virtqueue *vring_new_virtqueue(unsigned int index,
> >   				      unsigned int num,
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index d59adc4be068..c86ff02e0ca0 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -91,6 +91,9 @@ dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
> >   dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
> >   dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
> >
> > +int virtqueue_resize(struct virtqueue *vq, u32 num,
> > +		     void (*recycle)(struct virtqueue *vq, void *buf));
> > +
> >   /**
> >    * virtio_device - representation of a device using virtio
> >    * @index: unique position on the virtio bus
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 18/32] virtio_ring: introduce virtqueue_resize()

[Xuan Zhuo]

On Tue, 12 Apr 2022 14:41:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Introduce virtqueue_resize() to implement the resize of vring.
> > Based on these, the driver can dynamically adjust the size of the vring.
> > For example: ethtool -G.
> >
> > virtqueue_resize() implements resize based on the vq reset function. In
> > case of failure to allocate a new vring, it will give up resize and use
> > the original vring.
> >
> > During this process, if the re-enable reset vq fails, the vq can no
> > longer be used. Although the probability of this situation is not high.
> >
> > The parameter recycle is used to recycle the buffer that is no longer
> > used.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++++++++++
> >   include/linux/virtio.h       |  3 ++
> >   2 files changed, 72 insertions(+)
> >
> > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > index 06f66b15c86c..6250e19fc5bf 100644
> > --- a/drivers/virtio/virtio_ring.c
> > +++ b/drivers/virtio/virtio_ring.c
> > @@ -2554,6 +2554,75 @@ struct virtqueue *vring_create_virtqueue(
> >   }
> >   EXPORT_SYMBOL_GPL(vring_create_virtqueue);
> >
> > +/**
> > + * virtqueue_resize - resize the vring of vq
> > + * @_vq: the struct virtqueue we're talking about.
> > + * @num: new ring num
> > + * @recycle: callback for recycle the useless buffer
> > + *
> > + * When it is really necessary to create a new vring, it will set the current vq
> > + * into the reset state. Then call the passed callback to recycle the buffer
> > + * that is no longer used. Only after the new vring is successfully created, the
> > + * old vring will be released.
> > + *
> > + * Caller must ensure we don't call this with other virtqueue operations
> > + * at the same time (except where noted).
> > + *
> > + * Returns zero or a negative error.
>
>
> Should we document that the virtqueue is kept unchanged (still
> available) on (specific) failure?
>
>
> > + */
> > +int virtqueue_resize(struct virtqueue *_vq, u32 num,
> > +		     void (*recycle)(struct virtqueue *vq, void *buf))
> > +{
> > +	struct vring_virtqueue *vq = to_vvq(_vq);
> > +	struct virtio_device *vdev = vq->vq.vdev;
> > +	bool packed;
> > +	void *buf;
> > +	int err;
> > +
> > +	if (!vq->we_own_ring)
> > +		return -EINVAL;
> > +
> > +	if (num > vq->vq.num_max)
> > +		return -E2BIG;
> > +
> > +	if (!num)
> > +		return -EINVAL;
> > +
> > +	packed = virtio_has_feature(vdev, VIRTIO_F_RING_PACKED) ? true : false;
> > +
> > +	if ((packed ? vq->packed.vring.num : vq->split.vring.num) == num)
> > +		return 0;
> > +
> > +	if (!vdev->config->reset_vq)
> > +		return -ENOENT;
> > +
> > +	if (!vdev->config->enable_reset_vq)
> > +		return -ENOENT;
> > +
> > +	err = vdev->config->reset_vq(_vq);
> > +	if (err)
> > +		return err;
> > +
> > +	while ((buf = virtqueue_detach_unused_buf(_vq)) != NULL)
> > +		recycle(_vq, buf);
> > +
> > +	if (packed) {
> > +		err = virtqueue_resize_packed(_vq, num);
> > +		if (err)
> > +			virtqueue_reinit_packed(vq);
>
>
> Calling reinit here seems a little bit odd, it looks more like a reset


I also feel that this is a bit odd, I will put virtqueue_reinit_* into
virtqueue_resize_*.

Thanks.

> of the virtqueue. Consider we may re-use virtqueue reset for more
> purpose, I wonder if we need a helper like:
>
> virtqueue_resize() {
>      vdev->config->reset_vq(_vq);
>      if (packed)
>          virtqueue_reinit_packed(_vq)
>      else
>          virtqueue_reinit_split(_vq)
> }
>
> Thanks
>
>
> > +	} else {
> > +		err = virtqueue_resize_split(_vq, num);
> > +		if (err)
> > +			virtqueue_reinit_split(vq);
> > +	}
> > +
> > +	if (vdev->config->enable_reset_vq(_vq))
> > +		return -EBUSY;
> > +
> > +	return err;
> > +}
> > +EXPORT_SYMBOL_GPL(virtqueue_resize);
> > +
> >   /* Only available for split ring */
> >   struct virtqueue *vring_new_virtqueue(unsigned int index,
> >   				      unsigned int num,
> > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > index d59adc4be068..c86ff02e0ca0 100644
> > --- a/include/linux/virtio.h
> > +++ b/include/linux/virtio.h
> > @@ -91,6 +91,9 @@ dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
> >   dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
> >   dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
> >
> > +int virtqueue_resize(struct virtqueue *vq, u32 num,
> > +		     void (*recycle)(struct virtqueue *vq, void *buf));
> > +
> >   /**
> >    * virtio_device - representation of a device using virtio
> >    * @index: unique position on the virtio bus
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 22/32] virtio_pci: queue_reset: extract the logic of active vq for modern pci

[Xuan Zhuo]

On Tue, 12 Apr 2022 14:58:19 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Introduce vp_active_vq() to configure vring to backend after vq attach
> > vring. And configure vq vector if necessary.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_pci_modern.c | 46 ++++++++++++++++++------------
> >   1 file changed, 28 insertions(+), 18 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > index 86d301f272b8..49a4493732cf 100644
> > --- a/drivers/virtio/virtio_pci_modern.c
> > +++ b/drivers/virtio/virtio_pci_modern.c
> > @@ -176,6 +176,29 @@ static void vp_reset(struct virtio_device *vdev)
> >   	vp_disable_cbs(vdev);
> >   }
> >
> > +static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	unsigned long index;
> > +
> > +	index = vq->index;
> > +
> > +	/* activate the queue */
> > +	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > +	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > +				virtqueue_get_avail_addr(vq),
> > +				virtqueue_get_used_addr(vq));
> > +
> > +	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> > +		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> > +		if (msix_vec == VIRTIO_MSI_NO_VECTOR)
> > +			return -EBUSY;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> >   {
> >   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> > @@ -220,32 +243,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> >
> >   	vq->num_max = num;
> >
> > -	/* activate the queue */
> > -	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > -	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > -				virtqueue_get_avail_addr(vq),
> > -				virtqueue_get_used_addr(vq));
> > +	err = vp_active_vq(vq, msix_vec);
> > +	if (err)
> > +		goto err;
> >
> >   	vq->priv = (void __force *)vp_modern_map_vq_notify(mdev, index, NULL);
> >   	if (!vq->priv) {
> >   		err = -ENOMEM;
> > -		goto err_map_notify;
> > -	}
> > -
> > -	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> > -		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> > -		if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
> > -			err = -EBUSY;
> > -			goto err_assign_vector;
> > -		}
> > +		goto err;
> >   	}
> >
> >   	return vq;
> >
> > -err_assign_vector:
> > -	if (!mdev->notify_base)
> > -		pci_iounmap(mdev->pci_dev, (void __iomem __force *)vq->priv);
>
>
> We need keep this or anything I missed?

I think so, after modification, vp_modern_map_vq_notify is the last step before
returning vq. If it fails, then vq->priv is equal to NULL, so there is no need
to execute pci_iounmap.

Did I miss something?

Thanks.

>
> Thanks
>
>
> > -err_map_notify:
> > +err:
> >   	vring_del_virtqueue(vq);
> >   	return ERR_PTR(err);
> >   }
>

Re: [PATCH v9 22/32] virtio_pci: queue_reset: extract the logic of active vq for modern pci

[Xuan Zhuo]

On Tue, 12 Apr 2022 14:58:19 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Introduce vp_active_vq() to configure vring to backend after vq attach
> > vring. And configure vq vector if necessary.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_pci_modern.c | 46 ++++++++++++++++++------------
> >   1 file changed, 28 insertions(+), 18 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > index 86d301f272b8..49a4493732cf 100644
> > --- a/drivers/virtio/virtio_pci_modern.c
> > +++ b/drivers/virtio/virtio_pci_modern.c
> > @@ -176,6 +176,29 @@ static void vp_reset(struct virtio_device *vdev)
> >   	vp_disable_cbs(vdev);
> >   }
> >
> > +static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	unsigned long index;
> > +
> > +	index = vq->index;
> > +
> > +	/* activate the queue */
> > +	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > +	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > +				virtqueue_get_avail_addr(vq),
> > +				virtqueue_get_used_addr(vq));
> > +
> > +	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> > +		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> > +		if (msix_vec == VIRTIO_MSI_NO_VECTOR)
> > +			return -EBUSY;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> >   {
> >   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> > @@ -220,32 +243,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> >
> >   	vq->num_max = num;
> >
> > -	/* activate the queue */
> > -	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > -	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > -				virtqueue_get_avail_addr(vq),
> > -				virtqueue_get_used_addr(vq));
> > +	err = vp_active_vq(vq, msix_vec);
> > +	if (err)
> > +		goto err;
> >
> >   	vq->priv = (void __force *)vp_modern_map_vq_notify(mdev, index, NULL);
> >   	if (!vq->priv) {
> >   		err = -ENOMEM;
> > -		goto err_map_notify;
> > -	}
> > -
> > -	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> > -		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> > -		if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
> > -			err = -EBUSY;
> > -			goto err_assign_vector;
> > -		}
> > +		goto err;
> >   	}
> >
> >   	return vq;
> >
> > -err_assign_vector:
> > -	if (!mdev->notify_base)
> > -		pci_iounmap(mdev->pci_dev, (void __iomem __force *)vq->priv);
>
>
> We need keep this or anything I missed?

I think so, after modification, vp_modern_map_vq_notify is the last step before
returning vq. If it fails, then vq->priv is equal to NULL, so there is no need
to execute pci_iounmap.

Did I miss something?

Thanks.

>
> Thanks
>
>
> > -err_map_notify:
> > +err:
> >   	vring_del_virtqueue(vq);
> >   	return ERR_PTR(err);
> >   }
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 22/32] virtio_pci: queue_reset: extract the logic of active vq for modern pci

[Xuan Zhuo]

On Tue, 12 Apr 2022 14:58:19 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > Introduce vp_active_vq() to configure vring to backend after vq attach
> > vring. And configure vq vector if necessary.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/virtio/virtio_pci_modern.c | 46 ++++++++++++++++++------------
> >   1 file changed, 28 insertions(+), 18 deletions(-)
> >
> > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > index 86d301f272b8..49a4493732cf 100644
> > --- a/drivers/virtio/virtio_pci_modern.c
> > +++ b/drivers/virtio/virtio_pci_modern.c
> > @@ -176,6 +176,29 @@ static void vp_reset(struct virtio_device *vdev)
> >   	vp_disable_cbs(vdev);
> >   }
> >
> > +static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> > +{
> > +	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +	unsigned long index;
> > +
> > +	index = vq->index;
> > +
> > +	/* activate the queue */
> > +	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > +	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > +				virtqueue_get_avail_addr(vq),
> > +				virtqueue_get_used_addr(vq));
> > +
> > +	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> > +		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> > +		if (msix_vec == VIRTIO_MSI_NO_VECTOR)
> > +			return -EBUSY;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> >   {
> >   	return vp_modern_config_vector(&vp_dev->mdev, vector);
> > @@ -220,32 +243,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> >
> >   	vq->num_max = num;
> >
> > -	/* activate the queue */
> > -	vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > -	vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > -				virtqueue_get_avail_addr(vq),
> > -				virtqueue_get_used_addr(vq));
> > +	err = vp_active_vq(vq, msix_vec);
> > +	if (err)
> > +		goto err;
> >
> >   	vq->priv = (void __force *)vp_modern_map_vq_notify(mdev, index, NULL);
> >   	if (!vq->priv) {
> >   		err = -ENOMEM;
> > -		goto err_map_notify;
> > -	}
> > -
> > -	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> > -		msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> > -		if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
> > -			err = -EBUSY;
> > -			goto err_assign_vector;
> > -		}
> > +		goto err;
> >   	}
> >
> >   	return vq;
> >
> > -err_assign_vector:
> > -	if (!mdev->notify_base)
> > -		pci_iounmap(mdev->pci_dev, (void __iomem __force *)vq->priv);
>
>
> We need keep this or anything I missed?

I think so, after modification, vp_modern_map_vq_notify is the last step before
returning vq. If it fails, then vq->priv is equal to NULL, so there is no need
to execute pci_iounmap.

Did I miss something?

Thanks.

>
> Thanks
>
>
> > -err_map_notify:
> > +err:
> >   	vring_del_virtqueue(vq);
> >   	return ERR_PTR(err);
> >   }
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 01/32] virtio: add helper virtqueue_get_vring_max_size()

[Jason Wang]

On Wed, Apr 13, 2022 at 10:30 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 12 Apr 2022 10:41:03 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > Record the maximum queue num supported by the device.
> > >
> > > virtio-net can display the maximum (supported by hardware) ring size in
> > > ethtool -g eth0.
> > >
> > > When the subsequent patch implements vring reset, it can judge whether
> > > the ring size passed by the driver is legal based on this.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   arch/um/drivers/virtio_uml.c             |  1 +
> > >   drivers/platform/mellanox/mlxbf-tmfifo.c |  2 ++
> > >   drivers/remoteproc/remoteproc_virtio.c   |  2 ++
> > >   drivers/s390/virtio/virtio_ccw.c         |  3 +++
> > >   drivers/virtio/virtio_mmio.c             |  2 ++
> > >   drivers/virtio/virtio_pci_legacy.c       |  2 ++
> > >   drivers/virtio/virtio_pci_modern.c       |  2 ++
> > >   drivers/virtio/virtio_ring.c             | 14 ++++++++++++++
> > >   drivers/virtio/virtio_vdpa.c             |  2 ++
> > >   include/linux/virtio.h                   |  2 ++
> > >   10 files changed, 32 insertions(+)
> > >
> > > diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
> > > index ba562d68dc04..904993d15a85 100644
> > > --- a/arch/um/drivers/virtio_uml.c
> > > +++ b/arch/um/drivers/virtio_uml.c
> > > @@ -945,6 +945,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
> > >             goto error_create;
> > >     }
> > >     vq->priv = info;
> > > +   vq->num_max = num;
> > >     num = virtqueue_get_vring_size(vq);
> > >
> > >     if (vu_dev->protocol_features &
> > > diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
> > > index 38800e86ed8a..1ae3c56b66b0 100644
> > > --- a/drivers/platform/mellanox/mlxbf-tmfifo.c
> > > +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
> > > @@ -959,6 +959,8 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
> > >                     goto error;
> > >             }
> > >
> > > +           vq->num_max = vring->num;
> > > +
> > >             vqs[i] = vq;
> > >             vring->vq = vq;
> > >             vq->priv = vring;
> > > diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
> > > index 70ab496d0431..7611755d0ae2 100644
> > > --- a/drivers/remoteproc/remoteproc_virtio.c
> > > +++ b/drivers/remoteproc/remoteproc_virtio.c
> > > @@ -125,6 +125,8 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev,
> > >             return ERR_PTR(-ENOMEM);
> > >     }
> > >
> > > +   vq->num_max = len;
> >
> >
> > I wonder if this is correct.
> >
> > It looks to me len is counted in bytes:
> >
> > /**
> >   * struct rproc_vring - remoteproc vring state
> >   * @va: virtual address
> >   * @len: length, in bytes
> >   * @da: device address
> >   * @align: vring alignment
> >   * @notifyid: rproc-specific unique vring index
> >   * @rvdev: remote vdev
> >   * @vq: the virtqueue of this vring
> >   */
> > struct rproc_vring {
> >          void *va;
> >          int len;
> >          u32 da;
> >          u32 align;
> >          int notifyid;
> >          struct rproc_vdev *rvdev;
> >          struct virtqueue *vq;
> > };
> >
>
> I think this comment is incorrect because here len is passed as num to
> vring_new_virtqueue().
>
> There is also this usage:
>
>         /* actual size of vring (in bytes) */
>         size = PAGE_ALIGN(vring_size(rvring->len, rvring->align));
>
>
> And this value comes from here:
>
>         static int
>         rproc_parse_vring(struct rproc_vdev *rvdev, struct fw_rsc_vdev *rsc, int i)
>         {
>                 struct rproc *rproc = rvdev->rproc;
>                 struct device *dev = &rproc->dev;
>                 struct fw_rsc_vdev_vring *vring = &rsc->vring[i];
>                 struct rproc_vring *rvring = &rvdev->vring[i];
>
>                 dev_dbg(dev, "vdev rsc: vring%d: da 0x%x, qsz %d, align %d\n",
>                         i, vring->da, vring->num, vring->align);
>
>                 /* verify queue size and vring alignment are sane */
>                 if (!vring->num || !vring->align) {
>                         dev_err(dev, "invalid qsz (%d) or alignment (%d)\n",
>                                 vring->num, vring->align);
>                         return -EINVAL;
>                 }
>
>        >        rvring->len = vring->num;
>                 rvring->align = vring->align;
>                 rvring->rvdev = rvdev;
>
>                 return 0;
>         }
>
> /**
>  * struct fw_rsc_vdev_vring - vring descriptor entry
>  * @da: device address
>  * @align: the alignment between the consumer and producer parts of the vring
>  * @num: num of buffers supported by this vring (must be power of two)
>  * @notifyid: a unique rproc-wide notify index for this vring. This notify
>  * index is used when kicking a remote processor, to let it know that this
>  * vring is triggered.
>  * @pa: physical address
>  *
>  * This descriptor is not a resource entry by itself; it is part of the
>  * vdev resource type (see below).
>  *
>  * Note that @da should either contain the device address where
>  * the remote processor is expecting the vring, or indicate that
>  * dynamically allocation of the vring's device address is supported.
>  */
> struct fw_rsc_vdev_vring {
>         u32 da;
>         u32 align;
>         u32 num;
>         u32 notifyid;
>         u32 pa;
> } __packed;
>
> So I think the 'len' here may have changed its meaning in a version update.

I think you're right, let's have a patch to fix the comment (probably
with the name since len is confusing here).

Thanks

>
> Thanks.
>
> >
> > Other looks good.
> >
> > Thanks
> >
> >
> > > +
> > >     rvring->vq = vq;
> > >     vq->priv = rvring;
> > >
> > > diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
> > > index d35e7a3f7067..468da60b56c5 100644
> > > --- a/drivers/s390/virtio/virtio_ccw.c
> > > +++ b/drivers/s390/virtio/virtio_ccw.c
> > > @@ -529,6 +529,9 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,
> > >             err = -ENOMEM;
> > >             goto out_err;
> > >     }
> > > +
> > > +   vq->num_max = info->num;
> > > +
> > >     /* it may have been reduced */
> > >     info->num = virtqueue_get_vring_size(vq);
> > >
> > > diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> > > index 56128b9c46eb..a41abc8051b9 100644
> > > --- a/drivers/virtio/virtio_mmio.c
> > > +++ b/drivers/virtio/virtio_mmio.c
> > > @@ -390,6 +390,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
> > >             goto error_new_virtqueue;
> > >     }
> > >
> > > +   vq->num_max = num;
> > > +
> > >     /* Activate the queue */
> > >     writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
> > >     if (vm_dev->version == 1) {
> > > diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
> > > index 34141b9abe27..b68934fe6b5d 100644
> > > --- a/drivers/virtio/virtio_pci_legacy.c
> > > +++ b/drivers/virtio/virtio_pci_legacy.c
> > > @@ -135,6 +135,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> > >     if (!vq)
> > >             return ERR_PTR(-ENOMEM);
> > >
> > > +   vq->num_max = num;
> > > +
> > >     q_pfn = virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT;
> > >     if (q_pfn >> 32) {
> > >             dev_err(&vp_dev->pci_dev->dev,
> > > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > > index 5455bc041fb6..86d301f272b8 100644
> > > --- a/drivers/virtio/virtio_pci_modern.c
> > > +++ b/drivers/virtio/virtio_pci_modern.c
> > > @@ -218,6 +218,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> > >     if (!vq)
> > >             return ERR_PTR(-ENOMEM);
> > >
> > > +   vq->num_max = num;
> > > +
> > >     /* activate the queue */
> > >     vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > >     vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > > index 962f1477b1fa..b87130c8f312 100644
> > > --- a/drivers/virtio/virtio_ring.c
> > > +++ b/drivers/virtio/virtio_ring.c
> > > @@ -2371,6 +2371,20 @@ void vring_transport_features(struct virtio_device *vdev)
> > >   }
> > >   EXPORT_SYMBOL_GPL(vring_transport_features);
> > >
> > > +/**
> > > + * virtqueue_get_vring_max_size - return the max size of the virtqueue's vring
> > > + * @_vq: the struct virtqueue containing the vring of interest.
> > > + *
> > > + * Returns the max size of the vring.
> > > + *
> > > + * Unlike other operations, this need not be serialized.
> > > + */
> > > +unsigned int virtqueue_get_vring_max_size(struct virtqueue *_vq)
> > > +{
> > > +   return _vq->num_max;
> > > +}
> > > +EXPORT_SYMBOL_GPL(virtqueue_get_vring_max_size);
> > > +
> > >   /**
> > >    * virtqueue_get_vring_size - return the size of the virtqueue's vring
> > >    * @_vq: the struct virtqueue containing the vring of interest.
> > > diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
> > > index 7767a7f0119b..39e4c08eb0f2 100644
> > > --- a/drivers/virtio/virtio_vdpa.c
> > > +++ b/drivers/virtio/virtio_vdpa.c
> > > @@ -183,6 +183,8 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
> > >             goto error_new_virtqueue;
> > >     }
> > >
> > > +   vq->num_max = max_num;
> > > +
> > >     /* Setup virtqueue callback */
> > >     cb.callback = virtio_vdpa_virtqueue_cb;
> > >     cb.private = info;
> > > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > > index 72292a62cd90..d59adc4be068 100644
> > > --- a/include/linux/virtio.h
> > > +++ b/include/linux/virtio.h
> > > @@ -31,6 +31,7 @@ struct virtqueue {
> > >     struct virtio_device *vdev;
> > >     unsigned int index;
> > >     unsigned int num_free;
> > > +   unsigned int num_max;
> > >     void *priv;
> > >   };
> > >
> > > @@ -80,6 +81,7 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *vq);
> > >
> > >   void *virtqueue_detach_unused_buf(struct virtqueue *vq);
> > >
> > > +unsigned int virtqueue_get_vring_max_size(struct virtqueue *vq);
> > >   unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
> > >
> > >   bool virtqueue_is_broken(struct virtqueue *vq);
> >
>

Re: [PATCH v9 01/32] virtio: add helper virtqueue_get_vring_max_size()

[Jason Wang]

On Wed, Apr 13, 2022 at 10:30 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 12 Apr 2022 10:41:03 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > Record the maximum queue num supported by the device.
> > >
> > > virtio-net can display the maximum (supported by hardware) ring size in
> > > ethtool -g eth0.
> > >
> > > When the subsequent patch implements vring reset, it can judge whether
> > > the ring size passed by the driver is legal based on this.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   arch/um/drivers/virtio_uml.c             |  1 +
> > >   drivers/platform/mellanox/mlxbf-tmfifo.c |  2 ++
> > >   drivers/remoteproc/remoteproc_virtio.c   |  2 ++
> > >   drivers/s390/virtio/virtio_ccw.c         |  3 +++
> > >   drivers/virtio/virtio_mmio.c             |  2 ++
> > >   drivers/virtio/virtio_pci_legacy.c       |  2 ++
> > >   drivers/virtio/virtio_pci_modern.c       |  2 ++
> > >   drivers/virtio/virtio_ring.c             | 14 ++++++++++++++
> > >   drivers/virtio/virtio_vdpa.c             |  2 ++
> > >   include/linux/virtio.h                   |  2 ++
> > >   10 files changed, 32 insertions(+)
> > >
> > > diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
> > > index ba562d68dc04..904993d15a85 100644
> > > --- a/arch/um/drivers/virtio_uml.c
> > > +++ b/arch/um/drivers/virtio_uml.c
> > > @@ -945,6 +945,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
> > >             goto error_create;
> > >     }
> > >     vq->priv = info;
> > > +   vq->num_max = num;
> > >     num = virtqueue_get_vring_size(vq);
> > >
> > >     if (vu_dev->protocol_features &
> > > diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
> > > index 38800e86ed8a..1ae3c56b66b0 100644
> > > --- a/drivers/platform/mellanox/mlxbf-tmfifo.c
> > > +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
> > > @@ -959,6 +959,8 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
> > >                     goto error;
> > >             }
> > >
> > > +           vq->num_max = vring->num;
> > > +
> > >             vqs[i] = vq;
> > >             vring->vq = vq;
> > >             vq->priv = vring;
> > > diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
> > > index 70ab496d0431..7611755d0ae2 100644
> > > --- a/drivers/remoteproc/remoteproc_virtio.c
> > > +++ b/drivers/remoteproc/remoteproc_virtio.c
> > > @@ -125,6 +125,8 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev,
> > >             return ERR_PTR(-ENOMEM);
> > >     }
> > >
> > > +   vq->num_max = len;
> >
> >
> > I wonder if this is correct.
> >
> > It looks to me len is counted in bytes:
> >
> > /**
> >   * struct rproc_vring - remoteproc vring state
> >   * @va: virtual address
> >   * @len: length, in bytes
> >   * @da: device address
> >   * @align: vring alignment
> >   * @notifyid: rproc-specific unique vring index
> >   * @rvdev: remote vdev
> >   * @vq: the virtqueue of this vring
> >   */
> > struct rproc_vring {
> >          void *va;
> >          int len;
> >          u32 da;
> >          u32 align;
> >          int notifyid;
> >          struct rproc_vdev *rvdev;
> >          struct virtqueue *vq;
> > };
> >
>
> I think this comment is incorrect because here len is passed as num to
> vring_new_virtqueue().
>
> There is also this usage:
>
>         /* actual size of vring (in bytes) */
>         size = PAGE_ALIGN(vring_size(rvring->len, rvring->align));
>
>
> And this value comes from here:
>
>         static int
>         rproc_parse_vring(struct rproc_vdev *rvdev, struct fw_rsc_vdev *rsc, int i)
>         {
>                 struct rproc *rproc = rvdev->rproc;
>                 struct device *dev = &rproc->dev;
>                 struct fw_rsc_vdev_vring *vring = &rsc->vring[i];
>                 struct rproc_vring *rvring = &rvdev->vring[i];
>
>                 dev_dbg(dev, "vdev rsc: vring%d: da 0x%x, qsz %d, align %d\n",
>                         i, vring->da, vring->num, vring->align);
>
>                 /* verify queue size and vring alignment are sane */
>                 if (!vring->num || !vring->align) {
>                         dev_err(dev, "invalid qsz (%d) or alignment (%d)\n",
>                                 vring->num, vring->align);
>                         return -EINVAL;
>                 }
>
>        >        rvring->len = vring->num;
>                 rvring->align = vring->align;
>                 rvring->rvdev = rvdev;
>
>                 return 0;
>         }
>
> /**
>  * struct fw_rsc_vdev_vring - vring descriptor entry
>  * @da: device address
>  * @align: the alignment between the consumer and producer parts of the vring
>  * @num: num of buffers supported by this vring (must be power of two)
>  * @notifyid: a unique rproc-wide notify index for this vring. This notify
>  * index is used when kicking a remote processor, to let it know that this
>  * vring is triggered.
>  * @pa: physical address
>  *
>  * This descriptor is not a resource entry by itself; it is part of the
>  * vdev resource type (see below).
>  *
>  * Note that @da should either contain the device address where
>  * the remote processor is expecting the vring, or indicate that
>  * dynamically allocation of the vring's device address is supported.
>  */
> struct fw_rsc_vdev_vring {
>         u32 da;
>         u32 align;
>         u32 num;
>         u32 notifyid;
>         u32 pa;
> } __packed;
>
> So I think the 'len' here may have changed its meaning in a version update.

I think you're right, let's have a patch to fix the comment (probably
with the name since len is confusing here).

Thanks

>
> Thanks.
>
> >
> > Other looks good.
> >
> > Thanks
> >
> >
> > > +
> > >     rvring->vq = vq;
> > >     vq->priv = rvring;
> > >
> > > diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
> > > index d35e7a3f7067..468da60b56c5 100644
> > > --- a/drivers/s390/virtio/virtio_ccw.c
> > > +++ b/drivers/s390/virtio/virtio_ccw.c
> > > @@ -529,6 +529,9 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,
> > >             err = -ENOMEM;
> > >             goto out_err;
> > >     }
> > > +
> > > +   vq->num_max = info->num;
> > > +
> > >     /* it may have been reduced */
> > >     info->num = virtqueue_get_vring_size(vq);
> > >
> > > diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> > > index 56128b9c46eb..a41abc8051b9 100644
> > > --- a/drivers/virtio/virtio_mmio.c
> > > +++ b/drivers/virtio/virtio_mmio.c
> > > @@ -390,6 +390,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
> > >             goto error_new_virtqueue;
> > >     }
> > >
> > > +   vq->num_max = num;
> > > +
> > >     /* Activate the queue */
> > >     writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
> > >     if (vm_dev->version == 1) {
> > > diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
> > > index 34141b9abe27..b68934fe6b5d 100644
> > > --- a/drivers/virtio/virtio_pci_legacy.c
> > > +++ b/drivers/virtio/virtio_pci_legacy.c
> > > @@ -135,6 +135,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> > >     if (!vq)
> > >             return ERR_PTR(-ENOMEM);
> > >
> > > +   vq->num_max = num;
> > > +
> > >     q_pfn = virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT;
> > >     if (q_pfn >> 32) {
> > >             dev_err(&vp_dev->pci_dev->dev,
> > > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > > index 5455bc041fb6..86d301f272b8 100644
> > > --- a/drivers/virtio/virtio_pci_modern.c
> > > +++ b/drivers/virtio/virtio_pci_modern.c
> > > @@ -218,6 +218,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> > >     if (!vq)
> > >             return ERR_PTR(-ENOMEM);
> > >
> > > +   vq->num_max = num;
> > > +
> > >     /* activate the queue */
> > >     vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > >     vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > > index 962f1477b1fa..b87130c8f312 100644
> > > --- a/drivers/virtio/virtio_ring.c
> > > +++ b/drivers/virtio/virtio_ring.c
> > > @@ -2371,6 +2371,20 @@ void vring_transport_features(struct virtio_device *vdev)
> > >   }
> > >   EXPORT_SYMBOL_GPL(vring_transport_features);
> > >
> > > +/**
> > > + * virtqueue_get_vring_max_size - return the max size of the virtqueue's vring
> > > + * @_vq: the struct virtqueue containing the vring of interest.
> > > + *
> > > + * Returns the max size of the vring.
> > > + *
> > > + * Unlike other operations, this need not be serialized.
> > > + */
> > > +unsigned int virtqueue_get_vring_max_size(struct virtqueue *_vq)
> > > +{
> > > +   return _vq->num_max;
> > > +}
> > > +EXPORT_SYMBOL_GPL(virtqueue_get_vring_max_size);
> > > +
> > >   /**
> > >    * virtqueue_get_vring_size - return the size of the virtqueue's vring
> > >    * @_vq: the struct virtqueue containing the vring of interest.
> > > diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
> > > index 7767a7f0119b..39e4c08eb0f2 100644
> > > --- a/drivers/virtio/virtio_vdpa.c
> > > +++ b/drivers/virtio/virtio_vdpa.c
> > > @@ -183,6 +183,8 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
> > >             goto error_new_virtqueue;
> > >     }
> > >
> > > +   vq->num_max = max_num;
> > > +
> > >     /* Setup virtqueue callback */
> > >     cb.callback = virtio_vdpa_virtqueue_cb;
> > >     cb.private = info;
> > > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > > index 72292a62cd90..d59adc4be068 100644
> > > --- a/include/linux/virtio.h
> > > +++ b/include/linux/virtio.h
> > > @@ -31,6 +31,7 @@ struct virtqueue {
> > >     struct virtio_device *vdev;
> > >     unsigned int index;
> > >     unsigned int num_free;
> > > +   unsigned int num_max;
> > >     void *priv;
> > >   };
> > >
> > > @@ -80,6 +81,7 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *vq);
> > >
> > >   void *virtqueue_detach_unused_buf(struct virtqueue *vq);
> > >
> > > +unsigned int virtqueue_get_vring_max_size(struct virtqueue *vq);
> > >   unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
> > >
> > >   bool virtqueue_is_broken(struct virtqueue *vq);
> >
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 01/32] virtio: add helper virtqueue_get_vring_max_size()

[Jason Wang]

On Wed, Apr 13, 2022 at 10:30 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 12 Apr 2022 10:41:03 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > Record the maximum queue num supported by the device.
> > >
> > > virtio-net can display the maximum (supported by hardware) ring size in
> > > ethtool -g eth0.
> > >
> > > When the subsequent patch implements vring reset, it can judge whether
> > > the ring size passed by the driver is legal based on this.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   arch/um/drivers/virtio_uml.c             |  1 +
> > >   drivers/platform/mellanox/mlxbf-tmfifo.c |  2 ++
> > >   drivers/remoteproc/remoteproc_virtio.c   |  2 ++
> > >   drivers/s390/virtio/virtio_ccw.c         |  3 +++
> > >   drivers/virtio/virtio_mmio.c             |  2 ++
> > >   drivers/virtio/virtio_pci_legacy.c       |  2 ++
> > >   drivers/virtio/virtio_pci_modern.c       |  2 ++
> > >   drivers/virtio/virtio_ring.c             | 14 ++++++++++++++
> > >   drivers/virtio/virtio_vdpa.c             |  2 ++
> > >   include/linux/virtio.h                   |  2 ++
> > >   10 files changed, 32 insertions(+)
> > >
> > > diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c
> > > index ba562d68dc04..904993d15a85 100644
> > > --- a/arch/um/drivers/virtio_uml.c
> > > +++ b/arch/um/drivers/virtio_uml.c
> > > @@ -945,6 +945,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev,
> > >             goto error_create;
> > >     }
> > >     vq->priv = info;
> > > +   vq->num_max = num;
> > >     num = virtqueue_get_vring_size(vq);
> > >
> > >     if (vu_dev->protocol_features &
> > > diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c
> > > index 38800e86ed8a..1ae3c56b66b0 100644
> > > --- a/drivers/platform/mellanox/mlxbf-tmfifo.c
> > > +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c
> > > @@ -959,6 +959,8 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev,
> > >                     goto error;
> > >             }
> > >
> > > +           vq->num_max = vring->num;
> > > +
> > >             vqs[i] = vq;
> > >             vring->vq = vq;
> > >             vq->priv = vring;
> > > diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
> > > index 70ab496d0431..7611755d0ae2 100644
> > > --- a/drivers/remoteproc/remoteproc_virtio.c
> > > +++ b/drivers/remoteproc/remoteproc_virtio.c
> > > @@ -125,6 +125,8 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev,
> > >             return ERR_PTR(-ENOMEM);
> > >     }
> > >
> > > +   vq->num_max = len;
> >
> >
> > I wonder if this is correct.
> >
> > It looks to me len is counted in bytes:
> >
> > /**
> >   * struct rproc_vring - remoteproc vring state
> >   * @va: virtual address
> >   * @len: length, in bytes
> >   * @da: device address
> >   * @align: vring alignment
> >   * @notifyid: rproc-specific unique vring index
> >   * @rvdev: remote vdev
> >   * @vq: the virtqueue of this vring
> >   */
> > struct rproc_vring {
> >          void *va;
> >          int len;
> >          u32 da;
> >          u32 align;
> >          int notifyid;
> >          struct rproc_vdev *rvdev;
> >          struct virtqueue *vq;
> > };
> >
>
> I think this comment is incorrect because here len is passed as num to
> vring_new_virtqueue().
>
> There is also this usage:
>
>         /* actual size of vring (in bytes) */
>         size = PAGE_ALIGN(vring_size(rvring->len, rvring->align));
>
>
> And this value comes from here:
>
>         static int
>         rproc_parse_vring(struct rproc_vdev *rvdev, struct fw_rsc_vdev *rsc, int i)
>         {
>                 struct rproc *rproc = rvdev->rproc;
>                 struct device *dev = &rproc->dev;
>                 struct fw_rsc_vdev_vring *vring = &rsc->vring[i];
>                 struct rproc_vring *rvring = &rvdev->vring[i];
>
>                 dev_dbg(dev, "vdev rsc: vring%d: da 0x%x, qsz %d, align %d\n",
>                         i, vring->da, vring->num, vring->align);
>
>                 /* verify queue size and vring alignment are sane */
>                 if (!vring->num || !vring->align) {
>                         dev_err(dev, "invalid qsz (%d) or alignment (%d)\n",
>                                 vring->num, vring->align);
>                         return -EINVAL;
>                 }
>
>        >        rvring->len = vring->num;
>                 rvring->align = vring->align;
>                 rvring->rvdev = rvdev;
>
>                 return 0;
>         }
>
> /**
>  * struct fw_rsc_vdev_vring - vring descriptor entry
>  * @da: device address
>  * @align: the alignment between the consumer and producer parts of the vring
>  * @num: num of buffers supported by this vring (must be power of two)
>  * @notifyid: a unique rproc-wide notify index for this vring. This notify
>  * index is used when kicking a remote processor, to let it know that this
>  * vring is triggered.
>  * @pa: physical address
>  *
>  * This descriptor is not a resource entry by itself; it is part of the
>  * vdev resource type (see below).
>  *
>  * Note that @da should either contain the device address where
>  * the remote processor is expecting the vring, or indicate that
>  * dynamically allocation of the vring's device address is supported.
>  */
> struct fw_rsc_vdev_vring {
>         u32 da;
>         u32 align;
>         u32 num;
>         u32 notifyid;
>         u32 pa;
> } __packed;
>
> So I think the 'len' here may have changed its meaning in a version update.

I think you're right, let's have a patch to fix the comment (probably
with the name since len is confusing here).

Thanks

>
> Thanks.
>
> >
> > Other looks good.
> >
> > Thanks
> >
> >
> > > +
> > >     rvring->vq = vq;
> > >     vq->priv = rvring;
> > >
> > > diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c
> > > index d35e7a3f7067..468da60b56c5 100644
> > > --- a/drivers/s390/virtio/virtio_ccw.c
> > > +++ b/drivers/s390/virtio/virtio_ccw.c
> > > @@ -529,6 +529,9 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,
> > >             err = -ENOMEM;
> > >             goto out_err;
> > >     }
> > > +
> > > +   vq->num_max = info->num;
> > > +
> > >     /* it may have been reduced */
> > >     info->num = virtqueue_get_vring_size(vq);
> > >
> > > diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> > > index 56128b9c46eb..a41abc8051b9 100644
> > > --- a/drivers/virtio/virtio_mmio.c
> > > +++ b/drivers/virtio/virtio_mmio.c
> > > @@ -390,6 +390,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
> > >             goto error_new_virtqueue;
> > >     }
> > >
> > > +   vq->num_max = num;
> > > +
> > >     /* Activate the queue */
> > >     writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
> > >     if (vm_dev->version == 1) {
> > > diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
> > > index 34141b9abe27..b68934fe6b5d 100644
> > > --- a/drivers/virtio/virtio_pci_legacy.c
> > > +++ b/drivers/virtio/virtio_pci_legacy.c
> > > @@ -135,6 +135,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> > >     if (!vq)
> > >             return ERR_PTR(-ENOMEM);
> > >
> > > +   vq->num_max = num;
> > > +
> > >     q_pfn = virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT;
> > >     if (q_pfn >> 32) {
> > >             dev_err(&vp_dev->pci_dev->dev,
> > > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > > index 5455bc041fb6..86d301f272b8 100644
> > > --- a/drivers/virtio/virtio_pci_modern.c
> > > +++ b/drivers/virtio/virtio_pci_modern.c
> > > @@ -218,6 +218,8 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> > >     if (!vq)
> > >             return ERR_PTR(-ENOMEM);
> > >
> > > +   vq->num_max = num;
> > > +
> > >     /* activate the queue */
> > >     vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > >     vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > > index 962f1477b1fa..b87130c8f312 100644
> > > --- a/drivers/virtio/virtio_ring.c
> > > +++ b/drivers/virtio/virtio_ring.c
> > > @@ -2371,6 +2371,20 @@ void vring_transport_features(struct virtio_device *vdev)
> > >   }
> > >   EXPORT_SYMBOL_GPL(vring_transport_features);
> > >
> > > +/**
> > > + * virtqueue_get_vring_max_size - return the max size of the virtqueue's vring
> > > + * @_vq: the struct virtqueue containing the vring of interest.
> > > + *
> > > + * Returns the max size of the vring.
> > > + *
> > > + * Unlike other operations, this need not be serialized.
> > > + */
> > > +unsigned int virtqueue_get_vring_max_size(struct virtqueue *_vq)
> > > +{
> > > +   return _vq->num_max;
> > > +}
> > > +EXPORT_SYMBOL_GPL(virtqueue_get_vring_max_size);
> > > +
> > >   /**
> > >    * virtqueue_get_vring_size - return the size of the virtqueue's vring
> > >    * @_vq: the struct virtqueue containing the vring of interest.
> > > diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c
> > > index 7767a7f0119b..39e4c08eb0f2 100644
> > > --- a/drivers/virtio/virtio_vdpa.c
> > > +++ b/drivers/virtio/virtio_vdpa.c
> > > @@ -183,6 +183,8 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
> > >             goto error_new_virtqueue;
> > >     }
> > >
> > > +   vq->num_max = max_num;
> > > +
> > >     /* Setup virtqueue callback */
> > >     cb.callback = virtio_vdpa_virtqueue_cb;
> > >     cb.private = info;
> > > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > > index 72292a62cd90..d59adc4be068 100644
> > > --- a/include/linux/virtio.h
> > > +++ b/include/linux/virtio.h
> > > @@ -31,6 +31,7 @@ struct virtqueue {
> > >     struct virtio_device *vdev;
> > >     unsigned int index;
> > >     unsigned int num_free;
> > > +   unsigned int num_max;
> > >     void *priv;
> > >   };
> > >
> > > @@ -80,6 +81,7 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *vq);
> > >
> > >   void *virtqueue_detach_unused_buf(struct virtqueue *vq);
> > >
> > > +unsigned int virtqueue_get_vring_max_size(struct virtqueue *vq);
> > >   unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
> > >
> > >   bool virtqueue_is_broken(struct virtqueue *vq);
> >
>


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Jason Wang]

On Wed, Apr 13, 2022 at 11:23 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 12 Apr 2022 15:07:58 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > This patch implements virtio pci support for QUEUE RESET.
> > >
> > > Performing reset on a queue is divided into these steps:
> > >
> > >   1. notify the device to reset the queue
> > >   2. recycle the buffer submitted
> > >   3. reset the vring (may re-alloc)
> > >   4. mmap vring to device, and enable the queue
> > >
> > > This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
> > > pci scenario.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/virtio/virtio_pci_common.c |  8 +--
> > >   drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
> > >   drivers/virtio/virtio_ring.c       |  2 +
> > >   include/linux/virtio.h             |  1 +
> > >   4 files changed, 92 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> > > index fdbde1db5ec5..863d3a8a0956 100644
> > > --- a/drivers/virtio/virtio_pci_common.c
> > > +++ b/drivers/virtio/virtio_pci_common.c
> > > @@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
> > >     struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
> > >     unsigned long flags;
> > >
> > > -   spin_lock_irqsave(&vp_dev->lock, flags);
> > > -   list_del(&info->node);
> > > -   spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +   if (!vq->reset) {
> >
> >
> > On which condition that we may hit this path?
> >
> >
> > > +           spin_lock_irqsave(&vp_dev->lock, flags);
> > > +           list_del(&info->node);
> > > +           spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +   }
> > >
> > >     vp_dev->del_vq(info);
> > >     kfree(info);
> > > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > > index 49a4493732cf..cb5d38f1c9c8 100644
> > > --- a/drivers/virtio/virtio_pci_modern.c
> > > +++ b/drivers/virtio/virtio_pci_modern.c
> > > @@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
> > >     if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
> > >                     pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
> > >             __virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
> > > +
> > > +   if (features & BIT_ULL(VIRTIO_F_RING_RESET))
> > > +           __virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
> > >   }
> > >
> > >   /* virtio config->finalize_features() implementation */
> > > @@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> > >     return 0;
> > >   }
> > >
> > > +static int vp_modern_reset_vq(struct virtqueue *vq)
> > > +{
> > > +   struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > +   struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > +   struct virtio_pci_vq_info *info;
> > > +   unsigned long flags;
> > > +
> > > +   if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
> > > +           return -ENOENT;
> > > +
> > > +   vp_modern_set_queue_reset(mdev, vq->index);
> > > +
> > > +   info = vp_dev->vqs[vq->index];
> > > +
> > > +   /* delete vq from irq handler */
> > > +   spin_lock_irqsave(&vp_dev->lock, flags);
> > > +   list_del(&info->node);
> > > +   spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +
> > > +   INIT_LIST_HEAD(&info->node);
> > > +
> > > +   /* For the case where vq has an exclusive irq, to prevent the irq from
> > > +    * being received again and the pending irq, call disable_irq().
> > > +    *
> > > +    * In the scenario based on shared interrupts, vq will be searched from
> > > +    * the queue virtqueues. Since the previous list_del() has been deleted
> > > +    * from the queue, it is impossible for vq to be called in this case.
> > > +    * There is no need to close the corresponding interrupt.
> > > +    */
> > > +   if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > > +           disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> >
> >
> > See the previous discussion and the revert of the first try to harden
> > the interrupt. We probably can't use disable_irq() since it conflicts
> > with the affinity managed IRQ that is used by some drivers.
> >
> > We need to use synchonize_irq() and per virtqueue flag instead. As
> > mentioned in previous patches, this could be done on top of my rework on
> > the IRQ hardening .
>
> OK, the next version will contain hardened features by per virtqueue flag.

Actually, I'm working on a new version of hardening. I plan to switch
to the virtqueue flag so we will be fine here if we do the resize work
on top.

I will cc you in the new version.

Thanks

>
> Thanks.
>
> >
> >
> > > +
> > > +   vq->reset = true;
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > > +{
> > > +   struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > +   struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > +   struct virtio_pci_vq_info *info;
> > > +   unsigned long flags, index;
> > > +   int err;
> > > +
> > > +   if (!vq->reset)
> > > +           return -EBUSY;
> > > +
> > > +   index = vq->index;
> > > +   info = vp_dev->vqs[index];
> > > +
> > > +   /* check queue reset status */
> > > +   if (vp_modern_get_queue_reset(mdev, index) != 1)
> > > +           return -EBUSY;
> > > +
> > > +   err = vp_active_vq(vq, info->msix_vector);
> > > +   if (err)
> > > +           return err;
> > > +
> > > +   if (vq->callback) {
> > > +           spin_lock_irqsave(&vp_dev->lock, flags);
> > > +           list_add(&info->node, &vp_dev->virtqueues);
> > > +           spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +   } else {
> > > +           INIT_LIST_HEAD(&info->node);
> > > +   }
> > > +
> > > +   vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > > +
> > > +   if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > > +           enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> >
> >
> > We had the same issue as disable_irq().
> >
> > Thanks
> >
> >
> > > +
> > > +   vq->reset = false;
> > > +
> > > +   return 0;
> > > +}
> > > +
> > >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> > >   {
> > >     return vp_modern_config_vector(&vp_dev->mdev, vector);
> > > @@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
> > >     .set_vq_affinity = vp_set_vq_affinity,
> > >     .get_vq_affinity = vp_get_vq_affinity,
> > >     .get_shm_region  = vp_get_shm_region,
> > > +   .reset_vq        = vp_modern_reset_vq,
> > > +   .enable_reset_vq = vp_modern_enable_reset_vq,
> > >   };
> > >
> > >   static const struct virtio_config_ops virtio_pci_config_ops = {
> > > @@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
> > >     .set_vq_affinity = vp_set_vq_affinity,
> > >     .get_vq_affinity = vp_get_vq_affinity,
> > >     .get_shm_region  = vp_get_shm_region,
> > > +   .reset_vq        = vp_modern_reset_vq,
> > > +   .enable_reset_vq = vp_modern_enable_reset_vq,
> > >   };
> > >
> > >   /* the PCI probing function */
> > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > > index 6250e19fc5bf..91937e21edca 100644
> > > --- a/drivers/virtio/virtio_ring.c
> > > +++ b/drivers/virtio/virtio_ring.c
> > > @@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> > >     vq->vq.vdev = vdev;
> > >     vq->vq.name = name;
> > >     vq->vq.index = index;
> > > +   vq->vq.reset = false;
> > >     vq->notify = notify;
> > >     vq->weak_barriers = weak_barriers;
> > >
> > > @@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> > >     vq->vq.vdev = vdev;
> > >     vq->vq.name = name;
> > >     vq->vq.index = index;
> > > +   vq->vq.reset = false;
> > >     vq->notify = notify;
> > >     vq->weak_barriers = weak_barriers;
> > >
> > > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > > index c86ff02e0ca0..33ab003c5100 100644
> > > --- a/include/linux/virtio.h
> > > +++ b/include/linux/virtio.h
> > > @@ -33,6 +33,7 @@ struct virtqueue {
> > >     unsigned int num_free;
> > >     unsigned int num_max;
> > >     void *priv;
> > > +   bool reset;
> > >   };
> > >
> > >   int virtqueue_add_outbuf(struct virtqueue *vq,
> >
>

Re: [PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Jason Wang]

On Wed, Apr 13, 2022 at 11:23 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 12 Apr 2022 15:07:58 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > This patch implements virtio pci support for QUEUE RESET.
> > >
> > > Performing reset on a queue is divided into these steps:
> > >
> > >   1. notify the device to reset the queue
> > >   2. recycle the buffer submitted
> > >   3. reset the vring (may re-alloc)
> > >   4. mmap vring to device, and enable the queue
> > >
> > > This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
> > > pci scenario.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/virtio/virtio_pci_common.c |  8 +--
> > >   drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
> > >   drivers/virtio/virtio_ring.c       |  2 +
> > >   include/linux/virtio.h             |  1 +
> > >   4 files changed, 92 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> > > index fdbde1db5ec5..863d3a8a0956 100644
> > > --- a/drivers/virtio/virtio_pci_common.c
> > > +++ b/drivers/virtio/virtio_pci_common.c
> > > @@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
> > >     struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
> > >     unsigned long flags;
> > >
> > > -   spin_lock_irqsave(&vp_dev->lock, flags);
> > > -   list_del(&info->node);
> > > -   spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +   if (!vq->reset) {
> >
> >
> > On which condition that we may hit this path?
> >
> >
> > > +           spin_lock_irqsave(&vp_dev->lock, flags);
> > > +           list_del(&info->node);
> > > +           spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +   }
> > >
> > >     vp_dev->del_vq(info);
> > >     kfree(info);
> > > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > > index 49a4493732cf..cb5d38f1c9c8 100644
> > > --- a/drivers/virtio/virtio_pci_modern.c
> > > +++ b/drivers/virtio/virtio_pci_modern.c
> > > @@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
> > >     if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
> > >                     pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
> > >             __virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
> > > +
> > > +   if (features & BIT_ULL(VIRTIO_F_RING_RESET))
> > > +           __virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
> > >   }
> > >
> > >   /* virtio config->finalize_features() implementation */
> > > @@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> > >     return 0;
> > >   }
> > >
> > > +static int vp_modern_reset_vq(struct virtqueue *vq)
> > > +{
> > > +   struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > +   struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > +   struct virtio_pci_vq_info *info;
> > > +   unsigned long flags;
> > > +
> > > +   if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
> > > +           return -ENOENT;
> > > +
> > > +   vp_modern_set_queue_reset(mdev, vq->index);
> > > +
> > > +   info = vp_dev->vqs[vq->index];
> > > +
> > > +   /* delete vq from irq handler */
> > > +   spin_lock_irqsave(&vp_dev->lock, flags);
> > > +   list_del(&info->node);
> > > +   spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +
> > > +   INIT_LIST_HEAD(&info->node);
> > > +
> > > +   /* For the case where vq has an exclusive irq, to prevent the irq from
> > > +    * being received again and the pending irq, call disable_irq().
> > > +    *
> > > +    * In the scenario based on shared interrupts, vq will be searched from
> > > +    * the queue virtqueues. Since the previous list_del() has been deleted
> > > +    * from the queue, it is impossible for vq to be called in this case.
> > > +    * There is no need to close the corresponding interrupt.
> > > +    */
> > > +   if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > > +           disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> >
> >
> > See the previous discussion and the revert of the first try to harden
> > the interrupt. We probably can't use disable_irq() since it conflicts
> > with the affinity managed IRQ that is used by some drivers.
> >
> > We need to use synchonize_irq() and per virtqueue flag instead. As
> > mentioned in previous patches, this could be done on top of my rework on
> > the IRQ hardening .
>
> OK, the next version will contain hardened features by per virtqueue flag.

Actually, I'm working on a new version of hardening. I plan to switch
to the virtqueue flag so we will be fine here if we do the resize work
on top.

I will cc you in the new version.

Thanks

>
> Thanks.
>
> >
> >
> > > +
> > > +   vq->reset = true;
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > > +{
> > > +   struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > +   struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > +   struct virtio_pci_vq_info *info;
> > > +   unsigned long flags, index;
> > > +   int err;
> > > +
> > > +   if (!vq->reset)
> > > +           return -EBUSY;
> > > +
> > > +   index = vq->index;
> > > +   info = vp_dev->vqs[index];
> > > +
> > > +   /* check queue reset status */
> > > +   if (vp_modern_get_queue_reset(mdev, index) != 1)
> > > +           return -EBUSY;
> > > +
> > > +   err = vp_active_vq(vq, info->msix_vector);
> > > +   if (err)
> > > +           return err;
> > > +
> > > +   if (vq->callback) {
> > > +           spin_lock_irqsave(&vp_dev->lock, flags);
> > > +           list_add(&info->node, &vp_dev->virtqueues);
> > > +           spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +   } else {
> > > +           INIT_LIST_HEAD(&info->node);
> > > +   }
> > > +
> > > +   vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > > +
> > > +   if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > > +           enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> >
> >
> > We had the same issue as disable_irq().
> >
> > Thanks
> >
> >
> > > +
> > > +   vq->reset = false;
> > > +
> > > +   return 0;
> > > +}
> > > +
> > >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> > >   {
> > >     return vp_modern_config_vector(&vp_dev->mdev, vector);
> > > @@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
> > >     .set_vq_affinity = vp_set_vq_affinity,
> > >     .get_vq_affinity = vp_get_vq_affinity,
> > >     .get_shm_region  = vp_get_shm_region,
> > > +   .reset_vq        = vp_modern_reset_vq,
> > > +   .enable_reset_vq = vp_modern_enable_reset_vq,
> > >   };
> > >
> > >   static const struct virtio_config_ops virtio_pci_config_ops = {
> > > @@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
> > >     .set_vq_affinity = vp_set_vq_affinity,
> > >     .get_vq_affinity = vp_get_vq_affinity,
> > >     .get_shm_region  = vp_get_shm_region,
> > > +   .reset_vq        = vp_modern_reset_vq,
> > > +   .enable_reset_vq = vp_modern_enable_reset_vq,
> > >   };
> > >
> > >   /* the PCI probing function */
> > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > > index 6250e19fc5bf..91937e21edca 100644
> > > --- a/drivers/virtio/virtio_ring.c
> > > +++ b/drivers/virtio/virtio_ring.c
> > > @@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> > >     vq->vq.vdev = vdev;
> > >     vq->vq.name = name;
> > >     vq->vq.index = index;
> > > +   vq->vq.reset = false;
> > >     vq->notify = notify;
> > >     vq->weak_barriers = weak_barriers;
> > >
> > > @@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> > >     vq->vq.vdev = vdev;
> > >     vq->vq.name = name;
> > >     vq->vq.index = index;
> > > +   vq->vq.reset = false;
> > >     vq->notify = notify;
> > >     vq->weak_barriers = weak_barriers;
> > >
> > > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > > index c86ff02e0ca0..33ab003c5100 100644
> > > --- a/include/linux/virtio.h
> > > +++ b/include/linux/virtio.h
> > > @@ -33,6 +33,7 @@ struct virtqueue {
> > >     unsigned int num_free;
> > >     unsigned int num_max;
> > >     void *priv;
> > > +   bool reset;
> > >   };
> > >
> > >   int virtqueue_add_outbuf(struct virtqueue *vq,
> >
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 23/32] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET

[Jason Wang]

On Wed, Apr 13, 2022 at 11:23 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 12 Apr 2022 15:07:58 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > This patch implements virtio pci support for QUEUE RESET.
> > >
> > > Performing reset on a queue is divided into these steps:
> > >
> > >   1. notify the device to reset the queue
> > >   2. recycle the buffer submitted
> > >   3. reset the vring (may re-alloc)
> > >   4. mmap vring to device, and enable the queue
> > >
> > > This patch implements virtio_reset_vq(), virtio_enable_resetq() in the
> > > pci scenario.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/virtio/virtio_pci_common.c |  8 +--
> > >   drivers/virtio/virtio_pci_modern.c | 84 ++++++++++++++++++++++++++++++
> > >   drivers/virtio/virtio_ring.c       |  2 +
> > >   include/linux/virtio.h             |  1 +
> > >   4 files changed, 92 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> > > index fdbde1db5ec5..863d3a8a0956 100644
> > > --- a/drivers/virtio/virtio_pci_common.c
> > > +++ b/drivers/virtio/virtio_pci_common.c
> > > @@ -248,9 +248,11 @@ static void vp_del_vq(struct virtqueue *vq)
> > >     struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index];
> > >     unsigned long flags;
> > >
> > > -   spin_lock_irqsave(&vp_dev->lock, flags);
> > > -   list_del(&info->node);
> > > -   spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +   if (!vq->reset) {
> >
> >
> > On which condition that we may hit this path?
> >
> >
> > > +           spin_lock_irqsave(&vp_dev->lock, flags);
> > > +           list_del(&info->node);
> > > +           spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +   }
> > >
> > >     vp_dev->del_vq(info);
> > >     kfree(info);
> > > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > > index 49a4493732cf..cb5d38f1c9c8 100644
> > > --- a/drivers/virtio/virtio_pci_modern.c
> > > +++ b/drivers/virtio/virtio_pci_modern.c
> > > @@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
> > >     if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
> > >                     pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV))
> > >             __virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
> > > +
> > > +   if (features & BIT_ULL(VIRTIO_F_RING_RESET))
> > > +           __virtio_set_bit(vdev, VIRTIO_F_RING_RESET);
> > >   }
> > >
> > >   /* virtio config->finalize_features() implementation */
> > > @@ -199,6 +202,83 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> > >     return 0;
> > >   }
> > >
> > > +static int vp_modern_reset_vq(struct virtqueue *vq)
> > > +{
> > > +   struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > +   struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > +   struct virtio_pci_vq_info *info;
> > > +   unsigned long flags;
> > > +
> > > +   if (!virtio_has_feature(vq->vdev, VIRTIO_F_RING_RESET))
> > > +           return -ENOENT;
> > > +
> > > +   vp_modern_set_queue_reset(mdev, vq->index);
> > > +
> > > +   info = vp_dev->vqs[vq->index];
> > > +
> > > +   /* delete vq from irq handler */
> > > +   spin_lock_irqsave(&vp_dev->lock, flags);
> > > +   list_del(&info->node);
> > > +   spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +
> > > +   INIT_LIST_HEAD(&info->node);
> > > +
> > > +   /* For the case where vq has an exclusive irq, to prevent the irq from
> > > +    * being received again and the pending irq, call disable_irq().
> > > +    *
> > > +    * In the scenario based on shared interrupts, vq will be searched from
> > > +    * the queue virtqueues. Since the previous list_del() has been deleted
> > > +    * from the queue, it is impossible for vq to be called in this case.
> > > +    * There is no need to close the corresponding interrupt.
> > > +    */
> > > +   if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > > +           disable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> >
> >
> > See the previous discussion and the revert of the first try to harden
> > the interrupt. We probably can't use disable_irq() since it conflicts
> > with the affinity managed IRQ that is used by some drivers.
> >
> > We need to use synchonize_irq() and per virtqueue flag instead. As
> > mentioned in previous patches, this could be done on top of my rework on
> > the IRQ hardening .
>
> OK, the next version will contain hardened features by per virtqueue flag.

Actually, I'm working on a new version of hardening. I plan to switch
to the virtqueue flag so we will be fine here if we do the resize work
on top.

I will cc you in the new version.

Thanks

>
> Thanks.
>
> >
> >
> > > +
> > > +   vq->reset = true;
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > > +{
> > > +   struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > +   struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > +   struct virtio_pci_vq_info *info;
> > > +   unsigned long flags, index;
> > > +   int err;
> > > +
> > > +   if (!vq->reset)
> > > +           return -EBUSY;
> > > +
> > > +   index = vq->index;
> > > +   info = vp_dev->vqs[index];
> > > +
> > > +   /* check queue reset status */
> > > +   if (vp_modern_get_queue_reset(mdev, index) != 1)
> > > +           return -EBUSY;
> > > +
> > > +   err = vp_active_vq(vq, info->msix_vector);
> > > +   if (err)
> > > +           return err;
> > > +
> > > +   if (vq->callback) {
> > > +           spin_lock_irqsave(&vp_dev->lock, flags);
> > > +           list_add(&info->node, &vp_dev->virtqueues);
> > > +           spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +   } else {
> > > +           INIT_LIST_HEAD(&info->node);
> > > +   }
> > > +
> > > +   vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > > +
> > > +   if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > > +           enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> >
> >
> > We had the same issue as disable_irq().
> >
> > Thanks
> >
> >
> > > +
> > > +   vq->reset = false;
> > > +
> > > +   return 0;
> > > +}
> > > +
> > >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> > >   {
> > >     return vp_modern_config_vector(&vp_dev->mdev, vector);
> > > @@ -407,6 +487,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
> > >     .set_vq_affinity = vp_set_vq_affinity,
> > >     .get_vq_affinity = vp_get_vq_affinity,
> > >     .get_shm_region  = vp_get_shm_region,
> > > +   .reset_vq        = vp_modern_reset_vq,
> > > +   .enable_reset_vq = vp_modern_enable_reset_vq,
> > >   };
> > >
> > >   static const struct virtio_config_ops virtio_pci_config_ops = {
> > > @@ -425,6 +507,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = {
> > >     .set_vq_affinity = vp_set_vq_affinity,
> > >     .get_vq_affinity = vp_get_vq_affinity,
> > >     .get_shm_region  = vp_get_shm_region,
> > > +   .reset_vq        = vp_modern_reset_vq,
> > > +   .enable_reset_vq = vp_modern_enable_reset_vq,
> > >   };
> > >
> > >   /* the PCI probing function */
> > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > > index 6250e19fc5bf..91937e21edca 100644
> > > --- a/drivers/virtio/virtio_ring.c
> > > +++ b/drivers/virtio/virtio_ring.c
> > > @@ -2028,6 +2028,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> > >     vq->vq.vdev = vdev;
> > >     vq->vq.name = name;
> > >     vq->vq.index = index;
> > > +   vq->vq.reset = false;
> > >     vq->notify = notify;
> > >     vq->weak_barriers = weak_barriers;
> > >
> > > @@ -2508,6 +2509,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index,
> > >     vq->vq.vdev = vdev;
> > >     vq->vq.name = name;
> > >     vq->vq.index = index;
> > > +   vq->vq.reset = false;
> > >     vq->notify = notify;
> > >     vq->weak_barriers = weak_barriers;
> > >
> > > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > > index c86ff02e0ca0..33ab003c5100 100644
> > > --- a/include/linux/virtio.h
> > > +++ b/include/linux/virtio.h
> > > @@ -33,6 +33,7 @@ struct virtqueue {
> > >     unsigned int num_free;
> > >     unsigned int num_max;
> > >     void *priv;
> > > +   bool reset;
> > >   };
> > >
> > >   int virtqueue_add_outbuf(struct virtqueue *vq,
> >
>


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 12/32] virtio_ring: packed: extract the logic of alloc queue

[Jason Wang]

On Wed, Apr 13, 2022 at 11:26 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 12 Apr 2022 14:28:24 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > Separate the logic of packed to create vring queue.
> > >
> > > For the convenience of passing parameters, add a structure
> > > vring_packed.
> > >
> > > This feature is required for subsequent virtuqueue reset vring.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/virtio/virtio_ring.c | 70 ++++++++++++++++++++++++++++--------
> > >   1 file changed, 56 insertions(+), 14 deletions(-)
> > >
> > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > > index 33864134a744..ea451ae2aaef 100644
> > > --- a/drivers/virtio/virtio_ring.c
> > > +++ b/drivers/virtio/virtio_ring.c
> > > @@ -1817,19 +1817,17 @@ static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
> > >     return desc_extra;
> > >   }
> > >
> > > -static struct virtqueue *vring_create_virtqueue_packed(
> > > -   unsigned int index,
> > > -   unsigned int num,
> > > -   unsigned int vring_align,
> > > -   struct virtio_device *vdev,
> > > -   bool weak_barriers,
> > > -   bool may_reduce_num,
> > > -   bool context,
> > > -   bool (*notify)(struct virtqueue *),
> > > -   void (*callback)(struct virtqueue *),
> > > -   const char *name)
> > > +static int vring_alloc_queue_packed(struct virtio_device *vdev,
> > > +                               u32 num,
> > > +                               struct vring_packed_desc **_ring,
> > > +                               struct vring_packed_desc_event **_driver,
> > > +                               struct vring_packed_desc_event **_device,
> > > +                               dma_addr_t *_ring_dma_addr,
> > > +                               dma_addr_t *_driver_event_dma_addr,
> > > +                               dma_addr_t *_device_event_dma_addr,
> > > +                               size_t *_ring_size_in_bytes,
> > > +                               size_t *_event_size_in_bytes)
> > >   {
> > > -   struct vring_virtqueue *vq;
> > >     struct vring_packed_desc *ring;
> > >     struct vring_packed_desc_event *driver, *device;
> > >     dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> > > @@ -1857,6 +1855,52 @@ static struct virtqueue *vring_create_virtqueue_packed(
> > >     if (!device)
> > >             goto err_device;
> > >
> > > +   *_ring                   = ring;
> > > +   *_driver                 = driver;
> > > +   *_device                 = device;
> > > +   *_ring_dma_addr          = ring_dma_addr;
> > > +   *_driver_event_dma_addr  = driver_event_dma_addr;
> > > +   *_device_event_dma_addr  = device_event_dma_addr;
> > > +   *_ring_size_in_bytes     = ring_size_in_bytes;
> > > +   *_event_size_in_bytes    = event_size_in_bytes;
> >
> >
> > I wonder if we can simply factor out split and packed from struct
> > vring_virtqueue:
> >
> > struct vring_virtqueue {
> >      union {
> >          struct {} split;
> >          struct {} packed;
> >      };
> > };
> >
> > to
> >
> > struct vring_virtqueue_split {};
> > struct vring_virtqueue_packed {};
> >
> > Then we can do things like:
> >
> > vring_create_virtqueue_packed(struct virtio_device *vdev, u32 num,
> > struct vring_virtqueue_packed *packed);
> >
> > and
> >
> > vring_vritqueue_attach_packed(struct vring_virtqueue *vq, struct
> > vring_virtqueue_packed packed);
>
> This idea is very similar to my previous idea, just without introducing a new
> structure.

Yes, it's better to not introduce new structures if it's possible.

>
> I'd be more than happy to revise this.

Good to know this.

Thanks

>
> Thanks.
>
>
> >
> > Thanks
> >
> >
> > > +
> > > +   return 0;
> > > +
> > > +err_device:
> > > +   vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> > > +
> > > +err_driver:
> > > +   vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> > > +
> > > +err_ring:
> > > +   return -ENOMEM;
> > > +}
> > > +
> > > +static struct virtqueue *vring_create_virtqueue_packed(
> > > +   unsigned int index,
> > > +   unsigned int num,
> > > +   unsigned int vring_align,
> > > +   struct virtio_device *vdev,
> > > +   bool weak_barriers,
> > > +   bool may_reduce_num,
> > > +   bool context,
> > > +   bool (*notify)(struct virtqueue *),
> > > +   void (*callback)(struct virtqueue *),
> > > +   const char *name)
> > > +{
> > > +   dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> > > +   struct vring_packed_desc_event *driver, *device;
> > > +   size_t ring_size_in_bytes, event_size_in_bytes;
> > > +   struct vring_packed_desc *ring;
> > > +   struct vring_virtqueue *vq;
> > > +
> > > +   if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
> > > +                                &ring_dma_addr, &driver_event_dma_addr,
> > > +                                &device_event_dma_addr,
> > > +                                &ring_size_in_bytes,
> > > +                                &event_size_in_bytes))
> > > +           goto err_ring;
> > > +
> > >     vq = kmalloc(sizeof(*vq), GFP_KERNEL);
> > >     if (!vq)
> > >             goto err_vq;
> > > @@ -1939,9 +1983,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> > >     kfree(vq);
> > >   err_vq:
> > >     vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
> > > -err_device:
> > >     vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> > > -err_driver:
> > >     vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> > >   err_ring:
> > >     return NULL;
> >
>

Re: [PATCH v9 12/32] virtio_ring: packed: extract the logic of alloc queue

[Jason Wang]

On Wed, Apr 13, 2022 at 11:26 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 12 Apr 2022 14:28:24 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > Separate the logic of packed to create vring queue.
> > >
> > > For the convenience of passing parameters, add a structure
> > > vring_packed.
> > >
> > > This feature is required for subsequent virtuqueue reset vring.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/virtio/virtio_ring.c | 70 ++++++++++++++++++++++++++++--------
> > >   1 file changed, 56 insertions(+), 14 deletions(-)
> > >
> > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > > index 33864134a744..ea451ae2aaef 100644
> > > --- a/drivers/virtio/virtio_ring.c
> > > +++ b/drivers/virtio/virtio_ring.c
> > > @@ -1817,19 +1817,17 @@ static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
> > >     return desc_extra;
> > >   }
> > >
> > > -static struct virtqueue *vring_create_virtqueue_packed(
> > > -   unsigned int index,
> > > -   unsigned int num,
> > > -   unsigned int vring_align,
> > > -   struct virtio_device *vdev,
> > > -   bool weak_barriers,
> > > -   bool may_reduce_num,
> > > -   bool context,
> > > -   bool (*notify)(struct virtqueue *),
> > > -   void (*callback)(struct virtqueue *),
> > > -   const char *name)
> > > +static int vring_alloc_queue_packed(struct virtio_device *vdev,
> > > +                               u32 num,
> > > +                               struct vring_packed_desc **_ring,
> > > +                               struct vring_packed_desc_event **_driver,
> > > +                               struct vring_packed_desc_event **_device,
> > > +                               dma_addr_t *_ring_dma_addr,
> > > +                               dma_addr_t *_driver_event_dma_addr,
> > > +                               dma_addr_t *_device_event_dma_addr,
> > > +                               size_t *_ring_size_in_bytes,
> > > +                               size_t *_event_size_in_bytes)
> > >   {
> > > -   struct vring_virtqueue *vq;
> > >     struct vring_packed_desc *ring;
> > >     struct vring_packed_desc_event *driver, *device;
> > >     dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> > > @@ -1857,6 +1855,52 @@ static struct virtqueue *vring_create_virtqueue_packed(
> > >     if (!device)
> > >             goto err_device;
> > >
> > > +   *_ring                   = ring;
> > > +   *_driver                 = driver;
> > > +   *_device                 = device;
> > > +   *_ring_dma_addr          = ring_dma_addr;
> > > +   *_driver_event_dma_addr  = driver_event_dma_addr;
> > > +   *_device_event_dma_addr  = device_event_dma_addr;
> > > +   *_ring_size_in_bytes     = ring_size_in_bytes;
> > > +   *_event_size_in_bytes    = event_size_in_bytes;
> >
> >
> > I wonder if we can simply factor out split and packed from struct
> > vring_virtqueue:
> >
> > struct vring_virtqueue {
> >      union {
> >          struct {} split;
> >          struct {} packed;
> >      };
> > };
> >
> > to
> >
> > struct vring_virtqueue_split {};
> > struct vring_virtqueue_packed {};
> >
> > Then we can do things like:
> >
> > vring_create_virtqueue_packed(struct virtio_device *vdev, u32 num,
> > struct vring_virtqueue_packed *packed);
> >
> > and
> >
> > vring_vritqueue_attach_packed(struct vring_virtqueue *vq, struct
> > vring_virtqueue_packed packed);
>
> This idea is very similar to my previous idea, just without introducing a new
> structure.

Yes, it's better to not introduce new structures if it's possible.

>
> I'd be more than happy to revise this.

Good to know this.

Thanks

>
> Thanks.
>
>
> >
> > Thanks
> >
> >
> > > +
> > > +   return 0;
> > > +
> > > +err_device:
> > > +   vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> > > +
> > > +err_driver:
> > > +   vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> > > +
> > > +err_ring:
> > > +   return -ENOMEM;
> > > +}
> > > +
> > > +static struct virtqueue *vring_create_virtqueue_packed(
> > > +   unsigned int index,
> > > +   unsigned int num,
> > > +   unsigned int vring_align,
> > > +   struct virtio_device *vdev,
> > > +   bool weak_barriers,
> > > +   bool may_reduce_num,
> > > +   bool context,
> > > +   bool (*notify)(struct virtqueue *),
> > > +   void (*callback)(struct virtqueue *),
> > > +   const char *name)
> > > +{
> > > +   dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> > > +   struct vring_packed_desc_event *driver, *device;
> > > +   size_t ring_size_in_bytes, event_size_in_bytes;
> > > +   struct vring_packed_desc *ring;
> > > +   struct vring_virtqueue *vq;
> > > +
> > > +   if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
> > > +                                &ring_dma_addr, &driver_event_dma_addr,
> > > +                                &device_event_dma_addr,
> > > +                                &ring_size_in_bytes,
> > > +                                &event_size_in_bytes))
> > > +           goto err_ring;
> > > +
> > >     vq = kmalloc(sizeof(*vq), GFP_KERNEL);
> > >     if (!vq)
> > >             goto err_vq;
> > > @@ -1939,9 +1983,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> > >     kfree(vq);
> > >   err_vq:
> > >     vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
> > > -err_device:
> > >     vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> > > -err_driver:
> > >     vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> > >   err_ring:
> > >     return NULL;
> >
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 12/32] virtio_ring: packed: extract the logic of alloc queue

[Jason Wang]

On Wed, Apr 13, 2022 at 11:26 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 12 Apr 2022 14:28:24 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > Separate the logic of packed to create vring queue.
> > >
> > > For the convenience of passing parameters, add a structure
> > > vring_packed.
> > >
> > > This feature is required for subsequent virtuqueue reset vring.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/virtio/virtio_ring.c | 70 ++++++++++++++++++++++++++++--------
> > >   1 file changed, 56 insertions(+), 14 deletions(-)
> > >
> > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> > > index 33864134a744..ea451ae2aaef 100644
> > > --- a/drivers/virtio/virtio_ring.c
> > > +++ b/drivers/virtio/virtio_ring.c
> > > @@ -1817,19 +1817,17 @@ static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num)
> > >     return desc_extra;
> > >   }
> > >
> > > -static struct virtqueue *vring_create_virtqueue_packed(
> > > -   unsigned int index,
> > > -   unsigned int num,
> > > -   unsigned int vring_align,
> > > -   struct virtio_device *vdev,
> > > -   bool weak_barriers,
> > > -   bool may_reduce_num,
> > > -   bool context,
> > > -   bool (*notify)(struct virtqueue *),
> > > -   void (*callback)(struct virtqueue *),
> > > -   const char *name)
> > > +static int vring_alloc_queue_packed(struct virtio_device *vdev,
> > > +                               u32 num,
> > > +                               struct vring_packed_desc **_ring,
> > > +                               struct vring_packed_desc_event **_driver,
> > > +                               struct vring_packed_desc_event **_device,
> > > +                               dma_addr_t *_ring_dma_addr,
> > > +                               dma_addr_t *_driver_event_dma_addr,
> > > +                               dma_addr_t *_device_event_dma_addr,
> > > +                               size_t *_ring_size_in_bytes,
> > > +                               size_t *_event_size_in_bytes)
> > >   {
> > > -   struct vring_virtqueue *vq;
> > >     struct vring_packed_desc *ring;
> > >     struct vring_packed_desc_event *driver, *device;
> > >     dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> > > @@ -1857,6 +1855,52 @@ static struct virtqueue *vring_create_virtqueue_packed(
> > >     if (!device)
> > >             goto err_device;
> > >
> > > +   *_ring                   = ring;
> > > +   *_driver                 = driver;
> > > +   *_device                 = device;
> > > +   *_ring_dma_addr          = ring_dma_addr;
> > > +   *_driver_event_dma_addr  = driver_event_dma_addr;
> > > +   *_device_event_dma_addr  = device_event_dma_addr;
> > > +   *_ring_size_in_bytes     = ring_size_in_bytes;
> > > +   *_event_size_in_bytes    = event_size_in_bytes;
> >
> >
> > I wonder if we can simply factor out split and packed from struct
> > vring_virtqueue:
> >
> > struct vring_virtqueue {
> >      union {
> >          struct {} split;
> >          struct {} packed;
> >      };
> > };
> >
> > to
> >
> > struct vring_virtqueue_split {};
> > struct vring_virtqueue_packed {};
> >
> > Then we can do things like:
> >
> > vring_create_virtqueue_packed(struct virtio_device *vdev, u32 num,
> > struct vring_virtqueue_packed *packed);
> >
> > and
> >
> > vring_vritqueue_attach_packed(struct vring_virtqueue *vq, struct
> > vring_virtqueue_packed packed);
>
> This idea is very similar to my previous idea, just without introducing a new
> structure.

Yes, it's better to not introduce new structures if it's possible.

>
> I'd be more than happy to revise this.

Good to know this.

Thanks

>
> Thanks.
>
>
> >
> > Thanks
> >
> >
> > > +
> > > +   return 0;
> > > +
> > > +err_device:
> > > +   vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> > > +
> > > +err_driver:
> > > +   vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> > > +
> > > +err_ring:
> > > +   return -ENOMEM;
> > > +}
> > > +
> > > +static struct virtqueue *vring_create_virtqueue_packed(
> > > +   unsigned int index,
> > > +   unsigned int num,
> > > +   unsigned int vring_align,
> > > +   struct virtio_device *vdev,
> > > +   bool weak_barriers,
> > > +   bool may_reduce_num,
> > > +   bool context,
> > > +   bool (*notify)(struct virtqueue *),
> > > +   void (*callback)(struct virtqueue *),
> > > +   const char *name)
> > > +{
> > > +   dma_addr_t ring_dma_addr, driver_event_dma_addr, device_event_dma_addr;
> > > +   struct vring_packed_desc_event *driver, *device;
> > > +   size_t ring_size_in_bytes, event_size_in_bytes;
> > > +   struct vring_packed_desc *ring;
> > > +   struct vring_virtqueue *vq;
> > > +
> > > +   if (vring_alloc_queue_packed(vdev, num, &ring, &driver, &device,
> > > +                                &ring_dma_addr, &driver_event_dma_addr,
> > > +                                &device_event_dma_addr,
> > > +                                &ring_size_in_bytes,
> > > +                                &event_size_in_bytes))
> > > +           goto err_ring;
> > > +
> > >     vq = kmalloc(sizeof(*vq), GFP_KERNEL);
> > >     if (!vq)
> > >             goto err_vq;
> > > @@ -1939,9 +1983,7 @@ static struct virtqueue *vring_create_virtqueue_packed(
> > >     kfree(vq);
> > >   err_vq:
> > >     vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr);
> > > -err_device:
> > >     vring_free_queue(vdev, event_size_in_bytes, driver, driver_event_dma_addr);
> > > -err_driver:
> > >     vring_free_queue(vdev, ring_size_in_bytes, ring, ring_dma_addr);
> > >   err_ring:
> > >     return NULL;
> >
>


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > This patch implements the resize function of the rx, tx queues.
> > > Based on this function, it is possible to modify the ring num of the
> > > queue.
> > >
> > > There may be an exception during the resize process, the resize may
> > > fail, or the vq can no longer be used. Either way, we must execute
> > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > must be called after calling napi_disable.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > >   1 file changed, 81 insertions(+)
> > >
> > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > index b8bf00525177..ba6859f305f7 100644
> > > --- a/drivers/net/virtio_net.c
> > > +++ b/drivers/net/virtio_net.c
> > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > >     char padding[4];
> > >   };
> > >
> > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > +
> > >   static bool is_xdp_frame(void *ptr)
> > >   {
> > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > >   {
> > >     napi_enable(napi);
> > >
> > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > +    * be protected by napi. However, the protection of napi is only enabled
> > > +    * during the operation, and the protection of napi will end after the
> > > +    * operation is completed. If re-enable fails during the process, vq
> > > +    * will remain unavailable with reset state.
> > > +    */
> > > +   if (vq->reset)
> > > +           return;
> >
> >
> > I don't get when could we hit this condition.
>
>
> In patch 23, the code to implement re-enable vq is as follows:
>
> +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> +{
> +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> +       struct virtio_pci_vq_info *info;
> +       unsigned long flags, index;
> +       int err;
> +
> +       if (!vq->reset)
> +               return -EBUSY;
> +
> +       index = vq->index;
> +       info = vp_dev->vqs[index];
> +
> +       /* check queue reset status */
> +       if (vp_modern_get_queue_reset(mdev, index) != 1)
> +               return -EBUSY;
> +
> +       err = vp_active_vq(vq, info->msix_vector);
> +       if (err)
> +               return err;
> +
> +       if (vq->callback) {
> +               spin_lock_irqsave(&vp_dev->lock, flags);
> +               list_add(&info->node, &vp_dev->virtqueues);
> +               spin_unlock_irqrestore(&vp_dev->lock, flags);
> +       } else {
> +               INIT_LIST_HEAD(&info->node);
> +       }
> +
> +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> +
> +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> +
> +       vq->reset = false;
> +
> +       return 0;
> +}
>
>
> There are three situations where an error will be returned. These are the
> situations I want to handle.

Right, but it looks harmless if we just schedule the NAPI without the check.

>
> But I'm rethinking the question, and I feel like you're right, although the
> hardware setup may fail. We can no longer sync with the hardware. But using it
> as a normal vq doesn't have any problems.

Note that we should make sure the buggy(malicous) device won't crash
the codes by changing the queue_reset value at its will.

>
> >
> >
> > > +
> > >     /* If all buffers were filled by other side before we napi_enabled, we
> > >      * won't get another interrupt, so process any outstanding packets now.
> > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > >             struct receive_queue *rq = &vi->rq[i];
> > >
> > >             napi_disable(&rq->napi);
> > > +
> > > +           /* Check if vq is in reset state. See more in
> > > +            * virtnet_napi_enable()
> > > +            */
> > > +           if (rq->vq->reset) {
> > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +                   continue;
> > > +           }
> >
> >
> > Can we do something similar in virtnet_close() by canceling the work?
>
> I think there is no need to cancel the work here, because napi_disable will wait
> for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
> vq, this logic can be removed.

Actually I meant the part of virtnet_rx_resize().

If we don't synchronize with the refill work, it might enable NAPI unexpectedly?

Thanks

>
>
> >
> >
> > > +
> > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > >
> > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > >             return;
> > >
> > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > +   if (sq->vq->reset)
> > > +           return;
> >
> >
> > We've disabled TX napi, any chance we can still hit this?
>
> Same as above.
>
> >
> >
> > > +
> > >     if (__netif_tx_trylock(txq)) {
> > >             do {
> > >                     virtqueue_disable_cb(sq->vq);
> > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > >     return NETDEV_TX_OK;
> > >   }
> > >
> > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > +                        struct receive_queue *rq, u32 ring_num)
> > > +{
> > > +   int err;
> > > +
> > > +   napi_disable(&rq->napi);
> > > +
> > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > +   if (err)
> > > +           goto err;
> > > +
> > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > +           schedule_delayed_work(&vi->refill, 0);
> > > +
> > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +   return 0;
> > > +
> > > +err:
> > > +   netdev_err(vi->dev,
> > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > +              rq - vi->rq, err);
> > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +   return err;
> > > +}
> > > +
> > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > +                        struct send_queue *sq, u32 ring_num)
> > > +{
> > > +   struct netdev_queue *txq;
> > > +   int err, qindex;
> > > +
> > > +   qindex = sq - vi->sq;
> > > +
> > > +   virtnet_napi_tx_disable(&sq->napi);
> > > +
> > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > +   __netif_tx_lock_bh(txq);
> > > +   netif_stop_subqueue(vi->dev, qindex);
> > > +   __netif_tx_unlock_bh(txq);
> > > +
> > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > +   if (err)
> > > +           goto err;
> > > +
> > > +   netif_start_subqueue(vi->dev, qindex);
> > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > +   return 0;
> > > +
> > > +err:
> >
> >
> > I guess we can still start the queue in this case? (Since we don't
> > change the queue if resize fails).
>
> Yes, you are right.
>
> Thanks.
>
> >
> >
> > > +   netdev_err(vi->dev,
> > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > +              sq - vi->sq, err);
> > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > +   return err;
> > > +}
> > > +
> > >   /*
> > >    * Send command via the control virtqueue and check status.  Commands
> > >    * supported by the hypervisor, as indicated by feature bits, should
> >
>

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > This patch implements the resize function of the rx, tx queues.
> > > Based on this function, it is possible to modify the ring num of the
> > > queue.
> > >
> > > There may be an exception during the resize process, the resize may
> > > fail, or the vq can no longer be used. Either way, we must execute
> > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > must be called after calling napi_disable.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > >   1 file changed, 81 insertions(+)
> > >
> > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > index b8bf00525177..ba6859f305f7 100644
> > > --- a/drivers/net/virtio_net.c
> > > +++ b/drivers/net/virtio_net.c
> > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > >     char padding[4];
> > >   };
> > >
> > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > +
> > >   static bool is_xdp_frame(void *ptr)
> > >   {
> > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > >   {
> > >     napi_enable(napi);
> > >
> > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > +    * be protected by napi. However, the protection of napi is only enabled
> > > +    * during the operation, and the protection of napi will end after the
> > > +    * operation is completed. If re-enable fails during the process, vq
> > > +    * will remain unavailable with reset state.
> > > +    */
> > > +   if (vq->reset)
> > > +           return;
> >
> >
> > I don't get when could we hit this condition.
>
>
> In patch 23, the code to implement re-enable vq is as follows:
>
> +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> +{
> +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> +       struct virtio_pci_vq_info *info;
> +       unsigned long flags, index;
> +       int err;
> +
> +       if (!vq->reset)
> +               return -EBUSY;
> +
> +       index = vq->index;
> +       info = vp_dev->vqs[index];
> +
> +       /* check queue reset status */
> +       if (vp_modern_get_queue_reset(mdev, index) != 1)
> +               return -EBUSY;
> +
> +       err = vp_active_vq(vq, info->msix_vector);
> +       if (err)
> +               return err;
> +
> +       if (vq->callback) {
> +               spin_lock_irqsave(&vp_dev->lock, flags);
> +               list_add(&info->node, &vp_dev->virtqueues);
> +               spin_unlock_irqrestore(&vp_dev->lock, flags);
> +       } else {
> +               INIT_LIST_HEAD(&info->node);
> +       }
> +
> +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> +
> +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> +
> +       vq->reset = false;
> +
> +       return 0;
> +}
>
>
> There are three situations where an error will be returned. These are the
> situations I want to handle.

Right, but it looks harmless if we just schedule the NAPI without the check.

>
> But I'm rethinking the question, and I feel like you're right, although the
> hardware setup may fail. We can no longer sync with the hardware. But using it
> as a normal vq doesn't have any problems.

Note that we should make sure the buggy(malicous) device won't crash
the codes by changing the queue_reset value at its will.

>
> >
> >
> > > +
> > >     /* If all buffers were filled by other side before we napi_enabled, we
> > >      * won't get another interrupt, so process any outstanding packets now.
> > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > >             struct receive_queue *rq = &vi->rq[i];
> > >
> > >             napi_disable(&rq->napi);
> > > +
> > > +           /* Check if vq is in reset state. See more in
> > > +            * virtnet_napi_enable()
> > > +            */
> > > +           if (rq->vq->reset) {
> > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +                   continue;
> > > +           }
> >
> >
> > Can we do something similar in virtnet_close() by canceling the work?
>
> I think there is no need to cancel the work here, because napi_disable will wait
> for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
> vq, this logic can be removed.

Actually I meant the part of virtnet_rx_resize().

If we don't synchronize with the refill work, it might enable NAPI unexpectedly?

Thanks

>
>
> >
> >
> > > +
> > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > >
> > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > >             return;
> > >
> > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > +   if (sq->vq->reset)
> > > +           return;
> >
> >
> > We've disabled TX napi, any chance we can still hit this?
>
> Same as above.
>
> >
> >
> > > +
> > >     if (__netif_tx_trylock(txq)) {
> > >             do {
> > >                     virtqueue_disable_cb(sq->vq);
> > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > >     return NETDEV_TX_OK;
> > >   }
> > >
> > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > +                        struct receive_queue *rq, u32 ring_num)
> > > +{
> > > +   int err;
> > > +
> > > +   napi_disable(&rq->napi);
> > > +
> > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > +   if (err)
> > > +           goto err;
> > > +
> > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > +           schedule_delayed_work(&vi->refill, 0);
> > > +
> > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +   return 0;
> > > +
> > > +err:
> > > +   netdev_err(vi->dev,
> > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > +              rq - vi->rq, err);
> > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +   return err;
> > > +}
> > > +
> > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > +                        struct send_queue *sq, u32 ring_num)
> > > +{
> > > +   struct netdev_queue *txq;
> > > +   int err, qindex;
> > > +
> > > +   qindex = sq - vi->sq;
> > > +
> > > +   virtnet_napi_tx_disable(&sq->napi);
> > > +
> > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > +   __netif_tx_lock_bh(txq);
> > > +   netif_stop_subqueue(vi->dev, qindex);
> > > +   __netif_tx_unlock_bh(txq);
> > > +
> > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > +   if (err)
> > > +           goto err;
> > > +
> > > +   netif_start_subqueue(vi->dev, qindex);
> > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > +   return 0;
> > > +
> > > +err:
> >
> >
> > I guess we can still start the queue in this case? (Since we don't
> > change the queue if resize fails).
>
> Yes, you are right.
>
> Thanks.
>
> >
> >
> > > +   netdev_err(vi->dev,
> > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > +              sq - vi->sq, err);
> > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > +   return err;
> > > +}
> > > +
> > >   /*
> > >    * Send command via the control virtqueue and check status.  Commands
> > >    * supported by the hypervisor, as indicated by feature bits, should
> >
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > This patch implements the resize function of the rx, tx queues.
> > > Based on this function, it is possible to modify the ring num of the
> > > queue.
> > >
> > > There may be an exception during the resize process, the resize may
> > > fail, or the vq can no longer be used. Either way, we must execute
> > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > must be called after calling napi_disable.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > >   1 file changed, 81 insertions(+)
> > >
> > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > index b8bf00525177..ba6859f305f7 100644
> > > --- a/drivers/net/virtio_net.c
> > > +++ b/drivers/net/virtio_net.c
> > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > >     char padding[4];
> > >   };
> > >
> > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > +
> > >   static bool is_xdp_frame(void *ptr)
> > >   {
> > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > >   {
> > >     napi_enable(napi);
> > >
> > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > +    * be protected by napi. However, the protection of napi is only enabled
> > > +    * during the operation, and the protection of napi will end after the
> > > +    * operation is completed. If re-enable fails during the process, vq
> > > +    * will remain unavailable with reset state.
> > > +    */
> > > +   if (vq->reset)
> > > +           return;
> >
> >
> > I don't get when could we hit this condition.
>
>
> In patch 23, the code to implement re-enable vq is as follows:
>
> +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> +{
> +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> +       struct virtio_pci_vq_info *info;
> +       unsigned long flags, index;
> +       int err;
> +
> +       if (!vq->reset)
> +               return -EBUSY;
> +
> +       index = vq->index;
> +       info = vp_dev->vqs[index];
> +
> +       /* check queue reset status */
> +       if (vp_modern_get_queue_reset(mdev, index) != 1)
> +               return -EBUSY;
> +
> +       err = vp_active_vq(vq, info->msix_vector);
> +       if (err)
> +               return err;
> +
> +       if (vq->callback) {
> +               spin_lock_irqsave(&vp_dev->lock, flags);
> +               list_add(&info->node, &vp_dev->virtqueues);
> +               spin_unlock_irqrestore(&vp_dev->lock, flags);
> +       } else {
> +               INIT_LIST_HEAD(&info->node);
> +       }
> +
> +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> +
> +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> +
> +       vq->reset = false;
> +
> +       return 0;
> +}
>
>
> There are three situations where an error will be returned. These are the
> situations I want to handle.

Right, but it looks harmless if we just schedule the NAPI without the check.

>
> But I'm rethinking the question, and I feel like you're right, although the
> hardware setup may fail. We can no longer sync with the hardware. But using it
> as a normal vq doesn't have any problems.

Note that we should make sure the buggy(malicous) device won't crash
the codes by changing the queue_reset value at its will.

>
> >
> >
> > > +
> > >     /* If all buffers were filled by other side before we napi_enabled, we
> > >      * won't get another interrupt, so process any outstanding packets now.
> > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > >             struct receive_queue *rq = &vi->rq[i];
> > >
> > >             napi_disable(&rq->napi);
> > > +
> > > +           /* Check if vq is in reset state. See more in
> > > +            * virtnet_napi_enable()
> > > +            */
> > > +           if (rq->vq->reset) {
> > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +                   continue;
> > > +           }
> >
> >
> > Can we do something similar in virtnet_close() by canceling the work?
>
> I think there is no need to cancel the work here, because napi_disable will wait
> for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
> vq, this logic can be removed.

Actually I meant the part of virtnet_rx_resize().

If we don't synchronize with the refill work, it might enable NAPI unexpectedly?

Thanks

>
>
> >
> >
> > > +
> > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > >
> > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > >             return;
> > >
> > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > +   if (sq->vq->reset)
> > > +           return;
> >
> >
> > We've disabled TX napi, any chance we can still hit this?
>
> Same as above.
>
> >
> >
> > > +
> > >     if (__netif_tx_trylock(txq)) {
> > >             do {
> > >                     virtqueue_disable_cb(sq->vq);
> > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > >     return NETDEV_TX_OK;
> > >   }
> > >
> > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > +                        struct receive_queue *rq, u32 ring_num)
> > > +{
> > > +   int err;
> > > +
> > > +   napi_disable(&rq->napi);
> > > +
> > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > +   if (err)
> > > +           goto err;
> > > +
> > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > +           schedule_delayed_work(&vi->refill, 0);
> > > +
> > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +   return 0;
> > > +
> > > +err:
> > > +   netdev_err(vi->dev,
> > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > +              rq - vi->rq, err);
> > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +   return err;
> > > +}
> > > +
> > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > +                        struct send_queue *sq, u32 ring_num)
> > > +{
> > > +   struct netdev_queue *txq;
> > > +   int err, qindex;
> > > +
> > > +   qindex = sq - vi->sq;
> > > +
> > > +   virtnet_napi_tx_disable(&sq->napi);
> > > +
> > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > +   __netif_tx_lock_bh(txq);
> > > +   netif_stop_subqueue(vi->dev, qindex);
> > > +   __netif_tx_unlock_bh(txq);
> > > +
> > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > +   if (err)
> > > +           goto err;
> > > +
> > > +   netif_start_subqueue(vi->dev, qindex);
> > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > +   return 0;
> > > +
> > > +err:
> >
> >
> > I guess we can still start the queue in this case? (Since we don't
> > change the queue if resize fails).
>
> Yes, you are right.
>
> Thanks.
>
> >
> >
> > > +   netdev_err(vi->dev,
> > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > +              sq - vi->sq, err);
> > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > +   return err;
> > > +}
> > > +
> > >   /*
> > >    * Send command via the control virtqueue and check status.  Commands
> > >    * supported by the hypervisor, as indicated by feature bits, should
> >
>


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 22/32] virtio_pci: queue_reset: extract the logic of active vq for modern pci

[Jason Wang]

On Thu, Apr 14, 2022 at 2:25 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 12 Apr 2022 14:58:19 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > Introduce vp_active_vq() to configure vring to backend after vq attach
> > > vring. And configure vq vector if necessary.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/virtio/virtio_pci_modern.c | 46 ++++++++++++++++++------------
> > >   1 file changed, 28 insertions(+), 18 deletions(-)
> > >
> > > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > > index 86d301f272b8..49a4493732cf 100644
> > > --- a/drivers/virtio/virtio_pci_modern.c
> > > +++ b/drivers/virtio/virtio_pci_modern.c
> > > @@ -176,6 +176,29 @@ static void vp_reset(struct virtio_device *vdev)
> > >     vp_disable_cbs(vdev);
> > >   }
> > >
> > > +static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> > > +{
> > > +   struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > +   struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > +   unsigned long index;
> > > +
> > > +   index = vq->index;
> > > +
> > > +   /* activate the queue */
> > > +   vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > > +   vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > > +                           virtqueue_get_avail_addr(vq),
> > > +                           virtqueue_get_used_addr(vq));
> > > +
> > > +   if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> > > +           msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> > > +           if (msix_vec == VIRTIO_MSI_NO_VECTOR)
> > > +                   return -EBUSY;
> > > +   }
> > > +
> > > +   return 0;
> > > +}
> > > +
> > >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> > >   {
> > >     return vp_modern_config_vector(&vp_dev->mdev, vector);
> > > @@ -220,32 +243,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> > >
> > >     vq->num_max = num;
> > >
> > > -   /* activate the queue */
> > > -   vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > > -   vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > > -                           virtqueue_get_avail_addr(vq),
> > > -                           virtqueue_get_used_addr(vq));
> > > +   err = vp_active_vq(vq, msix_vec);
> > > +   if (err)
> > > +           goto err;
> > >
> > >     vq->priv = (void __force *)vp_modern_map_vq_notify(mdev, index, NULL);
> > >     if (!vq->priv) {
> > >             err = -ENOMEM;
> > > -           goto err_map_notify;
> > > -   }
> > > -
> > > -   if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> > > -           msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> > > -           if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
> > > -                   err = -EBUSY;
> > > -                   goto err_assign_vector;
> > > -           }
> > > +           goto err;
> > >     }
> > >
> > >     return vq;
> > >
> > > -err_assign_vector:
> > > -   if (!mdev->notify_base)
> > > -           pci_iounmap(mdev->pci_dev, (void __iomem __force *)vq->priv);
> >
> >
> > We need keep this or anything I missed?
>
> I think so, after modification, vp_modern_map_vq_notify is the last step before
> returning vq. If it fails, then vq->priv is equal to NULL, so there is no need
> to execute pci_iounmap.
>
> Did I miss something?

Nope I miss that the vector is configured before the mapping.

So

Acked-by: Jason Wang <jasowang@redhat.com>

Thanks

>
> Thanks.
>
> >
> > Thanks
> >
> >
> > > -err_map_notify:
> > > +err:
> > >     vring_del_virtqueue(vq);
> > >     return ERR_PTR(err);
> > >   }
> >
>

Re: [PATCH v9 22/32] virtio_pci: queue_reset: extract the logic of active vq for modern pci

[Jason Wang]

On Thu, Apr 14, 2022 at 2:25 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 12 Apr 2022 14:58:19 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > Introduce vp_active_vq() to configure vring to backend after vq attach
> > > vring. And configure vq vector if necessary.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/virtio/virtio_pci_modern.c | 46 ++++++++++++++++++------------
> > >   1 file changed, 28 insertions(+), 18 deletions(-)
> > >
> > > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > > index 86d301f272b8..49a4493732cf 100644
> > > --- a/drivers/virtio/virtio_pci_modern.c
> > > +++ b/drivers/virtio/virtio_pci_modern.c
> > > @@ -176,6 +176,29 @@ static void vp_reset(struct virtio_device *vdev)
> > >     vp_disable_cbs(vdev);
> > >   }
> > >
> > > +static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> > > +{
> > > +   struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > +   struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > +   unsigned long index;
> > > +
> > > +   index = vq->index;
> > > +
> > > +   /* activate the queue */
> > > +   vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > > +   vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > > +                           virtqueue_get_avail_addr(vq),
> > > +                           virtqueue_get_used_addr(vq));
> > > +
> > > +   if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> > > +           msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> > > +           if (msix_vec == VIRTIO_MSI_NO_VECTOR)
> > > +                   return -EBUSY;
> > > +   }
> > > +
> > > +   return 0;
> > > +}
> > > +
> > >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> > >   {
> > >     return vp_modern_config_vector(&vp_dev->mdev, vector);
> > > @@ -220,32 +243,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> > >
> > >     vq->num_max = num;
> > >
> > > -   /* activate the queue */
> > > -   vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > > -   vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > > -                           virtqueue_get_avail_addr(vq),
> > > -                           virtqueue_get_used_addr(vq));
> > > +   err = vp_active_vq(vq, msix_vec);
> > > +   if (err)
> > > +           goto err;
> > >
> > >     vq->priv = (void __force *)vp_modern_map_vq_notify(mdev, index, NULL);
> > >     if (!vq->priv) {
> > >             err = -ENOMEM;
> > > -           goto err_map_notify;
> > > -   }
> > > -
> > > -   if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> > > -           msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> > > -           if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
> > > -                   err = -EBUSY;
> > > -                   goto err_assign_vector;
> > > -           }
> > > +           goto err;
> > >     }
> > >
> > >     return vq;
> > >
> > > -err_assign_vector:
> > > -   if (!mdev->notify_base)
> > > -           pci_iounmap(mdev->pci_dev, (void __iomem __force *)vq->priv);
> >
> >
> > We need keep this or anything I missed?
>
> I think so, after modification, vp_modern_map_vq_notify is the last step before
> returning vq. If it fails, then vq->priv is equal to NULL, so there is no need
> to execute pci_iounmap.
>
> Did I miss something?

Nope I miss that the vector is configured before the mapping.

So

Acked-by: Jason Wang <jasowang@redhat.com>

Thanks

>
> Thanks.
>
> >
> > Thanks
> >
> >
> > > -err_map_notify:
> > > +err:
> > >     vring_del_virtqueue(vq);
> > >     return ERR_PTR(err);
> > >   }
> >
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 22/32] virtio_pci: queue_reset: extract the logic of active vq for modern pci

[Jason Wang]

On Thu, Apr 14, 2022 at 2:25 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 12 Apr 2022 14:58:19 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > Introduce vp_active_vq() to configure vring to backend after vq attach
> > > vring. And configure vq vector if necessary.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/virtio/virtio_pci_modern.c | 46 ++++++++++++++++++------------
> > >   1 file changed, 28 insertions(+), 18 deletions(-)
> > >
> > > diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> > > index 86d301f272b8..49a4493732cf 100644
> > > --- a/drivers/virtio/virtio_pci_modern.c
> > > +++ b/drivers/virtio/virtio_pci_modern.c
> > > @@ -176,6 +176,29 @@ static void vp_reset(struct virtio_device *vdev)
> > >     vp_disable_cbs(vdev);
> > >   }
> > >
> > > +static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
> > > +{
> > > +   struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > +   struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > +   unsigned long index;
> > > +
> > > +   index = vq->index;
> > > +
> > > +   /* activate the queue */
> > > +   vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > > +   vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > > +                           virtqueue_get_avail_addr(vq),
> > > +                           virtqueue_get_used_addr(vq));
> > > +
> > > +   if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> > > +           msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> > > +           if (msix_vec == VIRTIO_MSI_NO_VECTOR)
> > > +                   return -EBUSY;
> > > +   }
> > > +
> > > +   return 0;
> > > +}
> > > +
> > >   static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
> > >   {
> > >     return vp_modern_config_vector(&vp_dev->mdev, vector);
> > > @@ -220,32 +243,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
> > >
> > >     vq->num_max = num;
> > >
> > > -   /* activate the queue */
> > > -   vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
> > > -   vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
> > > -                           virtqueue_get_avail_addr(vq),
> > > -                           virtqueue_get_used_addr(vq));
> > > +   err = vp_active_vq(vq, msix_vec);
> > > +   if (err)
> > > +           goto err;
> > >
> > >     vq->priv = (void __force *)vp_modern_map_vq_notify(mdev, index, NULL);
> > >     if (!vq->priv) {
> > >             err = -ENOMEM;
> > > -           goto err_map_notify;
> > > -   }
> > > -
> > > -   if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
> > > -           msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
> > > -           if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
> > > -                   err = -EBUSY;
> > > -                   goto err_assign_vector;
> > > -           }
> > > +           goto err;
> > >     }
> > >
> > >     return vq;
> > >
> > > -err_assign_vector:
> > > -   if (!mdev->notify_base)
> > > -           pci_iounmap(mdev->pci_dev, (void __iomem __force *)vq->priv);
> >
> >
> > We need keep this or anything I missed?
>
> I think so, after modification, vp_modern_map_vq_notify is the last step before
> returning vq. If it fails, then vq->priv is equal to NULL, so there is no need
> to execute pci_iounmap.
>
> Did I miss something?

Nope I miss that the vector is configured before the mapping.

So

Acked-by: Jason Wang <jasowang@redhat.com>

Thanks

>
> Thanks.
>
> >
> > Thanks
> >
> >
> > > -err_map_notify:
> > > +err:
> > >     vring_del_virtqueue(vq);
> > >     return ERR_PTR(err);
> > >   }
> >
>


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Thu, 14 Apr 2022 17:30:02 +0800, Jason Wang <jasowang@redhat.com> wrote:
> On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> >
> > On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > >
> > > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > > This patch implements the resize function of the rx, tx queues.
> > > > Based on this function, it is possible to modify the ring num of the
> > > > queue.
> > > >
> > > > There may be an exception during the resize process, the resize may
> > > > fail, or the vq can no longer be used. Either way, we must execute
> > > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > > must be called after calling napi_disable.
> > > >
> > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > ---
> > > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > > >   1 file changed, 81 insertions(+)
> > > >
> > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > index b8bf00525177..ba6859f305f7 100644
> > > > --- a/drivers/net/virtio_net.c
> > > > +++ b/drivers/net/virtio_net.c
> > > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > > >     char padding[4];
> > > >   };
> > > >
> > > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > +
> > > >   static bool is_xdp_frame(void *ptr)
> > > >   {
> > > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > > >   {
> > > >     napi_enable(napi);
> > > >
> > > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > > +    * be protected by napi. However, the protection of napi is only enabled
> > > > +    * during the operation, and the protection of napi will end after the
> > > > +    * operation is completed. If re-enable fails during the process, vq
> > > > +    * will remain unavailable with reset state.
> > > > +    */
> > > > +   if (vq->reset)
> > > > +           return;
> > >
> > >
> > > I don't get when could we hit this condition.
> >
> >
> > In patch 23, the code to implement re-enable vq is as follows:
> >
> > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > +{
> > +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +       struct virtio_pci_vq_info *info;
> > +       unsigned long flags, index;
> > +       int err;
> > +
> > +       if (!vq->reset)
> > +               return -EBUSY;
> > +
> > +       index = vq->index;
> > +       info = vp_dev->vqs[index];
> > +
> > +       /* check queue reset status */
> > +       if (vp_modern_get_queue_reset(mdev, index) != 1)
> > +               return -EBUSY;
> > +
> > +       err = vp_active_vq(vq, info->msix_vector);
> > +       if (err)
> > +               return err;
> > +
> > +       if (vq->callback) {
> > +               spin_lock_irqsave(&vp_dev->lock, flags);
> > +               list_add(&info->node, &vp_dev->virtqueues);
> > +               spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +       } else {
> > +               INIT_LIST_HEAD(&info->node);
> > +       }
> > +
> > +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > +
> > +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> > +
> > +       vq->reset = false;
> > +
> > +       return 0;
> > +}
> >
> >
> > There are three situations where an error will be returned. These are the
> > situations I want to handle.
>
> Right, but it looks harmless if we just schedule the NAPI without the check.

Yes.

> >
> > But I'm rethinking the question, and I feel like you're right, although the
> > hardware setup may fail. We can no longer sync with the hardware. But using it
> > as a normal vq doesn't have any problems.
>
> Note that we should make sure the buggy(malicous) device won't crash
> the codes by changing the queue_reset value at its will.

I will keep an eye on this situation.

>
> >
> > >
> > >
> > > > +
> > > >     /* If all buffers were filled by other side before we napi_enabled, we
> > > >      * won't get another interrupt, so process any outstanding packets now.
> > > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > > >             struct receive_queue *rq = &vi->rq[i];
> > > >
> > > >             napi_disable(&rq->napi);
> > > > +
> > > > +           /* Check if vq is in reset state. See more in
> > > > +            * virtnet_napi_enable()
> > > > +            */
> > > > +           if (rq->vq->reset) {
> > > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > +                   continue;
> > > > +           }
> > >
> > >
> > > Can we do something similar in virtnet_close() by canceling the work?
> >
> > I think there is no need to cancel the work here, because napi_disable will wait
> > for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
> > vq, this logic can be removed.
>
> Actually I meant the part of virtnet_rx_resize().
>
> If we don't synchronize with the refill work, it might enable NAPI unexpectedly?

I don't think this situation will be encountered, because napi_disable is
mutually exclusive, so there will be no unexpected napi enable.

Is there something I misunderstood?

Thanks.

>
> Thanks
>
> >
> >
> > >
> > >
> > > > +
> > > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > > >
> > > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > > >             return;
> > > >
> > > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > > +   if (sq->vq->reset)
> > > > +           return;
> > >
> > >
> > > We've disabled TX napi, any chance we can still hit this?
> >
> > Same as above.
> >
> > >
> > >
> > > > +
> > > >     if (__netif_tx_trylock(txq)) {
> > > >             do {
> > > >                     virtqueue_disable_cb(sq->vq);
> > > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > > >     return NETDEV_TX_OK;
> > > >   }
> > > >
> > > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > > +                        struct receive_queue *rq, u32 ring_num)
> > > > +{
> > > > +   int err;
> > > > +
> > > > +   napi_disable(&rq->napi);
> > > > +
> > > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > > +   if (err)
> > > > +           goto err;
> > > > +
> > > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > > +           schedule_delayed_work(&vi->refill, 0);
> > > > +
> > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > +   return 0;
> > > > +
> > > > +err:
> > > > +   netdev_err(vi->dev,
> > > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > > +              rq - vi->rq, err);
> > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > +   return err;
> > > > +}
> > > > +
> > > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > > +                        struct send_queue *sq, u32 ring_num)
> > > > +{
> > > > +   struct netdev_queue *txq;
> > > > +   int err, qindex;
> > > > +
> > > > +   qindex = sq - vi->sq;
> > > > +
> > > > +   virtnet_napi_tx_disable(&sq->napi);
> > > > +
> > > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > > +   __netif_tx_lock_bh(txq);
> > > > +   netif_stop_subqueue(vi->dev, qindex);
> > > > +   __netif_tx_unlock_bh(txq);
> > > > +
> > > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > > +   if (err)
> > > > +           goto err;
> > > > +
> > > > +   netif_start_subqueue(vi->dev, qindex);
> > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > +   return 0;
> > > > +
> > > > +err:
> > >
> > >
> > > I guess we can still start the queue in this case? (Since we don't
> > > change the queue if resize fails).
> >
> > Yes, you are right.
> >
> > Thanks.
> >
> > >
> > >
> > > > +   netdev_err(vi->dev,
> > > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > > +              sq - vi->sq, err);
> > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > +   return err;
> > > > +}
> > > > +
> > > >   /*
> > > >    * Send command via the control virtqueue and check status.  Commands
> > > >    * supported by the hypervisor, as indicated by feature bits, should
> > >
> >
>

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Thu, 14 Apr 2022 17:30:02 +0800, Jason Wang <jasowang@redhat.com> wrote:
> On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> >
> > On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > >
> > > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > > This patch implements the resize function of the rx, tx queues.
> > > > Based on this function, it is possible to modify the ring num of the
> > > > queue.
> > > >
> > > > There may be an exception during the resize process, the resize may
> > > > fail, or the vq can no longer be used. Either way, we must execute
> > > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > > must be called after calling napi_disable.
> > > >
> > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > ---
> > > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > > >   1 file changed, 81 insertions(+)
> > > >
> > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > index b8bf00525177..ba6859f305f7 100644
> > > > --- a/drivers/net/virtio_net.c
> > > > +++ b/drivers/net/virtio_net.c
> > > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > > >     char padding[4];
> > > >   };
> > > >
> > > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > +
> > > >   static bool is_xdp_frame(void *ptr)
> > > >   {
> > > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > > >   {
> > > >     napi_enable(napi);
> > > >
> > > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > > +    * be protected by napi. However, the protection of napi is only enabled
> > > > +    * during the operation, and the protection of napi will end after the
> > > > +    * operation is completed. If re-enable fails during the process, vq
> > > > +    * will remain unavailable with reset state.
> > > > +    */
> > > > +   if (vq->reset)
> > > > +           return;
> > >
> > >
> > > I don't get when could we hit this condition.
> >
> >
> > In patch 23, the code to implement re-enable vq is as follows:
> >
> > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > +{
> > +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +       struct virtio_pci_vq_info *info;
> > +       unsigned long flags, index;
> > +       int err;
> > +
> > +       if (!vq->reset)
> > +               return -EBUSY;
> > +
> > +       index = vq->index;
> > +       info = vp_dev->vqs[index];
> > +
> > +       /* check queue reset status */
> > +       if (vp_modern_get_queue_reset(mdev, index) != 1)
> > +               return -EBUSY;
> > +
> > +       err = vp_active_vq(vq, info->msix_vector);
> > +       if (err)
> > +               return err;
> > +
> > +       if (vq->callback) {
> > +               spin_lock_irqsave(&vp_dev->lock, flags);
> > +               list_add(&info->node, &vp_dev->virtqueues);
> > +               spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +       } else {
> > +               INIT_LIST_HEAD(&info->node);
> > +       }
> > +
> > +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > +
> > +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> > +
> > +       vq->reset = false;
> > +
> > +       return 0;
> > +}
> >
> >
> > There are three situations where an error will be returned. These are the
> > situations I want to handle.
>
> Right, but it looks harmless if we just schedule the NAPI without the check.

Yes.

> >
> > But I'm rethinking the question, and I feel like you're right, although the
> > hardware setup may fail. We can no longer sync with the hardware. But using it
> > as a normal vq doesn't have any problems.
>
> Note that we should make sure the buggy(malicous) device won't crash
> the codes by changing the queue_reset value at its will.

I will keep an eye on this situation.

>
> >
> > >
> > >
> > > > +
> > > >     /* If all buffers were filled by other side before we napi_enabled, we
> > > >      * won't get another interrupt, so process any outstanding packets now.
> > > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > > >             struct receive_queue *rq = &vi->rq[i];
> > > >
> > > >             napi_disable(&rq->napi);
> > > > +
> > > > +           /* Check if vq is in reset state. See more in
> > > > +            * virtnet_napi_enable()
> > > > +            */
> > > > +           if (rq->vq->reset) {
> > > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > +                   continue;
> > > > +           }
> > >
> > >
> > > Can we do something similar in virtnet_close() by canceling the work?
> >
> > I think there is no need to cancel the work here, because napi_disable will wait
> > for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
> > vq, this logic can be removed.
>
> Actually I meant the part of virtnet_rx_resize().
>
> If we don't synchronize with the refill work, it might enable NAPI unexpectedly?

I don't think this situation will be encountered, because napi_disable is
mutually exclusive, so there will be no unexpected napi enable.

Is there something I misunderstood?

Thanks.

>
> Thanks
>
> >
> >
> > >
> > >
> > > > +
> > > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > > >
> > > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > > >             return;
> > > >
> > > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > > +   if (sq->vq->reset)
> > > > +           return;
> > >
> > >
> > > We've disabled TX napi, any chance we can still hit this?
> >
> > Same as above.
> >
> > >
> > >
> > > > +
> > > >     if (__netif_tx_trylock(txq)) {
> > > >             do {
> > > >                     virtqueue_disable_cb(sq->vq);
> > > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > > >     return NETDEV_TX_OK;
> > > >   }
> > > >
> > > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > > +                        struct receive_queue *rq, u32 ring_num)
> > > > +{
> > > > +   int err;
> > > > +
> > > > +   napi_disable(&rq->napi);
> > > > +
> > > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > > +   if (err)
> > > > +           goto err;
> > > > +
> > > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > > +           schedule_delayed_work(&vi->refill, 0);
> > > > +
> > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > +   return 0;
> > > > +
> > > > +err:
> > > > +   netdev_err(vi->dev,
> > > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > > +              rq - vi->rq, err);
> > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > +   return err;
> > > > +}
> > > > +
> > > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > > +                        struct send_queue *sq, u32 ring_num)
> > > > +{
> > > > +   struct netdev_queue *txq;
> > > > +   int err, qindex;
> > > > +
> > > > +   qindex = sq - vi->sq;
> > > > +
> > > > +   virtnet_napi_tx_disable(&sq->napi);
> > > > +
> > > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > > +   __netif_tx_lock_bh(txq);
> > > > +   netif_stop_subqueue(vi->dev, qindex);
> > > > +   __netif_tx_unlock_bh(txq);
> > > > +
> > > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > > +   if (err)
> > > > +           goto err;
> > > > +
> > > > +   netif_start_subqueue(vi->dev, qindex);
> > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > +   return 0;
> > > > +
> > > > +err:
> > >
> > >
> > > I guess we can still start the queue in this case? (Since we don't
> > > change the queue if resize fails).
> >
> > Yes, you are right.
> >
> > Thanks.
> >
> > >
> > >
> > > > +   netdev_err(vi->dev,
> > > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > > +              sq - vi->sq, err);
> > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > +   return err;
> > > > +}
> > > > +
> > > >   /*
> > > >    * Send command via the control virtqueue and check status.  Commands
> > > >    * supported by the hypervisor, as indicated by feature bits, should
> > >
> >
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Thu, 14 Apr 2022 17:30:02 +0800, Jason Wang <jasowang@redhat.com> wrote:
> On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> >
> > On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > >
> > > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > > This patch implements the resize function of the rx, tx queues.
> > > > Based on this function, it is possible to modify the ring num of the
> > > > queue.
> > > >
> > > > There may be an exception during the resize process, the resize may
> > > > fail, or the vq can no longer be used. Either way, we must execute
> > > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > > must be called after calling napi_disable.
> > > >
> > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > ---
> > > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > > >   1 file changed, 81 insertions(+)
> > > >
> > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > index b8bf00525177..ba6859f305f7 100644
> > > > --- a/drivers/net/virtio_net.c
> > > > +++ b/drivers/net/virtio_net.c
> > > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > > >     char padding[4];
> > > >   };
> > > >
> > > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > +
> > > >   static bool is_xdp_frame(void *ptr)
> > > >   {
> > > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > > >   {
> > > >     napi_enable(napi);
> > > >
> > > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > > +    * be protected by napi. However, the protection of napi is only enabled
> > > > +    * during the operation, and the protection of napi will end after the
> > > > +    * operation is completed. If re-enable fails during the process, vq
> > > > +    * will remain unavailable with reset state.
> > > > +    */
> > > > +   if (vq->reset)
> > > > +           return;
> > >
> > >
> > > I don't get when could we hit this condition.
> >
> >
> > In patch 23, the code to implement re-enable vq is as follows:
> >
> > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > +{
> > +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > +       struct virtio_pci_vq_info *info;
> > +       unsigned long flags, index;
> > +       int err;
> > +
> > +       if (!vq->reset)
> > +               return -EBUSY;
> > +
> > +       index = vq->index;
> > +       info = vp_dev->vqs[index];
> > +
> > +       /* check queue reset status */
> > +       if (vp_modern_get_queue_reset(mdev, index) != 1)
> > +               return -EBUSY;
> > +
> > +       err = vp_active_vq(vq, info->msix_vector);
> > +       if (err)
> > +               return err;
> > +
> > +       if (vq->callback) {
> > +               spin_lock_irqsave(&vp_dev->lock, flags);
> > +               list_add(&info->node, &vp_dev->virtqueues);
> > +               spin_unlock_irqrestore(&vp_dev->lock, flags);
> > +       } else {
> > +               INIT_LIST_HEAD(&info->node);
> > +       }
> > +
> > +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > +
> > +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> > +
> > +       vq->reset = false;
> > +
> > +       return 0;
> > +}
> >
> >
> > There are three situations where an error will be returned. These are the
> > situations I want to handle.
>
> Right, but it looks harmless if we just schedule the NAPI without the check.

Yes.

> >
> > But I'm rethinking the question, and I feel like you're right, although the
> > hardware setup may fail. We can no longer sync with the hardware. But using it
> > as a normal vq doesn't have any problems.
>
> Note that we should make sure the buggy(malicous) device won't crash
> the codes by changing the queue_reset value at its will.

I will keep an eye on this situation.

>
> >
> > >
> > >
> > > > +
> > > >     /* If all buffers were filled by other side before we napi_enabled, we
> > > >      * won't get another interrupt, so process any outstanding packets now.
> > > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > > >             struct receive_queue *rq = &vi->rq[i];
> > > >
> > > >             napi_disable(&rq->napi);
> > > > +
> > > > +           /* Check if vq is in reset state. See more in
> > > > +            * virtnet_napi_enable()
> > > > +            */
> > > > +           if (rq->vq->reset) {
> > > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > +                   continue;
> > > > +           }
> > >
> > >
> > > Can we do something similar in virtnet_close() by canceling the work?
> >
> > I think there is no need to cancel the work here, because napi_disable will wait
> > for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
> > vq, this logic can be removed.
>
> Actually I meant the part of virtnet_rx_resize().
>
> If we don't synchronize with the refill work, it might enable NAPI unexpectedly?

I don't think this situation will be encountered, because napi_disable is
mutually exclusive, so there will be no unexpected napi enable.

Is there something I misunderstood?

Thanks.

>
> Thanks
>
> >
> >
> > >
> > >
> > > > +
> > > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > > >
> > > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > > >             return;
> > > >
> > > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > > +   if (sq->vq->reset)
> > > > +           return;
> > >
> > >
> > > We've disabled TX napi, any chance we can still hit this?
> >
> > Same as above.
> >
> > >
> > >
> > > > +
> > > >     if (__netif_tx_trylock(txq)) {
> > > >             do {
> > > >                     virtqueue_disable_cb(sq->vq);
> > > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > > >     return NETDEV_TX_OK;
> > > >   }
> > > >
> > > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > > +                        struct receive_queue *rq, u32 ring_num)
> > > > +{
> > > > +   int err;
> > > > +
> > > > +   napi_disable(&rq->napi);
> > > > +
> > > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > > +   if (err)
> > > > +           goto err;
> > > > +
> > > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > > +           schedule_delayed_work(&vi->refill, 0);
> > > > +
> > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > +   return 0;
> > > > +
> > > > +err:
> > > > +   netdev_err(vi->dev,
> > > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > > +              rq - vi->rq, err);
> > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > +   return err;
> > > > +}
> > > > +
> > > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > > +                        struct send_queue *sq, u32 ring_num)
> > > > +{
> > > > +   struct netdev_queue *txq;
> > > > +   int err, qindex;
> > > > +
> > > > +   qindex = sq - vi->sq;
> > > > +
> > > > +   virtnet_napi_tx_disable(&sq->napi);
> > > > +
> > > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > > +   __netif_tx_lock_bh(txq);
> > > > +   netif_stop_subqueue(vi->dev, qindex);
> > > > +   __netif_tx_unlock_bh(txq);
> > > > +
> > > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > > +   if (err)
> > > > +           goto err;
> > > > +
> > > > +   netif_start_subqueue(vi->dev, qindex);
> > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > +   return 0;
> > > > +
> > > > +err:
> > >
> > >
> > > I guess we can still start the queue in this case? (Since we don't
> > > change the queue if resize fails).
> >
> > Yes, you are right.
> >
> > Thanks.
> >
> > >
> > >
> > > > +   netdev_err(vi->dev,
> > > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > > +              sq - vi->sq, err);
> > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > +   return err;
> > > > +}
> > > > +
> > > >   /*
> > > >    * Send command via the control virtqueue and check status.  Commands
> > > >    * supported by the hypervisor, as indicated by feature bits, should
> > >
> >
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

On Fri, Apr 15, 2022 at 10:23 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Thu, 14 Apr 2022 17:30:02 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> > >
> > > On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > > >
> > > > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > > > This patch implements the resize function of the rx, tx queues.
> > > > > Based on this function, it is possible to modify the ring num of the
> > > > > queue.
> > > > >
> > > > > There may be an exception during the resize process, the resize may
> > > > > fail, or the vq can no longer be used. Either way, we must execute
> > > > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > > > must be called after calling napi_disable.
> > > > >
> > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > > ---
> > > > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > > > >   1 file changed, 81 insertions(+)
> > > > >
> > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > > index b8bf00525177..ba6859f305f7 100644
> > > > > --- a/drivers/net/virtio_net.c
> > > > > +++ b/drivers/net/virtio_net.c
> > > > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > > > >     char padding[4];
> > > > >   };
> > > > >
> > > > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > > +
> > > > >   static bool is_xdp_frame(void *ptr)
> > > > >   {
> > > > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > > > >   {
> > > > >     napi_enable(napi);
> > > > >
> > > > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > > > +    * be protected by napi. However, the protection of napi is only enabled
> > > > > +    * during the operation, and the protection of napi will end after the
> > > > > +    * operation is completed. If re-enable fails during the process, vq
> > > > > +    * will remain unavailable with reset state.
> > > > > +    */
> > > > > +   if (vq->reset)
> > > > > +           return;
> > > >
> > > >
> > > > I don't get when could we hit this condition.
> > >
> > >
> > > In patch 23, the code to implement re-enable vq is as follows:
> > >
> > > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > > +{
> > > +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > +       struct virtio_pci_vq_info *info;
> > > +       unsigned long flags, index;
> > > +       int err;
> > > +
> > > +       if (!vq->reset)
> > > +               return -EBUSY;
> > > +
> > > +       index = vq->index;
> > > +       info = vp_dev->vqs[index];
> > > +
> > > +       /* check queue reset status */
> > > +       if (vp_modern_get_queue_reset(mdev, index) != 1)
> > > +               return -EBUSY;
> > > +
> > > +       err = vp_active_vq(vq, info->msix_vector);
> > > +       if (err)
> > > +               return err;
> > > +
> > > +       if (vq->callback) {
> > > +               spin_lock_irqsave(&vp_dev->lock, flags);
> > > +               list_add(&info->node, &vp_dev->virtqueues);
> > > +               spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +       } else {
> > > +               INIT_LIST_HEAD(&info->node);
> > > +       }
> > > +
> > > +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > > +
> > > +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > > +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> > > +
> > > +       vq->reset = false;
> > > +
> > > +       return 0;
> > > +}
> > >
> > >
> > > There are three situations where an error will be returned. These are the
> > > situations I want to handle.
> >
> > Right, but it looks harmless if we just schedule the NAPI without the check.
>
> Yes.
>
> > >
> > > But I'm rethinking the question, and I feel like you're right, although the
> > > hardware setup may fail. We can no longer sync with the hardware. But using it
> > > as a normal vq doesn't have any problems.
> >
> > Note that we should make sure the buggy(malicous) device won't crash
> > the codes by changing the queue_reset value at its will.
>
> I will keep an eye on this situation.
>
> >
> > >
> > > >
> > > >
> > > > > +
> > > > >     /* If all buffers were filled by other side before we napi_enabled, we
> > > > >      * won't get another interrupt, so process any outstanding packets now.
> > > > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > > > >             struct receive_queue *rq = &vi->rq[i];
> > > > >
> > > > >             napi_disable(&rq->napi);
> > > > > +
> > > > > +           /* Check if vq is in reset state. See more in
> > > > > +            * virtnet_napi_enable()
> > > > > +            */
> > > > > +           if (rq->vq->reset) {
> > > > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > +                   continue;
> > > > > +           }
> > > >
> > > >
> > > > Can we do something similar in virtnet_close() by canceling the work?
> > >
> > > I think there is no need to cancel the work here, because napi_disable will wait
> > > for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
> > > vq, this logic can be removed.
> >
> > Actually I meant the part of virtnet_rx_resize().
> >
> > If we don't synchronize with the refill work, it might enable NAPI unexpectedly?
>
> I don't think this situation will be encountered, because napi_disable is
> mutually exclusive, so there will be no unexpected napi enable.
>
> Is there something I misunderstood?

So in virtnet_rx_resize() we do:

napi_disable()
...
resize()
...
napi_enalbe()

How can we guarantee that the work is not run after the napi_disable()?

Thanks

>
> Thanks.
>
> >
> > Thanks
> >
> > >
> > >
> > > >
> > > >
> > > > > +
> > > > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > > > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > > > >
> > > > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > > > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > > > >             return;
> > > > >
> > > > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > > > +   if (sq->vq->reset)
> > > > > +           return;
> > > >
> > > >
> > > > We've disabled TX napi, any chance we can still hit this?
> > >
> > > Same as above.
> > >
> > > >
> > > >
> > > > > +
> > > > >     if (__netif_tx_trylock(txq)) {
> > > > >             do {
> > > > >                     virtqueue_disable_cb(sq->vq);
> > > > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > > > >     return NETDEV_TX_OK;
> > > > >   }
> > > > >
> > > > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > > > +                        struct receive_queue *rq, u32 ring_num)
> > > > > +{
> > > > > +   int err;
> > > > > +
> > > > > +   napi_disable(&rq->napi);
> > > > > +
> > > > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > > > +   if (err)
> > > > > +           goto err;
> > > > > +
> > > > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > > > +           schedule_delayed_work(&vi->refill, 0);
> > > > > +
> > > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > +   return 0;
> > > > > +
> > > > > +err:
> > > > > +   netdev_err(vi->dev,
> > > > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > > > +              rq - vi->rq, err);
> > > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > +   return err;
> > > > > +}
> > > > > +
> > > > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > > > +                        struct send_queue *sq, u32 ring_num)
> > > > > +{
> > > > > +   struct netdev_queue *txq;
> > > > > +   int err, qindex;
> > > > > +
> > > > > +   qindex = sq - vi->sq;
> > > > > +
> > > > > +   virtnet_napi_tx_disable(&sq->napi);
> > > > > +
> > > > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > > > +   __netif_tx_lock_bh(txq);
> > > > > +   netif_stop_subqueue(vi->dev, qindex);
> > > > > +   __netif_tx_unlock_bh(txq);
> > > > > +
> > > > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > > > +   if (err)
> > > > > +           goto err;
> > > > > +
> > > > > +   netif_start_subqueue(vi->dev, qindex);
> > > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > > +   return 0;
> > > > > +
> > > > > +err:
> > > >
> > > >
> > > > I guess we can still start the queue in this case? (Since we don't
> > > > change the queue if resize fails).
> > >
> > > Yes, you are right.
> > >
> > > Thanks.
> > >
> > > >
> > > >
> > > > > +   netdev_err(vi->dev,
> > > > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > > > +              sq - vi->sq, err);
> > > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > > +   return err;
> > > > > +}
> > > > > +
> > > > >   /*
> > > > >    * Send command via the control virtqueue and check status.  Commands
> > > > >    * supported by the hypervisor, as indicated by feature bits, should
> > > >
> > >
> >
>

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

On Fri, Apr 15, 2022 at 10:23 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Thu, 14 Apr 2022 17:30:02 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> > >
> > > On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > > >
> > > > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > > > This patch implements the resize function of the rx, tx queues.
> > > > > Based on this function, it is possible to modify the ring num of the
> > > > > queue.
> > > > >
> > > > > There may be an exception during the resize process, the resize may
> > > > > fail, or the vq can no longer be used. Either way, we must execute
> > > > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > > > must be called after calling napi_disable.
> > > > >
> > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > > ---
> > > > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > > > >   1 file changed, 81 insertions(+)
> > > > >
> > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > > index b8bf00525177..ba6859f305f7 100644
> > > > > --- a/drivers/net/virtio_net.c
> > > > > +++ b/drivers/net/virtio_net.c
> > > > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > > > >     char padding[4];
> > > > >   };
> > > > >
> > > > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > > +
> > > > >   static bool is_xdp_frame(void *ptr)
> > > > >   {
> > > > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > > > >   {
> > > > >     napi_enable(napi);
> > > > >
> > > > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > > > +    * be protected by napi. However, the protection of napi is only enabled
> > > > > +    * during the operation, and the protection of napi will end after the
> > > > > +    * operation is completed. If re-enable fails during the process, vq
> > > > > +    * will remain unavailable with reset state.
> > > > > +    */
> > > > > +   if (vq->reset)
> > > > > +           return;
> > > >
> > > >
> > > > I don't get when could we hit this condition.
> > >
> > >
> > > In patch 23, the code to implement re-enable vq is as follows:
> > >
> > > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > > +{
> > > +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > +       struct virtio_pci_vq_info *info;
> > > +       unsigned long flags, index;
> > > +       int err;
> > > +
> > > +       if (!vq->reset)
> > > +               return -EBUSY;
> > > +
> > > +       index = vq->index;
> > > +       info = vp_dev->vqs[index];
> > > +
> > > +       /* check queue reset status */
> > > +       if (vp_modern_get_queue_reset(mdev, index) != 1)
> > > +               return -EBUSY;
> > > +
> > > +       err = vp_active_vq(vq, info->msix_vector);
> > > +       if (err)
> > > +               return err;
> > > +
> > > +       if (vq->callback) {
> > > +               spin_lock_irqsave(&vp_dev->lock, flags);
> > > +               list_add(&info->node, &vp_dev->virtqueues);
> > > +               spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +       } else {
> > > +               INIT_LIST_HEAD(&info->node);
> > > +       }
> > > +
> > > +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > > +
> > > +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > > +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> > > +
> > > +       vq->reset = false;
> > > +
> > > +       return 0;
> > > +}
> > >
> > >
> > > There are three situations where an error will be returned. These are the
> > > situations I want to handle.
> >
> > Right, but it looks harmless if we just schedule the NAPI without the check.
>
> Yes.
>
> > >
> > > But I'm rethinking the question, and I feel like you're right, although the
> > > hardware setup may fail. We can no longer sync with the hardware. But using it
> > > as a normal vq doesn't have any problems.
> >
> > Note that we should make sure the buggy(malicous) device won't crash
> > the codes by changing the queue_reset value at its will.
>
> I will keep an eye on this situation.
>
> >
> > >
> > > >
> > > >
> > > > > +
> > > > >     /* If all buffers were filled by other side before we napi_enabled, we
> > > > >      * won't get another interrupt, so process any outstanding packets now.
> > > > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > > > >             struct receive_queue *rq = &vi->rq[i];
> > > > >
> > > > >             napi_disable(&rq->napi);
> > > > > +
> > > > > +           /* Check if vq is in reset state. See more in
> > > > > +            * virtnet_napi_enable()
> > > > > +            */
> > > > > +           if (rq->vq->reset) {
> > > > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > +                   continue;
> > > > > +           }
> > > >
> > > >
> > > > Can we do something similar in virtnet_close() by canceling the work?
> > >
> > > I think there is no need to cancel the work here, because napi_disable will wait
> > > for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
> > > vq, this logic can be removed.
> >
> > Actually I meant the part of virtnet_rx_resize().
> >
> > If we don't synchronize with the refill work, it might enable NAPI unexpectedly?
>
> I don't think this situation will be encountered, because napi_disable is
> mutually exclusive, so there will be no unexpected napi enable.
>
> Is there something I misunderstood?

So in virtnet_rx_resize() we do:

napi_disable()
...
resize()
...
napi_enalbe()

How can we guarantee that the work is not run after the napi_disable()?

Thanks

>
> Thanks.
>
> >
> > Thanks
> >
> > >
> > >
> > > >
> > > >
> > > > > +
> > > > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > > > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > > > >
> > > > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > > > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > > > >             return;
> > > > >
> > > > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > > > +   if (sq->vq->reset)
> > > > > +           return;
> > > >
> > > >
> > > > We've disabled TX napi, any chance we can still hit this?
> > >
> > > Same as above.
> > >
> > > >
> > > >
> > > > > +
> > > > >     if (__netif_tx_trylock(txq)) {
> > > > >             do {
> > > > >                     virtqueue_disable_cb(sq->vq);
> > > > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > > > >     return NETDEV_TX_OK;
> > > > >   }
> > > > >
> > > > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > > > +                        struct receive_queue *rq, u32 ring_num)
> > > > > +{
> > > > > +   int err;
> > > > > +
> > > > > +   napi_disable(&rq->napi);
> > > > > +
> > > > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > > > +   if (err)
> > > > > +           goto err;
> > > > > +
> > > > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > > > +           schedule_delayed_work(&vi->refill, 0);
> > > > > +
> > > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > +   return 0;
> > > > > +
> > > > > +err:
> > > > > +   netdev_err(vi->dev,
> > > > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > > > +              rq - vi->rq, err);
> > > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > +   return err;
> > > > > +}
> > > > > +
> > > > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > > > +                        struct send_queue *sq, u32 ring_num)
> > > > > +{
> > > > > +   struct netdev_queue *txq;
> > > > > +   int err, qindex;
> > > > > +
> > > > > +   qindex = sq - vi->sq;
> > > > > +
> > > > > +   virtnet_napi_tx_disable(&sq->napi);
> > > > > +
> > > > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > > > +   __netif_tx_lock_bh(txq);
> > > > > +   netif_stop_subqueue(vi->dev, qindex);
> > > > > +   __netif_tx_unlock_bh(txq);
> > > > > +
> > > > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > > > +   if (err)
> > > > > +           goto err;
> > > > > +
> > > > > +   netif_start_subqueue(vi->dev, qindex);
> > > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > > +   return 0;
> > > > > +
> > > > > +err:
> > > >
> > > >
> > > > I guess we can still start the queue in this case? (Since we don't
> > > > change the queue if resize fails).
> > >
> > > Yes, you are right.
> > >
> > > Thanks.
> > >
> > > >
> > > >
> > > > > +   netdev_err(vi->dev,
> > > > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > > > +              sq - vi->sq, err);
> > > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > > +   return err;
> > > > > +}
> > > > > +
> > > > >   /*
> > > > >    * Send command via the control virtqueue and check status.  Commands
> > > > >    * supported by the hypervisor, as indicated by feature bits, should
> > > >
> > >
> >
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

On Fri, Apr 15, 2022 at 10:23 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Thu, 14 Apr 2022 17:30:02 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> > >
> > > On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > > >
> > > > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > > > This patch implements the resize function of the rx, tx queues.
> > > > > Based on this function, it is possible to modify the ring num of the
> > > > > queue.
> > > > >
> > > > > There may be an exception during the resize process, the resize may
> > > > > fail, or the vq can no longer be used. Either way, we must execute
> > > > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > > > must be called after calling napi_disable.
> > > > >
> > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > > ---
> > > > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > > > >   1 file changed, 81 insertions(+)
> > > > >
> > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > > index b8bf00525177..ba6859f305f7 100644
> > > > > --- a/drivers/net/virtio_net.c
> > > > > +++ b/drivers/net/virtio_net.c
> > > > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > > > >     char padding[4];
> > > > >   };
> > > > >
> > > > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > > +
> > > > >   static bool is_xdp_frame(void *ptr)
> > > > >   {
> > > > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > > > >   {
> > > > >     napi_enable(napi);
> > > > >
> > > > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > > > +    * be protected by napi. However, the protection of napi is only enabled
> > > > > +    * during the operation, and the protection of napi will end after the
> > > > > +    * operation is completed. If re-enable fails during the process, vq
> > > > > +    * will remain unavailable with reset state.
> > > > > +    */
> > > > > +   if (vq->reset)
> > > > > +           return;
> > > >
> > > >
> > > > I don't get when could we hit this condition.
> > >
> > >
> > > In patch 23, the code to implement re-enable vq is as follows:
> > >
> > > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > > +{
> > > +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > +       struct virtio_pci_vq_info *info;
> > > +       unsigned long flags, index;
> > > +       int err;
> > > +
> > > +       if (!vq->reset)
> > > +               return -EBUSY;
> > > +
> > > +       index = vq->index;
> > > +       info = vp_dev->vqs[index];
> > > +
> > > +       /* check queue reset status */
> > > +       if (vp_modern_get_queue_reset(mdev, index) != 1)
> > > +               return -EBUSY;
> > > +
> > > +       err = vp_active_vq(vq, info->msix_vector);
> > > +       if (err)
> > > +               return err;
> > > +
> > > +       if (vq->callback) {
> > > +               spin_lock_irqsave(&vp_dev->lock, flags);
> > > +               list_add(&info->node, &vp_dev->virtqueues);
> > > +               spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > +       } else {
> > > +               INIT_LIST_HEAD(&info->node);
> > > +       }
> > > +
> > > +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > > +
> > > +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > > +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> > > +
> > > +       vq->reset = false;
> > > +
> > > +       return 0;
> > > +}
> > >
> > >
> > > There are three situations where an error will be returned. These are the
> > > situations I want to handle.
> >
> > Right, but it looks harmless if we just schedule the NAPI without the check.
>
> Yes.
>
> > >
> > > But I'm rethinking the question, and I feel like you're right, although the
> > > hardware setup may fail. We can no longer sync with the hardware. But using it
> > > as a normal vq doesn't have any problems.
> >
> > Note that we should make sure the buggy(malicous) device won't crash
> > the codes by changing the queue_reset value at its will.
>
> I will keep an eye on this situation.
>
> >
> > >
> > > >
> > > >
> > > > > +
> > > > >     /* If all buffers were filled by other side before we napi_enabled, we
> > > > >      * won't get another interrupt, so process any outstanding packets now.
> > > > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > > > >             struct receive_queue *rq = &vi->rq[i];
> > > > >
> > > > >             napi_disable(&rq->napi);
> > > > > +
> > > > > +           /* Check if vq is in reset state. See more in
> > > > > +            * virtnet_napi_enable()
> > > > > +            */
> > > > > +           if (rq->vq->reset) {
> > > > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > +                   continue;
> > > > > +           }
> > > >
> > > >
> > > > Can we do something similar in virtnet_close() by canceling the work?
> > >
> > > I think there is no need to cancel the work here, because napi_disable will wait
> > > for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
> > > vq, this logic can be removed.
> >
> > Actually I meant the part of virtnet_rx_resize().
> >
> > If we don't synchronize with the refill work, it might enable NAPI unexpectedly?
>
> I don't think this situation will be encountered, because napi_disable is
> mutually exclusive, so there will be no unexpected napi enable.
>
> Is there something I misunderstood?

So in virtnet_rx_resize() we do:

napi_disable()
...
resize()
...
napi_enalbe()

How can we guarantee that the work is not run after the napi_disable()?

Thanks

>
> Thanks.
>
> >
> > Thanks
> >
> > >
> > >
> > > >
> > > >
> > > > > +
> > > > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > > > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > > > >
> > > > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > > > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > > > >             return;
> > > > >
> > > > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > > > +   if (sq->vq->reset)
> > > > > +           return;
> > > >
> > > >
> > > > We've disabled TX napi, any chance we can still hit this?
> > >
> > > Same as above.
> > >
> > > >
> > > >
> > > > > +
> > > > >     if (__netif_tx_trylock(txq)) {
> > > > >             do {
> > > > >                     virtqueue_disable_cb(sq->vq);
> > > > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > > > >     return NETDEV_TX_OK;
> > > > >   }
> > > > >
> > > > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > > > +                        struct receive_queue *rq, u32 ring_num)
> > > > > +{
> > > > > +   int err;
> > > > > +
> > > > > +   napi_disable(&rq->napi);
> > > > > +
> > > > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > > > +   if (err)
> > > > > +           goto err;
> > > > > +
> > > > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > > > +           schedule_delayed_work(&vi->refill, 0);
> > > > > +
> > > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > +   return 0;
> > > > > +
> > > > > +err:
> > > > > +   netdev_err(vi->dev,
> > > > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > > > +              rq - vi->rq, err);
> > > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > +   return err;
> > > > > +}
> > > > > +
> > > > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > > > +                        struct send_queue *sq, u32 ring_num)
> > > > > +{
> > > > > +   struct netdev_queue *txq;
> > > > > +   int err, qindex;
> > > > > +
> > > > > +   qindex = sq - vi->sq;
> > > > > +
> > > > > +   virtnet_napi_tx_disable(&sq->napi);
> > > > > +
> > > > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > > > +   __netif_tx_lock_bh(txq);
> > > > > +   netif_stop_subqueue(vi->dev, qindex);
> > > > > +   __netif_tx_unlock_bh(txq);
> > > > > +
> > > > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > > > +   if (err)
> > > > > +           goto err;
> > > > > +
> > > > > +   netif_start_subqueue(vi->dev, qindex);
> > > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > > +   return 0;
> > > > > +
> > > > > +err:
> > > >
> > > >
> > > > I guess we can still start the queue in this case? (Since we don't
> > > > change the queue if resize fails).
> > >
> > > Yes, you are right.
> > >
> > > Thanks.
> > >
> > > >
> > > >
> > > > > +   netdev_err(vi->dev,
> > > > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > > > +              sq - vi->sq, err);
> > > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > > +   return err;
> > > > > +}
> > > > > +
> > > > >   /*
> > > > >    * Send command via the control virtqueue and check status.  Commands
> > > > >    * supported by the hypervisor, as indicated by feature bits, should
> > > >
> > >
> >
>


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Fri, 15 Apr 2022 13:53:54 +0800, Jason Wang <jasowang@redhat.com> wrote:
> On Fri, Apr 15, 2022 at 10:23 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> >
> > On Thu, 14 Apr 2022 17:30:02 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > > On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> > > >
> > > > On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > > > >
> > > > > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > > > > This patch implements the resize function of the rx, tx queues.
> > > > > > Based on this function, it is possible to modify the ring num of the
> > > > > > queue.
> > > > > >
> > > > > > There may be an exception during the resize process, the resize may
> > > > > > fail, or the vq can no longer be used. Either way, we must execute
> > > > > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > > > > must be called after calling napi_disable.
> > > > > >
> > > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > > > ---
> > > > > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > > > > >   1 file changed, 81 insertions(+)
> > > > > >
> > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > > > index b8bf00525177..ba6859f305f7 100644
> > > > > > --- a/drivers/net/virtio_net.c
> > > > > > +++ b/drivers/net/virtio_net.c
> > > > > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > > > > >     char padding[4];
> > > > > >   };
> > > > > >
> > > > > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > > > +
> > > > > >   static bool is_xdp_frame(void *ptr)
> > > > > >   {
> > > > > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > > > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > > > > >   {
> > > > > >     napi_enable(napi);
> > > > > >
> > > > > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > > > > +    * be protected by napi. However, the protection of napi is only enabled
> > > > > > +    * during the operation, and the protection of napi will end after the
> > > > > > +    * operation is completed. If re-enable fails during the process, vq
> > > > > > +    * will remain unavailable with reset state.
> > > > > > +    */
> > > > > > +   if (vq->reset)
> > > > > > +           return;
> > > > >
> > > > >
> > > > > I don't get when could we hit this condition.
> > > >
> > > >
> > > > In patch 23, the code to implement re-enable vq is as follows:
> > > >
> > > > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > > > +{
> > > > +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > > +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > > +       struct virtio_pci_vq_info *info;
> > > > +       unsigned long flags, index;
> > > > +       int err;
> > > > +
> > > > +       if (!vq->reset)
> > > > +               return -EBUSY;
> > > > +
> > > > +       index = vq->index;
> > > > +       info = vp_dev->vqs[index];
> > > > +
> > > > +       /* check queue reset status */
> > > > +       if (vp_modern_get_queue_reset(mdev, index) != 1)
> > > > +               return -EBUSY;
> > > > +
> > > > +       err = vp_active_vq(vq, info->msix_vector);
> > > > +       if (err)
> > > > +               return err;
> > > > +
> > > > +       if (vq->callback) {
> > > > +               spin_lock_irqsave(&vp_dev->lock, flags);
> > > > +               list_add(&info->node, &vp_dev->virtqueues);
> > > > +               spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > > +       } else {
> > > > +               INIT_LIST_HEAD(&info->node);
> > > > +       }
> > > > +
> > > > +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > > > +
> > > > +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > > > +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> > > > +
> > > > +       vq->reset = false;
> > > > +
> > > > +       return 0;
> > > > +}
> > > >
> > > >
> > > > There are three situations where an error will be returned. These are the
> > > > situations I want to handle.
> > >
> > > Right, but it looks harmless if we just schedule the NAPI without the check.
> >
> > Yes.
> >
> > > >
> > > > But I'm rethinking the question, and I feel like you're right, although the
> > > > hardware setup may fail. We can no longer sync with the hardware. But using it
> > > > as a normal vq doesn't have any problems.
> > >
> > > Note that we should make sure the buggy(malicous) device won't crash
> > > the codes by changing the queue_reset value at its will.
> >
> > I will keep an eye on this situation.
> >
> > >
> > > >
> > > > >
> > > > >
> > > > > > +
> > > > > >     /* If all buffers were filled by other side before we napi_enabled, we
> > > > > >      * won't get another interrupt, so process any outstanding packets now.
> > > > > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > > > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > > > > >             struct receive_queue *rq = &vi->rq[i];
> > > > > >
> > > > > >             napi_disable(&rq->napi);
> > > > > > +
> > > > > > +           /* Check if vq is in reset state. See more in
> > > > > > +            * virtnet_napi_enable()
> > > > > > +            */
> > > > > > +           if (rq->vq->reset) {
> > > > > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > > +                   continue;
> > > > > > +           }
> > > > >
> > > > >
> > > > > Can we do something similar in virtnet_close() by canceling the work?
> > > >
> > > > I think there is no need to cancel the work here, because napi_disable will wait
> > > > for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
> > > > vq, this logic can be removed.
> > >
> > > Actually I meant the part of virtnet_rx_resize().
> > >
> > > If we don't synchronize with the refill work, it might enable NAPI unexpectedly?
> >
> > I don't think this situation will be encountered, because napi_disable is
> > mutually exclusive, so there will be no unexpected napi enable.
> >
> > Is there something I misunderstood?
>
> So in virtnet_rx_resize() we do:
>
> napi_disable()
> ...
> resize()
> ...
> napi_enalbe()
>
> How can we guarantee that the work is not run after the napi_disable()?


I think you're talking about a situation like this:

virtnet_rx_resize          refill work
-----------------------------------------------------------
 napi_disable()
 ...                       napi_disable()
 resize()                      ...
                           napi_enable()
 ...
 napi_enalbe()


But in fact:

virtnet_rx_resize          refill work
-----------------------------------------------------------
 napi_disable()
 ...                       napi_disable() <----[0]
 resize()                       |
 ...                            |
 napi_enalbe()                  |
                           napi_disable() <---- [1] here success
                           napi_enable()

Because virtnet_rx_resize() has already executed napi_disable(), napi_disalbe()
of [0] will wait until [1] to complete.

I'm not sure if my understanding is correct.

Thanks.

>
> Thanks
>
> >
> > Thanks.
> >
> > >
> > > Thanks
> > >
> > > >
> > > >
> > > > >
> > > > >
> > > > > > +
> > > > > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > > > > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > >
> > > > > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > > > > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > > > > >             return;
> > > > > >
> > > > > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > > > > +   if (sq->vq->reset)
> > > > > > +           return;
> > > > >
> > > > >
> > > > > We've disabled TX napi, any chance we can still hit this?
> > > >
> > > > Same as above.
> > > >
> > > > >
> > > > >
> > > > > > +
> > > > > >     if (__netif_tx_trylock(txq)) {
> > > > > >             do {
> > > > > >                     virtqueue_disable_cb(sq->vq);
> > > > > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > > > > >     return NETDEV_TX_OK;
> > > > > >   }
> > > > > >
> > > > > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > > > > +                        struct receive_queue *rq, u32 ring_num)
> > > > > > +{
> > > > > > +   int err;
> > > > > > +
> > > > > > +   napi_disable(&rq->napi);
> > > > > > +
> > > > > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > > > > +   if (err)
> > > > > > +           goto err;
> > > > > > +
> > > > > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > > > > +           schedule_delayed_work(&vi->refill, 0);
> > > > > > +
> > > > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > > +   return 0;
> > > > > > +
> > > > > > +err:
> > > > > > +   netdev_err(vi->dev,
> > > > > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > > > > +              rq - vi->rq, err);
> > > > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > > +   return err;
> > > > > > +}
> > > > > > +
> > > > > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > > > > +                        struct send_queue *sq, u32 ring_num)
> > > > > > +{
> > > > > > +   struct netdev_queue *txq;
> > > > > > +   int err, qindex;
> > > > > > +
> > > > > > +   qindex = sq - vi->sq;
> > > > > > +
> > > > > > +   virtnet_napi_tx_disable(&sq->napi);
> > > > > > +
> > > > > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > > > > +   __netif_tx_lock_bh(txq);
> > > > > > +   netif_stop_subqueue(vi->dev, qindex);
> > > > > > +   __netif_tx_unlock_bh(txq);
> > > > > > +
> > > > > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > > > > +   if (err)
> > > > > > +           goto err;
> > > > > > +
> > > > > > +   netif_start_subqueue(vi->dev, qindex);
> > > > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > > > +   return 0;
> > > > > > +
> > > > > > +err:
> > > > >
> > > > >
> > > > > I guess we can still start the queue in this case? (Since we don't
> > > > > change the queue if resize fails).
> > > >
> > > > Yes, you are right.
> > > >
> > > > Thanks.
> > > >
> > > > >
> > > > >
> > > > > > +   netdev_err(vi->dev,
> > > > > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > > > > +              sq - vi->sq, err);
> > > > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > > > +   return err;
> > > > > > +}
> > > > > > +
> > > > > >   /*
> > > > > >    * Send command via the control virtqueue and check status.  Commands
> > > > > >    * supported by the hypervisor, as indicated by feature bits, should
> > > > >
> > > >
> > >
> >
>

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Fri, 15 Apr 2022 13:53:54 +0800, Jason Wang <jasowang@redhat.com> wrote:
> On Fri, Apr 15, 2022 at 10:23 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> >
> > On Thu, 14 Apr 2022 17:30:02 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > > On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> > > >
> > > > On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > > > >
> > > > > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > > > > This patch implements the resize function of the rx, tx queues.
> > > > > > Based on this function, it is possible to modify the ring num of the
> > > > > > queue.
> > > > > >
> > > > > > There may be an exception during the resize process, the resize may
> > > > > > fail, or the vq can no longer be used. Either way, we must execute
> > > > > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > > > > must be called after calling napi_disable.
> > > > > >
> > > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > > > ---
> > > > > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > > > > >   1 file changed, 81 insertions(+)
> > > > > >
> > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > > > index b8bf00525177..ba6859f305f7 100644
> > > > > > --- a/drivers/net/virtio_net.c
> > > > > > +++ b/drivers/net/virtio_net.c
> > > > > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > > > > >     char padding[4];
> > > > > >   };
> > > > > >
> > > > > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > > > +
> > > > > >   static bool is_xdp_frame(void *ptr)
> > > > > >   {
> > > > > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > > > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > > > > >   {
> > > > > >     napi_enable(napi);
> > > > > >
> > > > > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > > > > +    * be protected by napi. However, the protection of napi is only enabled
> > > > > > +    * during the operation, and the protection of napi will end after the
> > > > > > +    * operation is completed. If re-enable fails during the process, vq
> > > > > > +    * will remain unavailable with reset state.
> > > > > > +    */
> > > > > > +   if (vq->reset)
> > > > > > +           return;
> > > > >
> > > > >
> > > > > I don't get when could we hit this condition.
> > > >
> > > >
> > > > In patch 23, the code to implement re-enable vq is as follows:
> > > >
> > > > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > > > +{
> > > > +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > > +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > > +       struct virtio_pci_vq_info *info;
> > > > +       unsigned long flags, index;
> > > > +       int err;
> > > > +
> > > > +       if (!vq->reset)
> > > > +               return -EBUSY;
> > > > +
> > > > +       index = vq->index;
> > > > +       info = vp_dev->vqs[index];
> > > > +
> > > > +       /* check queue reset status */
> > > > +       if (vp_modern_get_queue_reset(mdev, index) != 1)
> > > > +               return -EBUSY;
> > > > +
> > > > +       err = vp_active_vq(vq, info->msix_vector);
> > > > +       if (err)
> > > > +               return err;
> > > > +
> > > > +       if (vq->callback) {
> > > > +               spin_lock_irqsave(&vp_dev->lock, flags);
> > > > +               list_add(&info->node, &vp_dev->virtqueues);
> > > > +               spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > > +       } else {
> > > > +               INIT_LIST_HEAD(&info->node);
> > > > +       }
> > > > +
> > > > +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > > > +
> > > > +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > > > +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> > > > +
> > > > +       vq->reset = false;
> > > > +
> > > > +       return 0;
> > > > +}
> > > >
> > > >
> > > > There are three situations where an error will be returned. These are the
> > > > situations I want to handle.
> > >
> > > Right, but it looks harmless if we just schedule the NAPI without the check.
> >
> > Yes.
> >
> > > >
> > > > But I'm rethinking the question, and I feel like you're right, although the
> > > > hardware setup may fail. We can no longer sync with the hardware. But using it
> > > > as a normal vq doesn't have any problems.
> > >
> > > Note that we should make sure the buggy(malicous) device won't crash
> > > the codes by changing the queue_reset value at its will.
> >
> > I will keep an eye on this situation.
> >
> > >
> > > >
> > > > >
> > > > >
> > > > > > +
> > > > > >     /* If all buffers were filled by other side before we napi_enabled, we
> > > > > >      * won't get another interrupt, so process any outstanding packets now.
> > > > > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > > > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > > > > >             struct receive_queue *rq = &vi->rq[i];
> > > > > >
> > > > > >             napi_disable(&rq->napi);
> > > > > > +
> > > > > > +           /* Check if vq is in reset state. See more in
> > > > > > +            * virtnet_napi_enable()
> > > > > > +            */
> > > > > > +           if (rq->vq->reset) {
> > > > > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > > +                   continue;
> > > > > > +           }
> > > > >
> > > > >
> > > > > Can we do something similar in virtnet_close() by canceling the work?
> > > >
> > > > I think there is no need to cancel the work here, because napi_disable will wait
> > > > for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
> > > > vq, this logic can be removed.
> > >
> > > Actually I meant the part of virtnet_rx_resize().
> > >
> > > If we don't synchronize with the refill work, it might enable NAPI unexpectedly?
> >
> > I don't think this situation will be encountered, because napi_disable is
> > mutually exclusive, so there will be no unexpected napi enable.
> >
> > Is there something I misunderstood?
>
> So in virtnet_rx_resize() we do:
>
> napi_disable()
> ...
> resize()
> ...
> napi_enalbe()
>
> How can we guarantee that the work is not run after the napi_disable()?


I think you're talking about a situation like this:

virtnet_rx_resize          refill work
-----------------------------------------------------------
 napi_disable()
 ...                       napi_disable()
 resize()                      ...
                           napi_enable()
 ...
 napi_enalbe()


But in fact:

virtnet_rx_resize          refill work
-----------------------------------------------------------
 napi_disable()
 ...                       napi_disable() <----[0]
 resize()                       |
 ...                            |
 napi_enalbe()                  |
                           napi_disable() <---- [1] here success
                           napi_enable()

Because virtnet_rx_resize() has already executed napi_disable(), napi_disalbe()
of [0] will wait until [1] to complete.

I'm not sure if my understanding is correct.

Thanks.

>
> Thanks
>
> >
> > Thanks.
> >
> > >
> > > Thanks
> > >
> > > >
> > > >
> > > > >
> > > > >
> > > > > > +
> > > > > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > > > > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > >
> > > > > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > > > > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > > > > >             return;
> > > > > >
> > > > > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > > > > +   if (sq->vq->reset)
> > > > > > +           return;
> > > > >
> > > > >
> > > > > We've disabled TX napi, any chance we can still hit this?
> > > >
> > > > Same as above.
> > > >
> > > > >
> > > > >
> > > > > > +
> > > > > >     if (__netif_tx_trylock(txq)) {
> > > > > >             do {
> > > > > >                     virtqueue_disable_cb(sq->vq);
> > > > > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > > > > >     return NETDEV_TX_OK;
> > > > > >   }
> > > > > >
> > > > > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > > > > +                        struct receive_queue *rq, u32 ring_num)
> > > > > > +{
> > > > > > +   int err;
> > > > > > +
> > > > > > +   napi_disable(&rq->napi);
> > > > > > +
> > > > > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > > > > +   if (err)
> > > > > > +           goto err;
> > > > > > +
> > > > > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > > > > +           schedule_delayed_work(&vi->refill, 0);
> > > > > > +
> > > > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > > +   return 0;
> > > > > > +
> > > > > > +err:
> > > > > > +   netdev_err(vi->dev,
> > > > > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > > > > +              rq - vi->rq, err);
> > > > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > > +   return err;
> > > > > > +}
> > > > > > +
> > > > > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > > > > +                        struct send_queue *sq, u32 ring_num)
> > > > > > +{
> > > > > > +   struct netdev_queue *txq;
> > > > > > +   int err, qindex;
> > > > > > +
> > > > > > +   qindex = sq - vi->sq;
> > > > > > +
> > > > > > +   virtnet_napi_tx_disable(&sq->napi);
> > > > > > +
> > > > > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > > > > +   __netif_tx_lock_bh(txq);
> > > > > > +   netif_stop_subqueue(vi->dev, qindex);
> > > > > > +   __netif_tx_unlock_bh(txq);
> > > > > > +
> > > > > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > > > > +   if (err)
> > > > > > +           goto err;
> > > > > > +
> > > > > > +   netif_start_subqueue(vi->dev, qindex);
> > > > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > > > +   return 0;
> > > > > > +
> > > > > > +err:
> > > > >
> > > > >
> > > > > I guess we can still start the queue in this case? (Since we don't
> > > > > change the queue if resize fails).
> > > >
> > > > Yes, you are right.
> > > >
> > > > Thanks.
> > > >
> > > > >
> > > > >
> > > > > > +   netdev_err(vi->dev,
> > > > > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > > > > +              sq - vi->sq, err);
> > > > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > > > +   return err;
> > > > > > +}
> > > > > > +
> > > > > >   /*
> > > > > >    * Send command via the control virtqueue and check status.  Commands
> > > > > >    * supported by the hypervisor, as indicated by feature bits, should
> > > > >
> > > >
> > >
> >
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Fri, 15 Apr 2022 13:53:54 +0800, Jason Wang <jasowang@redhat.com> wrote:
> On Fri, Apr 15, 2022 at 10:23 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> >
> > On Thu, 14 Apr 2022 17:30:02 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > > On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> > > >
> > > > On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > > > >
> > > > > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > > > > This patch implements the resize function of the rx, tx queues.
> > > > > > Based on this function, it is possible to modify the ring num of the
> > > > > > queue.
> > > > > >
> > > > > > There may be an exception during the resize process, the resize may
> > > > > > fail, or the vq can no longer be used. Either way, we must execute
> > > > > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > > > > must be called after calling napi_disable.
> > > > > >
> > > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > > > ---
> > > > > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > > > > >   1 file changed, 81 insertions(+)
> > > > > >
> > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > > > index b8bf00525177..ba6859f305f7 100644
> > > > > > --- a/drivers/net/virtio_net.c
> > > > > > +++ b/drivers/net/virtio_net.c
> > > > > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > > > > >     char padding[4];
> > > > > >   };
> > > > > >
> > > > > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > > > +
> > > > > >   static bool is_xdp_frame(void *ptr)
> > > > > >   {
> > > > > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > > > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > > > > >   {
> > > > > >     napi_enable(napi);
> > > > > >
> > > > > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > > > > +    * be protected by napi. However, the protection of napi is only enabled
> > > > > > +    * during the operation, and the protection of napi will end after the
> > > > > > +    * operation is completed. If re-enable fails during the process, vq
> > > > > > +    * will remain unavailable with reset state.
> > > > > > +    */
> > > > > > +   if (vq->reset)
> > > > > > +           return;
> > > > >
> > > > >
> > > > > I don't get when could we hit this condition.
> > > >
> > > >
> > > > In patch 23, the code to implement re-enable vq is as follows:
> > > >
> > > > +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
> > > > +{
> > > > +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
> > > > +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
> > > > +       struct virtio_pci_vq_info *info;
> > > > +       unsigned long flags, index;
> > > > +       int err;
> > > > +
> > > > +       if (!vq->reset)
> > > > +               return -EBUSY;
> > > > +
> > > > +       index = vq->index;
> > > > +       info = vp_dev->vqs[index];
> > > > +
> > > > +       /* check queue reset status */
> > > > +       if (vp_modern_get_queue_reset(mdev, index) != 1)
> > > > +               return -EBUSY;
> > > > +
> > > > +       err = vp_active_vq(vq, info->msix_vector);
> > > > +       if (err)
> > > > +               return err;
> > > > +
> > > > +       if (vq->callback) {
> > > > +               spin_lock_irqsave(&vp_dev->lock, flags);
> > > > +               list_add(&info->node, &vp_dev->virtqueues);
> > > > +               spin_unlock_irqrestore(&vp_dev->lock, flags);
> > > > +       } else {
> > > > +               INIT_LIST_HEAD(&info->node);
> > > > +       }
> > > > +
> > > > +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
> > > > +
> > > > +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
> > > > +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
> > > > +
> > > > +       vq->reset = false;
> > > > +
> > > > +       return 0;
> > > > +}
> > > >
> > > >
> > > > There are three situations where an error will be returned. These are the
> > > > situations I want to handle.
> > >
> > > Right, but it looks harmless if we just schedule the NAPI without the check.
> >
> > Yes.
> >
> > > >
> > > > But I'm rethinking the question, and I feel like you're right, although the
> > > > hardware setup may fail. We can no longer sync with the hardware. But using it
> > > > as a normal vq doesn't have any problems.
> > >
> > > Note that we should make sure the buggy(malicous) device won't crash
> > > the codes by changing the queue_reset value at its will.
> >
> > I will keep an eye on this situation.
> >
> > >
> > > >
> > > > >
> > > > >
> > > > > > +
> > > > > >     /* If all buffers were filled by other side before we napi_enabled, we
> > > > > >      * won't get another interrupt, so process any outstanding packets now.
> > > > > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > > > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > > > > >             struct receive_queue *rq = &vi->rq[i];
> > > > > >
> > > > > >             napi_disable(&rq->napi);
> > > > > > +
> > > > > > +           /* Check if vq is in reset state. See more in
> > > > > > +            * virtnet_napi_enable()
> > > > > > +            */
> > > > > > +           if (rq->vq->reset) {
> > > > > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > > +                   continue;
> > > > > > +           }
> > > > >
> > > > >
> > > > > Can we do something similar in virtnet_close() by canceling the work?
> > > >
> > > > I think there is no need to cancel the work here, because napi_disable will wait
> > > > for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
> > > > vq, this logic can be removed.
> > >
> > > Actually I meant the part of virtnet_rx_resize().
> > >
> > > If we don't synchronize with the refill work, it might enable NAPI unexpectedly?
> >
> > I don't think this situation will be encountered, because napi_disable is
> > mutually exclusive, so there will be no unexpected napi enable.
> >
> > Is there something I misunderstood?
>
> So in virtnet_rx_resize() we do:
>
> napi_disable()
> ...
> resize()
> ...
> napi_enalbe()
>
> How can we guarantee that the work is not run after the napi_disable()?


I think you're talking about a situation like this:

virtnet_rx_resize          refill work
-----------------------------------------------------------
 napi_disable()
 ...                       napi_disable()
 resize()                      ...
                           napi_enable()
 ...
 napi_enalbe()


But in fact:

virtnet_rx_resize          refill work
-----------------------------------------------------------
 napi_disable()
 ...                       napi_disable() <----[0]
 resize()                       |
 ...                            |
 napi_enalbe()                  |
                           napi_disable() <---- [1] here success
                           napi_enable()

Because virtnet_rx_resize() has already executed napi_disable(), napi_disalbe()
of [0] will wait until [1] to complete.

I'm not sure if my understanding is correct.

Thanks.

>
> Thanks
>
> >
> > Thanks.
> >
> > >
> > > Thanks
> > >
> > > >
> > > >
> > > > >
> > > > >
> > > > > > +
> > > > > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > > > > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > >
> > > > > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > > > > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > > > > >             return;
> > > > > >
> > > > > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > > > > +   if (sq->vq->reset)
> > > > > > +           return;
> > > > >
> > > > >
> > > > > We've disabled TX napi, any chance we can still hit this?
> > > >
> > > > Same as above.
> > > >
> > > > >
> > > > >
> > > > > > +
> > > > > >     if (__netif_tx_trylock(txq)) {
> > > > > >             do {
> > > > > >                     virtqueue_disable_cb(sq->vq);
> > > > > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > > > > >     return NETDEV_TX_OK;
> > > > > >   }
> > > > > >
> > > > > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > > > > +                        struct receive_queue *rq, u32 ring_num)
> > > > > > +{
> > > > > > +   int err;
> > > > > > +
> > > > > > +   napi_disable(&rq->napi);
> > > > > > +
> > > > > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > > > > +   if (err)
> > > > > > +           goto err;
> > > > > > +
> > > > > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > > > > +           schedule_delayed_work(&vi->refill, 0);
> > > > > > +
> > > > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > > +   return 0;
> > > > > > +
> > > > > > +err:
> > > > > > +   netdev_err(vi->dev,
> > > > > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > > > > +              rq - vi->rq, err);
> > > > > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > > > +   return err;
> > > > > > +}
> > > > > > +
> > > > > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > > > > +                        struct send_queue *sq, u32 ring_num)
> > > > > > +{
> > > > > > +   struct netdev_queue *txq;
> > > > > > +   int err, qindex;
> > > > > > +
> > > > > > +   qindex = sq - vi->sq;
> > > > > > +
> > > > > > +   virtnet_napi_tx_disable(&sq->napi);
> > > > > > +
> > > > > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > > > > +   __netif_tx_lock_bh(txq);
> > > > > > +   netif_stop_subqueue(vi->dev, qindex);
> > > > > > +   __netif_tx_unlock_bh(txq);
> > > > > > +
> > > > > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > > > > +   if (err)
> > > > > > +           goto err;
> > > > > > +
> > > > > > +   netif_start_subqueue(vi->dev, qindex);
> > > > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > > > +   return 0;
> > > > > > +
> > > > > > +err:
> > > > >
> > > > >
> > > > > I guess we can still start the queue in this case? (Since we don't
> > > > > change the queue if resize fails).
> > > >
> > > > Yes, you are right.
> > > >
> > > > Thanks.
> > > >
> > > > >
> > > > >
> > > > > > +   netdev_err(vi->dev,
> > > > > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > > > > +              sq - vi->sq, err);
> > > > > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > > > > +   return err;
> > > > > > +}
> > > > > > +
> > > > > >   /*
> > > > > >    * Send command via the control virtqueue and check status.  Commands
> > > > > >    * supported by the hypervisor, as indicated by feature bits, should
> > > > >
> > > >
> > >
> >
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > This patch implements the resize function of the rx, tx queues.
> > Based on this function, it is possible to modify the ring num of the
> > queue.
> >
> > There may be an exception during the resize process, the resize may
> > fail, or the vq can no longer be used. Either way, we must execute
> > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > must be called after calling napi_disable.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 81 insertions(+)
> >
> > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > index b8bf00525177..ba6859f305f7 100644
> > --- a/drivers/net/virtio_net.c
> > +++ b/drivers/net/virtio_net.c
> > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> >   	char padding[4];
> >   };
> >
> > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > +
> >   static bool is_xdp_frame(void *ptr)
> >   {
> >   	return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> >   {
> >   	napi_enable(napi);
> >
> > +	/* Check if vq is in reset state. The normal reset/resize process will
> > +	 * be protected by napi. However, the protection of napi is only enabled
> > +	 * during the operation, and the protection of napi will end after the
> > +	 * operation is completed. If re-enable fails during the process, vq
> > +	 * will remain unavailable with reset state.
> > +	 */
> > +	if (vq->reset)
> > +		return;
>
>
> I don't get when could we hit this condition.
>
>
> > +
> >   	/* If all buffers were filled by other side before we napi_enabled, we
> >   	 * won't get another interrupt, so process any outstanding packets now.
> >   	 * Call local_bh_enable after to trigger softIRQ processing.
> > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> >   		struct receive_queue *rq = &vi->rq[i];
> >
> >   		napi_disable(&rq->napi);
> > +
> > +		/* Check if vq is in reset state. See more in
> > +		 * virtnet_napi_enable()
> > +		 */
> > +		if (rq->vq->reset) {
> > +			virtnet_napi_enable(rq->vq, &rq->napi);
> > +			continue;
> > +		}
>
>
> Can we do something similar in virtnet_close() by canceling the work?
>
>
> > +
> >   		still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> >   		virtnet_napi_enable(rq->vq, &rq->napi);
> >
> > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> >   	if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> >   		return;
> >
> > +	/* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > +	if (sq->vq->reset)
> > +		return;
>
>
> We've disabled TX napi, any chance we can still hit this?


static int virtnet_poll(struct napi_struct *napi, int budget)
{
	struct receive_queue *rq =
		container_of(napi, struct receive_queue, napi);
	struct virtnet_info *vi = rq->vq->vdev->priv;
	struct send_queue *sq;
	unsigned int received;
	unsigned int xdp_xmit = 0;

	virtnet_poll_cleantx(rq);
...
}

This is called by rx poll. Although it is the logic of tx, it is not driven by
tx napi, but is called in rx poll.

Thanks.


>
>
> > +
> >   	if (__netif_tx_trylock(txq)) {
> >   		do {
> >   			virtqueue_disable_cb(sq->vq);
> > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> >   	return NETDEV_TX_OK;
> >   }
> >
> > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > +			     struct receive_queue *rq, u32 ring_num)
> > +{
> > +	int err;
> > +
> > +	napi_disable(&rq->napi);
> > +
> > +	err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > +	if (err)
> > +		goto err;
> > +
> > +	if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > +		schedule_delayed_work(&vi->refill, 0);
> > +
> > +	virtnet_napi_enable(rq->vq, &rq->napi);
> > +	return 0;
> > +
> > +err:
> > +	netdev_err(vi->dev,
> > +		   "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > +		   rq - vi->rq, err);
> > +	virtnet_napi_enable(rq->vq, &rq->napi);
> > +	return err;
> > +}
> > +
> > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > +			     struct send_queue *sq, u32 ring_num)
> > +{
> > +	struct netdev_queue *txq;
> > +	int err, qindex;
> > +
> > +	qindex = sq - vi->sq;
> > +
> > +	virtnet_napi_tx_disable(&sq->napi);
> > +
> > +	txq = netdev_get_tx_queue(vi->dev, qindex);
> > +	__netif_tx_lock_bh(txq);
> > +	netif_stop_subqueue(vi->dev, qindex);
> > +	__netif_tx_unlock_bh(txq);
> > +
> > +	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > +	if (err)
> > +		goto err;
> > +
> > +	netif_start_subqueue(vi->dev, qindex);
> > +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > +	return 0;
> > +
> > +err:
>
>
> I guess we can still start the queue in this case? (Since we don't
> change the queue if resize fails).
>
>
> > +	netdev_err(vi->dev,
> > +		   "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > +		   sq - vi->sq, err);
> > +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > +	return err;
> > +}
> > +
> >   /*
> >    * Send command via the control virtqueue and check status.  Commands
> >    * supported by the hypervisor, as indicated by feature bits, should
>

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > This patch implements the resize function of the rx, tx queues.
> > Based on this function, it is possible to modify the ring num of the
> > queue.
> >
> > There may be an exception during the resize process, the resize may
> > fail, or the vq can no longer be used. Either way, we must execute
> > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > must be called after calling napi_disable.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 81 insertions(+)
> >
> > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > index b8bf00525177..ba6859f305f7 100644
> > --- a/drivers/net/virtio_net.c
> > +++ b/drivers/net/virtio_net.c
> > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> >   	char padding[4];
> >   };
> >
> > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > +
> >   static bool is_xdp_frame(void *ptr)
> >   {
> >   	return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> >   {
> >   	napi_enable(napi);
> >
> > +	/* Check if vq is in reset state. The normal reset/resize process will
> > +	 * be protected by napi. However, the protection of napi is only enabled
> > +	 * during the operation, and the protection of napi will end after the
> > +	 * operation is completed. If re-enable fails during the process, vq
> > +	 * will remain unavailable with reset state.
> > +	 */
> > +	if (vq->reset)
> > +		return;
>
>
> I don't get when could we hit this condition.
>
>
> > +
> >   	/* If all buffers were filled by other side before we napi_enabled, we
> >   	 * won't get another interrupt, so process any outstanding packets now.
> >   	 * Call local_bh_enable after to trigger softIRQ processing.
> > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> >   		struct receive_queue *rq = &vi->rq[i];
> >
> >   		napi_disable(&rq->napi);
> > +
> > +		/* Check if vq is in reset state. See more in
> > +		 * virtnet_napi_enable()
> > +		 */
> > +		if (rq->vq->reset) {
> > +			virtnet_napi_enable(rq->vq, &rq->napi);
> > +			continue;
> > +		}
>
>
> Can we do something similar in virtnet_close() by canceling the work?
>
>
> > +
> >   		still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> >   		virtnet_napi_enable(rq->vq, &rq->napi);
> >
> > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> >   	if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> >   		return;
> >
> > +	/* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > +	if (sq->vq->reset)
> > +		return;
>
>
> We've disabled TX napi, any chance we can still hit this?


static int virtnet_poll(struct napi_struct *napi, int budget)
{
	struct receive_queue *rq =
		container_of(napi, struct receive_queue, napi);
	struct virtnet_info *vi = rq->vq->vdev->priv;
	struct send_queue *sq;
	unsigned int received;
	unsigned int xdp_xmit = 0;

	virtnet_poll_cleantx(rq);
...
}

This is called by rx poll. Although it is the logic of tx, it is not driven by
tx napi, but is called in rx poll.

Thanks.


>
>
> > +
> >   	if (__netif_tx_trylock(txq)) {
> >   		do {
> >   			virtqueue_disable_cb(sq->vq);
> > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> >   	return NETDEV_TX_OK;
> >   }
> >
> > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > +			     struct receive_queue *rq, u32 ring_num)
> > +{
> > +	int err;
> > +
> > +	napi_disable(&rq->napi);
> > +
> > +	err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > +	if (err)
> > +		goto err;
> > +
> > +	if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > +		schedule_delayed_work(&vi->refill, 0);
> > +
> > +	virtnet_napi_enable(rq->vq, &rq->napi);
> > +	return 0;
> > +
> > +err:
> > +	netdev_err(vi->dev,
> > +		   "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > +		   rq - vi->rq, err);
> > +	virtnet_napi_enable(rq->vq, &rq->napi);
> > +	return err;
> > +}
> > +
> > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > +			     struct send_queue *sq, u32 ring_num)
> > +{
> > +	struct netdev_queue *txq;
> > +	int err, qindex;
> > +
> > +	qindex = sq - vi->sq;
> > +
> > +	virtnet_napi_tx_disable(&sq->napi);
> > +
> > +	txq = netdev_get_tx_queue(vi->dev, qindex);
> > +	__netif_tx_lock_bh(txq);
> > +	netif_stop_subqueue(vi->dev, qindex);
> > +	__netif_tx_unlock_bh(txq);
> > +
> > +	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > +	if (err)
> > +		goto err;
> > +
> > +	netif_start_subqueue(vi->dev, qindex);
> > +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > +	return 0;
> > +
> > +err:
>
>
> I guess we can still start the queue in this case? (Since we don't
> change the queue if resize fails).
>
>
> > +	netdev_err(vi->dev,
> > +		   "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > +		   sq - vi->sq, err);
> > +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > +	return err;
> > +}
> > +
> >   /*
> >    * Send command via the control virtqueue and check status.  Commands
> >    * supported by the hypervisor, as indicated by feature bits, should
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > This patch implements the resize function of the rx, tx queues.
> > Based on this function, it is possible to modify the ring num of the
> > queue.
> >
> > There may be an exception during the resize process, the resize may
> > fail, or the vq can no longer be used. Either way, we must execute
> > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > must be called after calling napi_disable.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 81 insertions(+)
> >
> > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > index b8bf00525177..ba6859f305f7 100644
> > --- a/drivers/net/virtio_net.c
> > +++ b/drivers/net/virtio_net.c
> > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> >   	char padding[4];
> >   };
> >
> > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > +
> >   static bool is_xdp_frame(void *ptr)
> >   {
> >   	return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> >   {
> >   	napi_enable(napi);
> >
> > +	/* Check if vq is in reset state. The normal reset/resize process will
> > +	 * be protected by napi. However, the protection of napi is only enabled
> > +	 * during the operation, and the protection of napi will end after the
> > +	 * operation is completed. If re-enable fails during the process, vq
> > +	 * will remain unavailable with reset state.
> > +	 */
> > +	if (vq->reset)
> > +		return;
>
>
> I don't get when could we hit this condition.
>
>
> > +
> >   	/* If all buffers were filled by other side before we napi_enabled, we
> >   	 * won't get another interrupt, so process any outstanding packets now.
> >   	 * Call local_bh_enable after to trigger softIRQ processing.
> > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> >   		struct receive_queue *rq = &vi->rq[i];
> >
> >   		napi_disable(&rq->napi);
> > +
> > +		/* Check if vq is in reset state. See more in
> > +		 * virtnet_napi_enable()
> > +		 */
> > +		if (rq->vq->reset) {
> > +			virtnet_napi_enable(rq->vq, &rq->napi);
> > +			continue;
> > +		}
>
>
> Can we do something similar in virtnet_close() by canceling the work?
>
>
> > +
> >   		still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> >   		virtnet_napi_enable(rq->vq, &rq->napi);
> >
> > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> >   	if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> >   		return;
> >
> > +	/* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > +	if (sq->vq->reset)
> > +		return;
>
>
> We've disabled TX napi, any chance we can still hit this?


static int virtnet_poll(struct napi_struct *napi, int budget)
{
	struct receive_queue *rq =
		container_of(napi, struct receive_queue, napi);
	struct virtnet_info *vi = rq->vq->vdev->priv;
	struct send_queue *sq;
	unsigned int received;
	unsigned int xdp_xmit = 0;

	virtnet_poll_cleantx(rq);
...
}

This is called by rx poll. Although it is the logic of tx, it is not driven by
tx napi, but is called in rx poll.

Thanks.


>
>
> > +
> >   	if (__netif_tx_trylock(txq)) {
> >   		do {
> >   			virtqueue_disable_cb(sq->vq);
> > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> >   	return NETDEV_TX_OK;
> >   }
> >
> > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > +			     struct receive_queue *rq, u32 ring_num)
> > +{
> > +	int err;
> > +
> > +	napi_disable(&rq->napi);
> > +
> > +	err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > +	if (err)
> > +		goto err;
> > +
> > +	if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > +		schedule_delayed_work(&vi->refill, 0);
> > +
> > +	virtnet_napi_enable(rq->vq, &rq->napi);
> > +	return 0;
> > +
> > +err:
> > +	netdev_err(vi->dev,
> > +		   "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > +		   rq - vi->rq, err);
> > +	virtnet_napi_enable(rq->vq, &rq->napi);
> > +	return err;
> > +}
> > +
> > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > +			     struct send_queue *sq, u32 ring_num)
> > +{
> > +	struct netdev_queue *txq;
> > +	int err, qindex;
> > +
> > +	qindex = sq - vi->sq;
> > +
> > +	virtnet_napi_tx_disable(&sq->napi);
> > +
> > +	txq = netdev_get_tx_queue(vi->dev, qindex);
> > +	__netif_tx_lock_bh(txq);
> > +	netif_stop_subqueue(vi->dev, qindex);
> > +	__netif_tx_unlock_bh(txq);
> > +
> > +	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > +	if (err)
> > +		goto err;
> > +
> > +	netif_start_subqueue(vi->dev, qindex);
> > +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > +	return 0;
> > +
> > +err:
>
>
> I guess we can still start the queue in this case? (Since we don't
> change the queue if resize fails).
>
>
> > +	netdev_err(vi->dev,
> > +		   "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > +		   sq - vi->sq, err);
> > +	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > +	return err;
> > +}
> > +
> >   /*
> >    * Send command via the control virtqueue and check status.  Commands
> >    * supported by the hypervisor, as indicated by feature bits, should
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

On Mon, Apr 18, 2022 at 11:24 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > This patch implements the resize function of the rx, tx queues.
> > > Based on this function, it is possible to modify the ring num of the
> > > queue.
> > >
> > > There may be an exception during the resize process, the resize may
> > > fail, or the vq can no longer be used. Either way, we must execute
> > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > must be called after calling napi_disable.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > >   1 file changed, 81 insertions(+)
> > >
> > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > index b8bf00525177..ba6859f305f7 100644
> > > --- a/drivers/net/virtio_net.c
> > > +++ b/drivers/net/virtio_net.c
> > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > >     char padding[4];
> > >   };
> > >
> > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > +
> > >   static bool is_xdp_frame(void *ptr)
> > >   {
> > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > >   {
> > >     napi_enable(napi);
> > >
> > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > +    * be protected by napi. However, the protection of napi is only enabled
> > > +    * during the operation, and the protection of napi will end after the
> > > +    * operation is completed. If re-enable fails during the process, vq
> > > +    * will remain unavailable with reset state.
> > > +    */
> > > +   if (vq->reset)
> > > +           return;
> >
> >
> > I don't get when could we hit this condition.
> >
> >
> > > +
> > >     /* If all buffers were filled by other side before we napi_enabled, we
> > >      * won't get another interrupt, so process any outstanding packets now.
> > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > >             struct receive_queue *rq = &vi->rq[i];
> > >
> > >             napi_disable(&rq->napi);
> > > +
> > > +           /* Check if vq is in reset state. See more in
> > > +            * virtnet_napi_enable()
> > > +            */
> > > +           if (rq->vq->reset) {
> > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +                   continue;
> > > +           }
> >
> >
> > Can we do something similar in virtnet_close() by canceling the work?
> >
> >
> > > +
> > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > >
> > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > >             return;
> > >
> > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > +   if (sq->vq->reset)
> > > +           return;
> >
> >
> > We've disabled TX napi, any chance we can still hit this?
>
>
> static int virtnet_poll(struct napi_struct *napi, int budget)
> {
>         struct receive_queue *rq =
>                 container_of(napi, struct receive_queue, napi);
>         struct virtnet_info *vi = rq->vq->vdev->priv;
>         struct send_queue *sq;
>         unsigned int received;
>         unsigned int xdp_xmit = 0;
>
>         virtnet_poll_cleantx(rq);
> ...
> }
>
> This is called by rx poll. Although it is the logic of tx, it is not driven by
> tx napi, but is called in rx poll.

Ok, but we need guarantee the memory ordering in this case. Disable RX
napi could be a solution for this.

Thanks

>
> Thanks.
>
>
> >
> >
> > > +
> > >     if (__netif_tx_trylock(txq)) {
> > >             do {
> > >                     virtqueue_disable_cb(sq->vq);
> > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > >     return NETDEV_TX_OK;
> > >   }
> > >
> > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > +                        struct receive_queue *rq, u32 ring_num)
> > > +{
> > > +   int err;
> > > +
> > > +   napi_disable(&rq->napi);
> > > +
> > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > +   if (err)
> > > +           goto err;
> > > +
> > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > +           schedule_delayed_work(&vi->refill, 0);
> > > +
> > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +   return 0;
> > > +
> > > +err:
> > > +   netdev_err(vi->dev,
> > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > +              rq - vi->rq, err);
> > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +   return err;
> > > +}
> > > +
> > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > +                        struct send_queue *sq, u32 ring_num)
> > > +{
> > > +   struct netdev_queue *txq;
> > > +   int err, qindex;
> > > +
> > > +   qindex = sq - vi->sq;
> > > +
> > > +   virtnet_napi_tx_disable(&sq->napi);
> > > +
> > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > +   __netif_tx_lock_bh(txq);
> > > +   netif_stop_subqueue(vi->dev, qindex);
> > > +   __netif_tx_unlock_bh(txq);
> > > +
> > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > +   if (err)
> > > +           goto err;
> > > +
> > > +   netif_start_subqueue(vi->dev, qindex);
> > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > +   return 0;
> > > +
> > > +err:
> >
> >
> > I guess we can still start the queue in this case? (Since we don't
> > change the queue if resize fails).
> >
> >
> > > +   netdev_err(vi->dev,
> > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > +              sq - vi->sq, err);
> > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > +   return err;
> > > +}
> > > +
> > >   /*
> > >    * Send command via the control virtqueue and check status.  Commands
> > >    * supported by the hypervisor, as indicated by feature bits, should
> >
>

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

On Mon, Apr 18, 2022 at 11:24 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > This patch implements the resize function of the rx, tx queues.
> > > Based on this function, it is possible to modify the ring num of the
> > > queue.
> > >
> > > There may be an exception during the resize process, the resize may
> > > fail, or the vq can no longer be used. Either way, we must execute
> > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > must be called after calling napi_disable.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > >   1 file changed, 81 insertions(+)
> > >
> > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > index b8bf00525177..ba6859f305f7 100644
> > > --- a/drivers/net/virtio_net.c
> > > +++ b/drivers/net/virtio_net.c
> > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > >     char padding[4];
> > >   };
> > >
> > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > +
> > >   static bool is_xdp_frame(void *ptr)
> > >   {
> > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > >   {
> > >     napi_enable(napi);
> > >
> > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > +    * be protected by napi. However, the protection of napi is only enabled
> > > +    * during the operation, and the protection of napi will end after the
> > > +    * operation is completed. If re-enable fails during the process, vq
> > > +    * will remain unavailable with reset state.
> > > +    */
> > > +   if (vq->reset)
> > > +           return;
> >
> >
> > I don't get when could we hit this condition.
> >
> >
> > > +
> > >     /* If all buffers were filled by other side before we napi_enabled, we
> > >      * won't get another interrupt, so process any outstanding packets now.
> > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > >             struct receive_queue *rq = &vi->rq[i];
> > >
> > >             napi_disable(&rq->napi);
> > > +
> > > +           /* Check if vq is in reset state. See more in
> > > +            * virtnet_napi_enable()
> > > +            */
> > > +           if (rq->vq->reset) {
> > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +                   continue;
> > > +           }
> >
> >
> > Can we do something similar in virtnet_close() by canceling the work?
> >
> >
> > > +
> > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > >
> > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > >             return;
> > >
> > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > +   if (sq->vq->reset)
> > > +           return;
> >
> >
> > We've disabled TX napi, any chance we can still hit this?
>
>
> static int virtnet_poll(struct napi_struct *napi, int budget)
> {
>         struct receive_queue *rq =
>                 container_of(napi, struct receive_queue, napi);
>         struct virtnet_info *vi = rq->vq->vdev->priv;
>         struct send_queue *sq;
>         unsigned int received;
>         unsigned int xdp_xmit = 0;
>
>         virtnet_poll_cleantx(rq);
> ...
> }
>
> This is called by rx poll. Although it is the logic of tx, it is not driven by
> tx napi, but is called in rx poll.

Ok, but we need guarantee the memory ordering in this case. Disable RX
napi could be a solution for this.

Thanks

>
> Thanks.
>
>
> >
> >
> > > +
> > >     if (__netif_tx_trylock(txq)) {
> > >             do {
> > >                     virtqueue_disable_cb(sq->vq);
> > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > >     return NETDEV_TX_OK;
> > >   }
> > >
> > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > +                        struct receive_queue *rq, u32 ring_num)
> > > +{
> > > +   int err;
> > > +
> > > +   napi_disable(&rq->napi);
> > > +
> > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > +   if (err)
> > > +           goto err;
> > > +
> > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > +           schedule_delayed_work(&vi->refill, 0);
> > > +
> > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +   return 0;
> > > +
> > > +err:
> > > +   netdev_err(vi->dev,
> > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > +              rq - vi->rq, err);
> > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +   return err;
> > > +}
> > > +
> > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > +                        struct send_queue *sq, u32 ring_num)
> > > +{
> > > +   struct netdev_queue *txq;
> > > +   int err, qindex;
> > > +
> > > +   qindex = sq - vi->sq;
> > > +
> > > +   virtnet_napi_tx_disable(&sq->napi);
> > > +
> > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > +   __netif_tx_lock_bh(txq);
> > > +   netif_stop_subqueue(vi->dev, qindex);
> > > +   __netif_tx_unlock_bh(txq);
> > > +
> > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > +   if (err)
> > > +           goto err;
> > > +
> > > +   netif_start_subqueue(vi->dev, qindex);
> > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > +   return 0;
> > > +
> > > +err:
> >
> >
> > I guess we can still start the queue in this case? (Since we don't
> > change the queue if resize fails).
> >
> >
> > > +   netdev_err(vi->dev,
> > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > +              sq - vi->sq, err);
> > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > +   return err;
> > > +}
> > > +
> > >   /*
> > >    * Send command via the control virtqueue and check status.  Commands
> > >    * supported by the hypervisor, as indicated by feature bits, should
> >
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

On Mon, Apr 18, 2022 at 11:24 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > This patch implements the resize function of the rx, tx queues.
> > > Based on this function, it is possible to modify the ring num of the
> > > queue.
> > >
> > > There may be an exception during the resize process, the resize may
> > > fail, or the vq can no longer be used. Either way, we must execute
> > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > must be called after calling napi_disable.
> > >
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > ---
> > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > >   1 file changed, 81 insertions(+)
> > >
> > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > index b8bf00525177..ba6859f305f7 100644
> > > --- a/drivers/net/virtio_net.c
> > > +++ b/drivers/net/virtio_net.c
> > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > >     char padding[4];
> > >   };
> > >
> > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > +
> > >   static bool is_xdp_frame(void *ptr)
> > >   {
> > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > >   {
> > >     napi_enable(napi);
> > >
> > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > +    * be protected by napi. However, the protection of napi is only enabled
> > > +    * during the operation, and the protection of napi will end after the
> > > +    * operation is completed. If re-enable fails during the process, vq
> > > +    * will remain unavailable with reset state.
> > > +    */
> > > +   if (vq->reset)
> > > +           return;
> >
> >
> > I don't get when could we hit this condition.
> >
> >
> > > +
> > >     /* If all buffers were filled by other side before we napi_enabled, we
> > >      * won't get another interrupt, so process any outstanding packets now.
> > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > >             struct receive_queue *rq = &vi->rq[i];
> > >
> > >             napi_disable(&rq->napi);
> > > +
> > > +           /* Check if vq is in reset state. See more in
> > > +            * virtnet_napi_enable()
> > > +            */
> > > +           if (rq->vq->reset) {
> > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +                   continue;
> > > +           }
> >
> >
> > Can we do something similar in virtnet_close() by canceling the work?
> >
> >
> > > +
> > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > >
> > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > >             return;
> > >
> > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > +   if (sq->vq->reset)
> > > +           return;
> >
> >
> > We've disabled TX napi, any chance we can still hit this?
>
>
> static int virtnet_poll(struct napi_struct *napi, int budget)
> {
>         struct receive_queue *rq =
>                 container_of(napi, struct receive_queue, napi);
>         struct virtnet_info *vi = rq->vq->vdev->priv;
>         struct send_queue *sq;
>         unsigned int received;
>         unsigned int xdp_xmit = 0;
>
>         virtnet_poll_cleantx(rq);
> ...
> }
>
> This is called by rx poll. Although it is the logic of tx, it is not driven by
> tx napi, but is called in rx poll.

Ok, but we need guarantee the memory ordering in this case. Disable RX
napi could be a solution for this.

Thanks

>
> Thanks.
>
>
> >
> >
> > > +
> > >     if (__netif_tx_trylock(txq)) {
> > >             do {
> > >                     virtqueue_disable_cb(sq->vq);
> > > @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > >     return NETDEV_TX_OK;
> > >   }
> > >
> > > +static int virtnet_rx_resize(struct virtnet_info *vi,
> > > +                        struct receive_queue *rq, u32 ring_num)
> > > +{
> > > +   int err;
> > > +
> > > +   napi_disable(&rq->napi);
> > > +
> > > +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
> > > +   if (err)
> > > +           goto err;
> > > +
> > > +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
> > > +           schedule_delayed_work(&vi->refill, 0);
> > > +
> > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +   return 0;
> > > +
> > > +err:
> > > +   netdev_err(vi->dev,
> > > +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
> > > +              rq - vi->rq, err);
> > > +   virtnet_napi_enable(rq->vq, &rq->napi);
> > > +   return err;
> > > +}
> > > +
> > > +static int virtnet_tx_resize(struct virtnet_info *vi,
> > > +                        struct send_queue *sq, u32 ring_num)
> > > +{
> > > +   struct netdev_queue *txq;
> > > +   int err, qindex;
> > > +
> > > +   qindex = sq - vi->sq;
> > > +
> > > +   virtnet_napi_tx_disable(&sq->napi);
> > > +
> > > +   txq = netdev_get_tx_queue(vi->dev, qindex);
> > > +   __netif_tx_lock_bh(txq);
> > > +   netif_stop_subqueue(vi->dev, qindex);
> > > +   __netif_tx_unlock_bh(txq);
> > > +
> > > +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
> > > +   if (err)
> > > +           goto err;
> > > +
> > > +   netif_start_subqueue(vi->dev, qindex);
> > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > +   return 0;
> > > +
> > > +err:
> >
> >
> > I guess we can still start the queue in this case? (Since we don't
> > change the queue if resize fails).
> >
> >
> > > +   netdev_err(vi->dev,
> > > +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
> > > +              sq - vi->sq, err);
> > > +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
> > > +   return err;
> > > +}
> > > +
> > >   /*
> > >    * Send command via the control virtqueue and check status.  Commands
> > >    * supported by the hypervisor, as indicated by feature bits, should
> >
>


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

在 2022/4/15 17:17, Xuan Zhuo 写道:
> On Fri, 15 Apr 2022 13:53:54 +0800, Jason Wang <jasowang@redhat.com> wrote:
>> On Fri, Apr 15, 2022 at 10:23 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>>> On Thu, 14 Apr 2022 17:30:02 +0800, Jason Wang <jasowang@redhat.com> wrote:
>>>> On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>>>>> On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>>>>>> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
>>>>>>> This patch implements the resize function of the rx, tx queues.
>>>>>>> Based on this function, it is possible to modify the ring num of the
>>>>>>> queue.
>>>>>>>
>>>>>>> There may be an exception during the resize process, the resize may
>>>>>>> fail, or the vq can no longer be used. Either way, we must execute
>>>>>>> napi_enable(). Because napi_disable is similar to a lock, napi_enable
>>>>>>> must be called after calling napi_disable.
>>>>>>>
>>>>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>>>>> ---
>>>>>>>    drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
>>>>>>>    1 file changed, 81 insertions(+)
>>>>>>>
>>>>>>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
>>>>>>> index b8bf00525177..ba6859f305f7 100644
>>>>>>> --- a/drivers/net/virtio_net.c
>>>>>>> +++ b/drivers/net/virtio_net.c
>>>>>>> @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
>>>>>>>      char padding[4];
>>>>>>>    };
>>>>>>>
>>>>>>> +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
>>>>>>> +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
>>>>>>> +
>>>>>>>    static bool is_xdp_frame(void *ptr)
>>>>>>>    {
>>>>>>>      return (unsigned long)ptr & VIRTIO_XDP_FLAG;
>>>>>>> @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
>>>>>>>    {
>>>>>>>      napi_enable(napi);
>>>>>>>
>>>>>>> +   /* Check if vq is in reset state. The normal reset/resize process will
>>>>>>> +    * be protected by napi. However, the protection of napi is only enabled
>>>>>>> +    * during the operation, and the protection of napi will end after the
>>>>>>> +    * operation is completed. If re-enable fails during the process, vq
>>>>>>> +    * will remain unavailable with reset state.
>>>>>>> +    */
>>>>>>> +   if (vq->reset)
>>>>>>> +           return;
>>>>>>
>>>>>> I don't get when could we hit this condition.
>>>>>
>>>>> In patch 23, the code to implement re-enable vq is as follows:
>>>>>
>>>>> +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
>>>>> +{
>>>>> +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
>>>>> +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
>>>>> +       struct virtio_pci_vq_info *info;
>>>>> +       unsigned long flags, index;
>>>>> +       int err;
>>>>> +
>>>>> +       if (!vq->reset)
>>>>> +               return -EBUSY;
>>>>> +
>>>>> +       index = vq->index;
>>>>> +       info = vp_dev->vqs[index];
>>>>> +
>>>>> +       /* check queue reset status */
>>>>> +       if (vp_modern_get_queue_reset(mdev, index) != 1)
>>>>> +               return -EBUSY;
>>>>> +
>>>>> +       err = vp_active_vq(vq, info->msix_vector);
>>>>> +       if (err)
>>>>> +               return err;
>>>>> +
>>>>> +       if (vq->callback) {
>>>>> +               spin_lock_irqsave(&vp_dev->lock, flags);
>>>>> +               list_add(&info->node, &vp_dev->virtqueues);
>>>>> +               spin_unlock_irqrestore(&vp_dev->lock, flags);
>>>>> +       } else {
>>>>> +               INIT_LIST_HEAD(&info->node);
>>>>> +       }
>>>>> +
>>>>> +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
>>>>> +
>>>>> +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
>>>>> +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>>>>> +
>>>>> +       vq->reset = false;
>>>>> +
>>>>> +       return 0;
>>>>> +}
>>>>>
>>>>>
>>>>> There are three situations where an error will be returned. These are the
>>>>> situations I want to handle.
>>>> Right, but it looks harmless if we just schedule the NAPI without the check.
>>> Yes.
>>>
>>>>> But I'm rethinking the question, and I feel like you're right, although the
>>>>> hardware setup may fail. We can no longer sync with the hardware. But using it
>>>>> as a normal vq doesn't have any problems.
>>>> Note that we should make sure the buggy(malicous) device won't crash
>>>> the codes by changing the queue_reset value at its will.
>>> I will keep an eye on this situation.
>>>
>>>>>>
>>>>>>> +
>>>>>>>      /* If all buffers were filled by other side before we napi_enabled, we
>>>>>>>       * won't get another interrupt, so process any outstanding packets now.
>>>>>>>       * Call local_bh_enable after to trigger softIRQ processing.
>>>>>>> @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
>>>>>>>              struct receive_queue *rq = &vi->rq[i];
>>>>>>>
>>>>>>>              napi_disable(&rq->napi);
>>>>>>> +
>>>>>>> +           /* Check if vq is in reset state. See more in
>>>>>>> +            * virtnet_napi_enable()
>>>>>>> +            */
>>>>>>> +           if (rq->vq->reset) {
>>>>>>> +                   virtnet_napi_enable(rq->vq, &rq->napi);
>>>>>>> +                   continue;
>>>>>>> +           }
>>>>>>
>>>>>> Can we do something similar in virtnet_close() by canceling the work?
>>>>> I think there is no need to cancel the work here, because napi_disable will wait
>>>>> for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
>>>>> vq, this logic can be removed.
>>>> Actually I meant the part of virtnet_rx_resize().
>>>>
>>>> If we don't synchronize with the refill work, it might enable NAPI unexpectedly?
>>> I don't think this situation will be encountered, because napi_disable is
>>> mutually exclusive, so there will be no unexpected napi enable.
>>>
>>> Is there something I misunderstood?
>> So in virtnet_rx_resize() we do:
>>
>> napi_disable()
>> ...
>> resize()
>> ...
>> napi_enalbe()
>>
>> How can we guarantee that the work is not run after the napi_disable()?
>
> I think you're talking about a situation like this:
>
> virtnet_rx_resize          refill work
> -----------------------------------------------------------
>   napi_disable()
>   ...                       napi_disable()
>   resize()                      ...
>                             napi_enable()
>   ...
>   napi_enalbe()
>
>
> But in fact:
>
> virtnet_rx_resize          refill work
> -----------------------------------------------------------
>   napi_disable()
>   ...                       napi_disable() <----[0]
>   resize()                       |
>   ...                            |
>   napi_enalbe()                  |
>                             napi_disable() <---- [1] here success
>                             napi_enable()
>
> Because virtnet_rx_resize() has already executed napi_disable(), napi_disalbe()
> of [0] will wait until [1] to complete.
>
> I'm not sure if my understanding is correct.


I think you're right here.

Thanks


>
> Thanks.
>
>> Thanks
>>
>>> Thanks.
>>>
>>>> Thanks
>>>>
>>>>>
>>>>>>
>>>>>>> +
>>>>>>>              still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
>>>>>>>              virtnet_napi_enable(rq->vq, &rq->napi);
>>>>>>>
>>>>>>> @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
>>>>>>>      if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
>>>>>>>              return;
>>>>>>>
>>>>>>> +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
>>>>>>> +   if (sq->vq->reset)
>>>>>>> +           return;
>>>>>>
>>>>>> We've disabled TX napi, any chance we can still hit this?
>>>>> Same as above.
>>>>>
>>>>>>
>>>>>>> +
>>>>>>>      if (__netif_tx_trylock(txq)) {
>>>>>>>              do {
>>>>>>>                      virtqueue_disable_cb(sq->vq);
>>>>>>> @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
>>>>>>>      return NETDEV_TX_OK;
>>>>>>>    }
>>>>>>>
>>>>>>> +static int virtnet_rx_resize(struct virtnet_info *vi,
>>>>>>> +                        struct receive_queue *rq, u32 ring_num)
>>>>>>> +{
>>>>>>> +   int err;
>>>>>>> +
>>>>>>> +   napi_disable(&rq->napi);
>>>>>>> +
>>>>>>> +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
>>>>>>> +   if (err)
>>>>>>> +           goto err;
>>>>>>> +
>>>>>>> +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
>>>>>>> +           schedule_delayed_work(&vi->refill, 0);
>>>>>>> +
>>>>>>> +   virtnet_napi_enable(rq->vq, &rq->napi);
>>>>>>> +   return 0;
>>>>>>> +
>>>>>>> +err:
>>>>>>> +   netdev_err(vi->dev,
>>>>>>> +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
>>>>>>> +              rq - vi->rq, err);
>>>>>>> +   virtnet_napi_enable(rq->vq, &rq->napi);
>>>>>>> +   return err;
>>>>>>> +}
>>>>>>> +
>>>>>>> +static int virtnet_tx_resize(struct virtnet_info *vi,
>>>>>>> +                        struct send_queue *sq, u32 ring_num)
>>>>>>> +{
>>>>>>> +   struct netdev_queue *txq;
>>>>>>> +   int err, qindex;
>>>>>>> +
>>>>>>> +   qindex = sq - vi->sq;
>>>>>>> +
>>>>>>> +   virtnet_napi_tx_disable(&sq->napi);
>>>>>>> +
>>>>>>> +   txq = netdev_get_tx_queue(vi->dev, qindex);
>>>>>>> +   __netif_tx_lock_bh(txq);
>>>>>>> +   netif_stop_subqueue(vi->dev, qindex);
>>>>>>> +   __netif_tx_unlock_bh(txq);
>>>>>>> +
>>>>>>> +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
>>>>>>> +   if (err)
>>>>>>> +           goto err;
>>>>>>> +
>>>>>>> +   netif_start_subqueue(vi->dev, qindex);
>>>>>>> +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
>>>>>>> +   return 0;
>>>>>>> +
>>>>>>> +err:
>>>>>>
>>>>>> I guess we can still start the queue in this case? (Since we don't
>>>>>> change the queue if resize fails).
>>>>> Yes, you are right.
>>>>>
>>>>> Thanks.
>>>>>
>>>>>>
>>>>>>> +   netdev_err(vi->dev,
>>>>>>> +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
>>>>>>> +              sq - vi->sq, err);
>>>>>>> +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
>>>>>>> +   return err;
>>>>>>> +}
>>>>>>> +
>>>>>>>    /*
>>>>>>>     * Send command via the control virtqueue and check status.  Commands
>>>>>>>     * supported by the hypervisor, as indicated by feature bits, should

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

在 2022/4/15 17:17, Xuan Zhuo 写道:
> On Fri, 15 Apr 2022 13:53:54 +0800, Jason Wang <jasowang@redhat.com> wrote:
>> On Fri, Apr 15, 2022 at 10:23 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>>> On Thu, 14 Apr 2022 17:30:02 +0800, Jason Wang <jasowang@redhat.com> wrote:
>>>> On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>>>>> On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>>>>>> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
>>>>>>> This patch implements the resize function of the rx, tx queues.
>>>>>>> Based on this function, it is possible to modify the ring num of the
>>>>>>> queue.
>>>>>>>
>>>>>>> There may be an exception during the resize process, the resize may
>>>>>>> fail, or the vq can no longer be used. Either way, we must execute
>>>>>>> napi_enable(). Because napi_disable is similar to a lock, napi_enable
>>>>>>> must be called after calling napi_disable.
>>>>>>>
>>>>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>>>>> ---
>>>>>>>    drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
>>>>>>>    1 file changed, 81 insertions(+)
>>>>>>>
>>>>>>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
>>>>>>> index b8bf00525177..ba6859f305f7 100644
>>>>>>> --- a/drivers/net/virtio_net.c
>>>>>>> +++ b/drivers/net/virtio_net.c
>>>>>>> @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
>>>>>>>      char padding[4];
>>>>>>>    };
>>>>>>>
>>>>>>> +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
>>>>>>> +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
>>>>>>> +
>>>>>>>    static bool is_xdp_frame(void *ptr)
>>>>>>>    {
>>>>>>>      return (unsigned long)ptr & VIRTIO_XDP_FLAG;
>>>>>>> @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
>>>>>>>    {
>>>>>>>      napi_enable(napi);
>>>>>>>
>>>>>>> +   /* Check if vq is in reset state. The normal reset/resize process will
>>>>>>> +    * be protected by napi. However, the protection of napi is only enabled
>>>>>>> +    * during the operation, and the protection of napi will end after the
>>>>>>> +    * operation is completed. If re-enable fails during the process, vq
>>>>>>> +    * will remain unavailable with reset state.
>>>>>>> +    */
>>>>>>> +   if (vq->reset)
>>>>>>> +           return;
>>>>>>
>>>>>> I don't get when could we hit this condition.
>>>>>
>>>>> In patch 23, the code to implement re-enable vq is as follows:
>>>>>
>>>>> +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
>>>>> +{
>>>>> +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
>>>>> +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
>>>>> +       struct virtio_pci_vq_info *info;
>>>>> +       unsigned long flags, index;
>>>>> +       int err;
>>>>> +
>>>>> +       if (!vq->reset)
>>>>> +               return -EBUSY;
>>>>> +
>>>>> +       index = vq->index;
>>>>> +       info = vp_dev->vqs[index];
>>>>> +
>>>>> +       /* check queue reset status */
>>>>> +       if (vp_modern_get_queue_reset(mdev, index) != 1)
>>>>> +               return -EBUSY;
>>>>> +
>>>>> +       err = vp_active_vq(vq, info->msix_vector);
>>>>> +       if (err)
>>>>> +               return err;
>>>>> +
>>>>> +       if (vq->callback) {
>>>>> +               spin_lock_irqsave(&vp_dev->lock, flags);
>>>>> +               list_add(&info->node, &vp_dev->virtqueues);
>>>>> +               spin_unlock_irqrestore(&vp_dev->lock, flags);
>>>>> +       } else {
>>>>> +               INIT_LIST_HEAD(&info->node);
>>>>> +       }
>>>>> +
>>>>> +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
>>>>> +
>>>>> +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
>>>>> +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>>>>> +
>>>>> +       vq->reset = false;
>>>>> +
>>>>> +       return 0;
>>>>> +}
>>>>>
>>>>>
>>>>> There are three situations where an error will be returned. These are the
>>>>> situations I want to handle.
>>>> Right, but it looks harmless if we just schedule the NAPI without the check.
>>> Yes.
>>>
>>>>> But I'm rethinking the question, and I feel like you're right, although the
>>>>> hardware setup may fail. We can no longer sync with the hardware. But using it
>>>>> as a normal vq doesn't have any problems.
>>>> Note that we should make sure the buggy(malicous) device won't crash
>>>> the codes by changing the queue_reset value at its will.
>>> I will keep an eye on this situation.
>>>
>>>>>>
>>>>>>> +
>>>>>>>      /* If all buffers were filled by other side before we napi_enabled, we
>>>>>>>       * won't get another interrupt, so process any outstanding packets now.
>>>>>>>       * Call local_bh_enable after to trigger softIRQ processing.
>>>>>>> @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
>>>>>>>              struct receive_queue *rq = &vi->rq[i];
>>>>>>>
>>>>>>>              napi_disable(&rq->napi);
>>>>>>> +
>>>>>>> +           /* Check if vq is in reset state. See more in
>>>>>>> +            * virtnet_napi_enable()
>>>>>>> +            */
>>>>>>> +           if (rq->vq->reset) {
>>>>>>> +                   virtnet_napi_enable(rq->vq, &rq->napi);
>>>>>>> +                   continue;
>>>>>>> +           }
>>>>>>
>>>>>> Can we do something similar in virtnet_close() by canceling the work?
>>>>> I think there is no need to cancel the work here, because napi_disable will wait
>>>>> for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
>>>>> vq, this logic can be removed.
>>>> Actually I meant the part of virtnet_rx_resize().
>>>>
>>>> If we don't synchronize with the refill work, it might enable NAPI unexpectedly?
>>> I don't think this situation will be encountered, because napi_disable is
>>> mutually exclusive, so there will be no unexpected napi enable.
>>>
>>> Is there something I misunderstood?
>> So in virtnet_rx_resize() we do:
>>
>> napi_disable()
>> ...
>> resize()
>> ...
>> napi_enalbe()
>>
>> How can we guarantee that the work is not run after the napi_disable()?
>
> I think you're talking about a situation like this:
>
> virtnet_rx_resize          refill work
> -----------------------------------------------------------
>   napi_disable()
>   ...                       napi_disable()
>   resize()                      ...
>                             napi_enable()
>   ...
>   napi_enalbe()
>
>
> But in fact:
>
> virtnet_rx_resize          refill work
> -----------------------------------------------------------
>   napi_disable()
>   ...                       napi_disable() <----[0]
>   resize()                       |
>   ...                            |
>   napi_enalbe()                  |
>                             napi_disable() <---- [1] here success
>                             napi_enable()
>
> Because virtnet_rx_resize() has already executed napi_disable(), napi_disalbe()
> of [0] will wait until [1] to complete.
>
> I'm not sure if my understanding is correct.


I think you're right here.

Thanks


>
> Thanks.
>
>> Thanks
>>
>>> Thanks.
>>>
>>>> Thanks
>>>>
>>>>>
>>>>>>
>>>>>>> +
>>>>>>>              still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
>>>>>>>              virtnet_napi_enable(rq->vq, &rq->napi);
>>>>>>>
>>>>>>> @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
>>>>>>>      if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
>>>>>>>              return;
>>>>>>>
>>>>>>> +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
>>>>>>> +   if (sq->vq->reset)
>>>>>>> +           return;
>>>>>>
>>>>>> We've disabled TX napi, any chance we can still hit this?
>>>>> Same as above.
>>>>>
>>>>>>
>>>>>>> +
>>>>>>>      if (__netif_tx_trylock(txq)) {
>>>>>>>              do {
>>>>>>>                      virtqueue_disable_cb(sq->vq);
>>>>>>> @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
>>>>>>>      return NETDEV_TX_OK;
>>>>>>>    }
>>>>>>>
>>>>>>> +static int virtnet_rx_resize(struct virtnet_info *vi,
>>>>>>> +                        struct receive_queue *rq, u32 ring_num)
>>>>>>> +{
>>>>>>> +   int err;
>>>>>>> +
>>>>>>> +   napi_disable(&rq->napi);
>>>>>>> +
>>>>>>> +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
>>>>>>> +   if (err)
>>>>>>> +           goto err;
>>>>>>> +
>>>>>>> +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
>>>>>>> +           schedule_delayed_work(&vi->refill, 0);
>>>>>>> +
>>>>>>> +   virtnet_napi_enable(rq->vq, &rq->napi);
>>>>>>> +   return 0;
>>>>>>> +
>>>>>>> +err:
>>>>>>> +   netdev_err(vi->dev,
>>>>>>> +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
>>>>>>> +              rq - vi->rq, err);
>>>>>>> +   virtnet_napi_enable(rq->vq, &rq->napi);
>>>>>>> +   return err;
>>>>>>> +}
>>>>>>> +
>>>>>>> +static int virtnet_tx_resize(struct virtnet_info *vi,
>>>>>>> +                        struct send_queue *sq, u32 ring_num)
>>>>>>> +{
>>>>>>> +   struct netdev_queue *txq;
>>>>>>> +   int err, qindex;
>>>>>>> +
>>>>>>> +   qindex = sq - vi->sq;
>>>>>>> +
>>>>>>> +   virtnet_napi_tx_disable(&sq->napi);
>>>>>>> +
>>>>>>> +   txq = netdev_get_tx_queue(vi->dev, qindex);
>>>>>>> +   __netif_tx_lock_bh(txq);
>>>>>>> +   netif_stop_subqueue(vi->dev, qindex);
>>>>>>> +   __netif_tx_unlock_bh(txq);
>>>>>>> +
>>>>>>> +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
>>>>>>> +   if (err)
>>>>>>> +           goto err;
>>>>>>> +
>>>>>>> +   netif_start_subqueue(vi->dev, qindex);
>>>>>>> +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
>>>>>>> +   return 0;
>>>>>>> +
>>>>>>> +err:
>>>>>>
>>>>>> I guess we can still start the queue in this case? (Since we don't
>>>>>> change the queue if resize fails).
>>>>> Yes, you are right.
>>>>>
>>>>> Thanks.
>>>>>
>>>>>>
>>>>>>> +   netdev_err(vi->dev,
>>>>>>> +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
>>>>>>> +              sq - vi->sq, err);
>>>>>>> +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
>>>>>>> +   return err;
>>>>>>> +}
>>>>>>> +
>>>>>>>    /*
>>>>>>>     * Send command via the control virtqueue and check status.  Commands
>>>>>>>     * supported by the hypervisor, as indicated by feature bits, should

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Jason Wang]

在 2022/4/15 17:17, Xuan Zhuo 写道:
> On Fri, 15 Apr 2022 13:53:54 +0800, Jason Wang <jasowang@redhat.com> wrote:
>> On Fri, Apr 15, 2022 at 10:23 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>>> On Thu, 14 Apr 2022 17:30:02 +0800, Jason Wang <jasowang@redhat.com> wrote:
>>>> On Wed, Apr 13, 2022 at 4:47 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>>>>> On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
>>>>>> 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
>>>>>>> This patch implements the resize function of the rx, tx queues.
>>>>>>> Based on this function, it is possible to modify the ring num of the
>>>>>>> queue.
>>>>>>>
>>>>>>> There may be an exception during the resize process, the resize may
>>>>>>> fail, or the vq can no longer be used. Either way, we must execute
>>>>>>> napi_enable(). Because napi_disable is similar to a lock, napi_enable
>>>>>>> must be called after calling napi_disable.
>>>>>>>
>>>>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>>>>> ---
>>>>>>>    drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
>>>>>>>    1 file changed, 81 insertions(+)
>>>>>>>
>>>>>>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
>>>>>>> index b8bf00525177..ba6859f305f7 100644
>>>>>>> --- a/drivers/net/virtio_net.c
>>>>>>> +++ b/drivers/net/virtio_net.c
>>>>>>> @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
>>>>>>>      char padding[4];
>>>>>>>    };
>>>>>>>
>>>>>>> +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
>>>>>>> +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
>>>>>>> +
>>>>>>>    static bool is_xdp_frame(void *ptr)
>>>>>>>    {
>>>>>>>      return (unsigned long)ptr & VIRTIO_XDP_FLAG;
>>>>>>> @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
>>>>>>>    {
>>>>>>>      napi_enable(napi);
>>>>>>>
>>>>>>> +   /* Check if vq is in reset state. The normal reset/resize process will
>>>>>>> +    * be protected by napi. However, the protection of napi is only enabled
>>>>>>> +    * during the operation, and the protection of napi will end after the
>>>>>>> +    * operation is completed. If re-enable fails during the process, vq
>>>>>>> +    * will remain unavailable with reset state.
>>>>>>> +    */
>>>>>>> +   if (vq->reset)
>>>>>>> +           return;
>>>>>>
>>>>>> I don't get when could we hit this condition.
>>>>>
>>>>> In patch 23, the code to implement re-enable vq is as follows:
>>>>>
>>>>> +static int vp_modern_enable_reset_vq(struct virtqueue *vq)
>>>>> +{
>>>>> +       struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
>>>>> +       struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
>>>>> +       struct virtio_pci_vq_info *info;
>>>>> +       unsigned long flags, index;
>>>>> +       int err;
>>>>> +
>>>>> +       if (!vq->reset)
>>>>> +               return -EBUSY;
>>>>> +
>>>>> +       index = vq->index;
>>>>> +       info = vp_dev->vqs[index];
>>>>> +
>>>>> +       /* check queue reset status */
>>>>> +       if (vp_modern_get_queue_reset(mdev, index) != 1)
>>>>> +               return -EBUSY;
>>>>> +
>>>>> +       err = vp_active_vq(vq, info->msix_vector);
>>>>> +       if (err)
>>>>> +               return err;
>>>>> +
>>>>> +       if (vq->callback) {
>>>>> +               spin_lock_irqsave(&vp_dev->lock, flags);
>>>>> +               list_add(&info->node, &vp_dev->virtqueues);
>>>>> +               spin_unlock_irqrestore(&vp_dev->lock, flags);
>>>>> +       } else {
>>>>> +               INIT_LIST_HEAD(&info->node);
>>>>> +       }
>>>>> +
>>>>> +       vp_modern_set_queue_enable(&vp_dev->mdev, index, true);
>>>>> +
>>>>> +       if (vp_dev->per_vq_vectors && info->msix_vector != VIRTIO_MSI_NO_VECTOR)
>>>>> +               enable_irq(pci_irq_vector(vp_dev->pci_dev, info->msix_vector));
>>>>> +
>>>>> +       vq->reset = false;
>>>>> +
>>>>> +       return 0;
>>>>> +}
>>>>>
>>>>>
>>>>> There are three situations where an error will be returned. These are the
>>>>> situations I want to handle.
>>>> Right, but it looks harmless if we just schedule the NAPI without the check.
>>> Yes.
>>>
>>>>> But I'm rethinking the question, and I feel like you're right, although the
>>>>> hardware setup may fail. We can no longer sync with the hardware. But using it
>>>>> as a normal vq doesn't have any problems.
>>>> Note that we should make sure the buggy(malicous) device won't crash
>>>> the codes by changing the queue_reset value at its will.
>>> I will keep an eye on this situation.
>>>
>>>>>>
>>>>>>> +
>>>>>>>      /* If all buffers were filled by other side before we napi_enabled, we
>>>>>>>       * won't get another interrupt, so process any outstanding packets now.
>>>>>>>       * Call local_bh_enable after to trigger softIRQ processing.
>>>>>>> @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
>>>>>>>              struct receive_queue *rq = &vi->rq[i];
>>>>>>>
>>>>>>>              napi_disable(&rq->napi);
>>>>>>> +
>>>>>>> +           /* Check if vq is in reset state. See more in
>>>>>>> +            * virtnet_napi_enable()
>>>>>>> +            */
>>>>>>> +           if (rq->vq->reset) {
>>>>>>> +                   virtnet_napi_enable(rq->vq, &rq->napi);
>>>>>>> +                   continue;
>>>>>>> +           }
>>>>>>
>>>>>> Can we do something similar in virtnet_close() by canceling the work?
>>>>> I think there is no need to cancel the work here, because napi_disable will wait
>>>>> for the napi_enable of the resize. So if the re-enable failed vq is used as a normal
>>>>> vq, this logic can be removed.
>>>> Actually I meant the part of virtnet_rx_resize().
>>>>
>>>> If we don't synchronize with the refill work, it might enable NAPI unexpectedly?
>>> I don't think this situation will be encountered, because napi_disable is
>>> mutually exclusive, so there will be no unexpected napi enable.
>>>
>>> Is there something I misunderstood?
>> So in virtnet_rx_resize() we do:
>>
>> napi_disable()
>> ...
>> resize()
>> ...
>> napi_enalbe()
>>
>> How can we guarantee that the work is not run after the napi_disable()?
>
> I think you're talking about a situation like this:
>
> virtnet_rx_resize          refill work
> -----------------------------------------------------------
>   napi_disable()
>   ...                       napi_disable()
>   resize()                      ...
>                             napi_enable()
>   ...
>   napi_enalbe()
>
>
> But in fact:
>
> virtnet_rx_resize          refill work
> -----------------------------------------------------------
>   napi_disable()
>   ...                       napi_disable() <----[0]
>   resize()                       |
>   ...                            |
>   napi_enalbe()                  |
>                             napi_disable() <---- [1] here success
>                             napi_enable()
>
> Because virtnet_rx_resize() has already executed napi_disable(), napi_disalbe()
> of [0] will wait until [1] to complete.
>
> I'm not sure if my understanding is correct.


I think you're right here.

Thanks


>
> Thanks.
>
>> Thanks
>>
>>> Thanks.
>>>
>>>> Thanks
>>>>
>>>>>
>>>>>>
>>>>>>> +
>>>>>>>              still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
>>>>>>>              virtnet_napi_enable(rq->vq, &rq->napi);
>>>>>>>
>>>>>>> @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
>>>>>>>      if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
>>>>>>>              return;
>>>>>>>
>>>>>>> +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
>>>>>>> +   if (sq->vq->reset)
>>>>>>> +           return;
>>>>>>
>>>>>> We've disabled TX napi, any chance we can still hit this?
>>>>> Same as above.
>>>>>
>>>>>>
>>>>>>> +
>>>>>>>      if (__netif_tx_trylock(txq)) {
>>>>>>>              do {
>>>>>>>                      virtqueue_disable_cb(sq->vq);
>>>>>>> @@ -1769,6 +1794,62 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
>>>>>>>      return NETDEV_TX_OK;
>>>>>>>    }
>>>>>>>
>>>>>>> +static int virtnet_rx_resize(struct virtnet_info *vi,
>>>>>>> +                        struct receive_queue *rq, u32 ring_num)
>>>>>>> +{
>>>>>>> +   int err;
>>>>>>> +
>>>>>>> +   napi_disable(&rq->napi);
>>>>>>> +
>>>>>>> +   err = virtqueue_resize(rq->vq, ring_num, virtnet_rq_free_unused_buf);
>>>>>>> +   if (err)
>>>>>>> +           goto err;
>>>>>>> +
>>>>>>> +   if (!try_fill_recv(vi, rq, GFP_KERNEL))
>>>>>>> +           schedule_delayed_work(&vi->refill, 0);
>>>>>>> +
>>>>>>> +   virtnet_napi_enable(rq->vq, &rq->napi);
>>>>>>> +   return 0;
>>>>>>> +
>>>>>>> +err:
>>>>>>> +   netdev_err(vi->dev,
>>>>>>> +              "reset rx reset vq fail: rx queue index: %td err: %d\n",
>>>>>>> +              rq - vi->rq, err);
>>>>>>> +   virtnet_napi_enable(rq->vq, &rq->napi);
>>>>>>> +   return err;
>>>>>>> +}
>>>>>>> +
>>>>>>> +static int virtnet_tx_resize(struct virtnet_info *vi,
>>>>>>> +                        struct send_queue *sq, u32 ring_num)
>>>>>>> +{
>>>>>>> +   struct netdev_queue *txq;
>>>>>>> +   int err, qindex;
>>>>>>> +
>>>>>>> +   qindex = sq - vi->sq;
>>>>>>> +
>>>>>>> +   virtnet_napi_tx_disable(&sq->napi);
>>>>>>> +
>>>>>>> +   txq = netdev_get_tx_queue(vi->dev, qindex);
>>>>>>> +   __netif_tx_lock_bh(txq);
>>>>>>> +   netif_stop_subqueue(vi->dev, qindex);
>>>>>>> +   __netif_tx_unlock_bh(txq);
>>>>>>> +
>>>>>>> +   err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
>>>>>>> +   if (err)
>>>>>>> +           goto err;
>>>>>>> +
>>>>>>> +   netif_start_subqueue(vi->dev, qindex);
>>>>>>> +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
>>>>>>> +   return 0;
>>>>>>> +
>>>>>>> +err:
>>>>>>
>>>>>> I guess we can still start the queue in this case? (Since we don't
>>>>>> change the queue if resize fails).
>>>>> Yes, you are right.
>>>>>
>>>>> Thanks.
>>>>>
>>>>>>
>>>>>>> +   netdev_err(vi->dev,
>>>>>>> +              "reset tx reset vq fail: tx queue index: %td err: %d\n",
>>>>>>> +              sq - vi->sq, err);
>>>>>>> +   virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
>>>>>>> +   return err;
>>>>>>> +}
>>>>>>> +
>>>>>>>    /*
>>>>>>>     * Send command via the control virtqueue and check status.  Commands
>>>>>>>     * supported by the hypervisor, as indicated by feature bits, should


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Mon, 18 Apr 2022 15:49:29 +0800, Jason Wang <jasowang@redhat.com> wrote:
> On Mon, Apr 18, 2022 at 11:24 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> >
> > On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > >
> > > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > > This patch implements the resize function of the rx, tx queues.
> > > > Based on this function, it is possible to modify the ring num of the
> > > > queue.
> > > >
> > > > There may be an exception during the resize process, the resize may
> > > > fail, or the vq can no longer be used. Either way, we must execute
> > > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > > must be called after calling napi_disable.
> > > >
> > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > ---
> > > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > > >   1 file changed, 81 insertions(+)
> > > >
> > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > index b8bf00525177..ba6859f305f7 100644
> > > > --- a/drivers/net/virtio_net.c
> > > > +++ b/drivers/net/virtio_net.c
> > > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > > >     char padding[4];
> > > >   };
> > > >
> > > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > +
> > > >   static bool is_xdp_frame(void *ptr)
> > > >   {
> > > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > > >   {
> > > >     napi_enable(napi);
> > > >
> > > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > > +    * be protected by napi. However, the protection of napi is only enabled
> > > > +    * during the operation, and the protection of napi will end after the
> > > > +    * operation is completed. If re-enable fails during the process, vq
> > > > +    * will remain unavailable with reset state.
> > > > +    */
> > > > +   if (vq->reset)
> > > > +           return;
> > >
> > >
> > > I don't get when could we hit this condition.
> > >
> > >
> > > > +
> > > >     /* If all buffers were filled by other side before we napi_enabled, we
> > > >      * won't get another interrupt, so process any outstanding packets now.
> > > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > > >             struct receive_queue *rq = &vi->rq[i];
> > > >
> > > >             napi_disable(&rq->napi);
> > > > +
> > > > +           /* Check if vq is in reset state. See more in
> > > > +            * virtnet_napi_enable()
> > > > +            */
> > > > +           if (rq->vq->reset) {
> > > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > +                   continue;
> > > > +           }
> > >
> > >
> > > Can we do something similar in virtnet_close() by canceling the work?
> > >
> > >
> > > > +
> > > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > > >
> > > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > > >             return;
> > > >
> > > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > > +   if (sq->vq->reset)
> > > > +           return;
> > >
> > >
> > > We've disabled TX napi, any chance we can still hit this?
> >
> >
> > static int virtnet_poll(struct napi_struct *napi, int budget)
> > {
> >         struct receive_queue *rq =
> >                 container_of(napi, struct receive_queue, napi);
> >         struct virtnet_info *vi = rq->vq->vdev->priv;
> >         struct send_queue *sq;
> >         unsigned int received;
> >         unsigned int xdp_xmit = 0;
> >
> >         virtnet_poll_cleantx(rq);
> > ...
> > }
> >
> > This is called by rx poll. Although it is the logic of tx, it is not driven by
> > tx napi, but is called in rx poll.
>
> Ok, but we need guarantee the memory ordering in this case. Disable RX
> napi could be a solution for this.

Yes, I have realized this too. I have two solutions, disable rx napi or the
following.

Thanks.


diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 9bf1b6530b38..7764d1dcb831 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -135,6 +135,7 @@ struct send_queue {
 	struct virtnet_sq_stats stats;

 	struct napi_struct napi;
+	bool reset;
 };

 /* Internal representation of a receive virtqueue */
@@ -1583,6 +1587,11 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
 		return;

 	if (__netif_tx_trylock(txq)) {
+		if (sq->reset) {
+			__netif_tx_unlock(txq);
+			return;
+		}
+
 		do {
 			virtqueue_disable_cb(sq->vq);
 			free_old_xmit_skbs(sq, true);
@@ -1828,6 +1837,56 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
 	return NETDEV_TX_OK;
 }

+static int virtnet_tx_resize(struct virtnet_info *vi,
+			     struct send_queue *sq, u32 ring_num)
+{
+	struct netdev_queue *txq;
+	int err, qindex;
+
+	qindex = sq - vi->sq;
+
+	virtnet_napi_tx_disable(&sq->napi);
+
+	txq = netdev_get_tx_queue(vi->dev, qindex);
+
+	__netif_tx_lock_bh(txq);
+	netif_stop_subqueue(vi->dev, qindex);
+	sq->reset = true;
+	__netif_tx_unlock_bh(txq);
+
+	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
+	if (err)
+		netdev_err(vi->dev, "resize tx fail: tx queue index: %d err: %d\n", qindex, err);
+
+	__netif_tx_lock_bh(txq);
+	sq->reset = false;
+	netif_start_subqueue(vi->dev, qindex);
+	__netif_tx_unlock_bh(txq);
+
+	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
+	return err;
+}
+

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Mon, 18 Apr 2022 15:49:29 +0800, Jason Wang <jasowang@redhat.com> wrote:
> On Mon, Apr 18, 2022 at 11:24 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> >
> > On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > >
> > > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > > This patch implements the resize function of the rx, tx queues.
> > > > Based on this function, it is possible to modify the ring num of the
> > > > queue.
> > > >
> > > > There may be an exception during the resize process, the resize may
> > > > fail, or the vq can no longer be used. Either way, we must execute
> > > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > > must be called after calling napi_disable.
> > > >
> > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > ---
> > > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > > >   1 file changed, 81 insertions(+)
> > > >
> > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > index b8bf00525177..ba6859f305f7 100644
> > > > --- a/drivers/net/virtio_net.c
> > > > +++ b/drivers/net/virtio_net.c
> > > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > > >     char padding[4];
> > > >   };
> > > >
> > > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > +
> > > >   static bool is_xdp_frame(void *ptr)
> > > >   {
> > > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > > >   {
> > > >     napi_enable(napi);
> > > >
> > > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > > +    * be protected by napi. However, the protection of napi is only enabled
> > > > +    * during the operation, and the protection of napi will end after the
> > > > +    * operation is completed. If re-enable fails during the process, vq
> > > > +    * will remain unavailable with reset state.
> > > > +    */
> > > > +   if (vq->reset)
> > > > +           return;
> > >
> > >
> > > I don't get when could we hit this condition.
> > >
> > >
> > > > +
> > > >     /* If all buffers were filled by other side before we napi_enabled, we
> > > >      * won't get another interrupt, so process any outstanding packets now.
> > > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > > >             struct receive_queue *rq = &vi->rq[i];
> > > >
> > > >             napi_disable(&rq->napi);
> > > > +
> > > > +           /* Check if vq is in reset state. See more in
> > > > +            * virtnet_napi_enable()
> > > > +            */
> > > > +           if (rq->vq->reset) {
> > > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > +                   continue;
> > > > +           }
> > >
> > >
> > > Can we do something similar in virtnet_close() by canceling the work?
> > >
> > >
> > > > +
> > > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > > >
> > > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > > >             return;
> > > >
> > > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > > +   if (sq->vq->reset)
> > > > +           return;
> > >
> > >
> > > We've disabled TX napi, any chance we can still hit this?
> >
> >
> > static int virtnet_poll(struct napi_struct *napi, int budget)
> > {
> >         struct receive_queue *rq =
> >                 container_of(napi, struct receive_queue, napi);
> >         struct virtnet_info *vi = rq->vq->vdev->priv;
> >         struct send_queue *sq;
> >         unsigned int received;
> >         unsigned int xdp_xmit = 0;
> >
> >         virtnet_poll_cleantx(rq);
> > ...
> > }
> >
> > This is called by rx poll. Although it is the logic of tx, it is not driven by
> > tx napi, but is called in rx poll.
>
> Ok, but we need guarantee the memory ordering in this case. Disable RX
> napi could be a solution for this.

Yes, I have realized this too. I have two solutions, disable rx napi or the
following.

Thanks.


diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 9bf1b6530b38..7764d1dcb831 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -135,6 +135,7 @@ struct send_queue {
 	struct virtnet_sq_stats stats;

 	struct napi_struct napi;
+	bool reset;
 };

 /* Internal representation of a receive virtqueue */
@@ -1583,6 +1587,11 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
 		return;

 	if (__netif_tx_trylock(txq)) {
+		if (sq->reset) {
+			__netif_tx_unlock(txq);
+			return;
+		}
+
 		do {
 			virtqueue_disable_cb(sq->vq);
 			free_old_xmit_skbs(sq, true);
@@ -1828,6 +1837,56 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
 	return NETDEV_TX_OK;
 }

+static int virtnet_tx_resize(struct virtnet_info *vi,
+			     struct send_queue *sq, u32 ring_num)
+{
+	struct netdev_queue *txq;
+	int err, qindex;
+
+	qindex = sq - vi->sq;
+
+	virtnet_napi_tx_disable(&sq->napi);
+
+	txq = netdev_get_tx_queue(vi->dev, qindex);
+
+	__netif_tx_lock_bh(txq);
+	netif_stop_subqueue(vi->dev, qindex);
+	sq->reset = true;
+	__netif_tx_unlock_bh(txq);
+
+	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
+	if (err)
+		netdev_err(vi->dev, "resize tx fail: tx queue index: %d err: %d\n", qindex, err);
+
+	__netif_tx_lock_bh(txq);
+	sq->reset = false;
+	netif_start_subqueue(vi->dev, qindex);
+	__netif_tx_unlock_bh(txq);
+
+	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
+	return err;
+}
+

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 31/32] virtio_net: support rx/tx queue resize

[Xuan Zhuo]

On Mon, 18 Apr 2022 15:49:29 +0800, Jason Wang <jasowang@redhat.com> wrote:
> On Mon, Apr 18, 2022 at 11:24 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> >
> > On Wed, 13 Apr 2022 16:00:18 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > >
> > > 在 2022/4/6 上午11:43, Xuan Zhuo 写道:
> > > > This patch implements the resize function of the rx, tx queues.
> > > > Based on this function, it is possible to modify the ring num of the
> > > > queue.
> > > >
> > > > There may be an exception during the resize process, the resize may
> > > > fail, or the vq can no longer be used. Either way, we must execute
> > > > napi_enable(). Because napi_disable is similar to a lock, napi_enable
> > > > must be called after calling napi_disable.
> > > >
> > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > ---
> > > >   drivers/net/virtio_net.c | 81 ++++++++++++++++++++++++++++++++++++++++
> > > >   1 file changed, 81 insertions(+)
> > > >
> > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > index b8bf00525177..ba6859f305f7 100644
> > > > --- a/drivers/net/virtio_net.c
> > > > +++ b/drivers/net/virtio_net.c
> > > > @@ -251,6 +251,9 @@ struct padded_vnet_hdr {
> > > >     char padding[4];
> > > >   };
> > > >
> > > > +static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > +static void virtnet_rq_free_unused_buf(struct virtqueue *vq, void *buf);
> > > > +
> > > >   static bool is_xdp_frame(void *ptr)
> > > >   {
> > > >     return (unsigned long)ptr & VIRTIO_XDP_FLAG;
> > > > @@ -1369,6 +1372,15 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)
> > > >   {
> > > >     napi_enable(napi);
> > > >
> > > > +   /* Check if vq is in reset state. The normal reset/resize process will
> > > > +    * be protected by napi. However, the protection of napi is only enabled
> > > > +    * during the operation, and the protection of napi will end after the
> > > > +    * operation is completed. If re-enable fails during the process, vq
> > > > +    * will remain unavailable with reset state.
> > > > +    */
> > > > +   if (vq->reset)
> > > > +           return;
> > >
> > >
> > > I don't get when could we hit this condition.
> > >
> > >
> > > > +
> > > >     /* If all buffers were filled by other side before we napi_enabled, we
> > > >      * won't get another interrupt, so process any outstanding packets now.
> > > >      * Call local_bh_enable after to trigger softIRQ processing.
> > > > @@ -1413,6 +1425,15 @@ static void refill_work(struct work_struct *work)
> > > >             struct receive_queue *rq = &vi->rq[i];
> > > >
> > > >             napi_disable(&rq->napi);
> > > > +
> > > > +           /* Check if vq is in reset state. See more in
> > > > +            * virtnet_napi_enable()
> > > > +            */
> > > > +           if (rq->vq->reset) {
> > > > +                   virtnet_napi_enable(rq->vq, &rq->napi);
> > > > +                   continue;
> > > > +           }
> > >
> > >
> > > Can we do something similar in virtnet_close() by canceling the work?
> > >
> > >
> > > > +
> > > >             still_empty = !try_fill_recv(vi, rq, GFP_KERNEL);
> > > >             virtnet_napi_enable(rq->vq, &rq->napi);
> > > >
> > > > @@ -1523,6 +1544,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
> > > >     if (!sq->napi.weight || is_xdp_raw_buffer_queue(vi, index))
> > > >             return;
> > > >
> > > > +   /* Check if vq is in reset state. See more in virtnet_napi_enable() */
> > > > +   if (sq->vq->reset)
> > > > +           return;
> > >
> > >
> > > We've disabled TX napi, any chance we can still hit this?
> >
> >
> > static int virtnet_poll(struct napi_struct *napi, int budget)
> > {
> >         struct receive_queue *rq =
> >                 container_of(napi, struct receive_queue, napi);
> >         struct virtnet_info *vi = rq->vq->vdev->priv;
> >         struct send_queue *sq;
> >         unsigned int received;
> >         unsigned int xdp_xmit = 0;
> >
> >         virtnet_poll_cleantx(rq);
> > ...
> > }
> >
> > This is called by rx poll. Although it is the logic of tx, it is not driven by
> > tx napi, but is called in rx poll.
>
> Ok, but we need guarantee the memory ordering in this case. Disable RX
> napi could be a solution for this.

Yes, I have realized this too. I have two solutions, disable rx napi or the
following.

Thanks.


diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 9bf1b6530b38..7764d1dcb831 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -135,6 +135,7 @@ struct send_queue {
 	struct virtnet_sq_stats stats;

 	struct napi_struct napi;
+	bool reset;
 };

 /* Internal representation of a receive virtqueue */
@@ -1583,6 +1587,11 @@ static void virtnet_poll_cleantx(struct receive_queue *rq)
 		return;

 	if (__netif_tx_trylock(txq)) {
+		if (sq->reset) {
+			__netif_tx_unlock(txq);
+			return;
+		}
+
 		do {
 			virtqueue_disable_cb(sq->vq);
 			free_old_xmit_skbs(sq, true);
@@ -1828,6 +1837,56 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
 	return NETDEV_TX_OK;
 }

+static int virtnet_tx_resize(struct virtnet_info *vi,
+			     struct send_queue *sq, u32 ring_num)
+{
+	struct netdev_queue *txq;
+	int err, qindex;
+
+	qindex = sq - vi->sq;
+
+	virtnet_napi_tx_disable(&sq->napi);
+
+	txq = netdev_get_tx_queue(vi->dev, qindex);
+
+	__netif_tx_lock_bh(txq);
+	netif_stop_subqueue(vi->dev, qindex);
+	sq->reset = true;
+	__netif_tx_unlock_bh(txq);
+
+	err = virtqueue_resize(sq->vq, ring_num, virtnet_sq_free_unused_buf);
+	if (err)
+		netdev_err(vi->dev, "resize tx fail: tx queue index: %d err: %d\n", qindex, err);
+
+	__netif_tx_lock_bh(txq);
+	sq->reset = false;
+	netif_start_subqueue(vi->dev, qindex);
+	__netif_tx_unlock_bh(txq);
+
+	virtnet_napi_tx_enable(vi, sq->vq, &sq->napi);
+	return err;
+}
+


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 00/32] virtio pci support VIRTIO_F_RING_RESET (refactor vring)

[Michael S. Tsirkin]

On Wed, Apr 06, 2022 at 11:43:14AM +0800, Xuan Zhuo wrote:
> The virtio spec already supports the virtio queue reset function. This patch set
> is to add this function to the kernel. The relevant virtio spec information is
> here:
> 
>     https://github.com/oasis-tcs/virtio-spec/issues/124
> 
> Also regarding MMIO support for queue reset, I plan to support it after this
> patch is passed.

Regarding the spec, there's now an issue proposing
some changes to the interface. What do you think about that
proposal? Could you respond on that thread on the virtio TC mailing list?


> This patch set implements the refactoring of vring. Finally, the
> virtuque_resize() interface is provided based on the reset function of the
> transport layer.
> 
> Test environment:
>     Host: 4.19.91
>     Qemu: QEMU emulator version 6.2.50 (with vq reset support)
>     Test Cmd:  ethtool -G eth1 rx $1 tx $2; ethtool -g eth1
> 
>     The default is split mode, modify Qemu virtio-net to add PACKED feature to test
>     packed mode.
> 
> Qemu code:
>     https://github.com/fengidri/qemu/compare/89f3bfa3265554d1d591ee4d7f1197b6e3397e84...master
> 
> In order to simplify the review of this patch set, the function of reusing
> the old buffers after resize will be introduced in subsequent patch sets.
> 
> Please review. Thanks.
> 
> v9:
>   1. Provide a virtqueue_resize() interface directly
>   2. A patch set including vring resize, virtio pci reset, virtio-net resize
>   3. No more separate structs
> 
> v8:
>   1. Provide a virtqueue_reset() interface directly
>   2. Split the two patch sets, this is the first part
>   3. Add independent allocation helper for allocating state, extra
> 
> v7:
>   1. fix #6 subject typo
>   2. fix #6 ring_size_in_bytes is uninitialized
>   3. check by: make W=12
> 
> v6:
>   1. virtio_pci: use synchronize_irq(irq) to sync the irq callbacks
>   2. Introduce virtqueue_reset_vring() to implement the reset of vring during
>      the reset process. May use the old vring if num of the vq not change.
>   3. find_vqs() support sizes to special the max size of each vq
> 
> v5:
>   1. add virtio-net support set_ringparam
> 
> v4:
>   1. just the code of virtio, without virtio-net
>   2. Performing reset on a queue is divided into these steps:
>     1. reset_vq: reset one vq
>     2. recycle the buffer from vq by virtqueue_detach_unused_buf()
>     3. release the ring of the vq by vring_release_virtqueue()
>     4. enable_reset_vq: re-enable the reset queue
>   3. Simplify the parameters of enable_reset_vq()
>   4. add container structures for virtio_pci_common_cfg
> 
> v3:
>   1. keep vq, irq unreleased
> 
> Xuan Zhuo (32):
>   virtio: add helper virtqueue_get_vring_max_size()
>   virtio: struct virtio_config_ops add callbacks for queue_reset
>   virtio_ring: update the document of the virtqueue_detach_unused_buf
>     for queue reset
>   virtio_ring: remove the arg vq of vring_alloc_desc_extra()
>   virtio_ring: extract the logic of freeing vring
>   virtio_ring: split: extract the logic of alloc queue
>   virtio_ring: split: extract the logic of alloc state and extra
>   virtio_ring: split: extract the logic of attach vring
>   virtio_ring: split: extract the logic of vq init
>   virtio_ring: split: introduce virtqueue_reinit_split()
>   virtio_ring: split: introduce virtqueue_resize_split()
>   virtio_ring: packed: extract the logic of alloc queue
>   virtio_ring: packed: extract the logic of alloc state and extra
>   virtio_ring: packed: extract the logic of attach vring
>   virtio_ring: packed: extract the logic of vq init
>   virtio_ring: packed: introduce virtqueue_reinit_packed()
>   virtio_ring: packed: introduce virtqueue_resize_packed()
>   virtio_ring: introduce virtqueue_resize()
>   virtio_pci: struct virtio_pci_common_cfg add queue_notify_data
>   virtio: queue_reset: add VIRTIO_F_RING_RESET
>   virtio_pci: queue_reset: update struct virtio_pci_common_cfg and
>     option functions
>   virtio_pci: queue_reset: extract the logic of active vq for modern pci
>   virtio_pci: queue_reset: support VIRTIO_F_RING_RESET
>   virtio: find_vqs() add arg sizes
>   virtio_pci: support the arg sizes of find_vqs()
>   virtio_mmio: support the arg sizes of find_vqs()
>   virtio: add helper virtio_find_vqs_ctx_size()
>   virtio_net: set the default max ring size by find_vqs()
>   virtio_net: get ringparam by virtqueue_get_vring_max_size()
>   virtio_net: split free_unused_bufs()
>   virtio_net: support rx/tx queue resize
>   virtio_net: support set_ringparam
> 
>  arch/um/drivers/virtio_uml.c             |   3 +-
>  drivers/net/virtio_net.c                 | 219 +++++++-
>  drivers/platform/mellanox/mlxbf-tmfifo.c |   3 +
>  drivers/remoteproc/remoteproc_virtio.c   |   3 +
>  drivers/s390/virtio/virtio_ccw.c         |   4 +
>  drivers/virtio/virtio_mmio.c             |  11 +-
>  drivers/virtio/virtio_pci_common.c       |  28 +-
>  drivers/virtio/virtio_pci_common.h       |   3 +-
>  drivers/virtio/virtio_pci_legacy.c       |   8 +-
>  drivers/virtio/virtio_pci_modern.c       | 149 +++++-
>  drivers/virtio/virtio_pci_modern_dev.c   |  36 ++
>  drivers/virtio/virtio_ring.c             | 626 ++++++++++++++++++-----
>  drivers/virtio/virtio_vdpa.c             |   3 +
>  include/linux/virtio.h                   |   6 +
>  include/linux/virtio_config.h            |  38 +-
>  include/linux/virtio_pci_modern.h        |   2 +
>  include/uapi/linux/virtio_config.h       |   7 +-
>  include/uapi/linux/virtio_pci.h          |  14 +
>  18 files changed, 964 insertions(+), 199 deletions(-)
> 
> --
> 2.31.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 00/32] virtio pci support VIRTIO_F_RING_RESET (refactor vring)

[Michael S. Tsirkin]

On Wed, Apr 06, 2022 at 11:43:14AM +0800, Xuan Zhuo wrote:
> The virtio spec already supports the virtio queue reset function. This patch set
> is to add this function to the kernel. The relevant virtio spec information is
> here:
> 
>     https://github.com/oasis-tcs/virtio-spec/issues/124
> 
> Also regarding MMIO support for queue reset, I plan to support it after this
> patch is passed.

Regarding the spec, there's now an issue proposing
some changes to the interface. What do you think about that
proposal? Could you respond on that thread on the virtio TC mailing list?


> This patch set implements the refactoring of vring. Finally, the
> virtuque_resize() interface is provided based on the reset function of the
> transport layer.
> 
> Test environment:
>     Host: 4.19.91
>     Qemu: QEMU emulator version 6.2.50 (with vq reset support)
>     Test Cmd:  ethtool -G eth1 rx $1 tx $2; ethtool -g eth1
> 
>     The default is split mode, modify Qemu virtio-net to add PACKED feature to test
>     packed mode.
> 
> Qemu code:
>     https://github.com/fengidri/qemu/compare/89f3bfa3265554d1d591ee4d7f1197b6e3397e84...master
> 
> In order to simplify the review of this patch set, the function of reusing
> the old buffers after resize will be introduced in subsequent patch sets.
> 
> Please review. Thanks.
> 
> v9:
>   1. Provide a virtqueue_resize() interface directly
>   2. A patch set including vring resize, virtio pci reset, virtio-net resize
>   3. No more separate structs
> 
> v8:
>   1. Provide a virtqueue_reset() interface directly
>   2. Split the two patch sets, this is the first part
>   3. Add independent allocation helper for allocating state, extra
> 
> v7:
>   1. fix #6 subject typo
>   2. fix #6 ring_size_in_bytes is uninitialized
>   3. check by: make W=12
> 
> v6:
>   1. virtio_pci: use synchronize_irq(irq) to sync the irq callbacks
>   2. Introduce virtqueue_reset_vring() to implement the reset of vring during
>      the reset process. May use the old vring if num of the vq not change.
>   3. find_vqs() support sizes to special the max size of each vq
> 
> v5:
>   1. add virtio-net support set_ringparam
> 
> v4:
>   1. just the code of virtio, without virtio-net
>   2. Performing reset on a queue is divided into these steps:
>     1. reset_vq: reset one vq
>     2. recycle the buffer from vq by virtqueue_detach_unused_buf()
>     3. release the ring of the vq by vring_release_virtqueue()
>     4. enable_reset_vq: re-enable the reset queue
>   3. Simplify the parameters of enable_reset_vq()
>   4. add container structures for virtio_pci_common_cfg
> 
> v3:
>   1. keep vq, irq unreleased
> 
> Xuan Zhuo (32):
>   virtio: add helper virtqueue_get_vring_max_size()
>   virtio: struct virtio_config_ops add callbacks for queue_reset
>   virtio_ring: update the document of the virtqueue_detach_unused_buf
>     for queue reset
>   virtio_ring: remove the arg vq of vring_alloc_desc_extra()
>   virtio_ring: extract the logic of freeing vring
>   virtio_ring: split: extract the logic of alloc queue
>   virtio_ring: split: extract the logic of alloc state and extra
>   virtio_ring: split: extract the logic of attach vring
>   virtio_ring: split: extract the logic of vq init
>   virtio_ring: split: introduce virtqueue_reinit_split()
>   virtio_ring: split: introduce virtqueue_resize_split()
>   virtio_ring: packed: extract the logic of alloc queue
>   virtio_ring: packed: extract the logic of alloc state and extra
>   virtio_ring: packed: extract the logic of attach vring
>   virtio_ring: packed: extract the logic of vq init
>   virtio_ring: packed: introduce virtqueue_reinit_packed()
>   virtio_ring: packed: introduce virtqueue_resize_packed()
>   virtio_ring: introduce virtqueue_resize()
>   virtio_pci: struct virtio_pci_common_cfg add queue_notify_data
>   virtio: queue_reset: add VIRTIO_F_RING_RESET
>   virtio_pci: queue_reset: update struct virtio_pci_common_cfg and
>     option functions
>   virtio_pci: queue_reset: extract the logic of active vq for modern pci
>   virtio_pci: queue_reset: support VIRTIO_F_RING_RESET
>   virtio: find_vqs() add arg sizes
>   virtio_pci: support the arg sizes of find_vqs()
>   virtio_mmio: support the arg sizes of find_vqs()
>   virtio: add helper virtio_find_vqs_ctx_size()
>   virtio_net: set the default max ring size by find_vqs()
>   virtio_net: get ringparam by virtqueue_get_vring_max_size()
>   virtio_net: split free_unused_bufs()
>   virtio_net: support rx/tx queue resize
>   virtio_net: support set_ringparam
> 
>  arch/um/drivers/virtio_uml.c             |   3 +-
>  drivers/net/virtio_net.c                 | 219 +++++++-
>  drivers/platform/mellanox/mlxbf-tmfifo.c |   3 +
>  drivers/remoteproc/remoteproc_virtio.c   |   3 +
>  drivers/s390/virtio/virtio_ccw.c         |   4 +
>  drivers/virtio/virtio_mmio.c             |  11 +-
>  drivers/virtio/virtio_pci_common.c       |  28 +-
>  drivers/virtio/virtio_pci_common.h       |   3 +-
>  drivers/virtio/virtio_pci_legacy.c       |   8 +-
>  drivers/virtio/virtio_pci_modern.c       | 149 +++++-
>  drivers/virtio/virtio_pci_modern_dev.c   |  36 ++
>  drivers/virtio/virtio_ring.c             | 626 ++++++++++++++++++-----
>  drivers/virtio/virtio_vdpa.c             |   3 +
>  include/linux/virtio.h                   |   6 +
>  include/linux/virtio_config.h            |  38 +-
>  include/linux/virtio_pci_modern.h        |   2 +
>  include/uapi/linux/virtio_config.h       |   7 +-
>  include/uapi/linux/virtio_pci.h          |  14 +
>  18 files changed, 964 insertions(+), 199 deletions(-)
> 
> --
> 2.31.0

Re: [PATCH v9 00/32] virtio pci support VIRTIO_F_RING_RESET (refactor vring)

[Michael S. Tsirkin]

On Wed, Apr 06, 2022 at 11:43:14AM +0800, Xuan Zhuo wrote:
> The virtio spec already supports the virtio queue reset function. This patch set
> is to add this function to the kernel. The relevant virtio spec information is
> here:
> 
>     https://github.com/oasis-tcs/virtio-spec/issues/124
> 
> Also regarding MMIO support for queue reset, I plan to support it after this
> patch is passed.

Regarding the spec, there's now an issue proposing
some changes to the interface. What do you think about that
proposal? Could you respond on that thread on the virtio TC mailing list?


> This patch set implements the refactoring of vring. Finally, the
> virtuque_resize() interface is provided based on the reset function of the
> transport layer.
> 
> Test environment:
>     Host: 4.19.91
>     Qemu: QEMU emulator version 6.2.50 (with vq reset support)
>     Test Cmd:  ethtool -G eth1 rx $1 tx $2; ethtool -g eth1
> 
>     The default is split mode, modify Qemu virtio-net to add PACKED feature to test
>     packed mode.
> 
> Qemu code:
>     https://github.com/fengidri/qemu/compare/89f3bfa3265554d1d591ee4d7f1197b6e3397e84...master
> 
> In order to simplify the review of this patch set, the function of reusing
> the old buffers after resize will be introduced in subsequent patch sets.
> 
> Please review. Thanks.
> 
> v9:
>   1. Provide a virtqueue_resize() interface directly
>   2. A patch set including vring resize, virtio pci reset, virtio-net resize
>   3. No more separate structs
> 
> v8:
>   1. Provide a virtqueue_reset() interface directly
>   2. Split the two patch sets, this is the first part
>   3. Add independent allocation helper for allocating state, extra
> 
> v7:
>   1. fix #6 subject typo
>   2. fix #6 ring_size_in_bytes is uninitialized
>   3. check by: make W=12
> 
> v6:
>   1. virtio_pci: use synchronize_irq(irq) to sync the irq callbacks
>   2. Introduce virtqueue_reset_vring() to implement the reset of vring during
>      the reset process. May use the old vring if num of the vq not change.
>   3. find_vqs() support sizes to special the max size of each vq
> 
> v5:
>   1. add virtio-net support set_ringparam
> 
> v4:
>   1. just the code of virtio, without virtio-net
>   2. Performing reset on a queue is divided into these steps:
>     1. reset_vq: reset one vq
>     2. recycle the buffer from vq by virtqueue_detach_unused_buf()
>     3. release the ring of the vq by vring_release_virtqueue()
>     4. enable_reset_vq: re-enable the reset queue
>   3. Simplify the parameters of enable_reset_vq()
>   4. add container structures for virtio_pci_common_cfg
> 
> v3:
>   1. keep vq, irq unreleased
> 
> Xuan Zhuo (32):
>   virtio: add helper virtqueue_get_vring_max_size()
>   virtio: struct virtio_config_ops add callbacks for queue_reset
>   virtio_ring: update the document of the virtqueue_detach_unused_buf
>     for queue reset
>   virtio_ring: remove the arg vq of vring_alloc_desc_extra()
>   virtio_ring: extract the logic of freeing vring
>   virtio_ring: split: extract the logic of alloc queue
>   virtio_ring: split: extract the logic of alloc state and extra
>   virtio_ring: split: extract the logic of attach vring
>   virtio_ring: split: extract the logic of vq init
>   virtio_ring: split: introduce virtqueue_reinit_split()
>   virtio_ring: split: introduce virtqueue_resize_split()
>   virtio_ring: packed: extract the logic of alloc queue
>   virtio_ring: packed: extract the logic of alloc state and extra
>   virtio_ring: packed: extract the logic of attach vring
>   virtio_ring: packed: extract the logic of vq init
>   virtio_ring: packed: introduce virtqueue_reinit_packed()
>   virtio_ring: packed: introduce virtqueue_resize_packed()
>   virtio_ring: introduce virtqueue_resize()
>   virtio_pci: struct virtio_pci_common_cfg add queue_notify_data
>   virtio: queue_reset: add VIRTIO_F_RING_RESET
>   virtio_pci: queue_reset: update struct virtio_pci_common_cfg and
>     option functions
>   virtio_pci: queue_reset: extract the logic of active vq for modern pci
>   virtio_pci: queue_reset: support VIRTIO_F_RING_RESET
>   virtio: find_vqs() add arg sizes
>   virtio_pci: support the arg sizes of find_vqs()
>   virtio_mmio: support the arg sizes of find_vqs()
>   virtio: add helper virtio_find_vqs_ctx_size()
>   virtio_net: set the default max ring size by find_vqs()
>   virtio_net: get ringparam by virtqueue_get_vring_max_size()
>   virtio_net: split free_unused_bufs()
>   virtio_net: support rx/tx queue resize
>   virtio_net: support set_ringparam
> 
>  arch/um/drivers/virtio_uml.c             |   3 +-
>  drivers/net/virtio_net.c                 | 219 +++++++-
>  drivers/platform/mellanox/mlxbf-tmfifo.c |   3 +
>  drivers/remoteproc/remoteproc_virtio.c   |   3 +
>  drivers/s390/virtio/virtio_ccw.c         |   4 +
>  drivers/virtio/virtio_mmio.c             |  11 +-
>  drivers/virtio/virtio_pci_common.c       |  28 +-
>  drivers/virtio/virtio_pci_common.h       |   3 +-
>  drivers/virtio/virtio_pci_legacy.c       |   8 +-
>  drivers/virtio/virtio_pci_modern.c       | 149 +++++-
>  drivers/virtio/virtio_pci_modern_dev.c   |  36 ++
>  drivers/virtio/virtio_ring.c             | 626 ++++++++++++++++++-----
>  drivers/virtio/virtio_vdpa.c             |   3 +
>  include/linux/virtio.h                   |   6 +
>  include/linux/virtio_config.h            |  38 +-
>  include/linux/virtio_pci_modern.h        |   2 +
>  include/uapi/linux/virtio_config.h       |   7 +-
>  include/uapi/linux/virtio_pci.h          |  14 +
>  18 files changed, 964 insertions(+), 199 deletions(-)
> 
> --
> 2.31.0


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: [PATCH v9 00/32] virtio pci support VIRTIO_F_RING_RESET (refactor vring)

[Xuan Zhuo]

On Tue, 26 Apr 2022 05:55:41 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Wed, Apr 06, 2022 at 11:43:14AM +0800, Xuan Zhuo wrote:
> > The virtio spec already supports the virtio queue reset function. This patch set
> > is to add this function to the kernel. The relevant virtio spec information is
> > here:
> >
> >     https://github.com/oasis-tcs/virtio-spec/issues/124
> >
> > Also regarding MMIO support for queue reset, I plan to support it after this
> > patch is passed.
>
> Regarding the spec, there's now an issue proposing
> some changes to the interface. What do you think about that
> proposal? Could you respond on that thread on the virtio TC mailing list?

I have read that thread. And also asked some questions.  I will follow that
thread.

Grateful.

>
>
> > This patch set implements the refactoring of vring. Finally, the
> > virtuque_resize() interface is provided based on the reset function of the
> > transport layer.
> >
> > Test environment:
> >     Host: 4.19.91
> >     Qemu: QEMU emulator version 6.2.50 (with vq reset support)
> >     Test Cmd:  ethtool -G eth1 rx $1 tx $2; ethtool -g eth1
> >
> >     The default is split mode, modify Qemu virtio-net to add PACKED feature to test
> >     packed mode.
> >
> > Qemu code:
> >     https://github.com/fengidri/qemu/compare/89f3bfa3265554d1d591ee4d7f1197b6e3397e84...master
> >
> > In order to simplify the review of this patch set, the function of reusing
> > the old buffers after resize will be introduced in subsequent patch sets.
> >
> > Please review. Thanks.
> >
> > v9:
> >   1. Provide a virtqueue_resize() interface directly
> >   2. A patch set including vring resize, virtio pci reset, virtio-net resize
> >   3. No more separate structs
> >
> > v8:
> >   1. Provide a virtqueue_reset() interface directly
> >   2. Split the two patch sets, this is the first part
> >   3. Add independent allocation helper for allocating state, extra
> >
> > v7:
> >   1. fix #6 subject typo
> >   2. fix #6 ring_size_in_bytes is uninitialized
> >   3. check by: make W=12
> >
> > v6:
> >   1. virtio_pci: use synchronize_irq(irq) to sync the irq callbacks
> >   2. Introduce virtqueue_reset_vring() to implement the reset of vring during
> >      the reset process. May use the old vring if num of the vq not change.
> >   3. find_vqs() support sizes to special the max size of each vq
> >
> > v5:
> >   1. add virtio-net support set_ringparam
> >
> > v4:
> >   1. just the code of virtio, without virtio-net
> >   2. Performing reset on a queue is divided into these steps:
> >     1. reset_vq: reset one vq
> >     2. recycle the buffer from vq by virtqueue_detach_unused_buf()
> >     3. release the ring of the vq by vring_release_virtqueue()
> >     4. enable_reset_vq: re-enable the reset queue
> >   3. Simplify the parameters of enable_reset_vq()
> >   4. add container structures for virtio_pci_common_cfg
> >
> > v3:
> >   1. keep vq, irq unreleased
> >
> > Xuan Zhuo (32):
> >   virtio: add helper virtqueue_get_vring_max_size()
> >   virtio: struct virtio_config_ops add callbacks for queue_reset
> >   virtio_ring: update the document of the virtqueue_detach_unused_buf
> >     for queue reset
> >   virtio_ring: remove the arg vq of vring_alloc_desc_extra()
> >   virtio_ring: extract the logic of freeing vring
> >   virtio_ring: split: extract the logic of alloc queue
> >   virtio_ring: split: extract the logic of alloc state and extra
> >   virtio_ring: split: extract the logic of attach vring
> >   virtio_ring: split: extract the logic of vq init
> >   virtio_ring: split: introduce virtqueue_reinit_split()
> >   virtio_ring: split: introduce virtqueue_resize_split()
> >   virtio_ring: packed: extract the logic of alloc queue
> >   virtio_ring: packed: extract the logic of alloc state and extra
> >   virtio_ring: packed: extract the logic of attach vring
> >   virtio_ring: packed: extract the logic of vq init
> >   virtio_ring: packed: introduce virtqueue_reinit_packed()
> >   virtio_ring: packed: introduce virtqueue_resize_packed()
> >   virtio_ring: introduce virtqueue_resize()
> >   virtio_pci: struct virtio_pci_common_cfg add queue_notify_data
> >   virtio: queue_reset: add VIRTIO_F_RING_RESET
> >   virtio_pci: queue_reset: update struct virtio_pci_common_cfg and
> >     option functions
> >   virtio_pci: queue_reset: extract the logic of active vq for modern pci
> >   virtio_pci: queue_reset: support VIRTIO_F_RING_RESET
> >   virtio: find_vqs() add arg sizes
> >   virtio_pci: support the arg sizes of find_vqs()
> >   virtio_mmio: support the arg sizes of find_vqs()
> >   virtio: add helper virtio_find_vqs_ctx_size()
> >   virtio_net: set the default max ring size by find_vqs()
> >   virtio_net: get ringparam by virtqueue_get_vring_max_size()
> >   virtio_net: split free_unused_bufs()
> >   virtio_net: support rx/tx queue resize
> >   virtio_net: support set_ringparam
> >
> >  arch/um/drivers/virtio_uml.c             |   3 +-
> >  drivers/net/virtio_net.c                 | 219 +++++++-
> >  drivers/platform/mellanox/mlxbf-tmfifo.c |   3 +
> >  drivers/remoteproc/remoteproc_virtio.c   |   3 +
> >  drivers/s390/virtio/virtio_ccw.c         |   4 +
> >  drivers/virtio/virtio_mmio.c             |  11 +-
> >  drivers/virtio/virtio_pci_common.c       |  28 +-
> >  drivers/virtio/virtio_pci_common.h       |   3 +-
> >  drivers/virtio/virtio_pci_legacy.c       |   8 +-
> >  drivers/virtio/virtio_pci_modern.c       | 149 +++++-
> >  drivers/virtio/virtio_pci_modern_dev.c   |  36 ++
> >  drivers/virtio/virtio_ring.c             | 626 ++++++++++++++++++-----
> >  drivers/virtio/virtio_vdpa.c             |   3 +
> >  include/linux/virtio.h                   |   6 +
> >  include/linux/virtio_config.h            |  38 +-
> >  include/linux/virtio_pci_modern.h        |   2 +
> >  include/uapi/linux/virtio_config.h       |   7 +-
> >  include/uapi/linux/virtio_pci.h          |  14 +
> >  18 files changed, 964 insertions(+), 199 deletions(-)
> >
> > --
> > 2.31.0
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v9 00/32] virtio pci support VIRTIO_F_RING_RESET (refactor vring)

[Xuan Zhuo]

On Tue, 26 Apr 2022 05:55:41 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Wed, Apr 06, 2022 at 11:43:14AM +0800, Xuan Zhuo wrote:
> > The virtio spec already supports the virtio queue reset function. This patch set
> > is to add this function to the kernel. The relevant virtio spec information is
> > here:
> >
> >     https://github.com/oasis-tcs/virtio-spec/issues/124
> >
> > Also regarding MMIO support for queue reset, I plan to support it after this
> > patch is passed.
>
> Regarding the spec, there's now an issue proposing
> some changes to the interface. What do you think about that
> proposal? Could you respond on that thread on the virtio TC mailing list?

I have read that thread. And also asked some questions.  I will follow that
thread.

Grateful.

>
>
> > This patch set implements the refactoring of vring. Finally, the
> > virtuque_resize() interface is provided based on the reset function of the
> > transport layer.
> >
> > Test environment:
> >     Host: 4.19.91
> >     Qemu: QEMU emulator version 6.2.50 (with vq reset support)
> >     Test Cmd:  ethtool -G eth1 rx $1 tx $2; ethtool -g eth1
> >
> >     The default is split mode, modify Qemu virtio-net to add PACKED feature to test
> >     packed mode.
> >
> > Qemu code:
> >     https://github.com/fengidri/qemu/compare/89f3bfa3265554d1d591ee4d7f1197b6e3397e84...master
> >
> > In order to simplify the review of this patch set, the function of reusing
> > the old buffers after resize will be introduced in subsequent patch sets.
> >
> > Please review. Thanks.
> >
> > v9:
> >   1. Provide a virtqueue_resize() interface directly
> >   2. A patch set including vring resize, virtio pci reset, virtio-net resize
> >   3. No more separate structs
> >
> > v8:
> >   1. Provide a virtqueue_reset() interface directly
> >   2. Split the two patch sets, this is the first part
> >   3. Add independent allocation helper for allocating state, extra
> >
> > v7:
> >   1. fix #6 subject typo
> >   2. fix #6 ring_size_in_bytes is uninitialized
> >   3. check by: make W=12
> >
> > v6:
> >   1. virtio_pci: use synchronize_irq(irq) to sync the irq callbacks
> >   2. Introduce virtqueue_reset_vring() to implement the reset of vring during
> >      the reset process. May use the old vring if num of the vq not change.
> >   3. find_vqs() support sizes to special the max size of each vq
> >
> > v5:
> >   1. add virtio-net support set_ringparam
> >
> > v4:
> >   1. just the code of virtio, without virtio-net
> >   2. Performing reset on a queue is divided into these steps:
> >     1. reset_vq: reset one vq
> >     2. recycle the buffer from vq by virtqueue_detach_unused_buf()
> >     3. release the ring of the vq by vring_release_virtqueue()
> >     4. enable_reset_vq: re-enable the reset queue
> >   3. Simplify the parameters of enable_reset_vq()
> >   4. add container structures for virtio_pci_common_cfg
> >
> > v3:
> >   1. keep vq, irq unreleased
> >
> > Xuan Zhuo (32):
> >   virtio: add helper virtqueue_get_vring_max_size()
> >   virtio: struct virtio_config_ops add callbacks for queue_reset
> >   virtio_ring: update the document of the virtqueue_detach_unused_buf
> >     for queue reset
> >   virtio_ring: remove the arg vq of vring_alloc_desc_extra()
> >   virtio_ring: extract the logic of freeing vring
> >   virtio_ring: split: extract the logic of alloc queue
> >   virtio_ring: split: extract the logic of alloc state and extra
> >   virtio_ring: split: extract the logic of attach vring
> >   virtio_ring: split: extract the logic of vq init
> >   virtio_ring: split: introduce virtqueue_reinit_split()
> >   virtio_ring: split: introduce virtqueue_resize_split()
> >   virtio_ring: packed: extract the logic of alloc queue
> >   virtio_ring: packed: extract the logic of alloc state and extra
> >   virtio_ring: packed: extract the logic of attach vring
> >   virtio_ring: packed: extract the logic of vq init
> >   virtio_ring: packed: introduce virtqueue_reinit_packed()
> >   virtio_ring: packed: introduce virtqueue_resize_packed()
> >   virtio_ring: introduce virtqueue_resize()
> >   virtio_pci: struct virtio_pci_common_cfg add queue_notify_data
> >   virtio: queue_reset: add VIRTIO_F_RING_RESET
> >   virtio_pci: queue_reset: update struct virtio_pci_common_cfg and
> >     option functions
> >   virtio_pci: queue_reset: extract the logic of active vq for modern pci
> >   virtio_pci: queue_reset: support VIRTIO_F_RING_RESET
> >   virtio: find_vqs() add arg sizes
> >   virtio_pci: support the arg sizes of find_vqs()
> >   virtio_mmio: support the arg sizes of find_vqs()
> >   virtio: add helper virtio_find_vqs_ctx_size()
> >   virtio_net: set the default max ring size by find_vqs()
> >   virtio_net: get ringparam by virtqueue_get_vring_max_size()
> >   virtio_net: split free_unused_bufs()
> >   virtio_net: support rx/tx queue resize
> >   virtio_net: support set_ringparam
> >
> >  arch/um/drivers/virtio_uml.c             |   3 +-
> >  drivers/net/virtio_net.c                 | 219 +++++++-
> >  drivers/platform/mellanox/mlxbf-tmfifo.c |   3 +
> >  drivers/remoteproc/remoteproc_virtio.c   |   3 +
> >  drivers/s390/virtio/virtio_ccw.c         |   4 +
> >  drivers/virtio/virtio_mmio.c             |  11 +-
> >  drivers/virtio/virtio_pci_common.c       |  28 +-
> >  drivers/virtio/virtio_pci_common.h       |   3 +-
> >  drivers/virtio/virtio_pci_legacy.c       |   8 +-
> >  drivers/virtio/virtio_pci_modern.c       | 149 +++++-
> >  drivers/virtio/virtio_pci_modern_dev.c   |  36 ++
> >  drivers/virtio/virtio_ring.c             | 626 ++++++++++++++++++-----
> >  drivers/virtio/virtio_vdpa.c             |   3 +
> >  include/linux/virtio.h                   |   6 +
> >  include/linux/virtio_config.h            |  38 +-
> >  include/linux/virtio_pci_modern.h        |   2 +
> >  include/uapi/linux/virtio_config.h       |   7 +-
> >  include/uapi/linux/virtio_pci.h          |  14 +
> >  18 files changed, 964 insertions(+), 199 deletions(-)
> >
> > --
> > 2.31.0
>

Re: [PATCH v9 00/32] virtio pci support VIRTIO_F_RING_RESET (refactor vring)

[Xuan Zhuo]

On Tue, 26 Apr 2022 05:55:41 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Wed, Apr 06, 2022 at 11:43:14AM +0800, Xuan Zhuo wrote:
> > The virtio spec already supports the virtio queue reset function. This patch set
> > is to add this function to the kernel. The relevant virtio spec information is
> > here:
> >
> >     https://github.com/oasis-tcs/virtio-spec/issues/124
> >
> > Also regarding MMIO support for queue reset, I plan to support it after this
> > patch is passed.
>
> Regarding the spec, there's now an issue proposing
> some changes to the interface. What do you think about that
> proposal? Could you respond on that thread on the virtio TC mailing list?

I have read that thread. And also asked some questions.  I will follow that
thread.

Grateful.

>
>
> > This patch set implements the refactoring of vring. Finally, the
> > virtuque_resize() interface is provided based on the reset function of the
> > transport layer.
> >
> > Test environment:
> >     Host: 4.19.91
> >     Qemu: QEMU emulator version 6.2.50 (with vq reset support)
> >     Test Cmd:  ethtool -G eth1 rx $1 tx $2; ethtool -g eth1
> >
> >     The default is split mode, modify Qemu virtio-net to add PACKED feature to test
> >     packed mode.
> >
> > Qemu code:
> >     https://github.com/fengidri/qemu/compare/89f3bfa3265554d1d591ee4d7f1197b6e3397e84...master
> >
> > In order to simplify the review of this patch set, the function of reusing
> > the old buffers after resize will be introduced in subsequent patch sets.
> >
> > Please review. Thanks.
> >
> > v9:
> >   1. Provide a virtqueue_resize() interface directly
> >   2. A patch set including vring resize, virtio pci reset, virtio-net resize
> >   3. No more separate structs
> >
> > v8:
> >   1. Provide a virtqueue_reset() interface directly
> >   2. Split the two patch sets, this is the first part
> >   3. Add independent allocation helper for allocating state, extra
> >
> > v7:
> >   1. fix #6 subject typo
> >   2. fix #6 ring_size_in_bytes is uninitialized
> >   3. check by: make W=12
> >
> > v6:
> >   1. virtio_pci: use synchronize_irq(irq) to sync the irq callbacks
> >   2. Introduce virtqueue_reset_vring() to implement the reset of vring during
> >      the reset process. May use the old vring if num of the vq not change.
> >   3. find_vqs() support sizes to special the max size of each vq
> >
> > v5:
> >   1. add virtio-net support set_ringparam
> >
> > v4:
> >   1. just the code of virtio, without virtio-net
> >   2. Performing reset on a queue is divided into these steps:
> >     1. reset_vq: reset one vq
> >     2. recycle the buffer from vq by virtqueue_detach_unused_buf()
> >     3. release the ring of the vq by vring_release_virtqueue()
> >     4. enable_reset_vq: re-enable the reset queue
> >   3. Simplify the parameters of enable_reset_vq()
> >   4. add container structures for virtio_pci_common_cfg
> >
> > v3:
> >   1. keep vq, irq unreleased
> >
> > Xuan Zhuo (32):
> >   virtio: add helper virtqueue_get_vring_max_size()
> >   virtio: struct virtio_config_ops add callbacks for queue_reset
> >   virtio_ring: update the document of the virtqueue_detach_unused_buf
> >     for queue reset
> >   virtio_ring: remove the arg vq of vring_alloc_desc_extra()
> >   virtio_ring: extract the logic of freeing vring
> >   virtio_ring: split: extract the logic of alloc queue
> >   virtio_ring: split: extract the logic of alloc state and extra
> >   virtio_ring: split: extract the logic of attach vring
> >   virtio_ring: split: extract the logic of vq init
> >   virtio_ring: split: introduce virtqueue_reinit_split()
> >   virtio_ring: split: introduce virtqueue_resize_split()
> >   virtio_ring: packed: extract the logic of alloc queue
> >   virtio_ring: packed: extract the logic of alloc state and extra
> >   virtio_ring: packed: extract the logic of attach vring
> >   virtio_ring: packed: extract the logic of vq init
> >   virtio_ring: packed: introduce virtqueue_reinit_packed()
> >   virtio_ring: packed: introduce virtqueue_resize_packed()
> >   virtio_ring: introduce virtqueue_resize()
> >   virtio_pci: struct virtio_pci_common_cfg add queue_notify_data
> >   virtio: queue_reset: add VIRTIO_F_RING_RESET
> >   virtio_pci: queue_reset: update struct virtio_pci_common_cfg and
> >     option functions
> >   virtio_pci: queue_reset: extract the logic of active vq for modern pci
> >   virtio_pci: queue_reset: support VIRTIO_F_RING_RESET
> >   virtio: find_vqs() add arg sizes
> >   virtio_pci: support the arg sizes of find_vqs()
> >   virtio_mmio: support the arg sizes of find_vqs()
> >   virtio: add helper virtio_find_vqs_ctx_size()
> >   virtio_net: set the default max ring size by find_vqs()
> >   virtio_net: get ringparam by virtqueue_get_vring_max_size()
> >   virtio_net: split free_unused_bufs()
> >   virtio_net: support rx/tx queue resize
> >   virtio_net: support set_ringparam
> >
> >  arch/um/drivers/virtio_uml.c             |   3 +-
> >  drivers/net/virtio_net.c                 | 219 +++++++-
> >  drivers/platform/mellanox/mlxbf-tmfifo.c |   3 +
> >  drivers/remoteproc/remoteproc_virtio.c   |   3 +
> >  drivers/s390/virtio/virtio_ccw.c         |   4 +
> >  drivers/virtio/virtio_mmio.c             |  11 +-
> >  drivers/virtio/virtio_pci_common.c       |  28 +-
> >  drivers/virtio/virtio_pci_common.h       |   3 +-
> >  drivers/virtio/virtio_pci_legacy.c       |   8 +-
> >  drivers/virtio/virtio_pci_modern.c       | 149 +++++-
> >  drivers/virtio/virtio_pci_modern_dev.c   |  36 ++
> >  drivers/virtio/virtio_ring.c             | 626 ++++++++++++++++++-----
> >  drivers/virtio/virtio_vdpa.c             |   3 +
> >  include/linux/virtio.h                   |   6 +
> >  include/linux/virtio_config.h            |  38 +-
> >  include/linux/virtio_pci_modern.h        |   2 +
> >  include/uapi/linux/virtio_config.h       |   7 +-
> >  include/uapi/linux/virtio_pci.h          |  14 +
> >  18 files changed, 964 insertions(+), 199 deletions(-)
> >
> > --
> > 2.31.0
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um