[PATCH v2 0/2] net/virtio: vhost-vdpa fixes

[PATCH v2 0/2] net/virtio: vhost-vdpa fixes

[Maxime Coquelin]

While investigating vhost-vdpa initialization issue with mlx5
vDPA, we found two issues fixed by following patches.

In this v2, the control queue issue mentioned in v1 is
fixed. It turned out to the control queue being enabled
only if multiqueue was negotiated. It is fixed by enabling
it at device startup, and disabling it at stop time.

We still have an issue on one of our setup with mlx5, where
the mlx5 device sets VIRTIO_CONFIG_S_FAILED status, it is
currently being investigated.

Changes in v2:
--------------
- Fix cvq enablement
- Fix typo in commit message (David)


Maxime Coquelin (2):
  net/virtio: fix vDPA device init advertising control queue
  net/virtio: fix notification area initialization

 .../net/virtio/virtio_user/virtio_user_dev.c  | 27 +++++++++++++------
 1 file changed, 19 insertions(+), 8 deletions(-)

-- 
2.44.0

[PATCH v2 1/2] net/virtio: fix vDPA device init advertising control queue

[Maxime Coquelin]

If the vDPA device advertises control queue support, but
the user neither passes "cq=1" as devarg nor requests
multiple queues, the initialization fails because the
driver tries to setup the control queue without negotiating
related feature.

This patch enables the control queue at driver level as
soon as the device claims to support it, and not only when
multiple queue pairs are requested. Also, enable the
control queue event if multiqueue feature has not been
negotiated and device start time, and disable it at device
stop time.

Fixes: b277308e8868 ("net/virtio-user: advertise control VQ support with vDPA")
Cc: stable@dpdk.org

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 .../net/virtio/virtio_user/virtio_user_dev.c  | 22 ++++++++++++++-----
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index d395fc1676..54187fedf5 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -216,6 +216,12 @@ virtio_user_start_device(struct virtio_user_dev *dev)
 	if (ret < 0)
 		goto error;
 
+	if (dev->scvq) {
+		ret = dev->ops->cvq_enable(dev, 1);
+		if (ret < 0)
+			goto error;
+	}
+
 	dev->started = true;
 
 	pthread_mutex_unlock(&dev->mutex);
@@ -248,6 +254,12 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
 			goto err;
 	}
 
+	if (dev->scvq) {
+		ret = dev->ops->cvq_enable(dev, 0);
+		if (ret < 0)
+			goto err;
+	}
+
 	/* Stop the backend. */
 	for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
 		state.index = i;
@@ -752,7 +764,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, uint16_t queues,
 	if (virtio_user_dev_init_max_queue_pairs(dev, queues))
 		dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
 
-	if (dev->max_queue_pairs > 1)
+	if (dev->max_queue_pairs > 1 || dev->hw_cvq)
 		cq = 1;
 
 	if (!mrg_rxbuf)
@@ -770,8 +782,9 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, uint16_t queues,
 		dev->unsupported_features |= (1ull << VIRTIO_NET_F_MAC);
 
 	if (cq) {
-		/* device does not really need to know anything about CQ,
-		 * so if necessary, we just claim to support CQ
+		/* Except for vDPA, the device does not really need to know
+		 * anything about CQ, so if necessary, we just claim to support
+		 * control queue.
 		 */
 		dev->frontend_features |= (1ull << VIRTIO_NET_F_CTRL_VQ);
 	} else {
@@ -871,9 +884,6 @@ virtio_user_handle_mq(struct virtio_user_dev *dev, uint16_t q_pairs)
 	for (i = q_pairs; i < dev->max_queue_pairs; ++i)
 		ret |= dev->ops->enable_qp(dev, i, 0);
 
-	if (dev->scvq)
-		ret |= dev->ops->cvq_enable(dev, 1);
-
 	dev->queue_pairs = q_pairs;
 
 	return ret;
-- 
2.44.0

[PATCH v2 2/2] net/virtio: fix notification area initialization

[Maxime Coquelin]

Notification area is a Virtio feature that requires to be
negotiated because not all devices support it. Currently,
it is tried to be initialized as soon as the backend
implements the callback, so it assumes all Vhost-vDPA
devices support it.

This patch skips calling the notification area map callback
if the device does not advertise its support.

Fixes: 0fd2782660c8 ("net/virtio-user: support notification area mapping")

Reviewed-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 drivers/net/virtio/virtio_user/virtio_user_dev.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 54187fedf5..4fdfe70e7c 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -445,8 +445,9 @@ virtio_user_dev_init_notify(struct virtio_user_dev *dev)
 		dev->kickfds[i] = kickfd;
 	}
 
-	if (dev->ops->map_notification_area)
-		if (dev->ops->map_notification_area(dev))
+	if (dev->device_features & (1ULL << VIRTIO_F_NOTIFICATION_DATA))
+		if (dev->ops->map_notification_area &&
+				dev->ops->map_notification_area(dev))
 			goto err;
 
 	return 0;
-- 
2.44.0

Re: [PATCH v2 1/2] net/virtio: fix vDPA device init advertising control queue

[David Marchand]

On Wed, Mar 13, 2024 at 1:59 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> If the vDPA device advertises control queue support, but
> the user neither passes "cq=1" as devarg nor requests
> multiple queues, the initialization fails because the
> driver tries to setup the control queue without negotiating
> related feature.
>
> This patch enables the control queue at driver level as
> soon as the device claims to support it, and not only when
> multiple queue pairs are requested. Also, enable the
> control queue event if multiqueue feature has not been
> negotiated and device start time, and disable it at device
> stop time.
>
> Fixes: b277308e8868 ("net/virtio-user: advertise control VQ support with vDPA")
> Cc: stable@dpdk.org
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Reviewed-by: David Marchand <david.marchand@redhat.com>


-- 
David Marchand

Re: [PATCH v2 0/2] net/virtio: vhost-vdpa fixes

[David Marchand]

On Wed, Mar 13, 2024 at 1:59 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> While investigating vhost-vdpa initialization issue with mlx5
> vDPA, we found two issues fixed by following patches.
>
> In this v2, the control queue issue mentioned in v1 is
> fixed. It turned out to the control queue being enabled
> only if multiqueue was negotiated. It is fixed by enabling
> it at device startup, and disabling it at stop time.
>
> We still have an issue on one of our setup with mlx5, where
> the mlx5 device sets VIRTIO_CONFIG_S_FAILED status, it is
> currently being investigated.

v2 is working fine on my system, what else matters? :-)

The current fixes in this series make sense.
We may do followup fixes in the next release.


>
> Changes in v2:
> --------------
> - Fix cvq enablement
> - Fix typo in commit message (David)
>
>
> Maxime Coquelin (2):
>   net/virtio: fix vDPA device init advertising control queue
>   net/virtio: fix notification area initialization
>
>  .../net/virtio/virtio_user/virtio_user_dev.c  | 27 +++++++++++++------
>  1 file changed, 19 insertions(+), 8 deletions(-)
>

Series applied, thanks.


-- 
David Marchand