All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/5] Vhost-user: add Virtio RSS support
@ 2022-04-08 12:28 Maxime Coquelin
  2022-04-08 12:28 ` [PATCH 1/5] ebpf: pass and check RSS key length to the loader Maxime Coquelin
                   ` (5 more replies)
  0 siblings, 6 replies; 11+ messages in thread
From: Maxime Coquelin @ 2022-04-08 12:28 UTC (permalink / raw)
  To: qemu-devel, mst, jasowang, andrew, yuri.benditovich, dgilbert, quintela
  Cc: chenbo.xia, Maxime Coquelin, dmarchan, ktraynor

The goal of this series is to add support for Virtio RSS
feature to the Vhost-user backend.

First patches are preliminary reworks to support variable
RSS key and indirection table length. eBPF change only adds
checks on whether the key length is 40B, it does not add
support for longer keys.

Vhost-user implementation supports up to 52B RSS key, in
order to match with the maximum supported by physical
NICs (Intel E810). Idea is that it could be used for
application like Virtio-forwarder, by programming the
Virtio device RSS key into the physical NIC and let the
physical NIC do the packets distribution.

DPDK Vhost-user backend PoC implementing the new requests
can be found here [0], it only implements the messages
handling, it does not perform any RSS for now.

[0]: https://gitlab.com/mcoquelin/dpdk-next-virtio/-/commits/vhost_user_rss_poc/

Maxime Coquelin (5):
  ebpf: pass and check RSS key length to the loader
  virtio-net: prepare for variable RSS key and indir table lengths
  virtio-net: add RSS support for Vhost backends
  docs: introduce RSS support in Vhost-user specification
  vhost-user: add RSS support

 docs/interop/vhost-user.rst       |  57 ++++++++++++
 ebpf/ebpf_rss-stub.c              |   3 +-
 ebpf/ebpf_rss.c                   |  17 ++--
 ebpf/ebpf_rss.h                   |   3 +-
 hw/net/vhost_net-stub.c           |  10 ++
 hw/net/vhost_net.c                |  22 +++++
 hw/net/virtio-net.c               |  87 +++++++++++++-----
 hw/virtio/vhost-user.c            | 146 +++++++++++++++++++++++++++++-
 include/hw/virtio/vhost-backend.h |   7 ++
 include/hw/virtio/virtio-net.h    |  16 +++-
 include/migration/vmstate.h       |  10 ++
 include/net/vhost_net.h           |   4 +
 12 files changed, 344 insertions(+), 38 deletions(-)

-- 
2.35.1



^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH 1/5] ebpf: pass and check RSS key length to the loader
  2022-04-08 12:28 [PATCH 0/5] Vhost-user: add Virtio RSS support Maxime Coquelin
@ 2022-04-08 12:28 ` Maxime Coquelin
  2022-04-08 12:28 ` [PATCH 2/5] virtio-net: prepare for variable RSS key and indir table lengths Maxime Coquelin
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 11+ messages in thread
From: Maxime Coquelin @ 2022-04-08 12:28 UTC (permalink / raw)
  To: qemu-devel, mst, jasowang, andrew, yuri.benditovich, dgilbert, quintela
  Cc: chenbo.xia, Maxime Coquelin, dmarchan, ktraynor

This patch is preliminary rework to support RSS with
Vhost-user backends. The Vhost-user implementation will
allow RSS hash key of 40 bytes or more as allowed by the
Virtio specification, whereas the eBPF-based Vhost-kernel
solution only supports 40 bytes keys.

This patch adds the RSS key length to the loader, and
validate it is 40 bytes before copying it.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 ebpf/ebpf_rss-stub.c |  3 ++-
 ebpf/ebpf_rss.c      | 11 +++++++----
 ebpf/ebpf_rss.h      |  3 ++-
 hw/net/virtio-net.c  |  3 ++-
 4 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/ebpf/ebpf_rss-stub.c b/ebpf/ebpf_rss-stub.c
index e71e229190..ffc5c5574f 100644
--- a/ebpf/ebpf_rss-stub.c
+++ b/ebpf/ebpf_rss-stub.c
@@ -29,7 +29,8 @@ bool ebpf_rss_load(struct EBPFRSSContext *ctx)
 }
 
 bool ebpf_rss_set_all(struct EBPFRSSContext *ctx, struct EBPFRSSConfig *config,
-                      uint16_t *indirections_table, uint8_t *toeplitz_key)
+                      uint16_t *indirections_table, uint8_t *toeplitz_key,
+                      uint8_t key_len)
 {
     return false;
 }
diff --git a/ebpf/ebpf_rss.c b/ebpf/ebpf_rss.c
index 118c68da83..4a63854175 100644
--- a/ebpf/ebpf_rss.c
+++ b/ebpf/ebpf_rss.c
@@ -110,14 +110,16 @@ static bool ebpf_rss_set_indirections_table(struct EBPFRSSContext *ctx,
 }
 
 static bool ebpf_rss_set_toepliz_key(struct EBPFRSSContext *ctx,
-                                     uint8_t *toeplitz_key)
+                                     uint8_t *toeplitz_key,
+                                     size_t len)
 {
     uint32_t map_key = 0;
 
     /* prepare toeplitz key */
     uint8_t toe[VIRTIO_NET_RSS_MAX_KEY_SIZE] = {};
 
-    if (!ebpf_rss_is_loaded(ctx) || toeplitz_key == NULL) {
+    if (!ebpf_rss_is_loaded(ctx) || toeplitz_key == NULL ||
+            len != VIRTIO_NET_RSS_MAX_KEY_SIZE) {
         return false;
     }
     memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_MAX_KEY_SIZE);
@@ -131,7 +133,8 @@ static bool ebpf_rss_set_toepliz_key(struct EBPFRSSContext *ctx,
 }
 
 bool ebpf_rss_set_all(struct EBPFRSSContext *ctx, struct EBPFRSSConfig *config,
-                      uint16_t *indirections_table, uint8_t *toeplitz_key)
+                      uint16_t *indirections_table, uint8_t *toeplitz_key,
+                      uint8_t key_len)
 {
     if (!ebpf_rss_is_loaded(ctx) || config == NULL ||
         indirections_table == NULL || toeplitz_key == NULL) {
@@ -147,7 +150,7 @@ bool ebpf_rss_set_all(struct EBPFRSSContext *ctx, struct EBPFRSSConfig *config,
         return false;
     }
 
-    if (!ebpf_rss_set_toepliz_key(ctx, toeplitz_key)) {
+    if (!ebpf_rss_set_toepliz_key(ctx, toeplitz_key, key_len)) {
         return false;
     }
 
diff --git a/ebpf/ebpf_rss.h b/ebpf/ebpf_rss.h
index bf3f2572c7..db23ccd25f 100644
--- a/ebpf/ebpf_rss.h
+++ b/ebpf/ebpf_rss.h
@@ -37,7 +37,8 @@ bool ebpf_rss_is_loaded(struct EBPFRSSContext *ctx);
 bool ebpf_rss_load(struct EBPFRSSContext *ctx);
 
 bool ebpf_rss_set_all(struct EBPFRSSContext *ctx, struct EBPFRSSConfig *config,
-                      uint16_t *indirections_table, uint8_t *toeplitz_key);
+                      uint16_t *indirections_table, uint8_t *toeplitz_key,
+                      uint8_t key_len);
 
 void ebpf_rss_unload(struct EBPFRSSContext *ctx);
 
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 1067e72b39..73145d6390 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -1201,7 +1201,8 @@ static bool virtio_net_attach_epbf_rss(VirtIONet *n)
     rss_data_to_rss_config(&n->rss_data, &config);
 
     if (!ebpf_rss_set_all(&n->ebpf_rss, &config,
-                          n->rss_data.indirections_table, n->rss_data.key)) {
+                          n->rss_data.indirections_table, n->rss_data.key,
+                          VIRTIO_NET_RSS_MAX_KEY_SIZE)) {
         return false;
     }
 
-- 
2.35.1



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 2/5] virtio-net: prepare for variable RSS key and indir table lengths
  2022-04-08 12:28 [PATCH 0/5] Vhost-user: add Virtio RSS support Maxime Coquelin
  2022-04-08 12:28 ` [PATCH 1/5] ebpf: pass and check RSS key length to the loader Maxime Coquelin
@ 2022-04-08 12:28 ` Maxime Coquelin
  2022-04-15  5:39   ` Jason Wang
  2022-04-08 12:28 ` [PATCH 3/5] virtio-net: add RSS support for Vhost backends Maxime Coquelin
                   ` (3 subsequent siblings)
  5 siblings, 1 reply; 11+ messages in thread
From: Maxime Coquelin @ 2022-04-08 12:28 UTC (permalink / raw)
  To: qemu-devel, mst, jasowang, andrew, yuri.benditovich, dgilbert, quintela
  Cc: chenbo.xia, Maxime Coquelin, dmarchan, ktraynor

This patch is a preliminary rework to support RSS with
Vhost-user backends. It enables supporting different types
of hashes, key lengths and indirection table lengths.

This patch does not introduces behavioral changes.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 ebpf/ebpf_rss.c                |  8 ++++----
 hw/net/virtio-net.c            | 35 +++++++++++++++++++++++++---------
 include/hw/virtio/virtio-net.h | 16 +++++++++++++---
 include/migration/vmstate.h    | 10 ++++++++++
 4 files changed, 53 insertions(+), 16 deletions(-)

diff --git a/ebpf/ebpf_rss.c b/ebpf/ebpf_rss.c
index 4a63854175..f03be5f919 100644
--- a/ebpf/ebpf_rss.c
+++ b/ebpf/ebpf_rss.c
@@ -96,7 +96,7 @@ static bool ebpf_rss_set_indirections_table(struct EBPFRSSContext *ctx,
     uint32_t i = 0;
 
     if (!ebpf_rss_is_loaded(ctx) || indirections_table == NULL ||
-       len > VIRTIO_NET_RSS_MAX_TABLE_LEN) {
+       len > VIRTIO_NET_RSS_DEFAULT_TABLE_LEN) {
         return false;
     }
 
@@ -116,13 +116,13 @@ static bool ebpf_rss_set_toepliz_key(struct EBPFRSSContext *ctx,
     uint32_t map_key = 0;
 
     /* prepare toeplitz key */
-    uint8_t toe[VIRTIO_NET_RSS_MAX_KEY_SIZE] = {};
+    uint8_t toe[VIRTIO_NET_RSS_DEFAULT_KEY_SIZE] = {};
 
     if (!ebpf_rss_is_loaded(ctx) || toeplitz_key == NULL ||
-            len != VIRTIO_NET_RSS_MAX_KEY_SIZE) {
+            len != VIRTIO_NET_RSS_DEFAULT_KEY_SIZE) {
         return false;
     }
-    memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_MAX_KEY_SIZE);
+    memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_DEFAULT_KEY_SIZE);
     *(uint32_t *)toe = ntohl(*(uint32_t *)toe);
 
     if (bpf_map_update_elem(ctx->map_toeplitz_key, &map_key, toe,
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 73145d6390..38436e472b 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -137,12 +137,11 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
     memcpy(netcfg.mac, n->mac, ETH_ALEN);
     virtio_stl_p(vdev, &netcfg.speed, n->net_conf.speed);
     netcfg.duplex = n->net_conf.duplex;
-    netcfg.rss_max_key_size = VIRTIO_NET_RSS_MAX_KEY_SIZE;
+    netcfg.rss_max_key_size = n->rss_capa.max_key_size;
     virtio_stw_p(vdev, &netcfg.rss_max_indirection_table_length,
-                 virtio_host_has_feature(vdev, VIRTIO_NET_F_RSS) ?
-                 VIRTIO_NET_RSS_MAX_TABLE_LEN : 1);
+                 n->rss_capa.max_indirection_len);
     virtio_stl_p(vdev, &netcfg.supported_hash_types,
-                 VIRTIO_NET_RSS_SUPPORTED_HASHES);
+                 n->rss_capa.supported_hashes);
     memcpy(config, &netcfg, n->config_size);
 
     /*
@@ -1202,7 +1201,7 @@ static bool virtio_net_attach_epbf_rss(VirtIONet *n)
 
     if (!ebpf_rss_set_all(&n->ebpf_rss, &config,
                           n->rss_data.indirections_table, n->rss_data.key,
-                          VIRTIO_NET_RSS_MAX_KEY_SIZE)) {
+                          n->rss_data.key_len)) {
         return false;
     }
 
@@ -1277,7 +1276,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
         err_value = n->rss_data.indirections_len;
         goto error;
     }
-    if (n->rss_data.indirections_len > VIRTIO_NET_RSS_MAX_TABLE_LEN) {
+    if (n->rss_data.indirections_len > n->rss_capa.max_indirection_len) {
         err_msg = "Too large indirection table";
         err_value = n->rss_data.indirections_len;
         goto error;
@@ -1323,7 +1322,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
         err_value = queue_pairs;
         goto error;
     }
-    if (temp.b > VIRTIO_NET_RSS_MAX_KEY_SIZE) {
+    if (temp.b > n->rss_capa.max_key_size) {
         err_msg = "Invalid key size";
         err_value = temp.b;
         goto error;
@@ -1339,6 +1338,14 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
     }
     offset += size_get;
     size_get = temp.b;
+    n->rss_data.key_len = temp.b;
+    g_free(n->rss_data.key);
+    n->rss_data.key = g_malloc(size_get);
+    if (!n->rss_data.key) {
+        err_msg = "Can't allocate key";
+        err_value = n->rss_data.key_len;
+        goto error;
+    }
     s = iov_to_buf(iov, iov_cnt, offset, n->rss_data.key, size_get);
     if (s != size_get) {
         err_msg = "Can get key buffer";
@@ -3093,8 +3100,9 @@ static const VMStateDescription vmstate_virtio_net_rss = {
         VMSTATE_UINT32(rss_data.hash_types, VirtIONet),
         VMSTATE_UINT16(rss_data.indirections_len, VirtIONet),
         VMSTATE_UINT16(rss_data.default_queue, VirtIONet),
-        VMSTATE_UINT8_ARRAY(rss_data.key, VirtIONet,
-                            VIRTIO_NET_RSS_MAX_KEY_SIZE),
+        VMSTATE_VARRAY_UINT8_ALLOC(rss_data.key, VirtIONet,
+                                   rss_data.key_len, 0,
+                                   vmstate_info_uint8, uint8_t),
         VMSTATE_VARRAY_UINT16_ALLOC(rss_data.indirections_table, VirtIONet,
                                     rss_data.indirections_len, 0,
                                     vmstate_info_uint16, uint16_t),
@@ -3523,8 +3531,16 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
     net_rx_pkt_init(&n->rx_pkt, false);
 
     if (virtio_has_feature(n->host_features, VIRTIO_NET_F_RSS)) {
+        n->rss_capa.max_key_size = VIRTIO_NET_RSS_DEFAULT_KEY_SIZE;
+        n->rss_capa.max_indirection_len = VIRTIO_NET_RSS_DEFAULT_TABLE_LEN;
+        n->rss_capa.supported_hashes = VIRTIO_NET_RSS_SUPPORTED_HASHES;
+
         virtio_net_load_ebpf(n);
+    } else {
+        n->rss_capa.max_indirection_len = 1;
     }
+
+
 }
 
 static void virtio_net_device_unrealize(DeviceState *dev)
@@ -3567,6 +3583,7 @@ static void virtio_net_device_unrealize(DeviceState *dev)
     qemu_del_nic(n->nic);
     virtio_net_rsc_cleanup(n);
     g_free(n->rss_data.indirections_table);
+    g_free(n->rss_data.key);
     net_rx_pkt_uninit(n->rx_pkt);
     virtio_cleanup(vdev);
 }
diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h
index eb87032627..6794b354ad 100644
--- a/include/hw/virtio/virtio-net.h
+++ b/include/hw/virtio/virtio-net.h
@@ -127,8 +127,16 @@ typedef struct VirtioNetRscChain {
 /* Maximum packet size we can receive from tap device: header + 64k */
 #define VIRTIO_NET_MAX_BUFSIZE (sizeof(struct virtio_net_hdr) + (64 * KiB))
 
-#define VIRTIO_NET_RSS_MAX_KEY_SIZE     40
-#define VIRTIO_NET_RSS_MAX_TABLE_LEN    128
+typedef struct VirtioNetRssCapa {
+    uint8_t max_key_size;
+    uint16_t max_indirection_len;
+    uint32_t supported_hashes;
+} VirtioNetRssCapa;
+
+#define VIRTIO_NET_RSS_MIN_KEY_SIZE      40
+#define VIRTIO_NET_RSS_DEFAULT_KEY_SIZE  40
+#define VIRTIO_NET_RSS_MIN_TABLE_LEN     128
+#define VIRTIO_NET_RSS_DEFAULT_TABLE_LEN 128
 
 typedef struct VirtioNetRssData {
     bool    enabled;
@@ -136,7 +144,8 @@ typedef struct VirtioNetRssData {
     bool    redirect;
     bool    populate_hash;
     uint32_t hash_types;
-    uint8_t key[VIRTIO_NET_RSS_MAX_KEY_SIZE];
+    uint8_t key_len;
+    uint8_t *key;
     uint16_t indirections_len;
     uint16_t *indirections_table;
     uint16_t default_queue;
@@ -213,6 +222,7 @@ struct VirtIONet {
     QDict *primary_opts;
     bool primary_opts_from_json;
     Notifier migration_state;
+    VirtioNetRssCapa rss_capa;
     VirtioNetRssData rss_data;
     struct NetRxPkt *rx_pkt;
     struct EBPFRSSContext ebpf_rss;
diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index ad24aa1934..9398cdf803 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -448,6 +448,16 @@ extern const VMStateInfo vmstate_info_qlist;
     .offset     = vmstate_offset_varray(_state, _field, _type),      \
 }
 
+#define VMSTATE_VARRAY_UINT8_ALLOC(_field, _state, _field_num, _version, _info, _type) {\
+    .name       = (stringify(_field)),                               \
+    .version_id = (_version),                                        \
+    .num_offset = vmstate_offset_value(_state, _field_num, uint8_t),\
+    .info       = &(_info),                                          \
+    .size       = sizeof(_type),                                     \
+    .flags      = VMS_VARRAY_UINT8 | VMS_POINTER | VMS_ALLOC,       \
+    .offset     = vmstate_offset_pointer(_state, _field, _type),     \
+}
+
 #define VMSTATE_VSTRUCT_TEST(_field, _state, _test, _version, _vmsd, _type, _struct_version) { \
     .name         = (stringify(_field)),                             \
     .version_id   = (_version),                                      \
-- 
2.35.1



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 3/5] virtio-net: add RSS support for Vhost backends
  2022-04-08 12:28 [PATCH 0/5] Vhost-user: add Virtio RSS support Maxime Coquelin
  2022-04-08 12:28 ` [PATCH 1/5] ebpf: pass and check RSS key length to the loader Maxime Coquelin
  2022-04-08 12:28 ` [PATCH 2/5] virtio-net: prepare for variable RSS key and indir table lengths Maxime Coquelin
@ 2022-04-08 12:28 ` Maxime Coquelin
  2022-04-15  5:41   ` Jason Wang
  2022-04-08 12:28 ` [PATCH 4/5] docs: introduce RSS support in Vhost-user specification Maxime Coquelin
                   ` (2 subsequent siblings)
  5 siblings, 1 reply; 11+ messages in thread
From: Maxime Coquelin @ 2022-04-08 12:28 UTC (permalink / raw)
  To: qemu-devel, mst, jasowang, andrew, yuri.benditovich, dgilbert, quintela
  Cc: chenbo.xia, Maxime Coquelin, dmarchan, ktraynor

This patch introduces new Vhost backend callbacks to
support RSS, and makes them called in Virtio-net
device.

It will be used by Vhost-user backend implementation to
support RSS feature.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 hw/net/vhost_net-stub.c           | 10 ++++++
 hw/net/vhost_net.c                | 22 +++++++++++++
 hw/net/virtio-net.c               | 53 +++++++++++++++++++++----------
 include/hw/virtio/vhost-backend.h |  7 ++++
 include/net/vhost_net.h           |  4 +++
 5 files changed, 79 insertions(+), 17 deletions(-)

diff --git a/hw/net/vhost_net-stub.c b/hw/net/vhost_net-stub.c
index 89d71cfb8e..cc05e07c1f 100644
--- a/hw/net/vhost_net-stub.c
+++ b/hw/net/vhost_net-stub.c
@@ -101,3 +101,13 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)
 {
     return 0;
 }
+
+int vhost_net_get_rss(struct vhost_net *net, VirtioNetRssCapa *rss_capa)
+{
+    return 0;
+}
+
+int vhost_net_set_rss(struct vhost_net *net, VirtioNetRssData *rss_data)
+{
+    return 0;
+}
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 30379d2ca4..aa2a1e8e5f 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -512,3 +512,25 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)
 
     return vhost_ops->vhost_net_set_mtu(&net->dev, mtu);
 }
+
+int vhost_net_get_rss(struct vhost_net *net, VirtioNetRssCapa *rss_capa)
+{
+    const VhostOps *vhost_ops = net->dev.vhost_ops;
+
+    if (!vhost_ops->vhost_net_get_rss) {
+        return 0;
+    }
+
+    return vhost_ops->vhost_net_get_rss(&net->dev, rss_capa);
+}
+
+int vhost_net_set_rss(struct vhost_net *net, VirtioNetRssData *rss_data)
+{
+    const VhostOps *vhost_ops = net->dev.vhost_ops;
+
+    if (!vhost_ops->vhost_net_set_rss) {
+        return 0;
+    }
+
+    return vhost_ops->vhost_net_set_rss(&net->dev, rss_data);
+}
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 38436e472b..237bbdb1b3 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -741,8 +741,10 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features,
         return features;
     }
 
-    if (!ebpf_rss_is_loaded(&n->ebpf_rss)) {
-        virtio_clear_feature(&features, VIRTIO_NET_F_RSS);
+    if (nc->peer->info->type == NET_CLIENT_DRIVER_TAP) {
+        if (!ebpf_rss_is_loaded(&n->ebpf_rss)) {
+            virtio_clear_feature(&features, VIRTIO_NET_F_RSS);
+        }
     }
     features = vhost_net_get_features(get_vhost_net(nc->peer), features);
     vdev->backend_features = features;
@@ -1161,11 +1163,17 @@ static void virtio_net_detach_epbf_rss(VirtIONet *n);
 
 static void virtio_net_disable_rss(VirtIONet *n)
 {
+    NetClientState *nc = qemu_get_queue(n->nic);
+
     if (n->rss_data.enabled) {
         trace_virtio_net_rss_disable();
     }
     n->rss_data.enabled = false;
 
+    if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
+        vhost_net_set_rss(get_vhost_net(nc->peer), &n->rss_data);
+    }
+
     virtio_net_detach_epbf_rss(n);
 }
 
@@ -1239,6 +1247,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
                                       bool do_rss)
 {
     VirtIODevice *vdev = VIRTIO_DEVICE(n);
+    NetClientState *nc = qemu_get_queue(n->nic);
     struct virtio_net_rss_config cfg;
     size_t s, offset = 0, size_get;
     uint16_t queue_pairs, i;
@@ -1354,22 +1363,29 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
     }
     n->rss_data.enabled = true;
 
-    if (!n->rss_data.populate_hash) {
-        if (!virtio_net_attach_epbf_rss(n)) {
-            /* EBPF must be loaded for vhost */
-            if (get_vhost_net(qemu_get_queue(n->nic)->peer)) {
-                warn_report("Can't load eBPF RSS for vhost");
-                goto error;
+    if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
+        if (vhost_net_set_rss(get_vhost_net(nc->peer), &n->rss_data)) {
+            warn_report("Failed to configure RSS for vhost-user");
+            goto error;
+        }
+    } else {
+        if (!n->rss_data.populate_hash) {
+            if (!virtio_net_attach_epbf_rss(n)) {
+                /* EBPF must be loaded for vhost */
+                if (get_vhost_net(nc->peer)) {
+                    warn_report("Can't load eBPF RSS for vhost");
+                    goto error;
+                }
+                /* fallback to software RSS */
+                warn_report("Can't load eBPF RSS - fallback to software RSS");
+                n->rss_data.enabled_software_rss = true;
             }
-            /* fallback to software RSS */
-            warn_report("Can't load eBPF RSS - fallback to software RSS");
+        } else {
+            /* use software RSS for hash populating */
+            /* and detach eBPF if was loaded before */
+            virtio_net_detach_epbf_rss(n);
             n->rss_data.enabled_software_rss = true;
         }
-    } else {
-        /* use software RSS for hash populating */
-        /* and detach eBPF if was loaded before */
-        virtio_net_detach_epbf_rss(n);
-        n->rss_data.enabled_software_rss = true;
     }
 
     trace_virtio_net_rss_enable(n->rss_data.hash_types,
@@ -3534,8 +3550,11 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
         n->rss_capa.max_key_size = VIRTIO_NET_RSS_DEFAULT_KEY_SIZE;
         n->rss_capa.max_indirection_len = VIRTIO_NET_RSS_DEFAULT_TABLE_LEN;
         n->rss_capa.supported_hashes = VIRTIO_NET_RSS_SUPPORTED_HASHES;
-
-        virtio_net_load_ebpf(n);
+        if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
+            vhost_net_get_rss(get_vhost_net(nc->peer), &n->rss_capa);
+        } else {
+            virtio_net_load_ebpf(n);
+        }
     } else {
         n->rss_capa.max_indirection_len = 1;
     }
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index 81bf3109f8..0b9e2ea26e 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -12,6 +12,7 @@
 #define VHOST_BACKEND_H
 
 #include "exec/memory.h"
+#include "hw/virtio/virtio-net.h"
 
 typedef enum VhostBackendType {
     VHOST_BACKEND_TYPE_NONE = 0,
@@ -45,6 +46,10 @@ typedef int (*vhost_backend_memslots_limit)(struct vhost_dev *dev);
 typedef int (*vhost_net_set_backend_op)(struct vhost_dev *dev,
                                 struct vhost_vring_file *file);
 typedef int (*vhost_net_set_mtu_op)(struct vhost_dev *dev, uint16_t mtu);
+typedef int (*vhost_net_get_rss_op)(struct vhost_dev *dev,
+                                VirtioNetRssCapa *rss_capa);
+typedef int (*vhost_net_set_rss_op)(struct vhost_dev *dev,
+                                VirtioNetRssData *rss_data);
 typedef int (*vhost_scsi_set_endpoint_op)(struct vhost_dev *dev,
                                   struct vhost_scsi_target *target);
 typedef int (*vhost_scsi_clear_endpoint_op)(struct vhost_dev *dev,
@@ -133,6 +138,8 @@ typedef struct VhostOps {
     vhost_backend_memslots_limit vhost_backend_memslots_limit;
     vhost_net_set_backend_op vhost_net_set_backend;
     vhost_net_set_mtu_op vhost_net_set_mtu;
+    vhost_net_get_rss_op vhost_net_get_rss;
+    vhost_net_set_rss_op vhost_net_set_rss;
     vhost_scsi_set_endpoint_op vhost_scsi_set_endpoint;
     vhost_scsi_clear_endpoint_op vhost_scsi_clear_endpoint;
     vhost_scsi_get_abi_version_op vhost_scsi_get_abi_version;
diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
index 387e913e4e..9cf702e7e3 100644
--- a/include/net/vhost_net.h
+++ b/include/net/vhost_net.h
@@ -48,4 +48,8 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net);
 
 int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
 
+int vhost_net_get_rss(struct vhost_net *net, VirtioNetRssCapa *rss_capa);
+
+int vhost_net_set_rss(struct vhost_net *net, VirtioNetRssData *rss_data);
+
 #endif
-- 
2.35.1



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 4/5] docs: introduce RSS support in Vhost-user specification
  2022-04-08 12:28 [PATCH 0/5] Vhost-user: add Virtio RSS support Maxime Coquelin
                   ` (2 preceding siblings ...)
  2022-04-08 12:28 ` [PATCH 3/5] virtio-net: add RSS support for Vhost backends Maxime Coquelin
@ 2022-04-08 12:28 ` Maxime Coquelin
  2022-04-11 12:18   ` Dr. David Alan Gilbert
  2022-04-08 12:28 ` [PATCH 5/5] vhost-user: add RSS support Maxime Coquelin
  2022-04-15  5:43 ` [PATCH 0/5] Vhost-user: add Virtio " Jason Wang
  5 siblings, 1 reply; 11+ messages in thread
From: Maxime Coquelin @ 2022-04-08 12:28 UTC (permalink / raw)
  To: qemu-devel, mst, jasowang, andrew, yuri.benditovich, dgilbert, quintela
  Cc: chenbo.xia, Maxime Coquelin, dmarchan, ktraynor

This patch documents RSS feature in Vhost-user specification.
Two new requests are introduced backed by a dedicated
protocol feature.

First one is to query the Vhost-user slave RSS capabilities
such as supported hash types, maximum key length and
indirection table size.

The second one is to provide the slave with driver's RSS
configuration.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 docs/interop/vhost-user.rst | 57 +++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)

diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
index 4dbc84fd00..9de6297568 100644
--- a/docs/interop/vhost-user.rst
+++ b/docs/interop/vhost-user.rst
@@ -258,6 +258,42 @@ Inflight description
 
 :queue size: a 16-bit size of virtqueues
 
+RSS capabilities description
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
++----------------------+-------------+---------------+
+| supported hash types | max key len | max indir len |
++----------------------+-------------+---------------+
+
+:supported hash types: a 32-bit bitfield of supported hash types as defined
+                       in the Virtio specification
+
+:max key len: a 8-bit maximum size of the RSS key
+
+:max indir len: a 16-bits maximum size of the RSS indirection table
+
+RSS data description
+^^^^^^^^^^^^^^^^^^^^
+
++------------+---------+-----+-----------+-------------+---------------+
+| hash types | key len | key | indir len | indir table | default queue |
++------------+---------+-----+-----------+-------------+---------------+
+
+:hash types: a 32-bit bitfield of supported hash types as defined in the
+             Virtio specification
+
+:key len: 8-bit size of the RSS key
+
+:key: a 8-bit array of 52 elements containing the RSS key
+
+:indir len: a 16-bit size of the RSS indirection table
+
+:indir table: a 16-bit array of 512 elements containing the hash indirection
+              table
+
+:default queue: the default queue index for flows not matching requested hash
+                types
+
 C structure
 -----------
 
@@ -858,6 +894,7 @@ Protocol features
   #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS 14
   #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS  15
   #define VHOST_USER_PROTOCOL_F_STATUS               16
+  #define VHOST_USER_PROTOCOL_F_NET_RSS              17
 
 Master message types
 --------------------
@@ -1371,6 +1408,26 @@ Master message types
   query the backend for its device status as defined in the Virtio
   specification.
 
+``VHOST_USER_NET_GET_RSS``
+  :id: 41
+  :equivalent ioctl: N/A
+  :slave payload: RSS capabilities description
+  :master payload: N/A
+
+  When the ``VHOST_USER_PROTOCOL_F_NET_RSS`` protocol has been successfully
+  negotiated, this message is submitted by the master to get the RSS
+  capabilities of the slave.
+
+``VHOST_USER_NET_SET_RSS``
+  :id: 42
+  :equivalent ioctl: N/A
+  :slave payload: N/A
+  :master payload: RSS data description
+
+  When the ``VHOST_USER_PROTOCOL_F_NET_RSS`` protocol has been successfully
+  negotiated, this message is submitted by the master to set the RSS
+  configuration defined by the Virtio driver.
+
 
 Slave message types
 -------------------
-- 
2.35.1



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 5/5] vhost-user: add RSS support
  2022-04-08 12:28 [PATCH 0/5] Vhost-user: add Virtio RSS support Maxime Coquelin
                   ` (3 preceding siblings ...)
  2022-04-08 12:28 ` [PATCH 4/5] docs: introduce RSS support in Vhost-user specification Maxime Coquelin
@ 2022-04-08 12:28 ` Maxime Coquelin
  2022-04-15  5:43 ` [PATCH 0/5] Vhost-user: add Virtio " Jason Wang
  5 siblings, 0 replies; 11+ messages in thread
From: Maxime Coquelin @ 2022-04-08 12:28 UTC (permalink / raw)
  To: qemu-devel, mst, jasowang, andrew, yuri.benditovich, dgilbert, quintela
  Cc: chenbo.xia, Maxime Coquelin, dmarchan, ktraynor

This patch implements the RSS feature to the
Vhost-user backend.

The implementation supports up to 52 bytes RSS key length,
and 512 indirection table entries.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 hw/virtio/vhost-user.c | 146 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 145 insertions(+), 1 deletion(-)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 6abbc9da32..d047da81ba 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -81,6 +81,8 @@ enum VhostUserProtocolFeature {
     VHOST_USER_PROTOCOL_F_RESET_DEVICE = 13,
     /* Feature 14 reserved for VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS. */
     VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS = 15,
+    /* Feature 16 reserved for VHOST_USER_PROTOCOL_F_STATUS. */
+    VHOST_USER_PROTOCOL_F_NET_RSS = 17,
     VHOST_USER_PROTOCOL_F_MAX
 };
 
@@ -126,6 +128,10 @@ typedef enum VhostUserRequest {
     VHOST_USER_GET_MAX_MEM_SLOTS = 36,
     VHOST_USER_ADD_MEM_REG = 37,
     VHOST_USER_REM_MEM_REG = 38,
+    /* Message number 39 reserved for VHOST_USER_SET_STATUS. */
+    /* Message number 40 reserved for VHOST_USER_GET_STATUS. */
+    VHOST_USER_NET_GET_RSS = 41,
+    VHOST_USER_NET_SET_RSS = 42,
     VHOST_USER_MAX
 } VhostUserRequest;
 
@@ -196,6 +202,24 @@ typedef struct VhostUserInflight {
     uint16_t queue_size;
 } VhostUserInflight;
 
+typedef struct VhostUserRSSCapa {
+    uint32_t supported_hash_types;
+    uint8_t max_key_len;
+    uint16_t max_indir_len;
+} VhostUserRSSCapa;
+
+#define VHOST_USER_RSS_MAX_KEY_LEN    52
+#define VHOST_USER_RSS_MAX_INDIR_LEN  512
+
+typedef struct VhostUserRSSData {
+    uint32_t hash_types;
+    uint8_t key_len;
+    uint8_t key[VHOST_USER_RSS_MAX_KEY_LEN];
+    uint16_t indir_len;
+    uint16_t indir_table[VHOST_USER_RSS_MAX_INDIR_LEN];
+    uint16_t default_queue;
+} VhostUserRSSData;
+
 typedef struct {
     VhostUserRequest request;
 
@@ -220,6 +244,8 @@ typedef union {
         VhostUserCryptoSession session;
         VhostUserVringArea area;
         VhostUserInflight inflight;
+        VhostUserRSSCapa rss_capa;
+        VhostUserRSSData rss_data;
 } VhostUserPayload;
 
 typedef struct VhostUserMsg {
@@ -2178,7 +2204,123 @@ static int vhost_user_net_set_mtu(struct vhost_dev *dev, uint16_t mtu)
         return ret;
     }
 
-    /* If reply_ack supported, slave has to ack specified MTU is valid */
+    if (reply_supported) {
+        return process_message_reply(dev, &msg);
+    }
+
+    return 0;
+}
+
+static int vhost_user_net_get_rss(struct vhost_dev *dev,
+                                  VirtioNetRssCapa *rss_capa)
+{
+    int ret;
+    VhostUserMsg msg = {
+        .hdr.request = VHOST_USER_NET_GET_RSS,
+        .hdr.flags = VHOST_USER_VERSION,
+    };
+
+    if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_NET_RSS))) {
+        return -EPROTO;
+    }
+
+    ret = vhost_user_write(dev, &msg, NULL, 0);
+    if (ret < 0) {
+        return ret;
+    }
+
+    ret = vhost_user_read(dev, &msg);
+    if (ret < 0) {
+        return ret;
+    }
+
+    if (msg.hdr.request != VHOST_USER_NET_GET_RSS) {
+        error_report("Received unexpected msg type. Expected %d received %d",
+                     VHOST_USER_NET_GET_RSS, msg.hdr.request);
+        return -EPROTO;
+    }
+
+    if (msg.hdr.size != sizeof(msg.payload.rss_capa)) {
+        error_report("Received bad msg size.");
+        return -EPROTO;
+    }
+
+    if (msg.payload.rss_capa.max_key_len < VIRTIO_NET_RSS_MIN_KEY_SIZE) {
+        error_report("Invalid max RSS key len (%uB, minimum %uB).",
+                     msg.payload.rss_capa.max_key_len,
+                     VIRTIO_NET_RSS_MIN_KEY_SIZE);
+        return -EINVAL;
+    }
+
+    if (msg.payload.rss_capa.max_indir_len < VIRTIO_NET_RSS_MIN_TABLE_LEN) {
+        error_report("Invalid max RSS indir table entries (%u, minimum %u).",
+                     msg.payload.rss_capa.max_indir_len,
+                     VIRTIO_NET_RSS_MIN_TABLE_LEN);
+        return -EINVAL;
+    }
+
+    rss_capa->supported_hashes = msg.payload.rss_capa.supported_hash_types;
+    rss_capa->max_key_size = MIN(msg.payload.rss_capa.max_key_len,
+                                 VHOST_USER_RSS_MAX_KEY_LEN);
+    rss_capa->max_indirection_len = MIN(msg.payload.rss_capa.max_indir_len,
+                                        VHOST_USER_RSS_MAX_INDIR_LEN);
+
+    return 0;
+}
+
+static int vhost_user_net_set_rss(struct vhost_dev *dev,
+                                  VirtioNetRssData *rss_data)
+{
+    VhostUserMsg msg;
+    bool reply_supported = virtio_has_feature(dev->protocol_features,
+                                              VHOST_USER_PROTOCOL_F_REPLY_ACK);
+    int ret;
+
+    if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_NET_RSS))) {
+        return -EPROTO;
+    }
+
+    msg.hdr.request = VHOST_USER_NET_SET_RSS;
+    msg.hdr.size = sizeof(msg.payload.rss_data);
+    msg.hdr.flags = VHOST_USER_VERSION;
+    if (reply_supported) {
+        msg.hdr.flags |= VHOST_USER_NEED_REPLY_MASK;
+    }
+
+    msg.payload.rss_data.hash_types = rss_data->hash_types;
+
+    if (rss_data->key_len > VHOST_USER_RSS_MAX_KEY_LEN) {
+        error_report("RSS key length too long (%uB, max %uB).",
+                     rss_data->key_len, VHOST_USER_RSS_MAX_KEY_LEN);
+        return -EINVAL;
+    }
+
+    msg.payload.rss_data.key_len = rss_data->key_len;
+    memset(msg.payload.rss_data.key, 0, VHOST_USER_RSS_MAX_KEY_LEN);
+    memcpy(msg.payload.rss_data.key, rss_data->key, rss_data->key_len);
+
+    if (rss_data->indirections_len > VHOST_USER_RSS_MAX_INDIR_LEN) {
+        error_report("RSS indirection table too large (%u, max %u).",
+                     rss_data->indirections_len, VHOST_USER_RSS_MAX_INDIR_LEN);
+        return -EINVAL;
+    }
+
+    msg.payload.rss_data.indir_len = rss_data->indirections_len;
+    memset(msg.payload.rss_data.indir_table, 0,
+            VHOST_USER_RSS_MAX_INDIR_LEN *
+            sizeof(*msg.payload.rss_data.indir_table));
+    memcpy(msg.payload.rss_data.indir_table, rss_data->indirections_table,
+            msg.payload.rss_data.indir_len *
+            sizeof(*msg.payload.rss_data.indir_table));
+
+    msg.payload.rss_data.default_queue = rss_data->default_queue;
+
+    ret = vhost_user_write(dev, &msg, NULL, 0);
+    if (ret < 0) {
+        return ret;
+    }
+
+    /* If reply_ack supported, slave has to ack specified RSS conf is valid */
     if (reply_supported) {
         return process_message_reply(dev, &msg);
     }
@@ -2555,6 +2697,8 @@ const VhostOps user_ops = {
         .vhost_migration_done = vhost_user_migration_done,
         .vhost_backend_can_merge = vhost_user_can_merge,
         .vhost_net_set_mtu = vhost_user_net_set_mtu,
+        .vhost_net_get_rss = vhost_user_net_get_rss,
+        .vhost_net_set_rss = vhost_user_net_set_rss,
         .vhost_set_iotlb_callback = vhost_user_set_iotlb_callback,
         .vhost_send_device_iotlb_msg = vhost_user_send_device_iotlb_msg,
         .vhost_get_config = vhost_user_get_config,
-- 
2.35.1



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [PATCH 4/5] docs: introduce RSS support in Vhost-user specification
  2022-04-08 12:28 ` [PATCH 4/5] docs: introduce RSS support in Vhost-user specification Maxime Coquelin
@ 2022-04-11 12:18   ` Dr. David Alan Gilbert
  0 siblings, 0 replies; 11+ messages in thread
From: Dr. David Alan Gilbert @ 2022-04-11 12:18 UTC (permalink / raw)
  To: Maxime Coquelin
  Cc: chenbo.xia, andrew, quintela, jasowang, mst, qemu-devel,
	ktraynor, yuri.benditovich, dmarchan

* Maxime Coquelin (maxime.coquelin@redhat.com) wrote:
> This patch documents RSS feature in Vhost-user specification.
> Two new requests are introduced backed by a dedicated
> protocol feature.
> 
> First one is to query the Vhost-user slave RSS capabilities
> such as supported hash types, maximum key length and
> indirection table size.
> 
> The second one is to provide the slave with driver's RSS
> configuration.
> 
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Perhaps it would be worth adding a link to devel/ebpf_rss.rst somewhere
in there; I didn't know what RSS so had to dig.
Also, perhaps it's useful to update that file to point back to the
vhost-user or dpdk implementations.

Dave

> ---
>  docs/interop/vhost-user.rst | 57 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 57 insertions(+)
> 
> diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
> index 4dbc84fd00..9de6297568 100644
> --- a/docs/interop/vhost-user.rst
> +++ b/docs/interop/vhost-user.rst
> @@ -258,6 +258,42 @@ Inflight description
>  
>  :queue size: a 16-bit size of virtqueues
>  
> +RSS capabilities description
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> ++----------------------+-------------+---------------+
> +| supported hash types | max key len | max indir len |
> ++----------------------+-------------+---------------+
> +
> +:supported hash types: a 32-bit bitfield of supported hash types as defined
> +                       in the Virtio specification
> +
> +:max key len: a 8-bit maximum size of the RSS key
> +
> +:max indir len: a 16-bits maximum size of the RSS indirection table
> +
> +RSS data description
> +^^^^^^^^^^^^^^^^^^^^
> +
> ++------------+---------+-----+-----------+-------------+---------------+
> +| hash types | key len | key | indir len | indir table | default queue |
> ++------------+---------+-----+-----------+-------------+---------------+
> +
> +:hash types: a 32-bit bitfield of supported hash types as defined in the
> +             Virtio specification
> +
> +:key len: 8-bit size of the RSS key
> +
> +:key: a 8-bit array of 52 elements containing the RSS key
> +
> +:indir len: a 16-bit size of the RSS indirection table
> +
> +:indir table: a 16-bit array of 512 elements containing the hash indirection
> +              table
> +
> +:default queue: the default queue index for flows not matching requested hash
> +                types
> +
>  C structure
>  -----------
>  
> @@ -858,6 +894,7 @@ Protocol features
>    #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS 14
>    #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS  15
>    #define VHOST_USER_PROTOCOL_F_STATUS               16
> +  #define VHOST_USER_PROTOCOL_F_NET_RSS              17
>  
>  Master message types
>  --------------------
> @@ -1371,6 +1408,26 @@ Master message types
>    query the backend for its device status as defined in the Virtio
>    specification.
>  
> +``VHOST_USER_NET_GET_RSS``
> +  :id: 41
> +  :equivalent ioctl: N/A
> +  :slave payload: RSS capabilities description
> +  :master payload: N/A
> +
> +  When the ``VHOST_USER_PROTOCOL_F_NET_RSS`` protocol has been successfully
> +  negotiated, this message is submitted by the master to get the RSS
> +  capabilities of the slave.
> +
> +``VHOST_USER_NET_SET_RSS``
> +  :id: 42
> +  :equivalent ioctl: N/A
> +  :slave payload: N/A
> +  :master payload: RSS data description
> +
> +  When the ``VHOST_USER_PROTOCOL_F_NET_RSS`` protocol has been successfully
> +  negotiated, this message is submitted by the master to set the RSS
> +  configuration defined by the Virtio driver.
> +
>  
>  Slave message types
>  -------------------
> -- 
> 2.35.1
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 2/5] virtio-net: prepare for variable RSS key and indir table lengths
  2022-04-08 12:28 ` [PATCH 2/5] virtio-net: prepare for variable RSS key and indir table lengths Maxime Coquelin
@ 2022-04-15  5:39   ` Jason Wang
  2022-05-13 10:49     ` Michael S. Tsirkin
  0 siblings, 1 reply; 11+ messages in thread
From: Jason Wang @ 2022-04-15  5:39 UTC (permalink / raw)
  To: Maxime Coquelin, qemu-devel, mst, andrew, yuri.benditovich,
	dgilbert, quintela
  Cc: chenbo.xia, dmarchan, ktraynor


在 2022/4/8 20:28, Maxime Coquelin 写道:
> This patch is a preliminary rework to support RSS with
> Vhost-user backends. It enables supporting different types
> of hashes, key lengths and indirection table lengths.
>
> This patch does not introduces behavioral changes.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
>   ebpf/ebpf_rss.c                |  8 ++++----
>   hw/net/virtio-net.c            | 35 +++++++++++++++++++++++++---------
>   include/hw/virtio/virtio-net.h | 16 +++++++++++++---
>   include/migration/vmstate.h    | 10 ++++++++++
>   4 files changed, 53 insertions(+), 16 deletions(-)
>
> diff --git a/ebpf/ebpf_rss.c b/ebpf/ebpf_rss.c
> index 4a63854175..f03be5f919 100644
> --- a/ebpf/ebpf_rss.c
> +++ b/ebpf/ebpf_rss.c
> @@ -96,7 +96,7 @@ static bool ebpf_rss_set_indirections_table(struct EBPFRSSContext *ctx,
>       uint32_t i = 0;
>   
>       if (!ebpf_rss_is_loaded(ctx) || indirections_table == NULL ||
> -       len > VIRTIO_NET_RSS_MAX_TABLE_LEN) {
> +       len > VIRTIO_NET_RSS_DEFAULT_TABLE_LEN) {
>           return false;
>       }
>   
> @@ -116,13 +116,13 @@ static bool ebpf_rss_set_toepliz_key(struct EBPFRSSContext *ctx,
>       uint32_t map_key = 0;
>   
>       /* prepare toeplitz key */
> -    uint8_t toe[VIRTIO_NET_RSS_MAX_KEY_SIZE] = {};
> +    uint8_t toe[VIRTIO_NET_RSS_DEFAULT_KEY_SIZE] = {};
>   
>       if (!ebpf_rss_is_loaded(ctx) || toeplitz_key == NULL ||
> -            len != VIRTIO_NET_RSS_MAX_KEY_SIZE) {
> +            len != VIRTIO_NET_RSS_DEFAULT_KEY_SIZE) {
>           return false;
>       }
> -    memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_MAX_KEY_SIZE);
> +    memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_DEFAULT_KEY_SIZE);
>       *(uint32_t *)toe = ntohl(*(uint32_t *)toe);
>   
>       if (bpf_map_update_elem(ctx->map_toeplitz_key, &map_key, toe,
> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
> index 73145d6390..38436e472b 100644
> --- a/hw/net/virtio-net.c
> +++ b/hw/net/virtio-net.c
> @@ -137,12 +137,11 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
>       memcpy(netcfg.mac, n->mac, ETH_ALEN);
>       virtio_stl_p(vdev, &netcfg.speed, n->net_conf.speed);
>       netcfg.duplex = n->net_conf.duplex;
> -    netcfg.rss_max_key_size = VIRTIO_NET_RSS_MAX_KEY_SIZE;
> +    netcfg.rss_max_key_size = n->rss_capa.max_key_size;
>       virtio_stw_p(vdev, &netcfg.rss_max_indirection_table_length,
> -                 virtio_host_has_feature(vdev, VIRTIO_NET_F_RSS) ?
> -                 VIRTIO_NET_RSS_MAX_TABLE_LEN : 1);
> +                 n->rss_capa.max_indirection_len);
>       virtio_stl_p(vdev, &netcfg.supported_hash_types,
> -                 VIRTIO_NET_RSS_SUPPORTED_HASHES);
> +                 n->rss_capa.supported_hashes);
>       memcpy(config, &netcfg, n->config_size);
>   
>       /*
> @@ -1202,7 +1201,7 @@ static bool virtio_net_attach_epbf_rss(VirtIONet *n)
>   
>       if (!ebpf_rss_set_all(&n->ebpf_rss, &config,
>                             n->rss_data.indirections_table, n->rss_data.key,
> -                          VIRTIO_NET_RSS_MAX_KEY_SIZE)) {
> +                          n->rss_data.key_len)) {
>           return false;
>       }
>   
> @@ -1277,7 +1276,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
>           err_value = n->rss_data.indirections_len;
>           goto error;
>       }
> -    if (n->rss_data.indirections_len > VIRTIO_NET_RSS_MAX_TABLE_LEN) {
> +    if (n->rss_data.indirections_len > n->rss_capa.max_indirection_len) {
>           err_msg = "Too large indirection table";
>           err_value = n->rss_data.indirections_len;
>           goto error;
> @@ -1323,7 +1322,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
>           err_value = queue_pairs;
>           goto error;
>       }
> -    if (temp.b > VIRTIO_NET_RSS_MAX_KEY_SIZE) {
> +    if (temp.b > n->rss_capa.max_key_size) {
>           err_msg = "Invalid key size";
>           err_value = temp.b;
>           goto error;
> @@ -1339,6 +1338,14 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
>       }
>       offset += size_get;
>       size_get = temp.b;
> +    n->rss_data.key_len = temp.b;
> +    g_free(n->rss_data.key);
> +    n->rss_data.key = g_malloc(size_get);
> +    if (!n->rss_data.key) {
> +        err_msg = "Can't allocate key";
> +        err_value = n->rss_data.key_len;
> +        goto error;
> +    }
>       s = iov_to_buf(iov, iov_cnt, offset, n->rss_data.key, size_get);
>       if (s != size_get) {
>           err_msg = "Can get key buffer";
> @@ -3093,8 +3100,9 @@ static const VMStateDescription vmstate_virtio_net_rss = {
>           VMSTATE_UINT32(rss_data.hash_types, VirtIONet),
>           VMSTATE_UINT16(rss_data.indirections_len, VirtIONet),
>           VMSTATE_UINT16(rss_data.default_queue, VirtIONet),
> -        VMSTATE_UINT8_ARRAY(rss_data.key, VirtIONet,
> -                            VIRTIO_NET_RSS_MAX_KEY_SIZE),
> +        VMSTATE_VARRAY_UINT8_ALLOC(rss_data.key, VirtIONet,
> +                                   rss_data.key_len, 0,
> +                                   vmstate_info_uint8, uint8_t),


I wonder if we may break the migration compatibility here.

Thanks


>           VMSTATE_VARRAY_UINT16_ALLOC(rss_data.indirections_table, VirtIONet,
>                                       rss_data.indirections_len, 0,
>                                       vmstate_info_uint16, uint16_t),
> @@ -3523,8 +3531,16 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
>       net_rx_pkt_init(&n->rx_pkt, false);
>   
>       if (virtio_has_feature(n->host_features, VIRTIO_NET_F_RSS)) {
> +        n->rss_capa.max_key_size = VIRTIO_NET_RSS_DEFAULT_KEY_SIZE;
> +        n->rss_capa.max_indirection_len = VIRTIO_NET_RSS_DEFAULT_TABLE_LEN;
> +        n->rss_capa.supported_hashes = VIRTIO_NET_RSS_SUPPORTED_HASHES;
> +
>           virtio_net_load_ebpf(n);
> +    } else {
> +        n->rss_capa.max_indirection_len = 1;
>       }
> +
> +
>   }
>   
>   static void virtio_net_device_unrealize(DeviceState *dev)
> @@ -3567,6 +3583,7 @@ static void virtio_net_device_unrealize(DeviceState *dev)
>       qemu_del_nic(n->nic);
>       virtio_net_rsc_cleanup(n);
>       g_free(n->rss_data.indirections_table);
> +    g_free(n->rss_data.key);
>       net_rx_pkt_uninit(n->rx_pkt);
>       virtio_cleanup(vdev);
>   }
> diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h
> index eb87032627..6794b354ad 100644
> --- a/include/hw/virtio/virtio-net.h
> +++ b/include/hw/virtio/virtio-net.h
> @@ -127,8 +127,16 @@ typedef struct VirtioNetRscChain {
>   /* Maximum packet size we can receive from tap device: header + 64k */
>   #define VIRTIO_NET_MAX_BUFSIZE (sizeof(struct virtio_net_hdr) + (64 * KiB))
>   
> -#define VIRTIO_NET_RSS_MAX_KEY_SIZE     40
> -#define VIRTIO_NET_RSS_MAX_TABLE_LEN    128
> +typedef struct VirtioNetRssCapa {
> +    uint8_t max_key_size;
> +    uint16_t max_indirection_len;
> +    uint32_t supported_hashes;
> +} VirtioNetRssCapa;
> +
> +#define VIRTIO_NET_RSS_MIN_KEY_SIZE      40
> +#define VIRTIO_NET_RSS_DEFAULT_KEY_SIZE  40
> +#define VIRTIO_NET_RSS_MIN_TABLE_LEN     128
> +#define VIRTIO_NET_RSS_DEFAULT_TABLE_LEN 128
>   
>   typedef struct VirtioNetRssData {
>       bool    enabled;
> @@ -136,7 +144,8 @@ typedef struct VirtioNetRssData {
>       bool    redirect;
>       bool    populate_hash;
>       uint32_t hash_types;
> -    uint8_t key[VIRTIO_NET_RSS_MAX_KEY_SIZE];
> +    uint8_t key_len;
> +    uint8_t *key;
>       uint16_t indirections_len;
>       uint16_t *indirections_table;
>       uint16_t default_queue;
> @@ -213,6 +222,7 @@ struct VirtIONet {
>       QDict *primary_opts;
>       bool primary_opts_from_json;
>       Notifier migration_state;
> +    VirtioNetRssCapa rss_capa;
>       VirtioNetRssData rss_data;
>       struct NetRxPkt *rx_pkt;
>       struct EBPFRSSContext ebpf_rss;
> diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
> index ad24aa1934..9398cdf803 100644
> --- a/include/migration/vmstate.h
> +++ b/include/migration/vmstate.h
> @@ -448,6 +448,16 @@ extern const VMStateInfo vmstate_info_qlist;
>       .offset     = vmstate_offset_varray(_state, _field, _type),      \
>   }
>   
> +#define VMSTATE_VARRAY_UINT8_ALLOC(_field, _state, _field_num, _version, _info, _type) {\
> +    .name       = (stringify(_field)),                               \
> +    .version_id = (_version),                                        \
> +    .num_offset = vmstate_offset_value(_state, _field_num, uint8_t),\
> +    .info       = &(_info),                                          \
> +    .size       = sizeof(_type),                                     \
> +    .flags      = VMS_VARRAY_UINT8 | VMS_POINTER | VMS_ALLOC,       \
> +    .offset     = vmstate_offset_pointer(_state, _field, _type),     \
> +}
> +
>   #define VMSTATE_VSTRUCT_TEST(_field, _state, _test, _version, _vmsd, _type, _struct_version) { \
>       .name         = (stringify(_field)),                             \
>       .version_id   = (_version),                                      \



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 3/5] virtio-net: add RSS support for Vhost backends
  2022-04-08 12:28 ` [PATCH 3/5] virtio-net: add RSS support for Vhost backends Maxime Coquelin
@ 2022-04-15  5:41   ` Jason Wang
  0 siblings, 0 replies; 11+ messages in thread
From: Jason Wang @ 2022-04-15  5:41 UTC (permalink / raw)
  To: Maxime Coquelin, qemu-devel, mst, andrew, yuri.benditovich,
	dgilbert, quintela
  Cc: chenbo.xia, dmarchan, ktraynor


在 2022/4/8 20:28, Maxime Coquelin 写道:
> This patch introduces new Vhost backend callbacks to
> support RSS, and makes them called in Virtio-net
> device.
>
> It will be used by Vhost-user backend implementation to
> support RSS feature.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
>   hw/net/vhost_net-stub.c           | 10 ++++++
>   hw/net/vhost_net.c                | 22 +++++++++++++
>   hw/net/virtio-net.c               | 53 +++++++++++++++++++++----------
>   include/hw/virtio/vhost-backend.h |  7 ++++
>   include/net/vhost_net.h           |  4 +++
>   5 files changed, 79 insertions(+), 17 deletions(-)
>
> diff --git a/hw/net/vhost_net-stub.c b/hw/net/vhost_net-stub.c
> index 89d71cfb8e..cc05e07c1f 100644
> --- a/hw/net/vhost_net-stub.c
> +++ b/hw/net/vhost_net-stub.c
> @@ -101,3 +101,13 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)
>   {
>       return 0;
>   }
> +
> +int vhost_net_get_rss(struct vhost_net *net, VirtioNetRssCapa *rss_capa)
> +{
> +    return 0;
> +}
> +
> +int vhost_net_set_rss(struct vhost_net *net, VirtioNetRssData *rss_data)
> +{
> +    return 0;
> +}
> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> index 30379d2ca4..aa2a1e8e5f 100644
> --- a/hw/net/vhost_net.c
> +++ b/hw/net/vhost_net.c
> @@ -512,3 +512,25 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)
>   
>       return vhost_ops->vhost_net_set_mtu(&net->dev, mtu);
>   }
> +
> +int vhost_net_get_rss(struct vhost_net *net, VirtioNetRssCapa *rss_capa)
> +{
> +    const VhostOps *vhost_ops = net->dev.vhost_ops;
> +
> +    if (!vhost_ops->vhost_net_get_rss) {
> +        return 0;
> +    }
> +
> +    return vhost_ops->vhost_net_get_rss(&net->dev, rss_capa);
> +}
> +
> +int vhost_net_set_rss(struct vhost_net *net, VirtioNetRssData *rss_data)
> +{
> +    const VhostOps *vhost_ops = net->dev.vhost_ops;
> +
> +    if (!vhost_ops->vhost_net_set_rss) {
> +        return 0;
> +    }
> +
> +    return vhost_ops->vhost_net_set_rss(&net->dev, rss_data);
> +}
> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
> index 38436e472b..237bbdb1b3 100644
> --- a/hw/net/virtio-net.c
> +++ b/hw/net/virtio-net.c
> @@ -741,8 +741,10 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features,
>           return features;
>       }
>   
> -    if (!ebpf_rss_is_loaded(&n->ebpf_rss)) {
> -        virtio_clear_feature(&features, VIRTIO_NET_F_RSS);
> +    if (nc->peer->info->type == NET_CLIENT_DRIVER_TAP) {
> +        if (!ebpf_rss_is_loaded(&n->ebpf_rss)) {
> +            virtio_clear_feature(&features, VIRTIO_NET_F_RSS);
> +        }
>       }
>       features = vhost_net_get_features(get_vhost_net(nc->peer), features);
>       vdev->backend_features = features;
> @@ -1161,11 +1163,17 @@ static void virtio_net_detach_epbf_rss(VirtIONet *n);
>   
>   static void virtio_net_disable_rss(VirtIONet *n)
>   {
> +    NetClientState *nc = qemu_get_queue(n->nic);
> +
>       if (n->rss_data.enabled) {
>           trace_virtio_net_rss_disable();
>       }
>       n->rss_data.enabled = false;
>   
> +    if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
> +        vhost_net_set_rss(get_vhost_net(nc->peer), &n->rss_data);
> +    }
> +
>       virtio_net_detach_epbf_rss(n);
>   }
>   
> @@ -1239,6 +1247,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
>                                         bool do_rss)
>   {
>       VirtIODevice *vdev = VIRTIO_DEVICE(n);
> +    NetClientState *nc = qemu_get_queue(n->nic);
>       struct virtio_net_rss_config cfg;
>       size_t s, offset = 0, size_get;
>       uint16_t queue_pairs, i;
> @@ -1354,22 +1363,29 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
>       }
>       n->rss_data.enabled = true;
>   
> -    if (!n->rss_data.populate_hash) {
> -        if (!virtio_net_attach_epbf_rss(n)) {
> -            /* EBPF must be loaded for vhost */
> -            if (get_vhost_net(qemu_get_queue(n->nic)->peer)) {
> -                warn_report("Can't load eBPF RSS for vhost");
> -                goto error;
> +    if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
> +        if (vhost_net_set_rss(get_vhost_net(nc->peer), &n->rss_data)) {
> +            warn_report("Failed to configure RSS for vhost-user");
> +            goto error;
> +        }
> +    } else {
> +        if (!n->rss_data.populate_hash) {
> +            if (!virtio_net_attach_epbf_rss(n)) {
> +                /* EBPF must be loaded for vhost */
> +                if (get_vhost_net(nc->peer)) {
> +                    warn_report("Can't load eBPF RSS for vhost");
> +                    goto error;
> +                }
> +                /* fallback to software RSS */
> +                warn_report("Can't load eBPF RSS - fallback to software RSS");
> +                n->rss_data.enabled_software_rss = true;
>               }
> -            /* fallback to software RSS */
> -            warn_report("Can't load eBPF RSS - fallback to software RSS");
> +        } else {
> +            /* use software RSS for hash populating */
> +            /* and detach eBPF if was loaded before */
> +            virtio_net_detach_epbf_rss(n);
>               n->rss_data.enabled_software_rss = true;
>           }
> -    } else {
> -        /* use software RSS for hash populating */
> -        /* and detach eBPF if was loaded before */
> -        virtio_net_detach_epbf_rss(n);
> -        n->rss_data.enabled_software_rss = true;
>       }
>   
>       trace_virtio_net_rss_enable(n->rss_data.hash_types,
> @@ -3534,8 +3550,11 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
>           n->rss_capa.max_key_size = VIRTIO_NET_RSS_DEFAULT_KEY_SIZE;
>           n->rss_capa.max_indirection_len = VIRTIO_NET_RSS_DEFAULT_TABLE_LEN;
>           n->rss_capa.supported_hashes = VIRTIO_NET_RSS_SUPPORTED_HASHES;
> -
> -        virtio_net_load_ebpf(n);
> +        if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
> +            vhost_net_get_rss(get_vhost_net(nc->peer), &n->rss_capa);


I wonder if we need a command parameter for the capability instead of 
silently get those from the vhost-user backhand (since there's no 
guarantee that the capability in src and dst are matched).

Thanks



> +        } else {
> +            virtio_net_load_ebpf(n);
> +        }
>       } else {
>           n->rss_capa.max_indirection_len = 1;
>       }
> diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
> index 81bf3109f8..0b9e2ea26e 100644
> --- a/include/hw/virtio/vhost-backend.h
> +++ b/include/hw/virtio/vhost-backend.h
> @@ -12,6 +12,7 @@
>   #define VHOST_BACKEND_H
>   
>   #include "exec/memory.h"
> +#include "hw/virtio/virtio-net.h"
>   
>   typedef enum VhostBackendType {
>       VHOST_BACKEND_TYPE_NONE = 0,
> @@ -45,6 +46,10 @@ typedef int (*vhost_backend_memslots_limit)(struct vhost_dev *dev);
>   typedef int (*vhost_net_set_backend_op)(struct vhost_dev *dev,
>                                   struct vhost_vring_file *file);
>   typedef int (*vhost_net_set_mtu_op)(struct vhost_dev *dev, uint16_t mtu);
> +typedef int (*vhost_net_get_rss_op)(struct vhost_dev *dev,
> +                                VirtioNetRssCapa *rss_capa);
> +typedef int (*vhost_net_set_rss_op)(struct vhost_dev *dev,
> +                                VirtioNetRssData *rss_data);
>   typedef int (*vhost_scsi_set_endpoint_op)(struct vhost_dev *dev,
>                                     struct vhost_scsi_target *target);
>   typedef int (*vhost_scsi_clear_endpoint_op)(struct vhost_dev *dev,
> @@ -133,6 +138,8 @@ typedef struct VhostOps {
>       vhost_backend_memslots_limit vhost_backend_memslots_limit;
>       vhost_net_set_backend_op vhost_net_set_backend;
>       vhost_net_set_mtu_op vhost_net_set_mtu;
> +    vhost_net_get_rss_op vhost_net_get_rss;
> +    vhost_net_set_rss_op vhost_net_set_rss;
>       vhost_scsi_set_endpoint_op vhost_scsi_set_endpoint;
>       vhost_scsi_clear_endpoint_op vhost_scsi_clear_endpoint;
>       vhost_scsi_get_abi_version_op vhost_scsi_get_abi_version;
> diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
> index 387e913e4e..9cf702e7e3 100644
> --- a/include/net/vhost_net.h
> +++ b/include/net/vhost_net.h
> @@ -48,4 +48,8 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net);
>   
>   int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
>   
> +int vhost_net_get_rss(struct vhost_net *net, VirtioNetRssCapa *rss_capa);
> +
> +int vhost_net_set_rss(struct vhost_net *net, VirtioNetRssData *rss_data);
> +
>   #endif



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 0/5] Vhost-user: add Virtio RSS support
  2022-04-08 12:28 [PATCH 0/5] Vhost-user: add Virtio RSS support Maxime Coquelin
                   ` (4 preceding siblings ...)
  2022-04-08 12:28 ` [PATCH 5/5] vhost-user: add RSS support Maxime Coquelin
@ 2022-04-15  5:43 ` Jason Wang
  5 siblings, 0 replies; 11+ messages in thread
From: Jason Wang @ 2022-04-15  5:43 UTC (permalink / raw)
  To: Maxime Coquelin, qemu-devel, mst, andrew, yuri.benditovich,
	dgilbert, quintela
  Cc: chenbo.xia, dmarchan, ktraynor


在 2022/4/8 20:28, Maxime Coquelin 写道:
> The goal of this series is to add support for Virtio RSS
> feature to the Vhost-user backend.
>
> First patches are preliminary reworks to support variable
> RSS key and indirection table length. eBPF change only adds
> checks on whether the key length is 40B, it does not add
> support for longer keys.
>
> Vhost-user implementation supports up to 52B RSS key, in
> order to match with the maximum supported by physical
> NICs (Intel E810). Idea is that it could be used for
> application like Virtio-forwarder, by programming the
> Virtio device RSS key into the physical NIC and let the
> physical NIC do the packets distribution.
>
> DPDK Vhost-user backend PoC implementing the new requests
> can be found here [0], it only implements the messages
> handling, it does not perform any RSS for now.
>
> [0]: https://gitlab.com/mcoquelin/dpdk-next-virtio/-/commits/vhost_user_rss_poc/


Not directly related to this series. I wonder if vhost-user consider to 
support control virtqueue then all the RSS stuffs could be done at 
vhost-user backend without introducing new commands.

Thanks


>
> Maxime Coquelin (5):
>    ebpf: pass and check RSS key length to the loader
>    virtio-net: prepare for variable RSS key and indir table lengths
>    virtio-net: add RSS support for Vhost backends
>    docs: introduce RSS support in Vhost-user specification
>    vhost-user: add RSS support
>
>   docs/interop/vhost-user.rst       |  57 ++++++++++++
>   ebpf/ebpf_rss-stub.c              |   3 +-
>   ebpf/ebpf_rss.c                   |  17 ++--
>   ebpf/ebpf_rss.h                   |   3 +-
>   hw/net/vhost_net-stub.c           |  10 ++
>   hw/net/vhost_net.c                |  22 +++++
>   hw/net/virtio-net.c               |  87 +++++++++++++-----
>   hw/virtio/vhost-user.c            | 146 +++++++++++++++++++++++++++++-
>   include/hw/virtio/vhost-backend.h |   7 ++
>   include/hw/virtio/virtio-net.h    |  16 +++-
>   include/migration/vmstate.h       |  10 ++
>   include/net/vhost_net.h           |   4 +
>   12 files changed, 344 insertions(+), 38 deletions(-)
>



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 2/5] virtio-net: prepare for variable RSS key and indir table lengths
  2022-04-15  5:39   ` Jason Wang
@ 2022-05-13 10:49     ` Michael S. Tsirkin
  0 siblings, 0 replies; 11+ messages in thread
From: Michael S. Tsirkin @ 2022-05-13 10:49 UTC (permalink / raw)
  To: Jason Wang
  Cc: Maxime Coquelin, qemu-devel, andrew, yuri.benditovich, dgilbert,
	quintela, ktraynor, dmarchan, chenbo.xia

On Fri, Apr 15, 2022 at 01:39:04PM +0800, Jason Wang wrote:
> 
> 在 2022/4/8 20:28, Maxime Coquelin 写道:
> > This patch is a preliminary rework to support RSS with
> > Vhost-user backends. It enables supporting different types
> > of hashes, key lengths and indirection table lengths.
> > 
> > This patch does not introduces behavioral changes.
> > 
> > Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> > ---
> >   ebpf/ebpf_rss.c                |  8 ++++----
> >   hw/net/virtio-net.c            | 35 +++++++++++++++++++++++++---------
> >   include/hw/virtio/virtio-net.h | 16 +++++++++++++---
> >   include/migration/vmstate.h    | 10 ++++++++++
> >   4 files changed, 53 insertions(+), 16 deletions(-)
> > 
> > diff --git a/ebpf/ebpf_rss.c b/ebpf/ebpf_rss.c
> > index 4a63854175..f03be5f919 100644
> > --- a/ebpf/ebpf_rss.c
> > +++ b/ebpf/ebpf_rss.c
> > @@ -96,7 +96,7 @@ static bool ebpf_rss_set_indirections_table(struct EBPFRSSContext *ctx,
> >       uint32_t i = 0;
> >       if (!ebpf_rss_is_loaded(ctx) || indirections_table == NULL ||
> > -       len > VIRTIO_NET_RSS_MAX_TABLE_LEN) {
> > +       len > VIRTIO_NET_RSS_DEFAULT_TABLE_LEN) {
> >           return false;
> >       }
> > @@ -116,13 +116,13 @@ static bool ebpf_rss_set_toepliz_key(struct EBPFRSSContext *ctx,
> >       uint32_t map_key = 0;
> >       /* prepare toeplitz key */
> > -    uint8_t toe[VIRTIO_NET_RSS_MAX_KEY_SIZE] = {};
> > +    uint8_t toe[VIRTIO_NET_RSS_DEFAULT_KEY_SIZE] = {};
> >       if (!ebpf_rss_is_loaded(ctx) || toeplitz_key == NULL ||
> > -            len != VIRTIO_NET_RSS_MAX_KEY_SIZE) {
> > +            len != VIRTIO_NET_RSS_DEFAULT_KEY_SIZE) {
> >           return false;
> >       }
> > -    memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_MAX_KEY_SIZE);
> > +    memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_DEFAULT_KEY_SIZE);
> >       *(uint32_t *)toe = ntohl(*(uint32_t *)toe);
> >       if (bpf_map_update_elem(ctx->map_toeplitz_key, &map_key, toe,
> > diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
> > index 73145d6390..38436e472b 100644
> > --- a/hw/net/virtio-net.c
> > +++ b/hw/net/virtio-net.c
> > @@ -137,12 +137,11 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
> >       memcpy(netcfg.mac, n->mac, ETH_ALEN);
> >       virtio_stl_p(vdev, &netcfg.speed, n->net_conf.speed);
> >       netcfg.duplex = n->net_conf.duplex;
> > -    netcfg.rss_max_key_size = VIRTIO_NET_RSS_MAX_KEY_SIZE;
> > +    netcfg.rss_max_key_size = n->rss_capa.max_key_size;
> >       virtio_stw_p(vdev, &netcfg.rss_max_indirection_table_length,
> > -                 virtio_host_has_feature(vdev, VIRTIO_NET_F_RSS) ?
> > -                 VIRTIO_NET_RSS_MAX_TABLE_LEN : 1);
> > +                 n->rss_capa.max_indirection_len);
> >       virtio_stl_p(vdev, &netcfg.supported_hash_types,
> > -                 VIRTIO_NET_RSS_SUPPORTED_HASHES);
> > +                 n->rss_capa.supported_hashes);
> >       memcpy(config, &netcfg, n->config_size);
> >       /*
> > @@ -1202,7 +1201,7 @@ static bool virtio_net_attach_epbf_rss(VirtIONet *n)
> >       if (!ebpf_rss_set_all(&n->ebpf_rss, &config,
> >                             n->rss_data.indirections_table, n->rss_data.key,
> > -                          VIRTIO_NET_RSS_MAX_KEY_SIZE)) {
> > +                          n->rss_data.key_len)) {
> >           return false;
> >       }
> > @@ -1277,7 +1276,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
> >           err_value = n->rss_data.indirections_len;
> >           goto error;
> >       }
> > -    if (n->rss_data.indirections_len > VIRTIO_NET_RSS_MAX_TABLE_LEN) {
> > +    if (n->rss_data.indirections_len > n->rss_capa.max_indirection_len) {
> >           err_msg = "Too large indirection table";
> >           err_value = n->rss_data.indirections_len;
> >           goto error;
> > @@ -1323,7 +1322,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
> >           err_value = queue_pairs;
> >           goto error;
> >       }
> > -    if (temp.b > VIRTIO_NET_RSS_MAX_KEY_SIZE) {
> > +    if (temp.b > n->rss_capa.max_key_size) {
> >           err_msg = "Invalid key size";
> >           err_value = temp.b;
> >           goto error;
> > @@ -1339,6 +1338,14 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
> >       }
> >       offset += size_get;
> >       size_get = temp.b;
> > +    n->rss_data.key_len = temp.b;
> > +    g_free(n->rss_data.key);
> > +    n->rss_data.key = g_malloc(size_get);
> > +    if (!n->rss_data.key) {
> > +        err_msg = "Can't allocate key";
> > +        err_value = n->rss_data.key_len;
> > +        goto error;
> > +    }
> >       s = iov_to_buf(iov, iov_cnt, offset, n->rss_data.key, size_get);
> >       if (s != size_get) {
> >           err_msg = "Can get key buffer";
> > @@ -3093,8 +3100,9 @@ static const VMStateDescription vmstate_virtio_net_rss = {
> >           VMSTATE_UINT32(rss_data.hash_types, VirtIONet),
> >           VMSTATE_UINT16(rss_data.indirections_len, VirtIONet),
> >           VMSTATE_UINT16(rss_data.default_queue, VirtIONet),
> > -        VMSTATE_UINT8_ARRAY(rss_data.key, VirtIONet,
> > -                            VIRTIO_NET_RSS_MAX_KEY_SIZE),
> > +        VMSTATE_VARRAY_UINT8_ALLOC(rss_data.key, VirtIONet,
> > +                                   rss_data.key_len, 0,
> > +                                   vmstate_info_uint8, uint8_t),
> 
> 
> I wonder if we may break the migration compatibility here.
> 
> Thanks


To be more specific, we used to always send VIRTIO_NET_RSS_MAX_KEY_SIZE
elements, now we are sending rss_data.key_len.
Looks like we need to ensure old configs have rss_data.key_len equal
to VIRTIO_NET_RSS_MAX_KEY_SIZE, no?


> 
> >           VMSTATE_VARRAY_UINT16_ALLOC(rss_data.indirections_table, VirtIONet,
> >                                       rss_data.indirections_len, 0,
> >                                       vmstate_info_uint16, uint16_t),
> > @@ -3523,8 +3531,16 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
> >       net_rx_pkt_init(&n->rx_pkt, false);
> >       if (virtio_has_feature(n->host_features, VIRTIO_NET_F_RSS)) {
> > +        n->rss_capa.max_key_size = VIRTIO_NET_RSS_DEFAULT_KEY_SIZE;
> > +        n->rss_capa.max_indirection_len = VIRTIO_NET_RSS_DEFAULT_TABLE_LEN;
> > +        n->rss_capa.supported_hashes = VIRTIO_NET_RSS_SUPPORTED_HASHES;
> > +
> >           virtio_net_load_ebpf(n);
> > +    } else {
> > +        n->rss_capa.max_indirection_len = 1;
> >       }
> > +
> > +
> >   }
> >   static void virtio_net_device_unrealize(DeviceState *dev)
> > @@ -3567,6 +3583,7 @@ static void virtio_net_device_unrealize(DeviceState *dev)
> >       qemu_del_nic(n->nic);
> >       virtio_net_rsc_cleanup(n);
> >       g_free(n->rss_data.indirections_table);
> > +    g_free(n->rss_data.key);
> >       net_rx_pkt_uninit(n->rx_pkt);
> >       virtio_cleanup(vdev);
> >   }
> > diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h
> > index eb87032627..6794b354ad 100644
> > --- a/include/hw/virtio/virtio-net.h
> > +++ b/include/hw/virtio/virtio-net.h
> > @@ -127,8 +127,16 @@ typedef struct VirtioNetRscChain {
> >   /* Maximum packet size we can receive from tap device: header + 64k */
> >   #define VIRTIO_NET_MAX_BUFSIZE (sizeof(struct virtio_net_hdr) + (64 * KiB))
> > -#define VIRTIO_NET_RSS_MAX_KEY_SIZE     40
> > -#define VIRTIO_NET_RSS_MAX_TABLE_LEN    128
> > +typedef struct VirtioNetRssCapa {
> > +    uint8_t max_key_size;
> > +    uint16_t max_indirection_len;
> > +    uint32_t supported_hashes;
> > +} VirtioNetRssCapa;
> > +
> > +#define VIRTIO_NET_RSS_MIN_KEY_SIZE      40
> > +#define VIRTIO_NET_RSS_DEFAULT_KEY_SIZE  40
> > +#define VIRTIO_NET_RSS_MIN_TABLE_LEN     128
> > +#define VIRTIO_NET_RSS_DEFAULT_TABLE_LEN 128
> >   typedef struct VirtioNetRssData {
> >       bool    enabled;
> > @@ -136,7 +144,8 @@ typedef struct VirtioNetRssData {
> >       bool    redirect;
> >       bool    populate_hash;
> >       uint32_t hash_types;
> > -    uint8_t key[VIRTIO_NET_RSS_MAX_KEY_SIZE];
> > +    uint8_t key_len;
> > +    uint8_t *key;
> >       uint16_t indirections_len;
> >       uint16_t *indirections_table;
> >       uint16_t default_queue;
> > @@ -213,6 +222,7 @@ struct VirtIONet {
> >       QDict *primary_opts;
> >       bool primary_opts_from_json;
> >       Notifier migration_state;
> > +    VirtioNetRssCapa rss_capa;
> >       VirtioNetRssData rss_data;
> >       struct NetRxPkt *rx_pkt;
> >       struct EBPFRSSContext ebpf_rss;
> > diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
> > index ad24aa1934..9398cdf803 100644
> > --- a/include/migration/vmstate.h
> > +++ b/include/migration/vmstate.h
> > @@ -448,6 +448,16 @@ extern const VMStateInfo vmstate_info_qlist;
> >       .offset     = vmstate_offset_varray(_state, _field, _type),      \
> >   }
> > +#define VMSTATE_VARRAY_UINT8_ALLOC(_field, _state, _field_num, _version, _info, _type) {\
> > +    .name       = (stringify(_field)),                               \
> > +    .version_id = (_version),                                        \
> > +    .num_offset = vmstate_offset_value(_state, _field_num, uint8_t),\
> > +    .info       = &(_info),                                          \
> > +    .size       = sizeof(_type),                                     \
> > +    .flags      = VMS_VARRAY_UINT8 | VMS_POINTER | VMS_ALLOC,       \
> > +    .offset     = vmstate_offset_pointer(_state, _field, _type),     \
> > +}
> > +
> >   #define VMSTATE_VSTRUCT_TEST(_field, _state, _test, _version, _vmsd, _type, _struct_version) { \
> >       .name         = (stringify(_field)),                             \
> >       .version_id   = (_version),                                      \



^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2022-05-13 10:50 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-08 12:28 [PATCH 0/5] Vhost-user: add Virtio RSS support Maxime Coquelin
2022-04-08 12:28 ` [PATCH 1/5] ebpf: pass and check RSS key length to the loader Maxime Coquelin
2022-04-08 12:28 ` [PATCH 2/5] virtio-net: prepare for variable RSS key and indir table lengths Maxime Coquelin
2022-04-15  5:39   ` Jason Wang
2022-05-13 10:49     ` Michael S. Tsirkin
2022-04-08 12:28 ` [PATCH 3/5] virtio-net: add RSS support for Vhost backends Maxime Coquelin
2022-04-15  5:41   ` Jason Wang
2022-04-08 12:28 ` [PATCH 4/5] docs: introduce RSS support in Vhost-user specification Maxime Coquelin
2022-04-11 12:18   ` Dr. David Alan Gilbert
2022-04-08 12:28 ` [PATCH 5/5] vhost-user: add RSS support Maxime Coquelin
2022-04-15  5:43 ` [PATCH 0/5] Vhost-user: add Virtio " Jason Wang

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.