* [PATCH 1/5] ebpf: pass and check RSS key length to the loader
2022-04-08 12:28 [PATCH 0/5] Vhost-user: add Virtio RSS support Maxime Coquelin
@ 2022-04-08 12:28 ` Maxime Coquelin
2022-04-08 12:28 ` [PATCH 2/5] virtio-net: prepare for variable RSS key and indir table lengths Maxime Coquelin
` (4 subsequent siblings)
5 siblings, 0 replies; 11+ messages in thread
From: Maxime Coquelin @ 2022-04-08 12:28 UTC (permalink / raw)
To: qemu-devel, mst, jasowang, andrew, yuri.benditovich, dgilbert, quintela
Cc: chenbo.xia, Maxime Coquelin, dmarchan, ktraynor
This patch is preliminary rework to support RSS with
Vhost-user backends. The Vhost-user implementation will
allow RSS hash key of 40 bytes or more as allowed by the
Virtio specification, whereas the eBPF-based Vhost-kernel
solution only supports 40 bytes keys.
This patch adds the RSS key length to the loader, and
validate it is 40 bytes before copying it.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
| 3 ++-
| 11 +++++++----
| 3 ++-
hw/net/virtio-net.c | 3 ++-
4 files changed, 13 insertions(+), 7 deletions(-)
--git a/ebpf/ebpf_rss-stub.c b/ebpf/ebpf_rss-stub.c
index e71e229190..ffc5c5574f 100644
--- a/ebpf/ebpf_rss-stub.c
+++ b/ebpf/ebpf_rss-stub.c
@@ -29,7 +29,8 @@ bool ebpf_rss_load(struct EBPFRSSContext *ctx)
}
bool ebpf_rss_set_all(struct EBPFRSSContext *ctx, struct EBPFRSSConfig *config,
- uint16_t *indirections_table, uint8_t *toeplitz_key)
+ uint16_t *indirections_table, uint8_t *toeplitz_key,
+ uint8_t key_len)
{
return false;
}
--git a/ebpf/ebpf_rss.c b/ebpf/ebpf_rss.c
index 118c68da83..4a63854175 100644
--- a/ebpf/ebpf_rss.c
+++ b/ebpf/ebpf_rss.c
@@ -110,14 +110,16 @@ static bool ebpf_rss_set_indirections_table(struct EBPFRSSContext *ctx,
}
static bool ebpf_rss_set_toepliz_key(struct EBPFRSSContext *ctx,
- uint8_t *toeplitz_key)
+ uint8_t *toeplitz_key,
+ size_t len)
{
uint32_t map_key = 0;
/* prepare toeplitz key */
uint8_t toe[VIRTIO_NET_RSS_MAX_KEY_SIZE] = {};
- if (!ebpf_rss_is_loaded(ctx) || toeplitz_key == NULL) {
+ if (!ebpf_rss_is_loaded(ctx) || toeplitz_key == NULL ||
+ len != VIRTIO_NET_RSS_MAX_KEY_SIZE) {
return false;
}
memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_MAX_KEY_SIZE);
@@ -131,7 +133,8 @@ static bool ebpf_rss_set_toepliz_key(struct EBPFRSSContext *ctx,
}
bool ebpf_rss_set_all(struct EBPFRSSContext *ctx, struct EBPFRSSConfig *config,
- uint16_t *indirections_table, uint8_t *toeplitz_key)
+ uint16_t *indirections_table, uint8_t *toeplitz_key,
+ uint8_t key_len)
{
if (!ebpf_rss_is_loaded(ctx) || config == NULL ||
indirections_table == NULL || toeplitz_key == NULL) {
@@ -147,7 +150,7 @@ bool ebpf_rss_set_all(struct EBPFRSSContext *ctx, struct EBPFRSSConfig *config,
return false;
}
- if (!ebpf_rss_set_toepliz_key(ctx, toeplitz_key)) {
+ if (!ebpf_rss_set_toepliz_key(ctx, toeplitz_key, key_len)) {
return false;
}
--git a/ebpf/ebpf_rss.h b/ebpf/ebpf_rss.h
index bf3f2572c7..db23ccd25f 100644
--- a/ebpf/ebpf_rss.h
+++ b/ebpf/ebpf_rss.h
@@ -37,7 +37,8 @@ bool ebpf_rss_is_loaded(struct EBPFRSSContext *ctx);
bool ebpf_rss_load(struct EBPFRSSContext *ctx);
bool ebpf_rss_set_all(struct EBPFRSSContext *ctx, struct EBPFRSSConfig *config,
- uint16_t *indirections_table, uint8_t *toeplitz_key);
+ uint16_t *indirections_table, uint8_t *toeplitz_key,
+ uint8_t key_len);
void ebpf_rss_unload(struct EBPFRSSContext *ctx);
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 1067e72b39..73145d6390 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -1201,7 +1201,8 @@ static bool virtio_net_attach_epbf_rss(VirtIONet *n)
rss_data_to_rss_config(&n->rss_data, &config);
if (!ebpf_rss_set_all(&n->ebpf_rss, &config,
- n->rss_data.indirections_table, n->rss_data.key)) {
+ n->rss_data.indirections_table, n->rss_data.key,
+ VIRTIO_NET_RSS_MAX_KEY_SIZE)) {
return false;
}
--
2.35.1
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH 2/5] virtio-net: prepare for variable RSS key and indir table lengths
2022-04-08 12:28 [PATCH 0/5] Vhost-user: add Virtio RSS support Maxime Coquelin
2022-04-08 12:28 ` [PATCH 1/5] ebpf: pass and check RSS key length to the loader Maxime Coquelin
@ 2022-04-08 12:28 ` Maxime Coquelin
2022-04-15 5:39 ` Jason Wang
2022-04-08 12:28 ` [PATCH 3/5] virtio-net: add RSS support for Vhost backends Maxime Coquelin
` (3 subsequent siblings)
5 siblings, 1 reply; 11+ messages in thread
From: Maxime Coquelin @ 2022-04-08 12:28 UTC (permalink / raw)
To: qemu-devel, mst, jasowang, andrew, yuri.benditovich, dgilbert, quintela
Cc: chenbo.xia, Maxime Coquelin, dmarchan, ktraynor
This patch is a preliminary rework to support RSS with
Vhost-user backends. It enables supporting different types
of hashes, key lengths and indirection table lengths.
This patch does not introduces behavioral changes.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
| 8 ++++----
hw/net/virtio-net.c | 35 +++++++++++++++++++++++++---------
include/hw/virtio/virtio-net.h | 16 +++++++++++++---
include/migration/vmstate.h | 10 ++++++++++
4 files changed, 53 insertions(+), 16 deletions(-)
--git a/ebpf/ebpf_rss.c b/ebpf/ebpf_rss.c
index 4a63854175..f03be5f919 100644
--- a/ebpf/ebpf_rss.c
+++ b/ebpf/ebpf_rss.c
@@ -96,7 +96,7 @@ static bool ebpf_rss_set_indirections_table(struct EBPFRSSContext *ctx,
uint32_t i = 0;
if (!ebpf_rss_is_loaded(ctx) || indirections_table == NULL ||
- len > VIRTIO_NET_RSS_MAX_TABLE_LEN) {
+ len > VIRTIO_NET_RSS_DEFAULT_TABLE_LEN) {
return false;
}
@@ -116,13 +116,13 @@ static bool ebpf_rss_set_toepliz_key(struct EBPFRSSContext *ctx,
uint32_t map_key = 0;
/* prepare toeplitz key */
- uint8_t toe[VIRTIO_NET_RSS_MAX_KEY_SIZE] = {};
+ uint8_t toe[VIRTIO_NET_RSS_DEFAULT_KEY_SIZE] = {};
if (!ebpf_rss_is_loaded(ctx) || toeplitz_key == NULL ||
- len != VIRTIO_NET_RSS_MAX_KEY_SIZE) {
+ len != VIRTIO_NET_RSS_DEFAULT_KEY_SIZE) {
return false;
}
- memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_MAX_KEY_SIZE);
+ memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_DEFAULT_KEY_SIZE);
*(uint32_t *)toe = ntohl(*(uint32_t *)toe);
if (bpf_map_update_elem(ctx->map_toeplitz_key, &map_key, toe,
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 73145d6390..38436e472b 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -137,12 +137,11 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
memcpy(netcfg.mac, n->mac, ETH_ALEN);
virtio_stl_p(vdev, &netcfg.speed, n->net_conf.speed);
netcfg.duplex = n->net_conf.duplex;
- netcfg.rss_max_key_size = VIRTIO_NET_RSS_MAX_KEY_SIZE;
+ netcfg.rss_max_key_size = n->rss_capa.max_key_size;
virtio_stw_p(vdev, &netcfg.rss_max_indirection_table_length,
- virtio_host_has_feature(vdev, VIRTIO_NET_F_RSS) ?
- VIRTIO_NET_RSS_MAX_TABLE_LEN : 1);
+ n->rss_capa.max_indirection_len);
virtio_stl_p(vdev, &netcfg.supported_hash_types,
- VIRTIO_NET_RSS_SUPPORTED_HASHES);
+ n->rss_capa.supported_hashes);
memcpy(config, &netcfg, n->config_size);
/*
@@ -1202,7 +1201,7 @@ static bool virtio_net_attach_epbf_rss(VirtIONet *n)
if (!ebpf_rss_set_all(&n->ebpf_rss, &config,
n->rss_data.indirections_table, n->rss_data.key,
- VIRTIO_NET_RSS_MAX_KEY_SIZE)) {
+ n->rss_data.key_len)) {
return false;
}
@@ -1277,7 +1276,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
err_value = n->rss_data.indirections_len;
goto error;
}
- if (n->rss_data.indirections_len > VIRTIO_NET_RSS_MAX_TABLE_LEN) {
+ if (n->rss_data.indirections_len > n->rss_capa.max_indirection_len) {
err_msg = "Too large indirection table";
err_value = n->rss_data.indirections_len;
goto error;
@@ -1323,7 +1322,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
err_value = queue_pairs;
goto error;
}
- if (temp.b > VIRTIO_NET_RSS_MAX_KEY_SIZE) {
+ if (temp.b > n->rss_capa.max_key_size) {
err_msg = "Invalid key size";
err_value = temp.b;
goto error;
@@ -1339,6 +1338,14 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
}
offset += size_get;
size_get = temp.b;
+ n->rss_data.key_len = temp.b;
+ g_free(n->rss_data.key);
+ n->rss_data.key = g_malloc(size_get);
+ if (!n->rss_data.key) {
+ err_msg = "Can't allocate key";
+ err_value = n->rss_data.key_len;
+ goto error;
+ }
s = iov_to_buf(iov, iov_cnt, offset, n->rss_data.key, size_get);
if (s != size_get) {
err_msg = "Can get key buffer";
@@ -3093,8 +3100,9 @@ static const VMStateDescription vmstate_virtio_net_rss = {
VMSTATE_UINT32(rss_data.hash_types, VirtIONet),
VMSTATE_UINT16(rss_data.indirections_len, VirtIONet),
VMSTATE_UINT16(rss_data.default_queue, VirtIONet),
- VMSTATE_UINT8_ARRAY(rss_data.key, VirtIONet,
- VIRTIO_NET_RSS_MAX_KEY_SIZE),
+ VMSTATE_VARRAY_UINT8_ALLOC(rss_data.key, VirtIONet,
+ rss_data.key_len, 0,
+ vmstate_info_uint8, uint8_t),
VMSTATE_VARRAY_UINT16_ALLOC(rss_data.indirections_table, VirtIONet,
rss_data.indirections_len, 0,
vmstate_info_uint16, uint16_t),
@@ -3523,8 +3531,16 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
net_rx_pkt_init(&n->rx_pkt, false);
if (virtio_has_feature(n->host_features, VIRTIO_NET_F_RSS)) {
+ n->rss_capa.max_key_size = VIRTIO_NET_RSS_DEFAULT_KEY_SIZE;
+ n->rss_capa.max_indirection_len = VIRTIO_NET_RSS_DEFAULT_TABLE_LEN;
+ n->rss_capa.supported_hashes = VIRTIO_NET_RSS_SUPPORTED_HASHES;
+
virtio_net_load_ebpf(n);
+ } else {
+ n->rss_capa.max_indirection_len = 1;
}
+
+
}
static void virtio_net_device_unrealize(DeviceState *dev)
@@ -3567,6 +3583,7 @@ static void virtio_net_device_unrealize(DeviceState *dev)
qemu_del_nic(n->nic);
virtio_net_rsc_cleanup(n);
g_free(n->rss_data.indirections_table);
+ g_free(n->rss_data.key);
net_rx_pkt_uninit(n->rx_pkt);
virtio_cleanup(vdev);
}
diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h
index eb87032627..6794b354ad 100644
--- a/include/hw/virtio/virtio-net.h
+++ b/include/hw/virtio/virtio-net.h
@@ -127,8 +127,16 @@ typedef struct VirtioNetRscChain {
/* Maximum packet size we can receive from tap device: header + 64k */
#define VIRTIO_NET_MAX_BUFSIZE (sizeof(struct virtio_net_hdr) + (64 * KiB))
-#define VIRTIO_NET_RSS_MAX_KEY_SIZE 40
-#define VIRTIO_NET_RSS_MAX_TABLE_LEN 128
+typedef struct VirtioNetRssCapa {
+ uint8_t max_key_size;
+ uint16_t max_indirection_len;
+ uint32_t supported_hashes;
+} VirtioNetRssCapa;
+
+#define VIRTIO_NET_RSS_MIN_KEY_SIZE 40
+#define VIRTIO_NET_RSS_DEFAULT_KEY_SIZE 40
+#define VIRTIO_NET_RSS_MIN_TABLE_LEN 128
+#define VIRTIO_NET_RSS_DEFAULT_TABLE_LEN 128
typedef struct VirtioNetRssData {
bool enabled;
@@ -136,7 +144,8 @@ typedef struct VirtioNetRssData {
bool redirect;
bool populate_hash;
uint32_t hash_types;
- uint8_t key[VIRTIO_NET_RSS_MAX_KEY_SIZE];
+ uint8_t key_len;
+ uint8_t *key;
uint16_t indirections_len;
uint16_t *indirections_table;
uint16_t default_queue;
@@ -213,6 +222,7 @@ struct VirtIONet {
QDict *primary_opts;
bool primary_opts_from_json;
Notifier migration_state;
+ VirtioNetRssCapa rss_capa;
VirtioNetRssData rss_data;
struct NetRxPkt *rx_pkt;
struct EBPFRSSContext ebpf_rss;
diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index ad24aa1934..9398cdf803 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -448,6 +448,16 @@ extern const VMStateInfo vmstate_info_qlist;
.offset = vmstate_offset_varray(_state, _field, _type), \
}
+#define VMSTATE_VARRAY_UINT8_ALLOC(_field, _state, _field_num, _version, _info, _type) {\
+ .name = (stringify(_field)), \
+ .version_id = (_version), \
+ .num_offset = vmstate_offset_value(_state, _field_num, uint8_t),\
+ .info = &(_info), \
+ .size = sizeof(_type), \
+ .flags = VMS_VARRAY_UINT8 | VMS_POINTER | VMS_ALLOC, \
+ .offset = vmstate_offset_pointer(_state, _field, _type), \
+}
+
#define VMSTATE_VSTRUCT_TEST(_field, _state, _test, _version, _vmsd, _type, _struct_version) { \
.name = (stringify(_field)), \
.version_id = (_version), \
--
2.35.1
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH 2/5] virtio-net: prepare for variable RSS key and indir table lengths
2022-04-08 12:28 ` [PATCH 2/5] virtio-net: prepare for variable RSS key and indir table lengths Maxime Coquelin
@ 2022-04-15 5:39 ` Jason Wang
2022-05-13 10:49 ` Michael S. Tsirkin
0 siblings, 1 reply; 11+ messages in thread
From: Jason Wang @ 2022-04-15 5:39 UTC (permalink / raw)
To: Maxime Coquelin, qemu-devel, mst, andrew, yuri.benditovich,
dgilbert, quintela
Cc: chenbo.xia, dmarchan, ktraynor
在 2022/4/8 20:28, Maxime Coquelin 写道:
> This patch is a preliminary rework to support RSS with
> Vhost-user backends. It enables supporting different types
> of hashes, key lengths and indirection table lengths.
>
> This patch does not introduces behavioral changes.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> ebpf/ebpf_rss.c | 8 ++++----
> hw/net/virtio-net.c | 35 +++++++++++++++++++++++++---------
> include/hw/virtio/virtio-net.h | 16 +++++++++++++---
> include/migration/vmstate.h | 10 ++++++++++
> 4 files changed, 53 insertions(+), 16 deletions(-)
>
> diff --git a/ebpf/ebpf_rss.c b/ebpf/ebpf_rss.c
> index 4a63854175..f03be5f919 100644
> --- a/ebpf/ebpf_rss.c
> +++ b/ebpf/ebpf_rss.c
> @@ -96,7 +96,7 @@ static bool ebpf_rss_set_indirections_table(struct EBPFRSSContext *ctx,
> uint32_t i = 0;
>
> if (!ebpf_rss_is_loaded(ctx) || indirections_table == NULL ||
> - len > VIRTIO_NET_RSS_MAX_TABLE_LEN) {
> + len > VIRTIO_NET_RSS_DEFAULT_TABLE_LEN) {
> return false;
> }
>
> @@ -116,13 +116,13 @@ static bool ebpf_rss_set_toepliz_key(struct EBPFRSSContext *ctx,
> uint32_t map_key = 0;
>
> /* prepare toeplitz key */
> - uint8_t toe[VIRTIO_NET_RSS_MAX_KEY_SIZE] = {};
> + uint8_t toe[VIRTIO_NET_RSS_DEFAULT_KEY_SIZE] = {};
>
> if (!ebpf_rss_is_loaded(ctx) || toeplitz_key == NULL ||
> - len != VIRTIO_NET_RSS_MAX_KEY_SIZE) {
> + len != VIRTIO_NET_RSS_DEFAULT_KEY_SIZE) {
> return false;
> }
> - memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_MAX_KEY_SIZE);
> + memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_DEFAULT_KEY_SIZE);
> *(uint32_t *)toe = ntohl(*(uint32_t *)toe);
>
> if (bpf_map_update_elem(ctx->map_toeplitz_key, &map_key, toe,
> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
> index 73145d6390..38436e472b 100644
> --- a/hw/net/virtio-net.c
> +++ b/hw/net/virtio-net.c
> @@ -137,12 +137,11 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
> memcpy(netcfg.mac, n->mac, ETH_ALEN);
> virtio_stl_p(vdev, &netcfg.speed, n->net_conf.speed);
> netcfg.duplex = n->net_conf.duplex;
> - netcfg.rss_max_key_size = VIRTIO_NET_RSS_MAX_KEY_SIZE;
> + netcfg.rss_max_key_size = n->rss_capa.max_key_size;
> virtio_stw_p(vdev, &netcfg.rss_max_indirection_table_length,
> - virtio_host_has_feature(vdev, VIRTIO_NET_F_RSS) ?
> - VIRTIO_NET_RSS_MAX_TABLE_LEN : 1);
> + n->rss_capa.max_indirection_len);
> virtio_stl_p(vdev, &netcfg.supported_hash_types,
> - VIRTIO_NET_RSS_SUPPORTED_HASHES);
> + n->rss_capa.supported_hashes);
> memcpy(config, &netcfg, n->config_size);
>
> /*
> @@ -1202,7 +1201,7 @@ static bool virtio_net_attach_epbf_rss(VirtIONet *n)
>
> if (!ebpf_rss_set_all(&n->ebpf_rss, &config,
> n->rss_data.indirections_table, n->rss_data.key,
> - VIRTIO_NET_RSS_MAX_KEY_SIZE)) {
> + n->rss_data.key_len)) {
> return false;
> }
>
> @@ -1277,7 +1276,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
> err_value = n->rss_data.indirections_len;
> goto error;
> }
> - if (n->rss_data.indirections_len > VIRTIO_NET_RSS_MAX_TABLE_LEN) {
> + if (n->rss_data.indirections_len > n->rss_capa.max_indirection_len) {
> err_msg = "Too large indirection table";
> err_value = n->rss_data.indirections_len;
> goto error;
> @@ -1323,7 +1322,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
> err_value = queue_pairs;
> goto error;
> }
> - if (temp.b > VIRTIO_NET_RSS_MAX_KEY_SIZE) {
> + if (temp.b > n->rss_capa.max_key_size) {
> err_msg = "Invalid key size";
> err_value = temp.b;
> goto error;
> @@ -1339,6 +1338,14 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
> }
> offset += size_get;
> size_get = temp.b;
> + n->rss_data.key_len = temp.b;
> + g_free(n->rss_data.key);
> + n->rss_data.key = g_malloc(size_get);
> + if (!n->rss_data.key) {
> + err_msg = "Can't allocate key";
> + err_value = n->rss_data.key_len;
> + goto error;
> + }
> s = iov_to_buf(iov, iov_cnt, offset, n->rss_data.key, size_get);
> if (s != size_get) {
> err_msg = "Can get key buffer";
> @@ -3093,8 +3100,9 @@ static const VMStateDescription vmstate_virtio_net_rss = {
> VMSTATE_UINT32(rss_data.hash_types, VirtIONet),
> VMSTATE_UINT16(rss_data.indirections_len, VirtIONet),
> VMSTATE_UINT16(rss_data.default_queue, VirtIONet),
> - VMSTATE_UINT8_ARRAY(rss_data.key, VirtIONet,
> - VIRTIO_NET_RSS_MAX_KEY_SIZE),
> + VMSTATE_VARRAY_UINT8_ALLOC(rss_data.key, VirtIONet,
> + rss_data.key_len, 0,
> + vmstate_info_uint8, uint8_t),
I wonder if we may break the migration compatibility here.
Thanks
> VMSTATE_VARRAY_UINT16_ALLOC(rss_data.indirections_table, VirtIONet,
> rss_data.indirections_len, 0,
> vmstate_info_uint16, uint16_t),
> @@ -3523,8 +3531,16 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
> net_rx_pkt_init(&n->rx_pkt, false);
>
> if (virtio_has_feature(n->host_features, VIRTIO_NET_F_RSS)) {
> + n->rss_capa.max_key_size = VIRTIO_NET_RSS_DEFAULT_KEY_SIZE;
> + n->rss_capa.max_indirection_len = VIRTIO_NET_RSS_DEFAULT_TABLE_LEN;
> + n->rss_capa.supported_hashes = VIRTIO_NET_RSS_SUPPORTED_HASHES;
> +
> virtio_net_load_ebpf(n);
> + } else {
> + n->rss_capa.max_indirection_len = 1;
> }
> +
> +
> }
>
> static void virtio_net_device_unrealize(DeviceState *dev)
> @@ -3567,6 +3583,7 @@ static void virtio_net_device_unrealize(DeviceState *dev)
> qemu_del_nic(n->nic);
> virtio_net_rsc_cleanup(n);
> g_free(n->rss_data.indirections_table);
> + g_free(n->rss_data.key);
> net_rx_pkt_uninit(n->rx_pkt);
> virtio_cleanup(vdev);
> }
> diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h
> index eb87032627..6794b354ad 100644
> --- a/include/hw/virtio/virtio-net.h
> +++ b/include/hw/virtio/virtio-net.h
> @@ -127,8 +127,16 @@ typedef struct VirtioNetRscChain {
> /* Maximum packet size we can receive from tap device: header + 64k */
> #define VIRTIO_NET_MAX_BUFSIZE (sizeof(struct virtio_net_hdr) + (64 * KiB))
>
> -#define VIRTIO_NET_RSS_MAX_KEY_SIZE 40
> -#define VIRTIO_NET_RSS_MAX_TABLE_LEN 128
> +typedef struct VirtioNetRssCapa {
> + uint8_t max_key_size;
> + uint16_t max_indirection_len;
> + uint32_t supported_hashes;
> +} VirtioNetRssCapa;
> +
> +#define VIRTIO_NET_RSS_MIN_KEY_SIZE 40
> +#define VIRTIO_NET_RSS_DEFAULT_KEY_SIZE 40
> +#define VIRTIO_NET_RSS_MIN_TABLE_LEN 128
> +#define VIRTIO_NET_RSS_DEFAULT_TABLE_LEN 128
>
> typedef struct VirtioNetRssData {
> bool enabled;
> @@ -136,7 +144,8 @@ typedef struct VirtioNetRssData {
> bool redirect;
> bool populate_hash;
> uint32_t hash_types;
> - uint8_t key[VIRTIO_NET_RSS_MAX_KEY_SIZE];
> + uint8_t key_len;
> + uint8_t *key;
> uint16_t indirections_len;
> uint16_t *indirections_table;
> uint16_t default_queue;
> @@ -213,6 +222,7 @@ struct VirtIONet {
> QDict *primary_opts;
> bool primary_opts_from_json;
> Notifier migration_state;
> + VirtioNetRssCapa rss_capa;
> VirtioNetRssData rss_data;
> struct NetRxPkt *rx_pkt;
> struct EBPFRSSContext ebpf_rss;
> diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
> index ad24aa1934..9398cdf803 100644
> --- a/include/migration/vmstate.h
> +++ b/include/migration/vmstate.h
> @@ -448,6 +448,16 @@ extern const VMStateInfo vmstate_info_qlist;
> .offset = vmstate_offset_varray(_state, _field, _type), \
> }
>
> +#define VMSTATE_VARRAY_UINT8_ALLOC(_field, _state, _field_num, _version, _info, _type) {\
> + .name = (stringify(_field)), \
> + .version_id = (_version), \
> + .num_offset = vmstate_offset_value(_state, _field_num, uint8_t),\
> + .info = &(_info), \
> + .size = sizeof(_type), \
> + .flags = VMS_VARRAY_UINT8 | VMS_POINTER | VMS_ALLOC, \
> + .offset = vmstate_offset_pointer(_state, _field, _type), \
> +}
> +
> #define VMSTATE_VSTRUCT_TEST(_field, _state, _test, _version, _vmsd, _type, _struct_version) { \
> .name = (stringify(_field)), \
> .version_id = (_version), \
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH 2/5] virtio-net: prepare for variable RSS key and indir table lengths
2022-04-15 5:39 ` Jason Wang
@ 2022-05-13 10:49 ` Michael S. Tsirkin
0 siblings, 0 replies; 11+ messages in thread
From: Michael S. Tsirkin @ 2022-05-13 10:49 UTC (permalink / raw)
To: Jason Wang
Cc: Maxime Coquelin, qemu-devel, andrew, yuri.benditovich, dgilbert,
quintela, ktraynor, dmarchan, chenbo.xia
On Fri, Apr 15, 2022 at 01:39:04PM +0800, Jason Wang wrote:
>
> 在 2022/4/8 20:28, Maxime Coquelin 写道:
> > This patch is a preliminary rework to support RSS with
> > Vhost-user backends. It enables supporting different types
> > of hashes, key lengths and indirection table lengths.
> >
> > This patch does not introduces behavioral changes.
> >
> > Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> > ---
> > ebpf/ebpf_rss.c | 8 ++++----
> > hw/net/virtio-net.c | 35 +++++++++++++++++++++++++---------
> > include/hw/virtio/virtio-net.h | 16 +++++++++++++---
> > include/migration/vmstate.h | 10 ++++++++++
> > 4 files changed, 53 insertions(+), 16 deletions(-)
> >
> > diff --git a/ebpf/ebpf_rss.c b/ebpf/ebpf_rss.c
> > index 4a63854175..f03be5f919 100644
> > --- a/ebpf/ebpf_rss.c
> > +++ b/ebpf/ebpf_rss.c
> > @@ -96,7 +96,7 @@ static bool ebpf_rss_set_indirections_table(struct EBPFRSSContext *ctx,
> > uint32_t i = 0;
> > if (!ebpf_rss_is_loaded(ctx) || indirections_table == NULL ||
> > - len > VIRTIO_NET_RSS_MAX_TABLE_LEN) {
> > + len > VIRTIO_NET_RSS_DEFAULT_TABLE_LEN) {
> > return false;
> > }
> > @@ -116,13 +116,13 @@ static bool ebpf_rss_set_toepliz_key(struct EBPFRSSContext *ctx,
> > uint32_t map_key = 0;
> > /* prepare toeplitz key */
> > - uint8_t toe[VIRTIO_NET_RSS_MAX_KEY_SIZE] = {};
> > + uint8_t toe[VIRTIO_NET_RSS_DEFAULT_KEY_SIZE] = {};
> > if (!ebpf_rss_is_loaded(ctx) || toeplitz_key == NULL ||
> > - len != VIRTIO_NET_RSS_MAX_KEY_SIZE) {
> > + len != VIRTIO_NET_RSS_DEFAULT_KEY_SIZE) {
> > return false;
> > }
> > - memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_MAX_KEY_SIZE);
> > + memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_DEFAULT_KEY_SIZE);
> > *(uint32_t *)toe = ntohl(*(uint32_t *)toe);
> > if (bpf_map_update_elem(ctx->map_toeplitz_key, &map_key, toe,
> > diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
> > index 73145d6390..38436e472b 100644
> > --- a/hw/net/virtio-net.c
> > +++ b/hw/net/virtio-net.c
> > @@ -137,12 +137,11 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
> > memcpy(netcfg.mac, n->mac, ETH_ALEN);
> > virtio_stl_p(vdev, &netcfg.speed, n->net_conf.speed);
> > netcfg.duplex = n->net_conf.duplex;
> > - netcfg.rss_max_key_size = VIRTIO_NET_RSS_MAX_KEY_SIZE;
> > + netcfg.rss_max_key_size = n->rss_capa.max_key_size;
> > virtio_stw_p(vdev, &netcfg.rss_max_indirection_table_length,
> > - virtio_host_has_feature(vdev, VIRTIO_NET_F_RSS) ?
> > - VIRTIO_NET_RSS_MAX_TABLE_LEN : 1);
> > + n->rss_capa.max_indirection_len);
> > virtio_stl_p(vdev, &netcfg.supported_hash_types,
> > - VIRTIO_NET_RSS_SUPPORTED_HASHES);
> > + n->rss_capa.supported_hashes);
> > memcpy(config, &netcfg, n->config_size);
> > /*
> > @@ -1202,7 +1201,7 @@ static bool virtio_net_attach_epbf_rss(VirtIONet *n)
> > if (!ebpf_rss_set_all(&n->ebpf_rss, &config,
> > n->rss_data.indirections_table, n->rss_data.key,
> > - VIRTIO_NET_RSS_MAX_KEY_SIZE)) {
> > + n->rss_data.key_len)) {
> > return false;
> > }
> > @@ -1277,7 +1276,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
> > err_value = n->rss_data.indirections_len;
> > goto error;
> > }
> > - if (n->rss_data.indirections_len > VIRTIO_NET_RSS_MAX_TABLE_LEN) {
> > + if (n->rss_data.indirections_len > n->rss_capa.max_indirection_len) {
> > err_msg = "Too large indirection table";
> > err_value = n->rss_data.indirections_len;
> > goto error;
> > @@ -1323,7 +1322,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
> > err_value = queue_pairs;
> > goto error;
> > }
> > - if (temp.b > VIRTIO_NET_RSS_MAX_KEY_SIZE) {
> > + if (temp.b > n->rss_capa.max_key_size) {
> > err_msg = "Invalid key size";
> > err_value = temp.b;
> > goto error;
> > @@ -1339,6 +1338,14 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
> > }
> > offset += size_get;
> > size_get = temp.b;
> > + n->rss_data.key_len = temp.b;
> > + g_free(n->rss_data.key);
> > + n->rss_data.key = g_malloc(size_get);
> > + if (!n->rss_data.key) {
> > + err_msg = "Can't allocate key";
> > + err_value = n->rss_data.key_len;
> > + goto error;
> > + }
> > s = iov_to_buf(iov, iov_cnt, offset, n->rss_data.key, size_get);
> > if (s != size_get) {
> > err_msg = "Can get key buffer";
> > @@ -3093,8 +3100,9 @@ static const VMStateDescription vmstate_virtio_net_rss = {
> > VMSTATE_UINT32(rss_data.hash_types, VirtIONet),
> > VMSTATE_UINT16(rss_data.indirections_len, VirtIONet),
> > VMSTATE_UINT16(rss_data.default_queue, VirtIONet),
> > - VMSTATE_UINT8_ARRAY(rss_data.key, VirtIONet,
> > - VIRTIO_NET_RSS_MAX_KEY_SIZE),
> > + VMSTATE_VARRAY_UINT8_ALLOC(rss_data.key, VirtIONet,
> > + rss_data.key_len, 0,
> > + vmstate_info_uint8, uint8_t),
>
>
> I wonder if we may break the migration compatibility here.
>
> Thanks
To be more specific, we used to always send VIRTIO_NET_RSS_MAX_KEY_SIZE
elements, now we are sending rss_data.key_len.
Looks like we need to ensure old configs have rss_data.key_len equal
to VIRTIO_NET_RSS_MAX_KEY_SIZE, no?
>
> > VMSTATE_VARRAY_UINT16_ALLOC(rss_data.indirections_table, VirtIONet,
> > rss_data.indirections_len, 0,
> > vmstate_info_uint16, uint16_t),
> > @@ -3523,8 +3531,16 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
> > net_rx_pkt_init(&n->rx_pkt, false);
> > if (virtio_has_feature(n->host_features, VIRTIO_NET_F_RSS)) {
> > + n->rss_capa.max_key_size = VIRTIO_NET_RSS_DEFAULT_KEY_SIZE;
> > + n->rss_capa.max_indirection_len = VIRTIO_NET_RSS_DEFAULT_TABLE_LEN;
> > + n->rss_capa.supported_hashes = VIRTIO_NET_RSS_SUPPORTED_HASHES;
> > +
> > virtio_net_load_ebpf(n);
> > + } else {
> > + n->rss_capa.max_indirection_len = 1;
> > }
> > +
> > +
> > }
> > static void virtio_net_device_unrealize(DeviceState *dev)
> > @@ -3567,6 +3583,7 @@ static void virtio_net_device_unrealize(DeviceState *dev)
> > qemu_del_nic(n->nic);
> > virtio_net_rsc_cleanup(n);
> > g_free(n->rss_data.indirections_table);
> > + g_free(n->rss_data.key);
> > net_rx_pkt_uninit(n->rx_pkt);
> > virtio_cleanup(vdev);
> > }
> > diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h
> > index eb87032627..6794b354ad 100644
> > --- a/include/hw/virtio/virtio-net.h
> > +++ b/include/hw/virtio/virtio-net.h
> > @@ -127,8 +127,16 @@ typedef struct VirtioNetRscChain {
> > /* Maximum packet size we can receive from tap device: header + 64k */
> > #define VIRTIO_NET_MAX_BUFSIZE (sizeof(struct virtio_net_hdr) + (64 * KiB))
> > -#define VIRTIO_NET_RSS_MAX_KEY_SIZE 40
> > -#define VIRTIO_NET_RSS_MAX_TABLE_LEN 128
> > +typedef struct VirtioNetRssCapa {
> > + uint8_t max_key_size;
> > + uint16_t max_indirection_len;
> > + uint32_t supported_hashes;
> > +} VirtioNetRssCapa;
> > +
> > +#define VIRTIO_NET_RSS_MIN_KEY_SIZE 40
> > +#define VIRTIO_NET_RSS_DEFAULT_KEY_SIZE 40
> > +#define VIRTIO_NET_RSS_MIN_TABLE_LEN 128
> > +#define VIRTIO_NET_RSS_DEFAULT_TABLE_LEN 128
> > typedef struct VirtioNetRssData {
> > bool enabled;
> > @@ -136,7 +144,8 @@ typedef struct VirtioNetRssData {
> > bool redirect;
> > bool populate_hash;
> > uint32_t hash_types;
> > - uint8_t key[VIRTIO_NET_RSS_MAX_KEY_SIZE];
> > + uint8_t key_len;
> > + uint8_t *key;
> > uint16_t indirections_len;
> > uint16_t *indirections_table;
> > uint16_t default_queue;
> > @@ -213,6 +222,7 @@ struct VirtIONet {
> > QDict *primary_opts;
> > bool primary_opts_from_json;
> > Notifier migration_state;
> > + VirtioNetRssCapa rss_capa;
> > VirtioNetRssData rss_data;
> > struct NetRxPkt *rx_pkt;
> > struct EBPFRSSContext ebpf_rss;
> > diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
> > index ad24aa1934..9398cdf803 100644
> > --- a/include/migration/vmstate.h
> > +++ b/include/migration/vmstate.h
> > @@ -448,6 +448,16 @@ extern const VMStateInfo vmstate_info_qlist;
> > .offset = vmstate_offset_varray(_state, _field, _type), \
> > }
> > +#define VMSTATE_VARRAY_UINT8_ALLOC(_field, _state, _field_num, _version, _info, _type) {\
> > + .name = (stringify(_field)), \
> > + .version_id = (_version), \
> > + .num_offset = vmstate_offset_value(_state, _field_num, uint8_t),\
> > + .info = &(_info), \
> > + .size = sizeof(_type), \
> > + .flags = VMS_VARRAY_UINT8 | VMS_POINTER | VMS_ALLOC, \
> > + .offset = vmstate_offset_pointer(_state, _field, _type), \
> > +}
> > +
> > #define VMSTATE_VSTRUCT_TEST(_field, _state, _test, _version, _vmsd, _type, _struct_version) { \
> > .name = (stringify(_field)), \
> > .version_id = (_version), \
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 3/5] virtio-net: add RSS support for Vhost backends
2022-04-08 12:28 [PATCH 0/5] Vhost-user: add Virtio RSS support Maxime Coquelin
2022-04-08 12:28 ` [PATCH 1/5] ebpf: pass and check RSS key length to the loader Maxime Coquelin
2022-04-08 12:28 ` [PATCH 2/5] virtio-net: prepare for variable RSS key and indir table lengths Maxime Coquelin
@ 2022-04-08 12:28 ` Maxime Coquelin
2022-04-15 5:41 ` Jason Wang
2022-04-08 12:28 ` [PATCH 4/5] docs: introduce RSS support in Vhost-user specification Maxime Coquelin
` (2 subsequent siblings)
5 siblings, 1 reply; 11+ messages in thread
From: Maxime Coquelin @ 2022-04-08 12:28 UTC (permalink / raw)
To: qemu-devel, mst, jasowang, andrew, yuri.benditovich, dgilbert, quintela
Cc: chenbo.xia, Maxime Coquelin, dmarchan, ktraynor
This patch introduces new Vhost backend callbacks to
support RSS, and makes them called in Virtio-net
device.
It will be used by Vhost-user backend implementation to
support RSS feature.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
hw/net/vhost_net-stub.c | 10 ++++++
hw/net/vhost_net.c | 22 +++++++++++++
hw/net/virtio-net.c | 53 +++++++++++++++++++++----------
include/hw/virtio/vhost-backend.h | 7 ++++
include/net/vhost_net.h | 4 +++
5 files changed, 79 insertions(+), 17 deletions(-)
diff --git a/hw/net/vhost_net-stub.c b/hw/net/vhost_net-stub.c
index 89d71cfb8e..cc05e07c1f 100644
--- a/hw/net/vhost_net-stub.c
+++ b/hw/net/vhost_net-stub.c
@@ -101,3 +101,13 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)
{
return 0;
}
+
+int vhost_net_get_rss(struct vhost_net *net, VirtioNetRssCapa *rss_capa)
+{
+ return 0;
+}
+
+int vhost_net_set_rss(struct vhost_net *net, VirtioNetRssData *rss_data)
+{
+ return 0;
+}
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 30379d2ca4..aa2a1e8e5f 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -512,3 +512,25 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)
return vhost_ops->vhost_net_set_mtu(&net->dev, mtu);
}
+
+int vhost_net_get_rss(struct vhost_net *net, VirtioNetRssCapa *rss_capa)
+{
+ const VhostOps *vhost_ops = net->dev.vhost_ops;
+
+ if (!vhost_ops->vhost_net_get_rss) {
+ return 0;
+ }
+
+ return vhost_ops->vhost_net_get_rss(&net->dev, rss_capa);
+}
+
+int vhost_net_set_rss(struct vhost_net *net, VirtioNetRssData *rss_data)
+{
+ const VhostOps *vhost_ops = net->dev.vhost_ops;
+
+ if (!vhost_ops->vhost_net_set_rss) {
+ return 0;
+ }
+
+ return vhost_ops->vhost_net_set_rss(&net->dev, rss_data);
+}
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 38436e472b..237bbdb1b3 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -741,8 +741,10 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features,
return features;
}
- if (!ebpf_rss_is_loaded(&n->ebpf_rss)) {
- virtio_clear_feature(&features, VIRTIO_NET_F_RSS);
+ if (nc->peer->info->type == NET_CLIENT_DRIVER_TAP) {
+ if (!ebpf_rss_is_loaded(&n->ebpf_rss)) {
+ virtio_clear_feature(&features, VIRTIO_NET_F_RSS);
+ }
}
features = vhost_net_get_features(get_vhost_net(nc->peer), features);
vdev->backend_features = features;
@@ -1161,11 +1163,17 @@ static void virtio_net_detach_epbf_rss(VirtIONet *n);
static void virtio_net_disable_rss(VirtIONet *n)
{
+ NetClientState *nc = qemu_get_queue(n->nic);
+
if (n->rss_data.enabled) {
trace_virtio_net_rss_disable();
}
n->rss_data.enabled = false;
+ if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
+ vhost_net_set_rss(get_vhost_net(nc->peer), &n->rss_data);
+ }
+
virtio_net_detach_epbf_rss(n);
}
@@ -1239,6 +1247,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
bool do_rss)
{
VirtIODevice *vdev = VIRTIO_DEVICE(n);
+ NetClientState *nc = qemu_get_queue(n->nic);
struct virtio_net_rss_config cfg;
size_t s, offset = 0, size_get;
uint16_t queue_pairs, i;
@@ -1354,22 +1363,29 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
}
n->rss_data.enabled = true;
- if (!n->rss_data.populate_hash) {
- if (!virtio_net_attach_epbf_rss(n)) {
- /* EBPF must be loaded for vhost */
- if (get_vhost_net(qemu_get_queue(n->nic)->peer)) {
- warn_report("Can't load eBPF RSS for vhost");
- goto error;
+ if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
+ if (vhost_net_set_rss(get_vhost_net(nc->peer), &n->rss_data)) {
+ warn_report("Failed to configure RSS for vhost-user");
+ goto error;
+ }
+ } else {
+ if (!n->rss_data.populate_hash) {
+ if (!virtio_net_attach_epbf_rss(n)) {
+ /* EBPF must be loaded for vhost */
+ if (get_vhost_net(nc->peer)) {
+ warn_report("Can't load eBPF RSS for vhost");
+ goto error;
+ }
+ /* fallback to software RSS */
+ warn_report("Can't load eBPF RSS - fallback to software RSS");
+ n->rss_data.enabled_software_rss = true;
}
- /* fallback to software RSS */
- warn_report("Can't load eBPF RSS - fallback to software RSS");
+ } else {
+ /* use software RSS for hash populating */
+ /* and detach eBPF if was loaded before */
+ virtio_net_detach_epbf_rss(n);
n->rss_data.enabled_software_rss = true;
}
- } else {
- /* use software RSS for hash populating */
- /* and detach eBPF if was loaded before */
- virtio_net_detach_epbf_rss(n);
- n->rss_data.enabled_software_rss = true;
}
trace_virtio_net_rss_enable(n->rss_data.hash_types,
@@ -3534,8 +3550,11 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
n->rss_capa.max_key_size = VIRTIO_NET_RSS_DEFAULT_KEY_SIZE;
n->rss_capa.max_indirection_len = VIRTIO_NET_RSS_DEFAULT_TABLE_LEN;
n->rss_capa.supported_hashes = VIRTIO_NET_RSS_SUPPORTED_HASHES;
-
- virtio_net_load_ebpf(n);
+ if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
+ vhost_net_get_rss(get_vhost_net(nc->peer), &n->rss_capa);
+ } else {
+ virtio_net_load_ebpf(n);
+ }
} else {
n->rss_capa.max_indirection_len = 1;
}
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index 81bf3109f8..0b9e2ea26e 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -12,6 +12,7 @@
#define VHOST_BACKEND_H
#include "exec/memory.h"
+#include "hw/virtio/virtio-net.h"
typedef enum VhostBackendType {
VHOST_BACKEND_TYPE_NONE = 0,
@@ -45,6 +46,10 @@ typedef int (*vhost_backend_memslots_limit)(struct vhost_dev *dev);
typedef int (*vhost_net_set_backend_op)(struct vhost_dev *dev,
struct vhost_vring_file *file);
typedef int (*vhost_net_set_mtu_op)(struct vhost_dev *dev, uint16_t mtu);
+typedef int (*vhost_net_get_rss_op)(struct vhost_dev *dev,
+ VirtioNetRssCapa *rss_capa);
+typedef int (*vhost_net_set_rss_op)(struct vhost_dev *dev,
+ VirtioNetRssData *rss_data);
typedef int (*vhost_scsi_set_endpoint_op)(struct vhost_dev *dev,
struct vhost_scsi_target *target);
typedef int (*vhost_scsi_clear_endpoint_op)(struct vhost_dev *dev,
@@ -133,6 +138,8 @@ typedef struct VhostOps {
vhost_backend_memslots_limit vhost_backend_memslots_limit;
vhost_net_set_backend_op vhost_net_set_backend;
vhost_net_set_mtu_op vhost_net_set_mtu;
+ vhost_net_get_rss_op vhost_net_get_rss;
+ vhost_net_set_rss_op vhost_net_set_rss;
vhost_scsi_set_endpoint_op vhost_scsi_set_endpoint;
vhost_scsi_clear_endpoint_op vhost_scsi_clear_endpoint;
vhost_scsi_get_abi_version_op vhost_scsi_get_abi_version;
diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
index 387e913e4e..9cf702e7e3 100644
--- a/include/net/vhost_net.h
+++ b/include/net/vhost_net.h
@@ -48,4 +48,8 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net);
int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
+int vhost_net_get_rss(struct vhost_net *net, VirtioNetRssCapa *rss_capa);
+
+int vhost_net_set_rss(struct vhost_net *net, VirtioNetRssData *rss_data);
+
#endif
--
2.35.1
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH 3/5] virtio-net: add RSS support for Vhost backends
2022-04-08 12:28 ` [PATCH 3/5] virtio-net: add RSS support for Vhost backends Maxime Coquelin
@ 2022-04-15 5:41 ` Jason Wang
0 siblings, 0 replies; 11+ messages in thread
From: Jason Wang @ 2022-04-15 5:41 UTC (permalink / raw)
To: Maxime Coquelin, qemu-devel, mst, andrew, yuri.benditovich,
dgilbert, quintela
Cc: chenbo.xia, dmarchan, ktraynor
在 2022/4/8 20:28, Maxime Coquelin 写道:
> This patch introduces new Vhost backend callbacks to
> support RSS, and makes them called in Virtio-net
> device.
>
> It will be used by Vhost-user backend implementation to
> support RSS feature.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> hw/net/vhost_net-stub.c | 10 ++++++
> hw/net/vhost_net.c | 22 +++++++++++++
> hw/net/virtio-net.c | 53 +++++++++++++++++++++----------
> include/hw/virtio/vhost-backend.h | 7 ++++
> include/net/vhost_net.h | 4 +++
> 5 files changed, 79 insertions(+), 17 deletions(-)
>
> diff --git a/hw/net/vhost_net-stub.c b/hw/net/vhost_net-stub.c
> index 89d71cfb8e..cc05e07c1f 100644
> --- a/hw/net/vhost_net-stub.c
> +++ b/hw/net/vhost_net-stub.c
> @@ -101,3 +101,13 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)
> {
> return 0;
> }
> +
> +int vhost_net_get_rss(struct vhost_net *net, VirtioNetRssCapa *rss_capa)
> +{
> + return 0;
> +}
> +
> +int vhost_net_set_rss(struct vhost_net *net, VirtioNetRssData *rss_data)
> +{
> + return 0;
> +}
> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> index 30379d2ca4..aa2a1e8e5f 100644
> --- a/hw/net/vhost_net.c
> +++ b/hw/net/vhost_net.c
> @@ -512,3 +512,25 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)
>
> return vhost_ops->vhost_net_set_mtu(&net->dev, mtu);
> }
> +
> +int vhost_net_get_rss(struct vhost_net *net, VirtioNetRssCapa *rss_capa)
> +{
> + const VhostOps *vhost_ops = net->dev.vhost_ops;
> +
> + if (!vhost_ops->vhost_net_get_rss) {
> + return 0;
> + }
> +
> + return vhost_ops->vhost_net_get_rss(&net->dev, rss_capa);
> +}
> +
> +int vhost_net_set_rss(struct vhost_net *net, VirtioNetRssData *rss_data)
> +{
> + const VhostOps *vhost_ops = net->dev.vhost_ops;
> +
> + if (!vhost_ops->vhost_net_set_rss) {
> + return 0;
> + }
> +
> + return vhost_ops->vhost_net_set_rss(&net->dev, rss_data);
> +}
> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
> index 38436e472b..237bbdb1b3 100644
> --- a/hw/net/virtio-net.c
> +++ b/hw/net/virtio-net.c
> @@ -741,8 +741,10 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features,
> return features;
> }
>
> - if (!ebpf_rss_is_loaded(&n->ebpf_rss)) {
> - virtio_clear_feature(&features, VIRTIO_NET_F_RSS);
> + if (nc->peer->info->type == NET_CLIENT_DRIVER_TAP) {
> + if (!ebpf_rss_is_loaded(&n->ebpf_rss)) {
> + virtio_clear_feature(&features, VIRTIO_NET_F_RSS);
> + }
> }
> features = vhost_net_get_features(get_vhost_net(nc->peer), features);
> vdev->backend_features = features;
> @@ -1161,11 +1163,17 @@ static void virtio_net_detach_epbf_rss(VirtIONet *n);
>
> static void virtio_net_disable_rss(VirtIONet *n)
> {
> + NetClientState *nc = qemu_get_queue(n->nic);
> +
> if (n->rss_data.enabled) {
> trace_virtio_net_rss_disable();
> }
> n->rss_data.enabled = false;
>
> + if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
> + vhost_net_set_rss(get_vhost_net(nc->peer), &n->rss_data);
> + }
> +
> virtio_net_detach_epbf_rss(n);
> }
>
> @@ -1239,6 +1247,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
> bool do_rss)
> {
> VirtIODevice *vdev = VIRTIO_DEVICE(n);
> + NetClientState *nc = qemu_get_queue(n->nic);
> struct virtio_net_rss_config cfg;
> size_t s, offset = 0, size_get;
> uint16_t queue_pairs, i;
> @@ -1354,22 +1363,29 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n,
> }
> n->rss_data.enabled = true;
>
> - if (!n->rss_data.populate_hash) {
> - if (!virtio_net_attach_epbf_rss(n)) {
> - /* EBPF must be loaded for vhost */
> - if (get_vhost_net(qemu_get_queue(n->nic)->peer)) {
> - warn_report("Can't load eBPF RSS for vhost");
> - goto error;
> + if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
> + if (vhost_net_set_rss(get_vhost_net(nc->peer), &n->rss_data)) {
> + warn_report("Failed to configure RSS for vhost-user");
> + goto error;
> + }
> + } else {
> + if (!n->rss_data.populate_hash) {
> + if (!virtio_net_attach_epbf_rss(n)) {
> + /* EBPF must be loaded for vhost */
> + if (get_vhost_net(nc->peer)) {
> + warn_report("Can't load eBPF RSS for vhost");
> + goto error;
> + }
> + /* fallback to software RSS */
> + warn_report("Can't load eBPF RSS - fallback to software RSS");
> + n->rss_data.enabled_software_rss = true;
> }
> - /* fallback to software RSS */
> - warn_report("Can't load eBPF RSS - fallback to software RSS");
> + } else {
> + /* use software RSS for hash populating */
> + /* and detach eBPF if was loaded before */
> + virtio_net_detach_epbf_rss(n);
> n->rss_data.enabled_software_rss = true;
> }
> - } else {
> - /* use software RSS for hash populating */
> - /* and detach eBPF if was loaded before */
> - virtio_net_detach_epbf_rss(n);
> - n->rss_data.enabled_software_rss = true;
> }
>
> trace_virtio_net_rss_enable(n->rss_data.hash_types,
> @@ -3534,8 +3550,11 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
> n->rss_capa.max_key_size = VIRTIO_NET_RSS_DEFAULT_KEY_SIZE;
> n->rss_capa.max_indirection_len = VIRTIO_NET_RSS_DEFAULT_TABLE_LEN;
> n->rss_capa.supported_hashes = VIRTIO_NET_RSS_SUPPORTED_HASHES;
> -
> - virtio_net_load_ebpf(n);
> + if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
> + vhost_net_get_rss(get_vhost_net(nc->peer), &n->rss_capa);
I wonder if we need a command parameter for the capability instead of
silently get those from the vhost-user backhand (since there's no
guarantee that the capability in src and dst are matched).
Thanks
> + } else {
> + virtio_net_load_ebpf(n);
> + }
> } else {
> n->rss_capa.max_indirection_len = 1;
> }
> diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
> index 81bf3109f8..0b9e2ea26e 100644
> --- a/include/hw/virtio/vhost-backend.h
> +++ b/include/hw/virtio/vhost-backend.h
> @@ -12,6 +12,7 @@
> #define VHOST_BACKEND_H
>
> #include "exec/memory.h"
> +#include "hw/virtio/virtio-net.h"
>
> typedef enum VhostBackendType {
> VHOST_BACKEND_TYPE_NONE = 0,
> @@ -45,6 +46,10 @@ typedef int (*vhost_backend_memslots_limit)(struct vhost_dev *dev);
> typedef int (*vhost_net_set_backend_op)(struct vhost_dev *dev,
> struct vhost_vring_file *file);
> typedef int (*vhost_net_set_mtu_op)(struct vhost_dev *dev, uint16_t mtu);
> +typedef int (*vhost_net_get_rss_op)(struct vhost_dev *dev,
> + VirtioNetRssCapa *rss_capa);
> +typedef int (*vhost_net_set_rss_op)(struct vhost_dev *dev,
> + VirtioNetRssData *rss_data);
> typedef int (*vhost_scsi_set_endpoint_op)(struct vhost_dev *dev,
> struct vhost_scsi_target *target);
> typedef int (*vhost_scsi_clear_endpoint_op)(struct vhost_dev *dev,
> @@ -133,6 +138,8 @@ typedef struct VhostOps {
> vhost_backend_memslots_limit vhost_backend_memslots_limit;
> vhost_net_set_backend_op vhost_net_set_backend;
> vhost_net_set_mtu_op vhost_net_set_mtu;
> + vhost_net_get_rss_op vhost_net_get_rss;
> + vhost_net_set_rss_op vhost_net_set_rss;
> vhost_scsi_set_endpoint_op vhost_scsi_set_endpoint;
> vhost_scsi_clear_endpoint_op vhost_scsi_clear_endpoint;
> vhost_scsi_get_abi_version_op vhost_scsi_get_abi_version;
> diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
> index 387e913e4e..9cf702e7e3 100644
> --- a/include/net/vhost_net.h
> +++ b/include/net/vhost_net.h
> @@ -48,4 +48,8 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net);
>
> int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
>
> +int vhost_net_get_rss(struct vhost_net *net, VirtioNetRssCapa *rss_capa);
> +
> +int vhost_net_set_rss(struct vhost_net *net, VirtioNetRssData *rss_data);
> +
> #endif
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 4/5] docs: introduce RSS support in Vhost-user specification
2022-04-08 12:28 [PATCH 0/5] Vhost-user: add Virtio RSS support Maxime Coquelin
` (2 preceding siblings ...)
2022-04-08 12:28 ` [PATCH 3/5] virtio-net: add RSS support for Vhost backends Maxime Coquelin
@ 2022-04-08 12:28 ` Maxime Coquelin
2022-04-11 12:18 ` Dr. David Alan Gilbert
2022-04-08 12:28 ` [PATCH 5/5] vhost-user: add RSS support Maxime Coquelin
2022-04-15 5:43 ` [PATCH 0/5] Vhost-user: add Virtio " Jason Wang
5 siblings, 1 reply; 11+ messages in thread
From: Maxime Coquelin @ 2022-04-08 12:28 UTC (permalink / raw)
To: qemu-devel, mst, jasowang, andrew, yuri.benditovich, dgilbert, quintela
Cc: chenbo.xia, Maxime Coquelin, dmarchan, ktraynor
This patch documents RSS feature in Vhost-user specification.
Two new requests are introduced backed by a dedicated
protocol feature.
First one is to query the Vhost-user slave RSS capabilities
such as supported hash types, maximum key length and
indirection table size.
The second one is to provide the slave with driver's RSS
configuration.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
docs/interop/vhost-user.rst | 57 +++++++++++++++++++++++++++++++++++++
1 file changed, 57 insertions(+)
diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
index 4dbc84fd00..9de6297568 100644
--- a/docs/interop/vhost-user.rst
+++ b/docs/interop/vhost-user.rst
@@ -258,6 +258,42 @@ Inflight description
:queue size: a 16-bit size of virtqueues
+RSS capabilities description
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
++----------------------+-------------+---------------+
+| supported hash types | max key len | max indir len |
++----------------------+-------------+---------------+
+
+:supported hash types: a 32-bit bitfield of supported hash types as defined
+ in the Virtio specification
+
+:max key len: a 8-bit maximum size of the RSS key
+
+:max indir len: a 16-bits maximum size of the RSS indirection table
+
+RSS data description
+^^^^^^^^^^^^^^^^^^^^
+
++------------+---------+-----+-----------+-------------+---------------+
+| hash types | key len | key | indir len | indir table | default queue |
++------------+---------+-----+-----------+-------------+---------------+
+
+:hash types: a 32-bit bitfield of supported hash types as defined in the
+ Virtio specification
+
+:key len: 8-bit size of the RSS key
+
+:key: a 8-bit array of 52 elements containing the RSS key
+
+:indir len: a 16-bit size of the RSS indirection table
+
+:indir table: a 16-bit array of 512 elements containing the hash indirection
+ table
+
+:default queue: the default queue index for flows not matching requested hash
+ types
+
C structure
-----------
@@ -858,6 +894,7 @@ Protocol features
#define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS 14
#define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS 15
#define VHOST_USER_PROTOCOL_F_STATUS 16
+ #define VHOST_USER_PROTOCOL_F_NET_RSS 17
Master message types
--------------------
@@ -1371,6 +1408,26 @@ Master message types
query the backend for its device status as defined in the Virtio
specification.
+``VHOST_USER_NET_GET_RSS``
+ :id: 41
+ :equivalent ioctl: N/A
+ :slave payload: RSS capabilities description
+ :master payload: N/A
+
+ When the ``VHOST_USER_PROTOCOL_F_NET_RSS`` protocol has been successfully
+ negotiated, this message is submitted by the master to get the RSS
+ capabilities of the slave.
+
+``VHOST_USER_NET_SET_RSS``
+ :id: 42
+ :equivalent ioctl: N/A
+ :slave payload: N/A
+ :master payload: RSS data description
+
+ When the ``VHOST_USER_PROTOCOL_F_NET_RSS`` protocol has been successfully
+ negotiated, this message is submitted by the master to set the RSS
+ configuration defined by the Virtio driver.
+
Slave message types
-------------------
--
2.35.1
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH 4/5] docs: introduce RSS support in Vhost-user specification
2022-04-08 12:28 ` [PATCH 4/5] docs: introduce RSS support in Vhost-user specification Maxime Coquelin
@ 2022-04-11 12:18 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 11+ messages in thread
From: Dr. David Alan Gilbert @ 2022-04-11 12:18 UTC (permalink / raw)
To: Maxime Coquelin
Cc: chenbo.xia, andrew, quintela, jasowang, mst, qemu-devel,
ktraynor, yuri.benditovich, dmarchan
* Maxime Coquelin (maxime.coquelin@redhat.com) wrote:
> This patch documents RSS feature in Vhost-user specification.
> Two new requests are introduced backed by a dedicated
> protocol feature.
>
> First one is to query the Vhost-user slave RSS capabilities
> such as supported hash types, maximum key length and
> indirection table size.
>
> The second one is to provide the slave with driver's RSS
> configuration.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Perhaps it would be worth adding a link to devel/ebpf_rss.rst somewhere
in there; I didn't know what RSS so had to dig.
Also, perhaps it's useful to update that file to point back to the
vhost-user or dpdk implementations.
Dave
> ---
> docs/interop/vhost-user.rst | 57 +++++++++++++++++++++++++++++++++++++
> 1 file changed, 57 insertions(+)
>
> diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
> index 4dbc84fd00..9de6297568 100644
> --- a/docs/interop/vhost-user.rst
> +++ b/docs/interop/vhost-user.rst
> @@ -258,6 +258,42 @@ Inflight description
>
> :queue size: a 16-bit size of virtqueues
>
> +RSS capabilities description
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> ++----------------------+-------------+---------------+
> +| supported hash types | max key len | max indir len |
> ++----------------------+-------------+---------------+
> +
> +:supported hash types: a 32-bit bitfield of supported hash types as defined
> + in the Virtio specification
> +
> +:max key len: a 8-bit maximum size of the RSS key
> +
> +:max indir len: a 16-bits maximum size of the RSS indirection table
> +
> +RSS data description
> +^^^^^^^^^^^^^^^^^^^^
> +
> ++------------+---------+-----+-----------+-------------+---------------+
> +| hash types | key len | key | indir len | indir table | default queue |
> ++------------+---------+-----+-----------+-------------+---------------+
> +
> +:hash types: a 32-bit bitfield of supported hash types as defined in the
> + Virtio specification
> +
> +:key len: 8-bit size of the RSS key
> +
> +:key: a 8-bit array of 52 elements containing the RSS key
> +
> +:indir len: a 16-bit size of the RSS indirection table
> +
> +:indir table: a 16-bit array of 512 elements containing the hash indirection
> + table
> +
> +:default queue: the default queue index for flows not matching requested hash
> + types
> +
> C structure
> -----------
>
> @@ -858,6 +894,7 @@ Protocol features
> #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS 14
> #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS 15
> #define VHOST_USER_PROTOCOL_F_STATUS 16
> + #define VHOST_USER_PROTOCOL_F_NET_RSS 17
>
> Master message types
> --------------------
> @@ -1371,6 +1408,26 @@ Master message types
> query the backend for its device status as defined in the Virtio
> specification.
>
> +``VHOST_USER_NET_GET_RSS``
> + :id: 41
> + :equivalent ioctl: N/A
> + :slave payload: RSS capabilities description
> + :master payload: N/A
> +
> + When the ``VHOST_USER_PROTOCOL_F_NET_RSS`` protocol has been successfully
> + negotiated, this message is submitted by the master to get the RSS
> + capabilities of the slave.
> +
> +``VHOST_USER_NET_SET_RSS``
> + :id: 42
> + :equivalent ioctl: N/A
> + :slave payload: N/A
> + :master payload: RSS data description
> +
> + When the ``VHOST_USER_PROTOCOL_F_NET_RSS`` protocol has been successfully
> + negotiated, this message is submitted by the master to set the RSS
> + configuration defined by the Virtio driver.
> +
>
> Slave message types
> -------------------
> --
> 2.35.1
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 5/5] vhost-user: add RSS support
2022-04-08 12:28 [PATCH 0/5] Vhost-user: add Virtio RSS support Maxime Coquelin
` (3 preceding siblings ...)
2022-04-08 12:28 ` [PATCH 4/5] docs: introduce RSS support in Vhost-user specification Maxime Coquelin
@ 2022-04-08 12:28 ` Maxime Coquelin
2022-04-15 5:43 ` [PATCH 0/5] Vhost-user: add Virtio " Jason Wang
5 siblings, 0 replies; 11+ messages in thread
From: Maxime Coquelin @ 2022-04-08 12:28 UTC (permalink / raw)
To: qemu-devel, mst, jasowang, andrew, yuri.benditovich, dgilbert, quintela
Cc: chenbo.xia, Maxime Coquelin, dmarchan, ktraynor
This patch implements the RSS feature to the
Vhost-user backend.
The implementation supports up to 52 bytes RSS key length,
and 512 indirection table entries.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
hw/virtio/vhost-user.c | 146 ++++++++++++++++++++++++++++++++++++++++-
1 file changed, 145 insertions(+), 1 deletion(-)
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 6abbc9da32..d047da81ba 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -81,6 +81,8 @@ enum VhostUserProtocolFeature {
VHOST_USER_PROTOCOL_F_RESET_DEVICE = 13,
/* Feature 14 reserved for VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS. */
VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS = 15,
+ /* Feature 16 reserved for VHOST_USER_PROTOCOL_F_STATUS. */
+ VHOST_USER_PROTOCOL_F_NET_RSS = 17,
VHOST_USER_PROTOCOL_F_MAX
};
@@ -126,6 +128,10 @@ typedef enum VhostUserRequest {
VHOST_USER_GET_MAX_MEM_SLOTS = 36,
VHOST_USER_ADD_MEM_REG = 37,
VHOST_USER_REM_MEM_REG = 38,
+ /* Message number 39 reserved for VHOST_USER_SET_STATUS. */
+ /* Message number 40 reserved for VHOST_USER_GET_STATUS. */
+ VHOST_USER_NET_GET_RSS = 41,
+ VHOST_USER_NET_SET_RSS = 42,
VHOST_USER_MAX
} VhostUserRequest;
@@ -196,6 +202,24 @@ typedef struct VhostUserInflight {
uint16_t queue_size;
} VhostUserInflight;
+typedef struct VhostUserRSSCapa {
+ uint32_t supported_hash_types;
+ uint8_t max_key_len;
+ uint16_t max_indir_len;
+} VhostUserRSSCapa;
+
+#define VHOST_USER_RSS_MAX_KEY_LEN 52
+#define VHOST_USER_RSS_MAX_INDIR_LEN 512
+
+typedef struct VhostUserRSSData {
+ uint32_t hash_types;
+ uint8_t key_len;
+ uint8_t key[VHOST_USER_RSS_MAX_KEY_LEN];
+ uint16_t indir_len;
+ uint16_t indir_table[VHOST_USER_RSS_MAX_INDIR_LEN];
+ uint16_t default_queue;
+} VhostUserRSSData;
+
typedef struct {
VhostUserRequest request;
@@ -220,6 +244,8 @@ typedef union {
VhostUserCryptoSession session;
VhostUserVringArea area;
VhostUserInflight inflight;
+ VhostUserRSSCapa rss_capa;
+ VhostUserRSSData rss_data;
} VhostUserPayload;
typedef struct VhostUserMsg {
@@ -2178,7 +2204,123 @@ static int vhost_user_net_set_mtu(struct vhost_dev *dev, uint16_t mtu)
return ret;
}
- /* If reply_ack supported, slave has to ack specified MTU is valid */
+ if (reply_supported) {
+ return process_message_reply(dev, &msg);
+ }
+
+ return 0;
+}
+
+static int vhost_user_net_get_rss(struct vhost_dev *dev,
+ VirtioNetRssCapa *rss_capa)
+{
+ int ret;
+ VhostUserMsg msg = {
+ .hdr.request = VHOST_USER_NET_GET_RSS,
+ .hdr.flags = VHOST_USER_VERSION,
+ };
+
+ if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_NET_RSS))) {
+ return -EPROTO;
+ }
+
+ ret = vhost_user_write(dev, &msg, NULL, 0);
+ if (ret < 0) {
+ return ret;
+ }
+
+ ret = vhost_user_read(dev, &msg);
+ if (ret < 0) {
+ return ret;
+ }
+
+ if (msg.hdr.request != VHOST_USER_NET_GET_RSS) {
+ error_report("Received unexpected msg type. Expected %d received %d",
+ VHOST_USER_NET_GET_RSS, msg.hdr.request);
+ return -EPROTO;
+ }
+
+ if (msg.hdr.size != sizeof(msg.payload.rss_capa)) {
+ error_report("Received bad msg size.");
+ return -EPROTO;
+ }
+
+ if (msg.payload.rss_capa.max_key_len < VIRTIO_NET_RSS_MIN_KEY_SIZE) {
+ error_report("Invalid max RSS key len (%uB, minimum %uB).",
+ msg.payload.rss_capa.max_key_len,
+ VIRTIO_NET_RSS_MIN_KEY_SIZE);
+ return -EINVAL;
+ }
+
+ if (msg.payload.rss_capa.max_indir_len < VIRTIO_NET_RSS_MIN_TABLE_LEN) {
+ error_report("Invalid max RSS indir table entries (%u, minimum %u).",
+ msg.payload.rss_capa.max_indir_len,
+ VIRTIO_NET_RSS_MIN_TABLE_LEN);
+ return -EINVAL;
+ }
+
+ rss_capa->supported_hashes = msg.payload.rss_capa.supported_hash_types;
+ rss_capa->max_key_size = MIN(msg.payload.rss_capa.max_key_len,
+ VHOST_USER_RSS_MAX_KEY_LEN);
+ rss_capa->max_indirection_len = MIN(msg.payload.rss_capa.max_indir_len,
+ VHOST_USER_RSS_MAX_INDIR_LEN);
+
+ return 0;
+}
+
+static int vhost_user_net_set_rss(struct vhost_dev *dev,
+ VirtioNetRssData *rss_data)
+{
+ VhostUserMsg msg;
+ bool reply_supported = virtio_has_feature(dev->protocol_features,
+ VHOST_USER_PROTOCOL_F_REPLY_ACK);
+ int ret;
+
+ if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_NET_RSS))) {
+ return -EPROTO;
+ }
+
+ msg.hdr.request = VHOST_USER_NET_SET_RSS;
+ msg.hdr.size = sizeof(msg.payload.rss_data);
+ msg.hdr.flags = VHOST_USER_VERSION;
+ if (reply_supported) {
+ msg.hdr.flags |= VHOST_USER_NEED_REPLY_MASK;
+ }
+
+ msg.payload.rss_data.hash_types = rss_data->hash_types;
+
+ if (rss_data->key_len > VHOST_USER_RSS_MAX_KEY_LEN) {
+ error_report("RSS key length too long (%uB, max %uB).",
+ rss_data->key_len, VHOST_USER_RSS_MAX_KEY_LEN);
+ return -EINVAL;
+ }
+
+ msg.payload.rss_data.key_len = rss_data->key_len;
+ memset(msg.payload.rss_data.key, 0, VHOST_USER_RSS_MAX_KEY_LEN);
+ memcpy(msg.payload.rss_data.key, rss_data->key, rss_data->key_len);
+
+ if (rss_data->indirections_len > VHOST_USER_RSS_MAX_INDIR_LEN) {
+ error_report("RSS indirection table too large (%u, max %u).",
+ rss_data->indirections_len, VHOST_USER_RSS_MAX_INDIR_LEN);
+ return -EINVAL;
+ }
+
+ msg.payload.rss_data.indir_len = rss_data->indirections_len;
+ memset(msg.payload.rss_data.indir_table, 0,
+ VHOST_USER_RSS_MAX_INDIR_LEN *
+ sizeof(*msg.payload.rss_data.indir_table));
+ memcpy(msg.payload.rss_data.indir_table, rss_data->indirections_table,
+ msg.payload.rss_data.indir_len *
+ sizeof(*msg.payload.rss_data.indir_table));
+
+ msg.payload.rss_data.default_queue = rss_data->default_queue;
+
+ ret = vhost_user_write(dev, &msg, NULL, 0);
+ if (ret < 0) {
+ return ret;
+ }
+
+ /* If reply_ack supported, slave has to ack specified RSS conf is valid */
if (reply_supported) {
return process_message_reply(dev, &msg);
}
@@ -2555,6 +2697,8 @@ const VhostOps user_ops = {
.vhost_migration_done = vhost_user_migration_done,
.vhost_backend_can_merge = vhost_user_can_merge,
.vhost_net_set_mtu = vhost_user_net_set_mtu,
+ .vhost_net_get_rss = vhost_user_net_get_rss,
+ .vhost_net_set_rss = vhost_user_net_set_rss,
.vhost_set_iotlb_callback = vhost_user_set_iotlb_callback,
.vhost_send_device_iotlb_msg = vhost_user_send_device_iotlb_msg,
.vhost_get_config = vhost_user_get_config,
--
2.35.1
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH 0/5] Vhost-user: add Virtio RSS support
2022-04-08 12:28 [PATCH 0/5] Vhost-user: add Virtio RSS support Maxime Coquelin
` (4 preceding siblings ...)
2022-04-08 12:28 ` [PATCH 5/5] vhost-user: add RSS support Maxime Coquelin
@ 2022-04-15 5:43 ` Jason Wang
5 siblings, 0 replies; 11+ messages in thread
From: Jason Wang @ 2022-04-15 5:43 UTC (permalink / raw)
To: Maxime Coquelin, qemu-devel, mst, andrew, yuri.benditovich,
dgilbert, quintela
Cc: chenbo.xia, dmarchan, ktraynor
在 2022/4/8 20:28, Maxime Coquelin 写道:
> The goal of this series is to add support for Virtio RSS
> feature to the Vhost-user backend.
>
> First patches are preliminary reworks to support variable
> RSS key and indirection table length. eBPF change only adds
> checks on whether the key length is 40B, it does not add
> support for longer keys.
>
> Vhost-user implementation supports up to 52B RSS key, in
> order to match with the maximum supported by physical
> NICs (Intel E810). Idea is that it could be used for
> application like Virtio-forwarder, by programming the
> Virtio device RSS key into the physical NIC and let the
> physical NIC do the packets distribution.
>
> DPDK Vhost-user backend PoC implementing the new requests
> can be found here [0], it only implements the messages
> handling, it does not perform any RSS for now.
>
> [0]: https://gitlab.com/mcoquelin/dpdk-next-virtio/-/commits/vhost_user_rss_poc/
Not directly related to this series. I wonder if vhost-user consider to
support control virtqueue then all the RSS stuffs could be done at
vhost-user backend without introducing new commands.
Thanks
>
> Maxime Coquelin (5):
> ebpf: pass and check RSS key length to the loader
> virtio-net: prepare for variable RSS key and indir table lengths
> virtio-net: add RSS support for Vhost backends
> docs: introduce RSS support in Vhost-user specification
> vhost-user: add RSS support
>
> docs/interop/vhost-user.rst | 57 ++++++++++++
> ebpf/ebpf_rss-stub.c | 3 +-
> ebpf/ebpf_rss.c | 17 ++--
> ebpf/ebpf_rss.h | 3 +-
> hw/net/vhost_net-stub.c | 10 ++
> hw/net/vhost_net.c | 22 +++++
> hw/net/virtio-net.c | 87 +++++++++++++-----
> hw/virtio/vhost-user.c | 146 +++++++++++++++++++++++++++++-
> include/hw/virtio/vhost-backend.h | 7 ++
> include/hw/virtio/virtio-net.h | 16 +++-
> include/migration/vmstate.h | 10 ++
> include/net/vhost_net.h | 4 +
> 12 files changed, 344 insertions(+), 38 deletions(-)
>
^ permalink raw reply [flat|nested] 11+ messages in thread