* [PATCH 0/7] net/gve: RSS Support for GVE Driver
@ 2024-01-23 17:58 Joshua Washington
2024-01-23 17:58 ` [PATCH v2 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
` (10 more replies)
0 siblings, 11 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-23 17:58 UTC (permalink / raw)
Cc: dev, Ferruh Yigit, Rushil Gupta, Joshua Washington
This patch series introduces RSS support for the GVE poll-mode driver.
This series includes implementations of the following eth_dev_ops:
1) rss_hash_update
2) rss_hash_conf_get
3) reta_query
4) reta_update
In rss_hash_update, the GVE driver supports the following RSS hash
types:
* RTE_ETH_RSS_IPV4
* RTE_ETH_RSS_NONFRAG_IPV4_TCP
* RTE_ETH_RSS_NONFRAG_IPV4_UDP
* RTE_ETH_RSS_IPV6
* RTE_ETH_RSS_IPV6_EX
* RTE_ETH_RSS_NONFRAG_IPV6_TCP
* RTE_ETH_RSS_NONFRAG_IPV6_UDP
* RTE_ETH_RSS_IPV6_TCP_EX
* RTE_ETH_RSS_IPV6_UDP_EX
The hash key is 40B, and the lookup table has 128 entries. These values
are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set
with a key before the redriection table is set up. When the RSS hash is
configured, a default redirection table is generated based on the number
of queues. When the device is re-configured, the redirection table is
reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for
RSS redirection table entries, while the RTE API uses 16 bit integers.
However, this is unlikely to be an issue, as these values represent
receive queues, and the gVNIC device does not support anywhere near 64K
queues.
This series also updates the corresponding feature matrix ertries and
documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches
fixes.
Note: There is a warning about complex macros being parenthesized that
does not seem to be well-founded.
Joshua Washington (7):
net/gve: fully expose RSS offload support in dev_info
net/gve: RSS adminq command changes
net/gve: add gve_rss library for handling RSS-related behaviors
net/gve: RSS configuration update support
net/gve: RSS redirection table update support
net/gve: update gve.ini with RSS capabilities
net/gve: update GVE documentation with RSS support
doc/guides/nics/features/gve.ini | 3 +
doc/guides/nics/gve.rst | 16 ++-
drivers/net/gve/base/gve.h | 15 ++
drivers/net/gve/base/gve_adminq.c | 59 ++++++++
drivers/net/gve/base/gve_adminq.h | 21 +++
drivers/net/gve/gve_ethdev.c | 231 +++++++++++++++++++++++++++++-
drivers/net/gve/gve_ethdev.h | 17 +++
drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++
drivers/net/gve/gve_rss.h | 107 ++++++++++++++
drivers/net/gve/meson.build | 1 +
10 files changed, 667 insertions(+), 9 deletions(-)
create mode 100644 drivers/net/gve/gve_rss.c
create mode 100644 drivers/net/gve/gve_rss.h
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply [flat|nested] 44+ messages in thread
* [PATCH v2 1/7] net/gve: fully expose RSS offload support in dev_info
2024-01-23 17:58 [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
@ 2024-01-23 17:58 ` Joshua Washington
2024-01-23 17:58 ` [PATCH v2 2/7] net/gve: RSS adminq command changes Joshua Washington
` (9 subsequent siblings)
10 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-23 17:58 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch communicates that the GVE driver supports RSS, along with
the RSS offloads supported by the driver.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/gve_ethdev.c | 4 +++-
drivers/net/gve/gve_ethdev.h | 8 ++++++++
2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index d162fd3864..6acdb4e13b 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -405,7 +405,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = priv->max_mtu;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
- dev_info->rx_offload_capa = 0;
+ dev_info->rx_offload_capa = RTE_ETH_RX_OFFLOAD_RSS_HASH;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
@@ -442,6 +442,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.nb_align = 1,
};
+ dev_info->flow_type_rss_offloads = GVE_RSS_OFFLOAD_ALL;
+
return 0;
}
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 9893fcfee6..14c72ec91a 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -33,6 +33,14 @@
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
+#define GVE_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_IPV6_EX | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_IPV6_TCP_EX)
+
/* A list of pages registered with the device during setup and used by a queue
* as buffers
*/
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v2 2/7] net/gve: RSS adminq command changes
2024-01-23 17:58 [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
2024-01-23 17:58 ` [PATCH v2 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
@ 2024-01-23 17:58 ` Joshua Washington
2024-01-23 17:58 ` [PATCH v2 3/7] net/gve: add gve_rss library for handling RSS-related behaviors Joshua Washington
` (8 subsequent siblings)
10 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-23 17:58 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This change introduces admin queue changes that enable the configuration
of RSS parameters for the GVE driver.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/base/gve.h | 15 ++++++++
drivers/net/gve/base/gve_adminq.c | 59 +++++++++++++++++++++++++++++++
drivers/net/gve/base/gve_adminq.h | 29 +++++++++++++++
drivers/net/gve/gve_ethdev.h | 6 +++-
4 files changed, 108 insertions(+), 1 deletion(-)
diff --git a/drivers/net/gve/base/gve.h b/drivers/net/gve/base/gve.h
index f7b297e759..9c58fc4238 100644
--- a/drivers/net/gve/base/gve.h
+++ b/drivers/net/gve/base/gve.h
@@ -51,4 +51,19 @@ enum gve_state_flags_bit {
GVE_PRIV_FLAGS_NAPI_ENABLED = 4,
};
+enum gve_rss_hash_algorithm {
+ GVE_RSS_HASH_UNDEFINED = 0,
+ GVE_RSS_HASH_TOEPLITZ = 1,
+};
+
+struct gve_rss_config {
+ uint16_t hash_types;
+ enum gve_rss_hash_algorithm alg;
+ uint16_t key_size;
+ uint16_t indir_size;
+ uint8_t *key;
+ uint32_t *indir;
+};
+
+
#endif /* _GVE_H_ */
diff --git a/drivers/net/gve/base/gve_adminq.c b/drivers/net/gve/base/gve_adminq.c
index 343bd13d67..897c3dce03 100644
--- a/drivers/net/gve/base/gve_adminq.c
+++ b/drivers/net/gve/base/gve_adminq.c
@@ -389,6 +389,9 @@ static int gve_adminq_issue_cmd(struct gve_priv *priv,
case GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES:
priv->adminq_dcfg_device_resources_cnt++;
break;
+ case GVE_ADMINQ_CONFIGURE_RSS:
+ priv->adminq_cfg_rss_cnt++;
+ break;
case GVE_ADMINQ_SET_DRIVER_PARAMETER:
priv->adminq_set_driver_parameter_cnt++;
break;
@@ -938,3 +941,59 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,
gve_free_dma_mem(&ptype_map_dma_mem);
return err;
}
+
+int gve_adminq_configure_rss(struct gve_priv *priv,
+ struct gve_rss_config *rss_config)
+{
+ struct gve_dma_mem indirection_table_dma_mem;
+ struct gve_dma_mem rss_key_dma_mem;
+ union gve_adminq_command cmd;
+ __be32 *indir = NULL;
+ u8 *key = NULL;
+ int err = 0;
+ int i;
+
+ if (rss_config->indir_size) {
+ indir = gve_alloc_dma_mem(&indirection_table_dma_mem,
+ rss_config->indir_size *
+ sizeof(*rss_config->indir));
+ if (!indir) {
+ err = -ENOMEM;
+ goto out;
+ }
+ for (i = 0; i < rss_config->indir_size; i++)
+ indir[i] = cpu_to_be32(rss_config->indir[i]);
+ }
+
+ if (rss_config->key_size) {
+ key = gve_alloc_dma_mem(&rss_key_dma_mem,
+ rss_config->key_size *
+ sizeof(*rss_config->key));
+ if (!key) {
+ err = -ENOMEM;
+ goto out;
+ }
+ memcpy(key, rss_config->key, rss_config->key_size);
+ }
+
+ memset(&cmd, 0, sizeof(cmd));
+ cmd.opcode = cpu_to_be32(GVE_ADMINQ_CONFIGURE_RSS);
+ cmd.configure_rss = (struct gve_adminq_configure_rss) {
+ .hash_types = cpu_to_be16(rss_config->hash_types),
+ .halg = rss_config->alg,
+ .hkey_len = cpu_to_be16(rss_config->key_size),
+ .indir_len = cpu_to_be16(rss_config->indir_size),
+ .hkey_addr = cpu_to_be64(rss_key_dma_mem.pa),
+ .indir_addr = cpu_to_be64(indirection_table_dma_mem.pa),
+ };
+
+ err = gve_adminq_execute_cmd(priv, &cmd);
+
+out:
+ if (indir)
+ gve_free_dma_mem(&indirection_table_dma_mem);
+ if (key)
+ gve_free_dma_mem(&rss_key_dma_mem);
+ return err;
+}
+
diff --git a/drivers/net/gve/base/gve_adminq.h b/drivers/net/gve/base/gve_adminq.h
index f05362f85f..95f4960561 100644
--- a/drivers/net/gve/base/gve_adminq.h
+++ b/drivers/net/gve/base/gve_adminq.h
@@ -19,6 +19,7 @@ enum gve_adminq_opcodes {
GVE_ADMINQ_DESTROY_TX_QUEUE = 0x7,
GVE_ADMINQ_DESTROY_RX_QUEUE = 0x8,
GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES = 0x9,
+ GVE_ADMINQ_CONFIGURE_RSS = 0xA,
GVE_ADMINQ_SET_DRIVER_PARAMETER = 0xB,
GVE_ADMINQ_REPORT_STATS = 0xC,
GVE_ADMINQ_REPORT_LINK_SPEED = 0xD,
@@ -377,6 +378,27 @@ struct gve_adminq_get_ptype_map {
__be64 ptype_map_addr;
};
+#define GVE_RSS_HASH_IPV4 BIT(0)
+#define GVE_RSS_HASH_TCPV4 BIT(1)
+#define GVE_RSS_HASH_IPV6 BIT(2)
+#define GVE_RSS_HASH_IPV6_EX BIT(3)
+#define GVE_RSS_HASH_TCPV6 BIT(4)
+#define GVE_RSS_HASH_TCPV6_EX BIT(5)
+#define GVE_RSS_HASH_UDPV4 BIT(6)
+#define GVE_RSS_HASH_UDPV6 BIT(7)
+#define GVE_RSS_HASH_UDPV6_EX BIT(8)
+
+/* RSS configuration command */
+struct gve_adminq_configure_rss {
+ __be16 hash_types;
+ u8 halg; /* hash algorithm */
+ u8 reserved;
+ __be16 hkey_len;
+ __be16 indir_len;
+ __be64 hkey_addr;
+ __be64 indir_addr;
+};
+
union gve_adminq_command {
struct {
__be32 opcode;
@@ -391,6 +413,7 @@ union gve_adminq_command {
struct gve_adminq_describe_device describe_device;
struct gve_adminq_register_page_list reg_page_list;
struct gve_adminq_unregister_page_list unreg_page_list;
+ struct gve_adminq_configure_rss configure_rss;
struct gve_adminq_set_driver_parameter set_driver_param;
struct gve_adminq_report_stats report_stats;
struct gve_adminq_report_link_speed report_link_speed;
@@ -404,6 +427,8 @@ union gve_adminq_command {
GVE_CHECK_UNION_LEN(64, gve_adminq_command);
+struct gve_priv;
+struct gve_queue_page_list;
int gve_adminq_alloc(struct gve_priv *priv);
void gve_adminq_free(struct gve_priv *priv);
void gve_adminq_release(struct gve_priv *priv);
@@ -433,4 +458,8 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,
int gve_adminq_verify_driver_compatibility(struct gve_priv *priv,
u64 driver_info_len,
dma_addr_t driver_info_addr);
+
+int gve_adminq_configure_rss(struct gve_priv *priv,
+ struct gve_rss_config *rss_config);
+
#endif /* _GVE_ADMINQ_H */
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 14c72ec91a..f7635e829c 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -39,7 +39,10 @@
RTE_ETH_RSS_IPV6 | \
RTE_ETH_RSS_IPV6_EX | \
RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
- RTE_ETH_RSS_IPV6_TCP_EX)
+ RTE_ETH_RSS_IPV6_TCP_EX | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_IPV6_UDP_EX)
/* A list of pages registered with the device during setup and used by a queue
* as buffers
@@ -264,6 +267,7 @@ struct gve_priv {
uint32_t adminq_destroy_tx_queue_cnt;
uint32_t adminq_destroy_rx_queue_cnt;
uint32_t adminq_dcfg_device_resources_cnt;
+ uint32_t adminq_cfg_rss_cnt;
uint32_t adminq_set_driver_parameter_cnt;
uint32_t adminq_report_stats_cnt;
uint32_t adminq_report_link_speed_cnt;
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v2 3/7] net/gve: add gve_rss library for handling RSS-related behaviors
2024-01-23 17:58 [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
2024-01-23 17:58 ` [PATCH v2 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
2024-01-23 17:58 ` [PATCH v2 2/7] net/gve: RSS adminq command changes Joshua Washington
@ 2024-01-23 17:58 ` Joshua Washington
2024-01-23 17:58 ` [PATCH v2 4/7] net/gve: RSS configuration update support Joshua Washington
` (7 subsequent siblings)
10 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-23 17:58 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This change includes a number of helper functions to facilitate RSS
configuration on the GVE DPDK driver. These methods are declared in
gve_rss.h.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/base/gve_adminq.h | 10 +-
drivers/net/gve/gve_ethdev.c | 2 +-
drivers/net/gve/gve_ethdev.h | 4 +-
| 206 ++++++++++++++++++++++++++++++
| 107 ++++++++++++++++
drivers/net/gve/meson.build | 1 +
6 files changed, 319 insertions(+), 11 deletions(-)
create mode 100644 drivers/net/gve/gve_rss.c
create mode 100644 drivers/net/gve/gve_rss.h
diff --git a/drivers/net/gve/base/gve_adminq.h b/drivers/net/gve/base/gve_adminq.h
index 95f4960561..24abd945cc 100644
--- a/drivers/net/gve/base/gve_adminq.h
+++ b/drivers/net/gve/base/gve_adminq.h
@@ -378,15 +378,6 @@ struct gve_adminq_get_ptype_map {
__be64 ptype_map_addr;
};
-#define GVE_RSS_HASH_IPV4 BIT(0)
-#define GVE_RSS_HASH_TCPV4 BIT(1)
-#define GVE_RSS_HASH_IPV6 BIT(2)
-#define GVE_RSS_HASH_IPV6_EX BIT(3)
-#define GVE_RSS_HASH_TCPV6 BIT(4)
-#define GVE_RSS_HASH_TCPV6_EX BIT(5)
-#define GVE_RSS_HASH_UDPV4 BIT(6)
-#define GVE_RSS_HASH_UDPV6 BIT(7)
-#define GVE_RSS_HASH_UDPV6_EX BIT(8)
/* RSS configuration command */
struct gve_adminq_configure_rss {
@@ -428,6 +419,7 @@ union gve_adminq_command {
GVE_CHECK_UNION_LEN(64, gve_adminq_command);
struct gve_priv;
+struct gve_rss_config;
struct gve_queue_page_list;
int gve_adminq_alloc(struct gve_priv *priv);
void gve_adminq_free(struct gve_priv *priv);
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 6acdb4e13b..936ca22cb9 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -442,7 +442,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.nb_align = 1,
};
- dev_info->flow_type_rss_offloads = GVE_RSS_OFFLOAD_ALL;
+ dev_info->flow_type_rss_offloads = GVE_RTE_RSS_OFFLOAD_ALL;
return 0;
}
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index f7635e829c..bc486cb941 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -33,7 +33,7 @@
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
-#define GVE_RSS_OFFLOAD_ALL ( \
+#define GVE_RTE_RSS_OFFLOAD_ALL ( \
RTE_ETH_RSS_IPV4 | \
RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
RTE_ETH_RSS_IPV6 | \
@@ -290,6 +290,8 @@ struct gve_priv {
const struct rte_memzone *stats_report_mem;
uint16_t stats_start_idx; /* start index of array of stats written by NIC */
uint16_t stats_end_idx; /* end index of array of stats written by NIC */
+
+ struct gve_rss_config rss_config;
};
static inline bool
--git a/drivers/net/gve/gve_rss.c b/drivers/net/gve/gve_rss.c
new file mode 100644
index 0000000000..931180f8f2
--- /dev/null
+++ b/drivers/net/gve/gve_rss.c
@@ -0,0 +1,206 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Google LLC
+ */
+
+#include "gve_rss.h"
+
+int
+gve_generate_rss_reta(struct rte_eth_dev *dev, struct gve_rss_config *config)
+{
+ int i;
+ if (!config || !config->indir)
+ return -EINVAL;
+
+ for (i = 0; i < config->indir_size; i++)
+ config->indir[i] = i % dev->data->nb_rx_queues;
+
+ return 0;
+}
+
+
+int
+gve_init_rss_config(struct gve_rss_config *gve_rss_conf,
+ uint16_t key_size, uint16_t indir_size)
+{
+ int err;
+
+ gve_rss_conf->alg = GVE_RSS_HASH_TOEPLITZ;
+
+ gve_rss_conf->key_size = key_size;
+ gve_rss_conf->key = rte_zmalloc("rss key",
+ key_size * sizeof(*gve_rss_conf->key),
+ RTE_CACHE_LINE_SIZE);
+ if (!gve_rss_conf->key)
+ return -ENOMEM;
+
+ gve_rss_conf->indir_size = indir_size;
+ gve_rss_conf->indir = rte_zmalloc("rss reta",
+ indir_size * sizeof(*gve_rss_conf->indir),
+ RTE_CACHE_LINE_SIZE);
+ if (!gve_rss_conf->indir) {
+ err = -ENOMEM;
+ goto err_with_key;
+ }
+
+ return 0;
+err_with_key:
+ rte_free(gve_rss_conf->key);
+ return err;
+}
+
+int
+gve_init_rss_config_from_priv(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf)
+{
+ int err = gve_init_rss_config(gve_rss_conf, priv->rss_config.key_size,
+ priv->rss_config.indir_size);
+ if (err)
+ return err;
+
+ gve_rss_conf->hash_types = priv->rss_config.hash_types;
+ gve_rss_conf->alg = priv->rss_config.alg;
+ memcpy(gve_rss_conf->key, priv->rss_config.key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ memcpy(gve_rss_conf->indir, priv->rss_config.indir,
+ gve_rss_conf->indir_size * sizeof(*gve_rss_conf->indir));
+
+ return 0;
+}
+
+void
+gve_free_rss_config(struct gve_rss_config *gve_rss_conf)
+{
+ rte_free(gve_rss_conf->indir);
+ gve_rss_conf->indir = NULL;
+ rte_free(gve_rss_conf->key);
+ gve_rss_conf->key = NULL;
+}
+
+int
+gve_update_priv_rss_config(struct gve_priv *priv, struct gve_rss_config *config)
+{
+ struct gve_rss_config *priv_config = &priv->rss_config;
+ int key_bytes, indir_bytes;
+
+ if (!config)
+ return -EINVAL;
+ if (config->key_size == 0 || !config->key)
+ return -EINVAL;
+ if (config->indir_size == 0 || !config->indir)
+ return -EINVAL;
+
+ priv_config->hash_types = config->hash_types;
+ priv_config->alg = config->alg;
+
+ priv_config->key_size = config->key_size;
+ key_bytes = priv_config->key_size * sizeof(*priv_config->key);
+ if (!priv_config->key)
+ priv_config->key = rte_zmalloc("priv rss key", key_bytes,
+ RTE_CACHE_LINE_SIZE);
+ else
+ priv_config->key = rte_realloc(priv_config->key, key_bytes,
+ RTE_CACHE_LINE_SIZE);
+ if (!priv_config->key)
+ return -ENOMEM;
+
+ priv_config->indir_size = config->indir_size;
+ indir_bytes = priv_config->indir_size * sizeof(*priv_config->indir);
+ if (!priv_config->indir)
+ priv_config->indir = rte_zmalloc("priv rss reta", indir_bytes,
+ RTE_CACHE_LINE_SIZE);
+ else
+ priv_config->indir = rte_realloc(priv_config->indir,
+ indir_bytes, RTE_CACHE_LINE_SIZE);
+
+ if (!priv_config->indir)
+ return -ENOMEM;
+
+ memcpy(priv_config->indir, config->indir,
+ config->indir_size * sizeof(*priv_config->indir));
+ memcpy(priv_config->key, config->key,
+ config->key_size * sizeof(*priv_config->key));
+
+ return 0;
+}
+
+int
+gve_update_rss_key(struct gve_priv *priv, struct gve_rss_config *gve_rss_conf,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ if (rss_conf->rss_key_len && rss_conf->rss_key) {
+ gve_rss_conf->key_size = rss_conf->rss_key_len;
+ memcpy(gve_rss_conf->key, rss_conf->rss_key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ } else if (priv->rss_config.key_size && priv->rss_config.key) {
+ gve_rss_conf->key_size = priv->rss_config.key_size;
+ memcpy(gve_rss_conf->key, priv->rss_config.key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ } else {
+ PMD_DRV_LOG(ERR, "RSS key must be set as part of initial RSS "
+ "configuration.");
+ return -EINVAL;
+ }
+ return 0;
+}
+
+int
+gve_update_rss_hash_types(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf, struct rte_eth_rss_conf *rss_conf)
+{
+ /* Initialize to 0 before modifying. */
+ gve_rss_conf->hash_types = 0;
+ if (rss_conf->rss_hf)
+ rte_to_gve_rss_hf(rss_conf->rss_hf, gve_rss_conf);
+ else if (priv->rss_config.key_size && priv->rss_config.key)
+ gve_rss_conf->hash_types = priv->rss_config.hash_types;
+ else
+ gve_rss_conf->hash_types = GVE_RSS_OFFLOAD_DEFAULT;
+ return 0;
+}
+
+void
+rte_to_gve_rss_hf(uint64_t rte_rss_hf, struct gve_rss_config *gve_rss_conf)
+{
+ if (rte_rss_hf & RTE_ETH_RSS_IPV4)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_TCP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV6_EX;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_TCP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_TCP_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV6_EX;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_UDP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_UDP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_UDP_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV6_EX;
+}
+
+void
+gve_to_rte_rss_hf(uint16_t gve_rss_types, struct rte_eth_rss_conf *rss_conf)
+{
+ if (gve_rss_types & GVE_RSS_HASH_IPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV4;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV4_TCP;
+ if (gve_rss_types & GVE_RSS_HASH_IPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6;
+ if (gve_rss_types & GVE_RSS_HASH_IPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_EX;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV6_TCP;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_TCP_EX;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV4_UDP;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV6_UDP;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_UDP_EX;
+}
+
--git a/drivers/net/gve/gve_rss.h b/drivers/net/gve/gve_rss.h
new file mode 100644
index 0000000000..0cc235a19a
--- /dev/null
+++ b/drivers/net/gve/gve_rss.h
@@ -0,0 +1,107 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Google LLC
+ */
+
+#include "gve_adminq.h"
+#include "gve_ethdev.h"
+
+#define GVE_RSS_HASH_IPV4 BIT(0)
+#define GVE_RSS_HASH_TCPV4 BIT(1)
+#define GVE_RSS_HASH_IPV6 BIT(2)
+#define GVE_RSS_HASH_IPV6_EX BIT(3)
+#define GVE_RSS_HASH_TCPV6 BIT(4)
+#define GVE_RSS_HASH_TCPV6_EX BIT(5)
+#define GVE_RSS_HASH_UDPV4 BIT(6)
+#define GVE_RSS_HASH_UDPV6 BIT(7)
+#define GVE_RSS_HASH_UDPV6_EX BIT(8)
+
+#define GVE_RSS_OFFLOAD_DEFAULT ( \
+ GVE_RSS_HASH_IPV4 | \
+ GVE_RSS_HASH_TCPV4 | \
+ GVE_RSS_HASH_IPV6 | \
+ GVE_RSS_HASH_IPV6_EX | \
+ GVE_RSS_HASH_TCPV6 | \
+ GVE_RSS_HASH_TCPV6_EX | \
+ GVE_RSS_HASH_UDPV4 | \
+ GVE_RSS_HASH_UDPV6 | \
+ GVE_RSS_HASH_UDPV6_EX)
+
+/**
+ * Generates default RSS redirection table based on the number of queues the
+ * device is configured with. This assigns hash values to queues in a
+ * round-robin manner.
+ */
+int
+gve_generate_rss_reta(struct rte_eth_dev *dev, struct gve_rss_config *config);
+
+/**
+ * Initializes `gve_rss_conf`, setting the fields to default values and
+ * allocating memory for the RSS key and redirection table.
+ */
+int
+gve_init_rss_config(struct gve_rss_config *gve_rss_conf,
+ uint16_t key_size, uint16_t indir_size);
+
+/**
+ * Initializes `gve_rss_conf` based on the RSS configuration stored in `priv`.
+ */
+int
+gve_init_rss_config_from_priv(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Frees RSS key and redriection table pointers stored in `gve_rss_conf`.
+ */
+void
+gve_free_rss_config(struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Updates the rss_config stored in `priv` with the contents of `config`.
+ */
+int
+gve_update_priv_rss_config(struct gve_priv *priv,
+ struct gve_rss_config *config);
+
+/**
+ * Updates the RSS key stored in `gve_rss_conf`. It is prioritized as follows:
+ * 1) retrieve from `rss_conf`, if non-null
+ * 2) retrieve from `priv`, if non-null
+ * If keys from both sources are unset, return -EINVAL.
+ */
+int
+gve_update_rss_key(struct gve_priv *priv, struct gve_rss_config *gve_rss_conf,
+ struct rte_eth_rss_conf *rss_conf);
+
+/**
+ * Updates the RSS hash types stored in `gve_rss_conf`. It is prioritized as
+ * follows:
+ * 1) retrieve from `rss_conf`, if set
+ * 2) retrieve from priv, if RSS has been configured
+ * 3) set default RSS offload
+ */
+int
+gve_update_rss_hash_types(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf, struct rte_eth_rss_conf *rss_conf);
+
+/**
+ * Ensures that only supported RSS hash fields are set in `rte_rss_hf`.
+ */
+static inline int
+gve_validate_rss_hf(uint64_t rte_rss_hf) {
+ return rte_rss_hf & ~GVE_RTE_RSS_OFFLOAD_ALL;
+}
+
+/**
+ * Converts RSS hash types from RTE values to GVE values, storing them in
+ * `gve_rss_conf`.
+ */
+void
+rte_to_gve_rss_hf(uint64_t rte_rss_hf, struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Converts RSS hash types from GVE values to RTE values, storing them in
+ * `rss_conf`.
+ */
+void
+gve_to_rte_rss_hf(uint16_t gve_rss_types, struct rte_eth_rss_conf *rss_conf);
+
diff --git a/drivers/net/gve/meson.build b/drivers/net/gve/meson.build
index 61d195009c..6da13a8406 100644
--- a/drivers/net/gve/meson.build
+++ b/drivers/net/gve/meson.build
@@ -15,5 +15,6 @@ sources = files(
'gve_tx_dqo.c',
'gve_ethdev.c',
'gve_version.c',
+ 'gve_rss.c',
)
includes += include_directories('base')
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v2 4/7] net/gve: RSS configuration update support
2024-01-23 17:58 [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (2 preceding siblings ...)
2024-01-23 17:58 ` [PATCH v2 3/7] net/gve: add gve_rss library for handling RSS-related behaviors Joshua Washington
@ 2024-01-23 17:58 ` Joshua Washington
2024-01-23 17:58 ` [PATCH v2 5/7] net/gve: RSS redirection table " Joshua Washington
` (6 subsequent siblings)
10 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-23 17:58 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch adds support for updating the RSS hash key and hash fields
in the GVE PMD through the implementation of rss_hash_update and
rss_hash_conf_get.
The RSS hash key for gVNIC is required to be 40 bytes. On initial
configuration of the RSS hash key, the RSS redirection table will be
set to a static default, using a round-robin approach for all queues.
Note, however, that this patch does not include support for setting the
redirection table explicitly. In dev_configure, if the static
redirection table has been set, it will be updated to reflect the new
queue count, if it has changed.
The RSS key must be set before any other RSS configuration can happen.
As such, an attempt to set the hash types before the key is configured
will fail.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/gve_ethdev.c | 132 ++++++++++++++++++++++++++++++++++-
drivers/net/gve/gve_ethdev.h | 3 +
2 files changed, 133 insertions(+), 2 deletions(-)
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 936ca22cb9..2a68d31808 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -1,5 +1,6 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(C) 2022 Intel Corporation
+ * Copyright(C) 2022-2023 Intel Corporation
+ * Copyright(C) 2023 Google LLC
*/
#include "gve_ethdev.h"
@@ -8,6 +9,7 @@
#include "base/gve_osdep.h"
#include "gve_version.h"
#include "rte_ether.h"
+#include "gve_rss.h"
static void
gve_write_version(uint8_t *driver_version_register)
@@ -88,12 +90,31 @@ gve_dev_configure(struct rte_eth_dev *dev)
{
struct gve_priv *priv = dev->data->dev_private;
- if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) {
dev->data->dev_conf.rxmode.offloads |= RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ priv->rss_config.alg = GVE_RSS_HASH_TOEPLITZ;
+ }
if (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO)
priv->enable_rsc = 1;
+ /* Reset RSS RETA in case number of queues changed. */
+ if (priv->rss_config.indir) {
+ struct gve_rss_config update_reta_config;
+ gve_init_rss_config_from_priv(priv, &update_reta_config);
+ gve_generate_rss_reta(dev, &update_reta_config);
+
+ int err = gve_adminq_configure_rss(priv, &update_reta_config);
+ if (err)
+ PMD_DRV_LOG(ERR,
+ "Could not reconfigure RSS redirection table.");
+ else
+ gve_update_priv_rss_config(priv, &update_reta_config);
+
+ gve_free_rss_config(&update_reta_config);
+ return err;
+ }
+
return 0;
}
@@ -443,6 +464,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
};
dev_info->flow_type_rss_offloads = GVE_RTE_RSS_OFFLOAD_ALL;
+ dev_info->hash_key_size = GVE_RSS_HASH_KEY_SIZE;
+ dev_info->reta_size = GVE_RSS_INDIR_SIZE;
return 0;
}
@@ -646,6 +669,107 @@ gve_xstats_get_names(struct rte_eth_dev *dev,
return count;
}
+
+static int
+gve_rss_hash_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ struct gve_rss_config gve_rss_conf;
+ int rss_reta_size;
+ int err;
+
+ if (gve_validate_rss_hf(rss_conf->rss_hf)) {
+ PMD_DRV_LOG(ERR, "Unsupported hash function.");
+ return -EINVAL;
+ }
+
+ if (rss_conf->algorithm != RTE_ETH_HASH_FUNCTION_TOEPLITZ &&
+ rss_conf->algorithm != RTE_ETH_HASH_FUNCTION_DEFAULT) {
+ PMD_DRV_LOG(ERR, "Device only supports Toeplitz algorithm.");
+ return -EINVAL;
+ }
+
+ if (rss_conf->rss_key_len) {
+ if (rss_conf->rss_key_len != GVE_RSS_HASH_KEY_SIZE) {
+ PMD_DRV_LOG(ERR,
+ "Invalid hash key size. Only RSS hash key size "
+ "of %u supported", GVE_RSS_HASH_KEY_SIZE);
+ return -EINVAL;
+ }
+
+ if (!rss_conf->rss_key) {
+ PMD_DRV_LOG(ERR, "RSS key must be non-null.");
+ return -EINVAL;
+ }
+ } else {
+ if (!priv->rss_config.key_size) {
+ PMD_DRV_LOG(ERR, "RSS key must be initialized before "
+ "any other configuration.");
+ return -EINVAL;
+ }
+ rss_conf->rss_key_len = priv->rss_config.key_size;
+ }
+
+ rss_reta_size = priv->rss_config.indir ?
+ priv->rss_config.indir_size :
+ GVE_RSS_INDIR_SIZE;
+ err = gve_init_rss_config(&gve_rss_conf, rss_conf->rss_key_len,
+ rss_reta_size);
+ if (err)
+ return err;
+
+ gve_rss_conf.alg = GVE_RSS_HASH_TOEPLITZ;
+ err = gve_update_rss_hash_types(priv, &gve_rss_conf, rss_conf);
+ if (err)
+ goto err;
+ err = gve_update_rss_key(priv, &gve_rss_conf, rss_conf);
+ if (err)
+ goto err;
+
+ /* Set redirection table to default or preexisting. */
+ if (!priv->rss_config.indir)
+ gve_generate_rss_reta(dev, &gve_rss_conf);
+ else
+ memcpy(gve_rss_conf.indir, priv->rss_config.indir,
+ gve_rss_conf.indir_size * sizeof(*priv->rss_config.indir));
+
+ err = gve_adminq_configure_rss(priv, &gve_rss_conf);
+ if (!err)
+ gve_update_priv_rss_config(priv, &gve_rss_conf);
+
+err:
+ gve_free_rss_config(&gve_rss_conf);
+ return err;
+}
+
+static int
+gve_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+
+ if (!(dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)) {
+ PMD_DRV_LOG(ERR, "RSS not configured.");
+ return -ENOTSUP;
+ }
+
+
+ gve_to_rte_rss_hf(priv->rss_config.hash_types, rss_conf);
+ rss_conf->rss_key_len = priv->rss_config.key_size;
+ if (rss_conf->rss_key) {
+ if (!priv->rss_config.key) {
+ PMD_DRV_LOG(ERR, "Unable to retrieve default RSS hash key.");
+ return -ENOTSUP;
+ }
+ memcpy(rss_conf->rss_key, priv->rss_config.key,
+ rss_conf->rss_key_len * sizeof(*rss_conf->rss_key));
+ }
+
+ return 0;
+}
+
static const struct eth_dev_ops gve_eth_dev_ops = {
.dev_configure = gve_dev_configure,
.dev_start = gve_dev_start,
@@ -666,6 +790,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = {
.mtu_set = gve_dev_mtu_set,
.xstats_get = gve_xstats_get,
.xstats_get_names = gve_xstats_get_names,
+ .rss_hash_update = gve_rss_hash_update,
+ .rss_hash_conf_get = gve_rss_hash_conf_get,
};
static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -688,6 +814,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
.mtu_set = gve_dev_mtu_set,
.xstats_get = gve_xstats_get,
.xstats_get_names = gve_xstats_get_names,
+ .rss_hash_update = gve_rss_hash_update,
+ .rss_hash_conf_get = gve_rss_hash_conf_get,
};
static void
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index bc486cb941..d713657d10 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -29,6 +29,9 @@
#define GVE_RX_MIN_BUF_SIZE_GQI 2048
#define GVE_RX_MAX_BUF_SIZE_GQI 4096
+#define GVE_RSS_HASH_KEY_SIZE 40
+#define GVE_RSS_INDIR_SIZE 128
+
#define GVE_TX_CKSUM_OFFLOAD_MASK ( \
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v2 5/7] net/gve: RSS redirection table update support
2024-01-23 17:58 [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (3 preceding siblings ...)
2024-01-23 17:58 ` [PATCH v2 4/7] net/gve: RSS configuration update support Joshua Washington
@ 2024-01-23 17:58 ` Joshua Washington
2024-01-23 17:58 ` [PATCH v2 6/7] net/gve: update gve.ini with RSS capabilities Joshua Washington
` (5 subsequent siblings)
10 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-23 17:58 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch introduces support for updating the RSS redirection table in
the GVE PMD through the implementation of rss_reta_update and
rss_reta_query.
Due to an infrastructure limitation, the RSS hash key must be manually
configured before the redirection table can be updated or queried. The
redirection table is expected to be exactly 128 bytes.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/gve_ethdev.c | 95 ++++++++++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 2a68d31808..9c39ce286a 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -770,6 +770,97 @@ gve_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
}
+static int
+gve_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ struct gve_rss_config gve_rss_conf;
+ int table_id;
+ int err;
+ int i;
+
+ /* RSS key must be set before the redirection table can be set. */
+ if (!priv->rss_config.key || priv->rss_config.key_size == 0) {
+ PMD_DRV_LOG(ERR, "RSS hash key msut be set before the "
+ "redirection table can be updated.");
+ return -ENOTSUP;
+ }
+
+ if (reta_size != GVE_RSS_INDIR_SIZE) {
+ PMD_DRV_LOG(ERR, "Redirection table must have %hu elements",
+ GVE_RSS_INDIR_SIZE);
+ return -EINVAL;
+ }
+
+ err = gve_init_rss_config_from_priv(priv, &gve_rss_conf);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Error allocating new RSS config.");
+ return err;
+ }
+
+ table_id = 0;
+ for (i = 0; i < priv->rss_config.indir_size; i++) {
+ int table_entry = i % RTE_ETH_RETA_GROUP_SIZE;
+ if (reta_conf[table_id].mask & (1ULL << table_entry))
+ gve_rss_conf.indir[i] =
+ reta_conf[table_id].reta[table_entry];
+
+ if (table_entry == RTE_ETH_RETA_GROUP_SIZE - 1)
+ table_id++;
+ }
+
+ err = gve_adminq_configure_rss(priv, &gve_rss_conf);
+ if (err)
+ PMD_DRV_LOG(ERR, "Problem configuring RSS with device.");
+ else
+ gve_update_priv_rss_config(priv, &gve_rss_conf);
+
+ gve_free_rss_config(&gve_rss_conf);
+ return err;
+}
+
+static int
+gve_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ int table_id;
+ int i;
+
+ if (!(dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)) {
+ PMD_DRV_LOG(ERR, "RSS not configured.");
+ return -ENOTSUP;
+ }
+
+ /* RSS key must be set before the redirection table can be queried. */
+ if (!priv->rss_config.key) {
+ PMD_DRV_LOG(ERR, "RSS hash key must be set before the "
+ "redirection table can be initialized.");
+ return -ENOTSUP;
+ }
+
+ if (reta_size != priv->rss_config.indir_size) {
+ PMD_DRV_LOG(ERR, "RSS redirection table must have %d entries.",
+ priv->rss_config.indir_size);
+ return -EINVAL;
+ }
+
+ table_id = 0;
+ for (i = 0; i < priv->rss_config.indir_size; i++) {
+ int table_entry = i % RTE_ETH_RETA_GROUP_SIZE;
+ if (reta_conf[table_id].mask & (1ULL << table_entry))
+ reta_conf[table_id].reta[table_entry] =
+ priv->rss_config.indir[i];
+
+ if (table_entry == RTE_ETH_RETA_GROUP_SIZE - 1)
+ table_id++;
+ }
+
+ return 0;
+}
+
static const struct eth_dev_ops gve_eth_dev_ops = {
.dev_configure = gve_dev_configure,
.dev_start = gve_dev_start,
@@ -792,6 +883,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = {
.xstats_get_names = gve_xstats_get_names,
.rss_hash_update = gve_rss_hash_update,
.rss_hash_conf_get = gve_rss_hash_conf_get,
+ .reta_update = gve_rss_reta_update,
+ .reta_query = gve_rss_reta_query,
};
static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -816,6 +909,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
.xstats_get_names = gve_xstats_get_names,
.rss_hash_update = gve_rss_hash_update,
.rss_hash_conf_get = gve_rss_hash_conf_get,
+ .reta_update = gve_rss_reta_update,
+ .reta_query = gve_rss_reta_query,
};
static void
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v2 6/7] net/gve: update gve.ini with RSS capabilities
2024-01-23 17:58 [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (4 preceding siblings ...)
2024-01-23 17:58 ` [PATCH v2 5/7] net/gve: RSS redirection table " Joshua Washington
@ 2024-01-23 17:58 ` Joshua Washington
2024-01-23 17:58 ` [PATCH v2 7/7] net/gve: update GVE documentation with RSS support Joshua Washington
` (4 subsequent siblings)
10 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-23 17:58 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch updates the DPDK feature matrix to expose that the GVE driver
supports RSS hash, RSS key update, and RSS reta update.
Signed-off-by: Joshua Washington <joshwash@google.com>
---
doc/guides/nics/features/gve.ini | 3 +++
1 file changed, 3 insertions(+)
diff --git a/doc/guides/nics/features/gve.ini b/doc/guides/nics/features/gve.ini
index 838edd456a..bc08648dbc 100644
--- a/doc/guides/nics/features/gve.ini
+++ b/doc/guides/nics/features/gve.ini
@@ -15,3 +15,6 @@ Linux = Y
x86-32 = Y
x86-64 = Y
Usage doc = Y
+RSS hash = Y
+RSS key update = Y
+RSS reta update = Y
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v2 7/7] net/gve: update GVE documentation with RSS support
2024-01-23 17:58 [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (5 preceding siblings ...)
2024-01-23 17:58 ` [PATCH v2 6/7] net/gve: update gve.ini with RSS capabilities Joshua Washington
@ 2024-01-23 17:58 ` Joshua Washington
2024-01-24 0:04 ` [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (3 subsequent siblings)
10 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-23 17:58 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch updates the GVE doc page to communicate that GVE now supports
RSS configuration and explains the limitations.
Signed-off-by: Joshua Washington <joshwash@google.com>
---
doc/guides/nics/gve.rst | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/doc/guides/nics/gve.rst b/doc/guides/nics/gve.rst
index 1c3eaf03ef..80991e70cf 100644
--- a/doc/guides/nics/gve.rst
+++ b/doc/guides/nics/gve.rst
@@ -70,6 +70,8 @@ Supported features of the GVE PMD are:
- Link state information
- Tx multi-segments (Scatter Tx)
- Tx UDP/TCP/SCTP Checksum
+- RSS hash configuration
+- RSS redirection table update and query
Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD.
Jumbo Frame is not supported in PMD for now.
@@ -77,10 +79,12 @@ It'll be added in a future DPDK release.
Also, only GQI_QPL queue format is in use on GCP
since GQI_RDA hasn't been released in production.
-Currently, setting MTU with value larger than 1460 is not supported.
+RSS
+^^^
-Currently, only "RSS hash" is force enabled
-so that the backend hardware device calculated hash values
-could be shared with applications.
-But for RSS, there is no such API to config RSS hash function or RETA table.
-So, limited RSS is supported only with default config/setting.
+GVE RSS can be enabled and configured using the standard interfaces. The driver
+does not support querying the initial RSS configuration.
+
+The RSS hash key must be exactly 40 bytes, and the redirection table must have
+128 entries. The RSS hash key must be configured before the redirection table
+can be set up.
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH 0/7] net/gve: RSS Support for GVE Driver
2024-01-23 17:58 [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (6 preceding siblings ...)
2024-01-23 17:58 ` [PATCH v2 7/7] net/gve: update GVE documentation with RSS support Joshua Washington
@ 2024-01-24 0:04 ` Joshua Washington
2024-01-24 0:04 ` [PATCH v3 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
` (6 more replies)
2024-01-24 0:09 ` [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (2 subsequent siblings)
10 siblings, 7 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:04 UTC (permalink / raw)
Cc: dev, Ferruh Yigit, Rushil Gupta, Joshua Washington
This patch series introduces RSS support for the GVE poll-mode driver.
This series includes implementations of the following eth_dev_ops:
1) rss_hash_update
2) rss_hash_conf_get
3) reta_query
4) reta_update
In rss_hash_update, the GVE driver supports the following RSS hash
types:
* RTE_ETH_RSS_IPV4
* RTE_ETH_RSS_NONFRAG_IPV4_TCP
* RTE_ETH_RSS_NONFRAG_IPV4_UDP
* RTE_ETH_RSS_IPV6
* RTE_ETH_RSS_IPV6_EX
* RTE_ETH_RSS_NONFRAG_IPV6_TCP
* RTE_ETH_RSS_NONFRAG_IPV6_UDP
* RTE_ETH_RSS_IPV6_TCP_EX
* RTE_ETH_RSS_IPV6_UDP_EX
The hash key is 40B, and the lookup table has 128 entries. These values
are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set
with a key before the redriection table is set up. When the RSS hash is
configured, a default redirection table is generated based on the number
of queues. When the device is re-configured, the redirection table is
reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for
RSS redirection table entries, while the RTE API uses 16 bit integers.
However, this is unlikely to be an issue, as these values represent
receive queues, and the gVNIC device does not support anywhere near 64K
queues.
This series also updates the corresponding feature matrix ertries and
documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches
fixes.
Note: There is a warning about complex macros being parenthesized that
does not seem to be well-founded.
v3:
Fix build warnings that come up on certain distros.
Joshua Washington (7):
net/gve: fully expose RSS offload support in dev_info
net/gve: RSS adminq command changes
net/gve: add gve_rss library for handling RSS-related behaviors
net/gve: RSS configuration update support
net/gve: RSS redirection table update support
net/gve: update gve.ini with RSS capabilities
net/gve: update GVE documentation with RSS support
doc/guides/nics/features/gve.ini | 3 +
doc/guides/nics/gve.rst | 16 ++-
drivers/net/gve/base/gve.h | 15 ++
drivers/net/gve/base/gve_adminq.c | 59 ++++++++
drivers/net/gve/base/gve_adminq.h | 21 +++
drivers/net/gve/gve_ethdev.c | 231 +++++++++++++++++++++++++++++-
drivers/net/gve/gve_ethdev.h | 17 +++
drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++
drivers/net/gve/gve_rss.h | 107 ++++++++++++++
drivers/net/gve/meson.build | 1 +
10 files changed, 667 insertions(+), 9 deletions(-)
create mode 100644 drivers/net/gve/gve_rss.c
create mode 100644 drivers/net/gve/gve_rss.h
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply [flat|nested] 44+ messages in thread
* [PATCH v3 1/7] net/gve: fully expose RSS offload support in dev_info
2024-01-24 0:04 ` [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
@ 2024-01-24 0:04 ` Joshua Washington
2024-01-24 0:04 ` [PATCH v3 2/7] net/gve: RSS adminq command changes Joshua Washington
` (5 subsequent siblings)
6 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:04 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch communicates that the GVE driver supports RSS, along with
the RSS offloads supported by the driver.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/gve_ethdev.c | 4 +++-
drivers/net/gve/gve_ethdev.h | 8 ++++++++
2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index d162fd3864..6acdb4e13b 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -405,7 +405,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = priv->max_mtu;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
- dev_info->rx_offload_capa = 0;
+ dev_info->rx_offload_capa = RTE_ETH_RX_OFFLOAD_RSS_HASH;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
@@ -442,6 +442,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.nb_align = 1,
};
+ dev_info->flow_type_rss_offloads = GVE_RSS_OFFLOAD_ALL;
+
return 0;
}
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 9893fcfee6..14c72ec91a 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -33,6 +33,14 @@
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
+#define GVE_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_IPV6_EX | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_IPV6_TCP_EX)
+
/* A list of pages registered with the device during setup and used by a queue
* as buffers
*/
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v3 2/7] net/gve: RSS adminq command changes
2024-01-24 0:04 ` [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
2024-01-24 0:04 ` [PATCH v3 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
@ 2024-01-24 0:04 ` Joshua Washington
2024-01-24 0:04 ` [PATCH v3 3/7] net/gve: add gve_rss library for handling RSS-related behaviors Joshua Washington
` (4 subsequent siblings)
6 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:04 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This change introduces admin queue changes that enable the configuration
of RSS parameters for the GVE driver.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/base/gve.h | 15 ++++++++
drivers/net/gve/base/gve_adminq.c | 58 +++++++++++++++++++++++++++++++
drivers/net/gve/base/gve_adminq.h | 29 ++++++++++++++++
drivers/net/gve/gve_ethdev.h | 6 +++-
4 files changed, 107 insertions(+), 1 deletion(-)
diff --git a/drivers/net/gve/base/gve.h b/drivers/net/gve/base/gve.h
index f7b297e759..9c58fc4238 100644
--- a/drivers/net/gve/base/gve.h
+++ b/drivers/net/gve/base/gve.h
@@ -51,4 +51,19 @@ enum gve_state_flags_bit {
GVE_PRIV_FLAGS_NAPI_ENABLED = 4,
};
+enum gve_rss_hash_algorithm {
+ GVE_RSS_HASH_UNDEFINED = 0,
+ GVE_RSS_HASH_TOEPLITZ = 1,
+};
+
+struct gve_rss_config {
+ uint16_t hash_types;
+ enum gve_rss_hash_algorithm alg;
+ uint16_t key_size;
+ uint16_t indir_size;
+ uint8_t *key;
+ uint32_t *indir;
+};
+
+
#endif /* _GVE_H_ */
diff --git a/drivers/net/gve/base/gve_adminq.c b/drivers/net/gve/base/gve_adminq.c
index 343bd13d67..fede0a4b82 100644
--- a/drivers/net/gve/base/gve_adminq.c
+++ b/drivers/net/gve/base/gve_adminq.c
@@ -389,6 +389,9 @@ static int gve_adminq_issue_cmd(struct gve_priv *priv,
case GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES:
priv->adminq_dcfg_device_resources_cnt++;
break;
+ case GVE_ADMINQ_CONFIGURE_RSS:
+ priv->adminq_cfg_rss_cnt++;
+ break;
case GVE_ADMINQ_SET_DRIVER_PARAMETER:
priv->adminq_set_driver_parameter_cnt++;
break;
@@ -938,3 +941,58 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,
gve_free_dma_mem(&ptype_map_dma_mem);
return err;
}
+
+int gve_adminq_configure_rss(struct gve_priv *priv,
+ struct gve_rss_config *rss_config)
+{
+ struct gve_dma_mem indirection_table_dma_mem;
+ struct gve_dma_mem rss_key_dma_mem;
+ union gve_adminq_command cmd;
+ __be32 *indir = NULL;
+ u8 *key = NULL;
+ int err = 0;
+ int i;
+
+ if (!rss_config->indir_size || !rss_config->key_size)
+ return -EINVAL;
+
+ indir = gve_alloc_dma_mem(&indirection_table_dma_mem,
+ rss_config->indir_size *
+ sizeof(*rss_config->indir));
+ if (!indir) {
+ err = -ENOMEM;
+ goto out;
+ }
+ for (i = 0; i < rss_config->indir_size; i++)
+ indir[i] = cpu_to_be32(rss_config->indir[i]);
+
+ key = gve_alloc_dma_mem(&rss_key_dma_mem,
+ rss_config->key_size *
+ sizeof(*rss_config->key));
+ if (!key) {
+ err = -ENOMEM;
+ goto out;
+ }
+ memcpy(key, rss_config->key, rss_config->key_size);
+
+ memset(&cmd, 0, sizeof(cmd));
+ cmd.opcode = cpu_to_be32(GVE_ADMINQ_CONFIGURE_RSS);
+ cmd.configure_rss = (struct gve_adminq_configure_rss) {
+ .hash_types = cpu_to_be16(rss_config->hash_types),
+ .halg = rss_config->alg,
+ .hkey_len = cpu_to_be16(rss_config->key_size),
+ .indir_len = cpu_to_be16(rss_config->indir_size),
+ .hkey_addr = cpu_to_be64(rss_key_dma_mem.pa),
+ .indir_addr = cpu_to_be64(indirection_table_dma_mem.pa),
+ };
+
+ err = gve_adminq_execute_cmd(priv, &cmd);
+
+out:
+ if (indir)
+ gve_free_dma_mem(&indirection_table_dma_mem);
+ if (key)
+ gve_free_dma_mem(&rss_key_dma_mem);
+ return err;
+}
+
diff --git a/drivers/net/gve/base/gve_adminq.h b/drivers/net/gve/base/gve_adminq.h
index f05362f85f..95f4960561 100644
--- a/drivers/net/gve/base/gve_adminq.h
+++ b/drivers/net/gve/base/gve_adminq.h
@@ -19,6 +19,7 @@ enum gve_adminq_opcodes {
GVE_ADMINQ_DESTROY_TX_QUEUE = 0x7,
GVE_ADMINQ_DESTROY_RX_QUEUE = 0x8,
GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES = 0x9,
+ GVE_ADMINQ_CONFIGURE_RSS = 0xA,
GVE_ADMINQ_SET_DRIVER_PARAMETER = 0xB,
GVE_ADMINQ_REPORT_STATS = 0xC,
GVE_ADMINQ_REPORT_LINK_SPEED = 0xD,
@@ -377,6 +378,27 @@ struct gve_adminq_get_ptype_map {
__be64 ptype_map_addr;
};
+#define GVE_RSS_HASH_IPV4 BIT(0)
+#define GVE_RSS_HASH_TCPV4 BIT(1)
+#define GVE_RSS_HASH_IPV6 BIT(2)
+#define GVE_RSS_HASH_IPV6_EX BIT(3)
+#define GVE_RSS_HASH_TCPV6 BIT(4)
+#define GVE_RSS_HASH_TCPV6_EX BIT(5)
+#define GVE_RSS_HASH_UDPV4 BIT(6)
+#define GVE_RSS_HASH_UDPV6 BIT(7)
+#define GVE_RSS_HASH_UDPV6_EX BIT(8)
+
+/* RSS configuration command */
+struct gve_adminq_configure_rss {
+ __be16 hash_types;
+ u8 halg; /* hash algorithm */
+ u8 reserved;
+ __be16 hkey_len;
+ __be16 indir_len;
+ __be64 hkey_addr;
+ __be64 indir_addr;
+};
+
union gve_adminq_command {
struct {
__be32 opcode;
@@ -391,6 +413,7 @@ union gve_adminq_command {
struct gve_adminq_describe_device describe_device;
struct gve_adminq_register_page_list reg_page_list;
struct gve_adminq_unregister_page_list unreg_page_list;
+ struct gve_adminq_configure_rss configure_rss;
struct gve_adminq_set_driver_parameter set_driver_param;
struct gve_adminq_report_stats report_stats;
struct gve_adminq_report_link_speed report_link_speed;
@@ -404,6 +427,8 @@ union gve_adminq_command {
GVE_CHECK_UNION_LEN(64, gve_adminq_command);
+struct gve_priv;
+struct gve_queue_page_list;
int gve_adminq_alloc(struct gve_priv *priv);
void gve_adminq_free(struct gve_priv *priv);
void gve_adminq_release(struct gve_priv *priv);
@@ -433,4 +458,8 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,
int gve_adminq_verify_driver_compatibility(struct gve_priv *priv,
u64 driver_info_len,
dma_addr_t driver_info_addr);
+
+int gve_adminq_configure_rss(struct gve_priv *priv,
+ struct gve_rss_config *rss_config);
+
#endif /* _GVE_ADMINQ_H */
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 14c72ec91a..f7635e829c 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -39,7 +39,10 @@
RTE_ETH_RSS_IPV6 | \
RTE_ETH_RSS_IPV6_EX | \
RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
- RTE_ETH_RSS_IPV6_TCP_EX)
+ RTE_ETH_RSS_IPV6_TCP_EX | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_IPV6_UDP_EX)
/* A list of pages registered with the device during setup and used by a queue
* as buffers
@@ -264,6 +267,7 @@ struct gve_priv {
uint32_t adminq_destroy_tx_queue_cnt;
uint32_t adminq_destroy_rx_queue_cnt;
uint32_t adminq_dcfg_device_resources_cnt;
+ uint32_t adminq_cfg_rss_cnt;
uint32_t adminq_set_driver_parameter_cnt;
uint32_t adminq_report_stats_cnt;
uint32_t adminq_report_link_speed_cnt;
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v3 3/7] net/gve: add gve_rss library for handling RSS-related behaviors
2024-01-24 0:04 ` [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
2024-01-24 0:04 ` [PATCH v3 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
2024-01-24 0:04 ` [PATCH v3 2/7] net/gve: RSS adminq command changes Joshua Washington
@ 2024-01-24 0:04 ` Joshua Washington
2024-01-24 0:04 ` [PATCH v3 4/7] net/gve: RSS configuration update support Joshua Washington
` (3 subsequent siblings)
6 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:04 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This change includes a number of helper functions to facilitate RSS
configuration on the GVE DPDK driver. These methods are declared in
gve_rss.h.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/base/gve_adminq.h | 10 +-
drivers/net/gve/gve_ethdev.c | 2 +-
drivers/net/gve/gve_ethdev.h | 4 +-
| 206 ++++++++++++++++++++++++++++++
| 107 ++++++++++++++++
drivers/net/gve/meson.build | 1 +
6 files changed, 319 insertions(+), 11 deletions(-)
create mode 100644 drivers/net/gve/gve_rss.c
create mode 100644 drivers/net/gve/gve_rss.h
diff --git a/drivers/net/gve/base/gve_adminq.h b/drivers/net/gve/base/gve_adminq.h
index 95f4960561..24abd945cc 100644
--- a/drivers/net/gve/base/gve_adminq.h
+++ b/drivers/net/gve/base/gve_adminq.h
@@ -378,15 +378,6 @@ struct gve_adminq_get_ptype_map {
__be64 ptype_map_addr;
};
-#define GVE_RSS_HASH_IPV4 BIT(0)
-#define GVE_RSS_HASH_TCPV4 BIT(1)
-#define GVE_RSS_HASH_IPV6 BIT(2)
-#define GVE_RSS_HASH_IPV6_EX BIT(3)
-#define GVE_RSS_HASH_TCPV6 BIT(4)
-#define GVE_RSS_HASH_TCPV6_EX BIT(5)
-#define GVE_RSS_HASH_UDPV4 BIT(6)
-#define GVE_RSS_HASH_UDPV6 BIT(7)
-#define GVE_RSS_HASH_UDPV6_EX BIT(8)
/* RSS configuration command */
struct gve_adminq_configure_rss {
@@ -428,6 +419,7 @@ union gve_adminq_command {
GVE_CHECK_UNION_LEN(64, gve_adminq_command);
struct gve_priv;
+struct gve_rss_config;
struct gve_queue_page_list;
int gve_adminq_alloc(struct gve_priv *priv);
void gve_adminq_free(struct gve_priv *priv);
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 6acdb4e13b..936ca22cb9 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -442,7 +442,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.nb_align = 1,
};
- dev_info->flow_type_rss_offloads = GVE_RSS_OFFLOAD_ALL;
+ dev_info->flow_type_rss_offloads = GVE_RTE_RSS_OFFLOAD_ALL;
return 0;
}
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index f7635e829c..bc486cb941 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -33,7 +33,7 @@
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
-#define GVE_RSS_OFFLOAD_ALL ( \
+#define GVE_RTE_RSS_OFFLOAD_ALL ( \
RTE_ETH_RSS_IPV4 | \
RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
RTE_ETH_RSS_IPV6 | \
@@ -290,6 +290,8 @@ struct gve_priv {
const struct rte_memzone *stats_report_mem;
uint16_t stats_start_idx; /* start index of array of stats written by NIC */
uint16_t stats_end_idx; /* end index of array of stats written by NIC */
+
+ struct gve_rss_config rss_config;
};
static inline bool
--git a/drivers/net/gve/gve_rss.c b/drivers/net/gve/gve_rss.c
new file mode 100644
index 0000000000..931180f8f2
--- /dev/null
+++ b/drivers/net/gve/gve_rss.c
@@ -0,0 +1,206 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Google LLC
+ */
+
+#include "gve_rss.h"
+
+int
+gve_generate_rss_reta(struct rte_eth_dev *dev, struct gve_rss_config *config)
+{
+ int i;
+ if (!config || !config->indir)
+ return -EINVAL;
+
+ for (i = 0; i < config->indir_size; i++)
+ config->indir[i] = i % dev->data->nb_rx_queues;
+
+ return 0;
+}
+
+
+int
+gve_init_rss_config(struct gve_rss_config *gve_rss_conf,
+ uint16_t key_size, uint16_t indir_size)
+{
+ int err;
+
+ gve_rss_conf->alg = GVE_RSS_HASH_TOEPLITZ;
+
+ gve_rss_conf->key_size = key_size;
+ gve_rss_conf->key = rte_zmalloc("rss key",
+ key_size * sizeof(*gve_rss_conf->key),
+ RTE_CACHE_LINE_SIZE);
+ if (!gve_rss_conf->key)
+ return -ENOMEM;
+
+ gve_rss_conf->indir_size = indir_size;
+ gve_rss_conf->indir = rte_zmalloc("rss reta",
+ indir_size * sizeof(*gve_rss_conf->indir),
+ RTE_CACHE_LINE_SIZE);
+ if (!gve_rss_conf->indir) {
+ err = -ENOMEM;
+ goto err_with_key;
+ }
+
+ return 0;
+err_with_key:
+ rte_free(gve_rss_conf->key);
+ return err;
+}
+
+int
+gve_init_rss_config_from_priv(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf)
+{
+ int err = gve_init_rss_config(gve_rss_conf, priv->rss_config.key_size,
+ priv->rss_config.indir_size);
+ if (err)
+ return err;
+
+ gve_rss_conf->hash_types = priv->rss_config.hash_types;
+ gve_rss_conf->alg = priv->rss_config.alg;
+ memcpy(gve_rss_conf->key, priv->rss_config.key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ memcpy(gve_rss_conf->indir, priv->rss_config.indir,
+ gve_rss_conf->indir_size * sizeof(*gve_rss_conf->indir));
+
+ return 0;
+}
+
+void
+gve_free_rss_config(struct gve_rss_config *gve_rss_conf)
+{
+ rte_free(gve_rss_conf->indir);
+ gve_rss_conf->indir = NULL;
+ rte_free(gve_rss_conf->key);
+ gve_rss_conf->key = NULL;
+}
+
+int
+gve_update_priv_rss_config(struct gve_priv *priv, struct gve_rss_config *config)
+{
+ struct gve_rss_config *priv_config = &priv->rss_config;
+ int key_bytes, indir_bytes;
+
+ if (!config)
+ return -EINVAL;
+ if (config->key_size == 0 || !config->key)
+ return -EINVAL;
+ if (config->indir_size == 0 || !config->indir)
+ return -EINVAL;
+
+ priv_config->hash_types = config->hash_types;
+ priv_config->alg = config->alg;
+
+ priv_config->key_size = config->key_size;
+ key_bytes = priv_config->key_size * sizeof(*priv_config->key);
+ if (!priv_config->key)
+ priv_config->key = rte_zmalloc("priv rss key", key_bytes,
+ RTE_CACHE_LINE_SIZE);
+ else
+ priv_config->key = rte_realloc(priv_config->key, key_bytes,
+ RTE_CACHE_LINE_SIZE);
+ if (!priv_config->key)
+ return -ENOMEM;
+
+ priv_config->indir_size = config->indir_size;
+ indir_bytes = priv_config->indir_size * sizeof(*priv_config->indir);
+ if (!priv_config->indir)
+ priv_config->indir = rte_zmalloc("priv rss reta", indir_bytes,
+ RTE_CACHE_LINE_SIZE);
+ else
+ priv_config->indir = rte_realloc(priv_config->indir,
+ indir_bytes, RTE_CACHE_LINE_SIZE);
+
+ if (!priv_config->indir)
+ return -ENOMEM;
+
+ memcpy(priv_config->indir, config->indir,
+ config->indir_size * sizeof(*priv_config->indir));
+ memcpy(priv_config->key, config->key,
+ config->key_size * sizeof(*priv_config->key));
+
+ return 0;
+}
+
+int
+gve_update_rss_key(struct gve_priv *priv, struct gve_rss_config *gve_rss_conf,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ if (rss_conf->rss_key_len && rss_conf->rss_key) {
+ gve_rss_conf->key_size = rss_conf->rss_key_len;
+ memcpy(gve_rss_conf->key, rss_conf->rss_key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ } else if (priv->rss_config.key_size && priv->rss_config.key) {
+ gve_rss_conf->key_size = priv->rss_config.key_size;
+ memcpy(gve_rss_conf->key, priv->rss_config.key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ } else {
+ PMD_DRV_LOG(ERR, "RSS key must be set as part of initial RSS "
+ "configuration.");
+ return -EINVAL;
+ }
+ return 0;
+}
+
+int
+gve_update_rss_hash_types(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf, struct rte_eth_rss_conf *rss_conf)
+{
+ /* Initialize to 0 before modifying. */
+ gve_rss_conf->hash_types = 0;
+ if (rss_conf->rss_hf)
+ rte_to_gve_rss_hf(rss_conf->rss_hf, gve_rss_conf);
+ else if (priv->rss_config.key_size && priv->rss_config.key)
+ gve_rss_conf->hash_types = priv->rss_config.hash_types;
+ else
+ gve_rss_conf->hash_types = GVE_RSS_OFFLOAD_DEFAULT;
+ return 0;
+}
+
+void
+rte_to_gve_rss_hf(uint64_t rte_rss_hf, struct gve_rss_config *gve_rss_conf)
+{
+ if (rte_rss_hf & RTE_ETH_RSS_IPV4)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_TCP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV6_EX;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_TCP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_TCP_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV6_EX;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_UDP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_UDP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_UDP_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV6_EX;
+}
+
+void
+gve_to_rte_rss_hf(uint16_t gve_rss_types, struct rte_eth_rss_conf *rss_conf)
+{
+ if (gve_rss_types & GVE_RSS_HASH_IPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV4;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV4_TCP;
+ if (gve_rss_types & GVE_RSS_HASH_IPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6;
+ if (gve_rss_types & GVE_RSS_HASH_IPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_EX;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV6_TCP;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_TCP_EX;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV4_UDP;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV6_UDP;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_UDP_EX;
+}
+
--git a/drivers/net/gve/gve_rss.h b/drivers/net/gve/gve_rss.h
new file mode 100644
index 0000000000..0cc235a19a
--- /dev/null
+++ b/drivers/net/gve/gve_rss.h
@@ -0,0 +1,107 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Google LLC
+ */
+
+#include "gve_adminq.h"
+#include "gve_ethdev.h"
+
+#define GVE_RSS_HASH_IPV4 BIT(0)
+#define GVE_RSS_HASH_TCPV4 BIT(1)
+#define GVE_RSS_HASH_IPV6 BIT(2)
+#define GVE_RSS_HASH_IPV6_EX BIT(3)
+#define GVE_RSS_HASH_TCPV6 BIT(4)
+#define GVE_RSS_HASH_TCPV6_EX BIT(5)
+#define GVE_RSS_HASH_UDPV4 BIT(6)
+#define GVE_RSS_HASH_UDPV6 BIT(7)
+#define GVE_RSS_HASH_UDPV6_EX BIT(8)
+
+#define GVE_RSS_OFFLOAD_DEFAULT ( \
+ GVE_RSS_HASH_IPV4 | \
+ GVE_RSS_HASH_TCPV4 | \
+ GVE_RSS_HASH_IPV6 | \
+ GVE_RSS_HASH_IPV6_EX | \
+ GVE_RSS_HASH_TCPV6 | \
+ GVE_RSS_HASH_TCPV6_EX | \
+ GVE_RSS_HASH_UDPV4 | \
+ GVE_RSS_HASH_UDPV6 | \
+ GVE_RSS_HASH_UDPV6_EX)
+
+/**
+ * Generates default RSS redirection table based on the number of queues the
+ * device is configured with. This assigns hash values to queues in a
+ * round-robin manner.
+ */
+int
+gve_generate_rss_reta(struct rte_eth_dev *dev, struct gve_rss_config *config);
+
+/**
+ * Initializes `gve_rss_conf`, setting the fields to default values and
+ * allocating memory for the RSS key and redirection table.
+ */
+int
+gve_init_rss_config(struct gve_rss_config *gve_rss_conf,
+ uint16_t key_size, uint16_t indir_size);
+
+/**
+ * Initializes `gve_rss_conf` based on the RSS configuration stored in `priv`.
+ */
+int
+gve_init_rss_config_from_priv(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Frees RSS key and redriection table pointers stored in `gve_rss_conf`.
+ */
+void
+gve_free_rss_config(struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Updates the rss_config stored in `priv` with the contents of `config`.
+ */
+int
+gve_update_priv_rss_config(struct gve_priv *priv,
+ struct gve_rss_config *config);
+
+/**
+ * Updates the RSS key stored in `gve_rss_conf`. It is prioritized as follows:
+ * 1) retrieve from `rss_conf`, if non-null
+ * 2) retrieve from `priv`, if non-null
+ * If keys from both sources are unset, return -EINVAL.
+ */
+int
+gve_update_rss_key(struct gve_priv *priv, struct gve_rss_config *gve_rss_conf,
+ struct rte_eth_rss_conf *rss_conf);
+
+/**
+ * Updates the RSS hash types stored in `gve_rss_conf`. It is prioritized as
+ * follows:
+ * 1) retrieve from `rss_conf`, if set
+ * 2) retrieve from priv, if RSS has been configured
+ * 3) set default RSS offload
+ */
+int
+gve_update_rss_hash_types(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf, struct rte_eth_rss_conf *rss_conf);
+
+/**
+ * Ensures that only supported RSS hash fields are set in `rte_rss_hf`.
+ */
+static inline int
+gve_validate_rss_hf(uint64_t rte_rss_hf) {
+ return rte_rss_hf & ~GVE_RTE_RSS_OFFLOAD_ALL;
+}
+
+/**
+ * Converts RSS hash types from RTE values to GVE values, storing them in
+ * `gve_rss_conf`.
+ */
+void
+rte_to_gve_rss_hf(uint64_t rte_rss_hf, struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Converts RSS hash types from GVE values to RTE values, storing them in
+ * `rss_conf`.
+ */
+void
+gve_to_rte_rss_hf(uint16_t gve_rss_types, struct rte_eth_rss_conf *rss_conf);
+
diff --git a/drivers/net/gve/meson.build b/drivers/net/gve/meson.build
index 61d195009c..6da13a8406 100644
--- a/drivers/net/gve/meson.build
+++ b/drivers/net/gve/meson.build
@@ -15,5 +15,6 @@ sources = files(
'gve_tx_dqo.c',
'gve_ethdev.c',
'gve_version.c',
+ 'gve_rss.c',
)
includes += include_directories('base')
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v3 4/7] net/gve: RSS configuration update support
2024-01-24 0:04 ` [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (2 preceding siblings ...)
2024-01-24 0:04 ` [PATCH v3 3/7] net/gve: add gve_rss library for handling RSS-related behaviors Joshua Washington
@ 2024-01-24 0:04 ` Joshua Washington
2024-01-24 0:04 ` [PATCH v3 5/7] net/gve: RSS redirection table " Joshua Washington
` (2 subsequent siblings)
6 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:04 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch adds support for updating the RSS hash key and hash fields
in the GVE PMD through the implementation of rss_hash_update and
rss_hash_conf_get.
The RSS hash key for gVNIC is required to be 40 bytes. On initial
configuration of the RSS hash key, the RSS redirection table will be
set to a static default, using a round-robin approach for all queues.
Note, however, that this patch does not include support for setting the
redirection table explicitly. In dev_configure, if the static
redirection table has been set, it will be updated to reflect the new
queue count, if it has changed.
The RSS key must be set before any other RSS configuration can happen.
As such, an attempt to set the hash types before the key is configured
will fail.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/gve_ethdev.c | 132 ++++++++++++++++++++++++++++++++++-
drivers/net/gve/gve_ethdev.h | 3 +
2 files changed, 133 insertions(+), 2 deletions(-)
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 936ca22cb9..2a68d31808 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -1,5 +1,6 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(C) 2022 Intel Corporation
+ * Copyright(C) 2022-2023 Intel Corporation
+ * Copyright(C) 2023 Google LLC
*/
#include "gve_ethdev.h"
@@ -8,6 +9,7 @@
#include "base/gve_osdep.h"
#include "gve_version.h"
#include "rte_ether.h"
+#include "gve_rss.h"
static void
gve_write_version(uint8_t *driver_version_register)
@@ -88,12 +90,31 @@ gve_dev_configure(struct rte_eth_dev *dev)
{
struct gve_priv *priv = dev->data->dev_private;
- if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) {
dev->data->dev_conf.rxmode.offloads |= RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ priv->rss_config.alg = GVE_RSS_HASH_TOEPLITZ;
+ }
if (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO)
priv->enable_rsc = 1;
+ /* Reset RSS RETA in case number of queues changed. */
+ if (priv->rss_config.indir) {
+ struct gve_rss_config update_reta_config;
+ gve_init_rss_config_from_priv(priv, &update_reta_config);
+ gve_generate_rss_reta(dev, &update_reta_config);
+
+ int err = gve_adminq_configure_rss(priv, &update_reta_config);
+ if (err)
+ PMD_DRV_LOG(ERR,
+ "Could not reconfigure RSS redirection table.");
+ else
+ gve_update_priv_rss_config(priv, &update_reta_config);
+
+ gve_free_rss_config(&update_reta_config);
+ return err;
+ }
+
return 0;
}
@@ -443,6 +464,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
};
dev_info->flow_type_rss_offloads = GVE_RTE_RSS_OFFLOAD_ALL;
+ dev_info->hash_key_size = GVE_RSS_HASH_KEY_SIZE;
+ dev_info->reta_size = GVE_RSS_INDIR_SIZE;
return 0;
}
@@ -646,6 +669,107 @@ gve_xstats_get_names(struct rte_eth_dev *dev,
return count;
}
+
+static int
+gve_rss_hash_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ struct gve_rss_config gve_rss_conf;
+ int rss_reta_size;
+ int err;
+
+ if (gve_validate_rss_hf(rss_conf->rss_hf)) {
+ PMD_DRV_LOG(ERR, "Unsupported hash function.");
+ return -EINVAL;
+ }
+
+ if (rss_conf->algorithm != RTE_ETH_HASH_FUNCTION_TOEPLITZ &&
+ rss_conf->algorithm != RTE_ETH_HASH_FUNCTION_DEFAULT) {
+ PMD_DRV_LOG(ERR, "Device only supports Toeplitz algorithm.");
+ return -EINVAL;
+ }
+
+ if (rss_conf->rss_key_len) {
+ if (rss_conf->rss_key_len != GVE_RSS_HASH_KEY_SIZE) {
+ PMD_DRV_LOG(ERR,
+ "Invalid hash key size. Only RSS hash key size "
+ "of %u supported", GVE_RSS_HASH_KEY_SIZE);
+ return -EINVAL;
+ }
+
+ if (!rss_conf->rss_key) {
+ PMD_DRV_LOG(ERR, "RSS key must be non-null.");
+ return -EINVAL;
+ }
+ } else {
+ if (!priv->rss_config.key_size) {
+ PMD_DRV_LOG(ERR, "RSS key must be initialized before "
+ "any other configuration.");
+ return -EINVAL;
+ }
+ rss_conf->rss_key_len = priv->rss_config.key_size;
+ }
+
+ rss_reta_size = priv->rss_config.indir ?
+ priv->rss_config.indir_size :
+ GVE_RSS_INDIR_SIZE;
+ err = gve_init_rss_config(&gve_rss_conf, rss_conf->rss_key_len,
+ rss_reta_size);
+ if (err)
+ return err;
+
+ gve_rss_conf.alg = GVE_RSS_HASH_TOEPLITZ;
+ err = gve_update_rss_hash_types(priv, &gve_rss_conf, rss_conf);
+ if (err)
+ goto err;
+ err = gve_update_rss_key(priv, &gve_rss_conf, rss_conf);
+ if (err)
+ goto err;
+
+ /* Set redirection table to default or preexisting. */
+ if (!priv->rss_config.indir)
+ gve_generate_rss_reta(dev, &gve_rss_conf);
+ else
+ memcpy(gve_rss_conf.indir, priv->rss_config.indir,
+ gve_rss_conf.indir_size * sizeof(*priv->rss_config.indir));
+
+ err = gve_adminq_configure_rss(priv, &gve_rss_conf);
+ if (!err)
+ gve_update_priv_rss_config(priv, &gve_rss_conf);
+
+err:
+ gve_free_rss_config(&gve_rss_conf);
+ return err;
+}
+
+static int
+gve_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+
+ if (!(dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)) {
+ PMD_DRV_LOG(ERR, "RSS not configured.");
+ return -ENOTSUP;
+ }
+
+
+ gve_to_rte_rss_hf(priv->rss_config.hash_types, rss_conf);
+ rss_conf->rss_key_len = priv->rss_config.key_size;
+ if (rss_conf->rss_key) {
+ if (!priv->rss_config.key) {
+ PMD_DRV_LOG(ERR, "Unable to retrieve default RSS hash key.");
+ return -ENOTSUP;
+ }
+ memcpy(rss_conf->rss_key, priv->rss_config.key,
+ rss_conf->rss_key_len * sizeof(*rss_conf->rss_key));
+ }
+
+ return 0;
+}
+
static const struct eth_dev_ops gve_eth_dev_ops = {
.dev_configure = gve_dev_configure,
.dev_start = gve_dev_start,
@@ -666,6 +790,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = {
.mtu_set = gve_dev_mtu_set,
.xstats_get = gve_xstats_get,
.xstats_get_names = gve_xstats_get_names,
+ .rss_hash_update = gve_rss_hash_update,
+ .rss_hash_conf_get = gve_rss_hash_conf_get,
};
static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -688,6 +814,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
.mtu_set = gve_dev_mtu_set,
.xstats_get = gve_xstats_get,
.xstats_get_names = gve_xstats_get_names,
+ .rss_hash_update = gve_rss_hash_update,
+ .rss_hash_conf_get = gve_rss_hash_conf_get,
};
static void
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index bc486cb941..d713657d10 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -29,6 +29,9 @@
#define GVE_RX_MIN_BUF_SIZE_GQI 2048
#define GVE_RX_MAX_BUF_SIZE_GQI 4096
+#define GVE_RSS_HASH_KEY_SIZE 40
+#define GVE_RSS_INDIR_SIZE 128
+
#define GVE_TX_CKSUM_OFFLOAD_MASK ( \
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v3 5/7] net/gve: RSS redirection table update support
2024-01-24 0:04 ` [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (3 preceding siblings ...)
2024-01-24 0:04 ` [PATCH v3 4/7] net/gve: RSS configuration update support Joshua Washington
@ 2024-01-24 0:04 ` Joshua Washington
2024-01-24 0:04 ` [PATCH v3 6/7] net/gve: update gve.ini with RSS capabilities Joshua Washington
2024-01-24 0:04 ` [PATCH v3 7/7] net/gve: update GVE documentation with RSS support Joshua Washington
6 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:04 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch introduces support for updating the RSS redirection table in
the GVE PMD through the implementation of rss_reta_update and
rss_reta_query.
Due to an infrastructure limitation, the RSS hash key must be manually
configured before the redirection table can be updated or queried. The
redirection table is expected to be exactly 128 bytes.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/gve_ethdev.c | 95 ++++++++++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 2a68d31808..3b8ec5872d 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -770,6 +770,97 @@ gve_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
}
+static int
+gve_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ struct gve_rss_config gve_rss_conf;
+ int table_id;
+ int err;
+ int i;
+
+ /* RSS key must be set before the redirection table can be set. */
+ if (!priv->rss_config.key || priv->rss_config.key_size == 0) {
+ PMD_DRV_LOG(ERR, "RSS hash key msut be set before the "
+ "redirection table can be updated.");
+ return -ENOTSUP;
+ }
+
+ if (reta_size != GVE_RSS_INDIR_SIZE) {
+ PMD_DRV_LOG(ERR, "Redirection table must have %hu elements",
+ (uint16_t)GVE_RSS_INDIR_SIZE);
+ return -EINVAL;
+ }
+
+ err = gve_init_rss_config_from_priv(priv, &gve_rss_conf);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Error allocating new RSS config.");
+ return err;
+ }
+
+ table_id = 0;
+ for (i = 0; i < priv->rss_config.indir_size; i++) {
+ int table_entry = i % RTE_ETH_RETA_GROUP_SIZE;
+ if (reta_conf[table_id].mask & (1ULL << table_entry))
+ gve_rss_conf.indir[i] =
+ reta_conf[table_id].reta[table_entry];
+
+ if (table_entry == RTE_ETH_RETA_GROUP_SIZE - 1)
+ table_id++;
+ }
+
+ err = gve_adminq_configure_rss(priv, &gve_rss_conf);
+ if (err)
+ PMD_DRV_LOG(ERR, "Problem configuring RSS with device.");
+ else
+ gve_update_priv_rss_config(priv, &gve_rss_conf);
+
+ gve_free_rss_config(&gve_rss_conf);
+ return err;
+}
+
+static int
+gve_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ int table_id;
+ int i;
+
+ if (!(dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)) {
+ PMD_DRV_LOG(ERR, "RSS not configured.");
+ return -ENOTSUP;
+ }
+
+ /* RSS key must be set before the redirection table can be queried. */
+ if (!priv->rss_config.key) {
+ PMD_DRV_LOG(ERR, "RSS hash key must be set before the "
+ "redirection table can be initialized.");
+ return -ENOTSUP;
+ }
+
+ if (reta_size != priv->rss_config.indir_size) {
+ PMD_DRV_LOG(ERR, "RSS redirection table must have %d entries.",
+ priv->rss_config.indir_size);
+ return -EINVAL;
+ }
+
+ table_id = 0;
+ for (i = 0; i < priv->rss_config.indir_size; i++) {
+ int table_entry = i % RTE_ETH_RETA_GROUP_SIZE;
+ if (reta_conf[table_id].mask & (1ULL << table_entry))
+ reta_conf[table_id].reta[table_entry] =
+ priv->rss_config.indir[i];
+
+ if (table_entry == RTE_ETH_RETA_GROUP_SIZE - 1)
+ table_id++;
+ }
+
+ return 0;
+}
+
static const struct eth_dev_ops gve_eth_dev_ops = {
.dev_configure = gve_dev_configure,
.dev_start = gve_dev_start,
@@ -792,6 +883,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = {
.xstats_get_names = gve_xstats_get_names,
.rss_hash_update = gve_rss_hash_update,
.rss_hash_conf_get = gve_rss_hash_conf_get,
+ .reta_update = gve_rss_reta_update,
+ .reta_query = gve_rss_reta_query,
};
static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -816,6 +909,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
.xstats_get_names = gve_xstats_get_names,
.rss_hash_update = gve_rss_hash_update,
.rss_hash_conf_get = gve_rss_hash_conf_get,
+ .reta_update = gve_rss_reta_update,
+ .reta_query = gve_rss_reta_query,
};
static void
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v3 6/7] net/gve: update gve.ini with RSS capabilities
2024-01-24 0:04 ` [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (4 preceding siblings ...)
2024-01-24 0:04 ` [PATCH v3 5/7] net/gve: RSS redirection table " Joshua Washington
@ 2024-01-24 0:04 ` Joshua Washington
2024-01-24 0:04 ` [PATCH v3 7/7] net/gve: update GVE documentation with RSS support Joshua Washington
6 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:04 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch updates the DPDK feature matrix to expose that the GVE driver
supports RSS hash, RSS key update, and RSS reta update.
Signed-off-by: Joshua Washington <joshwash@google.com>
---
doc/guides/nics/features/gve.ini | 3 +++
1 file changed, 3 insertions(+)
diff --git a/doc/guides/nics/features/gve.ini b/doc/guides/nics/features/gve.ini
index 838edd456a..bc08648dbc 100644
--- a/doc/guides/nics/features/gve.ini
+++ b/doc/guides/nics/features/gve.ini
@@ -15,3 +15,6 @@ Linux = Y
x86-32 = Y
x86-64 = Y
Usage doc = Y
+RSS hash = Y
+RSS key update = Y
+RSS reta update = Y
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v3 7/7] net/gve: update GVE documentation with RSS support
2024-01-24 0:04 ` [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (5 preceding siblings ...)
2024-01-24 0:04 ` [PATCH v3 6/7] net/gve: update gve.ini with RSS capabilities Joshua Washington
@ 2024-01-24 0:04 ` Joshua Washington
6 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:04 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch updates the GVE doc page to communicate that GVE now supports
RSS configuration and explains the limitations.
Signed-off-by: Joshua Washington <joshwash@google.com>
---
doc/guides/nics/gve.rst | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/doc/guides/nics/gve.rst b/doc/guides/nics/gve.rst
index 1c3eaf03ef..80991e70cf 100644
--- a/doc/guides/nics/gve.rst
+++ b/doc/guides/nics/gve.rst
@@ -70,6 +70,8 @@ Supported features of the GVE PMD are:
- Link state information
- Tx multi-segments (Scatter Tx)
- Tx UDP/TCP/SCTP Checksum
+- RSS hash configuration
+- RSS redirection table update and query
Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD.
Jumbo Frame is not supported in PMD for now.
@@ -77,10 +79,12 @@ It'll be added in a future DPDK release.
Also, only GQI_QPL queue format is in use on GCP
since GQI_RDA hasn't been released in production.
-Currently, setting MTU with value larger than 1460 is not supported.
+RSS
+^^^
-Currently, only "RSS hash" is force enabled
-so that the backend hardware device calculated hash values
-could be shared with applications.
-But for RSS, there is no such API to config RSS hash function or RETA table.
-So, limited RSS is supported only with default config/setting.
+GVE RSS can be enabled and configured using the standard interfaces. The driver
+does not support querying the initial RSS configuration.
+
+The RSS hash key must be exactly 40 bytes, and the redirection table must have
+128 entries. The RSS hash key must be configured before the redirection table
+can be set up.
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH 0/7] net/gve: RSS Support for GVE Driver
2024-01-23 17:58 [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (7 preceding siblings ...)
2024-01-24 0:04 ` [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
@ 2024-01-24 0:09 ` Joshua Washington
2024-01-24 0:13 ` [PATCH v3 " Joshua Washington
2024-01-24 0:14 ` Joshua Washington
10 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:09 UTC (permalink / raw)
Cc: dev, Ferruh Yigit, Rushil Gupta, Joshua Washington
This patch series introduces RSS support for the GVE poll-mode driver.
This series includes implementations of the following eth_dev_ops:
1) rss_hash_update
2) rss_hash_conf_get
3) reta_query
4) reta_update
In rss_hash_update, the GVE driver supports the following RSS hash
types:
* RTE_ETH_RSS_IPV4
* RTE_ETH_RSS_NONFRAG_IPV4_TCP
* RTE_ETH_RSS_NONFRAG_IPV4_UDP
* RTE_ETH_RSS_IPV6
* RTE_ETH_RSS_IPV6_EX
* RTE_ETH_RSS_NONFRAG_IPV6_TCP
* RTE_ETH_RSS_NONFRAG_IPV6_UDP
* RTE_ETH_RSS_IPV6_TCP_EX
* RTE_ETH_RSS_IPV6_UDP_EX
The hash key is 40B, and the lookup table has 128 entries. These values
are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set
with a key before the redriection table is set up. When the RSS hash is
configured, a default redirection table is generated based on the number
of queues. When the device is re-configured, the redirection table is
reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for
RSS redirection table entries, while the RTE API uses 16 bit integers.
However, this is unlikely to be an issue, as these values represent
receive queues, and the gVNIC device does not support anywhere near 64K
queues.
This series also updates the corresponding feature matrix ertries and
documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches
fixes.
Note: There is a warning about complex macros being parenthesized that
does not seem to be well-founded.
v3:
Fix build warnings that come up on certain distros.
Joshua Washington (7):
net/gve: fully expose RSS offload support in dev_info
net/gve: RSS adminq command changes
net/gve: add gve_rss library for handling RSS-related behaviors
net/gve: RSS configuration update support
net/gve: RSS redirection table update support
net/gve: update gve.ini with RSS capabilities
net/gve: update GVE documentation with RSS support
doc/guides/nics/features/gve.ini | 3 +
doc/guides/nics/gve.rst | 16 ++-
drivers/net/gve/base/gve.h | 15 ++
drivers/net/gve/base/gve_adminq.c | 59 ++++++++
drivers/net/gve/base/gve_adminq.h | 21 +++
drivers/net/gve/gve_ethdev.c | 231 +++++++++++++++++++++++++++++-
drivers/net/gve/gve_ethdev.h | 17 +++
drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++
drivers/net/gve/gve_rss.h | 107 ++++++++++++++
drivers/net/gve/meson.build | 1 +
10 files changed, 667 insertions(+), 9 deletions(-)
create mode 100644 drivers/net/gve/gve_rss.c
create mode 100644 drivers/net/gve/gve_rss.h
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply [flat|nested] 44+ messages in thread
* [PATCH v3 0/7] net/gve: RSS Support for GVE Driver
2024-01-23 17:58 [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (8 preceding siblings ...)
2024-01-24 0:09 ` [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
@ 2024-01-24 0:13 ` Joshua Washington
2024-01-24 0:14 ` Joshua Washington
10 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:13 UTC (permalink / raw)
Cc: dev, Ferruh Yigit, Rushil Gupta, Joshua Washington
This patch series introduces RSS support for the GVE poll-mode driver.
This series includes implementations of the following eth_dev_ops:
1) rss_hash_update
2) rss_hash_conf_get
3) reta_query
4) reta_update
In rss_hash_update, the GVE driver supports the following RSS hash
types:
* RTE_ETH_RSS_IPV4
* RTE_ETH_RSS_NONFRAG_IPV4_TCP
* RTE_ETH_RSS_NONFRAG_IPV4_UDP
* RTE_ETH_RSS_IPV6
* RTE_ETH_RSS_IPV6_EX
* RTE_ETH_RSS_NONFRAG_IPV6_TCP
* RTE_ETH_RSS_NONFRAG_IPV6_UDP
* RTE_ETH_RSS_IPV6_TCP_EX
* RTE_ETH_RSS_IPV6_UDP_EX
The hash key is 40B, and the lookup table has 128 entries. These values
are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set
with a key before the redriection table is set up. When the RSS hash is
configured, a default redirection table is generated based on the number
of queues. When the device is re-configured, the redirection table is
reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for
RSS redirection table entries, while the RTE API uses 16 bit integers.
However, this is unlikely to be an issue, as these values represent
receive queues, and the gVNIC device does not support anywhere near 64K
queues.
This series also updates the corresponding feature matrix ertries and
documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches
fixes.
Note: There is a warning about complex macros being parenthesized that
does not seem to be well-founded.
v3:
Fix build warnings that come up on certain distros.
Joshua Washington (7):
net/gve: fully expose RSS offload support in dev_info
net/gve: RSS adminq command changes
net/gve: add gve_rss library for handling RSS-related behaviors
net/gve: RSS configuration update support
net/gve: RSS redirection table update support
net/gve: update gve.ini with RSS capabilities
net/gve: update GVE documentation with RSS support
doc/guides/nics/features/gve.ini | 3 +
doc/guides/nics/gve.rst | 16 ++-
drivers/net/gve/base/gve.h | 15 ++
drivers/net/gve/base/gve_adminq.c | 59 ++++++++
drivers/net/gve/base/gve_adminq.h | 21 +++
drivers/net/gve/gve_ethdev.c | 231 +++++++++++++++++++++++++++++-
drivers/net/gve/gve_ethdev.h | 17 +++
drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++
drivers/net/gve/gve_rss.h | 107 ++++++++++++++
drivers/net/gve/meson.build | 1 +
10 files changed, 667 insertions(+), 9 deletions(-)
create mode 100644 drivers/net/gve/gve_rss.c
create mode 100644 drivers/net/gve/gve_rss.h
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply [flat|nested] 44+ messages in thread
* [PATCH v3 0/7] net/gve: RSS Support for GVE Driver
2024-01-23 17:58 [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
` (9 preceding siblings ...)
2024-01-24 0:13 ` [PATCH v3 " Joshua Washington
@ 2024-01-24 0:14 ` Joshua Washington
2024-01-24 0:14 ` [PATCH v3 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
` (7 more replies)
10 siblings, 8 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:14 UTC (permalink / raw)
Cc: dev, Ferruh Yigit, Rushil Gupta, Joshua Washington
This patch series introduces RSS support for the GVE poll-mode driver.
This series includes implementations of the following eth_dev_ops:
1) rss_hash_update
2) rss_hash_conf_get
3) reta_query
4) reta_update
In rss_hash_update, the GVE driver supports the following RSS hash
types:
* RTE_ETH_RSS_IPV4
* RTE_ETH_RSS_NONFRAG_IPV4_TCP
* RTE_ETH_RSS_NONFRAG_IPV4_UDP
* RTE_ETH_RSS_IPV6
* RTE_ETH_RSS_IPV6_EX
* RTE_ETH_RSS_NONFRAG_IPV6_TCP
* RTE_ETH_RSS_NONFRAG_IPV6_UDP
* RTE_ETH_RSS_IPV6_TCP_EX
* RTE_ETH_RSS_IPV6_UDP_EX
The hash key is 40B, and the lookup table has 128 entries. These values
are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set
with a key before the redriection table is set up. When the RSS hash is
configured, a default redirection table is generated based on the number
of queues. When the device is re-configured, the redirection table is
reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for
RSS redirection table entries, while the RTE API uses 16 bit integers.
However, this is unlikely to be an issue, as these values represent
receive queues, and the gVNIC device does not support anywhere near 64K
queues.
This series also updates the corresponding feature matrix ertries and
documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches
fixes.
Note: There is a warning about complex macros being parenthesized that
does not seem to be well-founded.
v3:
Fix build warnings that come up on certain distros.
Joshua Washington (7):
net/gve: fully expose RSS offload support in dev_info
net/gve: RSS adminq command changes
net/gve: add gve_rss library for handling RSS-related behaviors
net/gve: RSS configuration update support
net/gve: RSS redirection table update support
net/gve: update gve.ini with RSS capabilities
net/gve: update GVE documentation with RSS support
doc/guides/nics/features/gve.ini | 3 +
doc/guides/nics/gve.rst | 16 ++-
drivers/net/gve/base/gve.h | 15 ++
drivers/net/gve/base/gve_adminq.c | 59 ++++++++
drivers/net/gve/base/gve_adminq.h | 21 +++
drivers/net/gve/gve_ethdev.c | 231 +++++++++++++++++++++++++++++-
drivers/net/gve/gve_ethdev.h | 17 +++
drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++
drivers/net/gve/gve_rss.h | 107 ++++++++++++++
drivers/net/gve/meson.build | 1 +
10 files changed, 667 insertions(+), 9 deletions(-)
create mode 100644 drivers/net/gve/gve_rss.c
create mode 100644 drivers/net/gve/gve_rss.h
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply [flat|nested] 44+ messages in thread
* [PATCH v3 1/7] net/gve: fully expose RSS offload support in dev_info
2024-01-24 0:14 ` Joshua Washington
@ 2024-01-24 0:14 ` Joshua Washington
2024-01-24 0:14 ` [PATCH v3 2/7] net/gve: RSS adminq command changes Joshua Washington
` (6 subsequent siblings)
7 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:14 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch communicates that the GVE driver supports RSS, along with
the RSS offloads supported by the driver.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/gve_ethdev.c | 4 +++-
drivers/net/gve/gve_ethdev.h | 8 ++++++++
2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index d162fd3864..6acdb4e13b 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -405,7 +405,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = priv->max_mtu;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
- dev_info->rx_offload_capa = 0;
+ dev_info->rx_offload_capa = RTE_ETH_RX_OFFLOAD_RSS_HASH;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
@@ -442,6 +442,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.nb_align = 1,
};
+ dev_info->flow_type_rss_offloads = GVE_RSS_OFFLOAD_ALL;
+
return 0;
}
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 9893fcfee6..14c72ec91a 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -33,6 +33,14 @@
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
+#define GVE_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_IPV6_EX | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_IPV6_TCP_EX)
+
/* A list of pages registered with the device during setup and used by a queue
* as buffers
*/
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v3 2/7] net/gve: RSS adminq command changes
2024-01-24 0:14 ` Joshua Washington
2024-01-24 0:14 ` [PATCH v3 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
@ 2024-01-24 0:14 ` Joshua Washington
2024-01-24 0:14 ` [PATCH v3 3/7] net/gve: add gve_rss library for handling RSS-related behaviors Joshua Washington
` (5 subsequent siblings)
7 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:14 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This change introduces admin queue changes that enable the configuration
of RSS parameters for the GVE driver.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/base/gve.h | 15 ++++++++
drivers/net/gve/base/gve_adminq.c | 58 +++++++++++++++++++++++++++++++
drivers/net/gve/base/gve_adminq.h | 29 ++++++++++++++++
drivers/net/gve/gve_ethdev.h | 6 +++-
4 files changed, 107 insertions(+), 1 deletion(-)
diff --git a/drivers/net/gve/base/gve.h b/drivers/net/gve/base/gve.h
index f7b297e759..9c58fc4238 100644
--- a/drivers/net/gve/base/gve.h
+++ b/drivers/net/gve/base/gve.h
@@ -51,4 +51,19 @@ enum gve_state_flags_bit {
GVE_PRIV_FLAGS_NAPI_ENABLED = 4,
};
+enum gve_rss_hash_algorithm {
+ GVE_RSS_HASH_UNDEFINED = 0,
+ GVE_RSS_HASH_TOEPLITZ = 1,
+};
+
+struct gve_rss_config {
+ uint16_t hash_types;
+ enum gve_rss_hash_algorithm alg;
+ uint16_t key_size;
+ uint16_t indir_size;
+ uint8_t *key;
+ uint32_t *indir;
+};
+
+
#endif /* _GVE_H_ */
diff --git a/drivers/net/gve/base/gve_adminq.c b/drivers/net/gve/base/gve_adminq.c
index 343bd13d67..fede0a4b82 100644
--- a/drivers/net/gve/base/gve_adminq.c
+++ b/drivers/net/gve/base/gve_adminq.c
@@ -389,6 +389,9 @@ static int gve_adminq_issue_cmd(struct gve_priv *priv,
case GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES:
priv->adminq_dcfg_device_resources_cnt++;
break;
+ case GVE_ADMINQ_CONFIGURE_RSS:
+ priv->adminq_cfg_rss_cnt++;
+ break;
case GVE_ADMINQ_SET_DRIVER_PARAMETER:
priv->adminq_set_driver_parameter_cnt++;
break;
@@ -938,3 +941,58 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,
gve_free_dma_mem(&ptype_map_dma_mem);
return err;
}
+
+int gve_adminq_configure_rss(struct gve_priv *priv,
+ struct gve_rss_config *rss_config)
+{
+ struct gve_dma_mem indirection_table_dma_mem;
+ struct gve_dma_mem rss_key_dma_mem;
+ union gve_adminq_command cmd;
+ __be32 *indir = NULL;
+ u8 *key = NULL;
+ int err = 0;
+ int i;
+
+ if (!rss_config->indir_size || !rss_config->key_size)
+ return -EINVAL;
+
+ indir = gve_alloc_dma_mem(&indirection_table_dma_mem,
+ rss_config->indir_size *
+ sizeof(*rss_config->indir));
+ if (!indir) {
+ err = -ENOMEM;
+ goto out;
+ }
+ for (i = 0; i < rss_config->indir_size; i++)
+ indir[i] = cpu_to_be32(rss_config->indir[i]);
+
+ key = gve_alloc_dma_mem(&rss_key_dma_mem,
+ rss_config->key_size *
+ sizeof(*rss_config->key));
+ if (!key) {
+ err = -ENOMEM;
+ goto out;
+ }
+ memcpy(key, rss_config->key, rss_config->key_size);
+
+ memset(&cmd, 0, sizeof(cmd));
+ cmd.opcode = cpu_to_be32(GVE_ADMINQ_CONFIGURE_RSS);
+ cmd.configure_rss = (struct gve_adminq_configure_rss) {
+ .hash_types = cpu_to_be16(rss_config->hash_types),
+ .halg = rss_config->alg,
+ .hkey_len = cpu_to_be16(rss_config->key_size),
+ .indir_len = cpu_to_be16(rss_config->indir_size),
+ .hkey_addr = cpu_to_be64(rss_key_dma_mem.pa),
+ .indir_addr = cpu_to_be64(indirection_table_dma_mem.pa),
+ };
+
+ err = gve_adminq_execute_cmd(priv, &cmd);
+
+out:
+ if (indir)
+ gve_free_dma_mem(&indirection_table_dma_mem);
+ if (key)
+ gve_free_dma_mem(&rss_key_dma_mem);
+ return err;
+}
+
diff --git a/drivers/net/gve/base/gve_adminq.h b/drivers/net/gve/base/gve_adminq.h
index f05362f85f..95f4960561 100644
--- a/drivers/net/gve/base/gve_adminq.h
+++ b/drivers/net/gve/base/gve_adminq.h
@@ -19,6 +19,7 @@ enum gve_adminq_opcodes {
GVE_ADMINQ_DESTROY_TX_QUEUE = 0x7,
GVE_ADMINQ_DESTROY_RX_QUEUE = 0x8,
GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES = 0x9,
+ GVE_ADMINQ_CONFIGURE_RSS = 0xA,
GVE_ADMINQ_SET_DRIVER_PARAMETER = 0xB,
GVE_ADMINQ_REPORT_STATS = 0xC,
GVE_ADMINQ_REPORT_LINK_SPEED = 0xD,
@@ -377,6 +378,27 @@ struct gve_adminq_get_ptype_map {
__be64 ptype_map_addr;
};
+#define GVE_RSS_HASH_IPV4 BIT(0)
+#define GVE_RSS_HASH_TCPV4 BIT(1)
+#define GVE_RSS_HASH_IPV6 BIT(2)
+#define GVE_RSS_HASH_IPV6_EX BIT(3)
+#define GVE_RSS_HASH_TCPV6 BIT(4)
+#define GVE_RSS_HASH_TCPV6_EX BIT(5)
+#define GVE_RSS_HASH_UDPV4 BIT(6)
+#define GVE_RSS_HASH_UDPV6 BIT(7)
+#define GVE_RSS_HASH_UDPV6_EX BIT(8)
+
+/* RSS configuration command */
+struct gve_adminq_configure_rss {
+ __be16 hash_types;
+ u8 halg; /* hash algorithm */
+ u8 reserved;
+ __be16 hkey_len;
+ __be16 indir_len;
+ __be64 hkey_addr;
+ __be64 indir_addr;
+};
+
union gve_adminq_command {
struct {
__be32 opcode;
@@ -391,6 +413,7 @@ union gve_adminq_command {
struct gve_adminq_describe_device describe_device;
struct gve_adminq_register_page_list reg_page_list;
struct gve_adminq_unregister_page_list unreg_page_list;
+ struct gve_adminq_configure_rss configure_rss;
struct gve_adminq_set_driver_parameter set_driver_param;
struct gve_adminq_report_stats report_stats;
struct gve_adminq_report_link_speed report_link_speed;
@@ -404,6 +427,8 @@ union gve_adminq_command {
GVE_CHECK_UNION_LEN(64, gve_adminq_command);
+struct gve_priv;
+struct gve_queue_page_list;
int gve_adminq_alloc(struct gve_priv *priv);
void gve_adminq_free(struct gve_priv *priv);
void gve_adminq_release(struct gve_priv *priv);
@@ -433,4 +458,8 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,
int gve_adminq_verify_driver_compatibility(struct gve_priv *priv,
u64 driver_info_len,
dma_addr_t driver_info_addr);
+
+int gve_adminq_configure_rss(struct gve_priv *priv,
+ struct gve_rss_config *rss_config);
+
#endif /* _GVE_ADMINQ_H */
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 14c72ec91a..f7635e829c 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -39,7 +39,10 @@
RTE_ETH_RSS_IPV6 | \
RTE_ETH_RSS_IPV6_EX | \
RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
- RTE_ETH_RSS_IPV6_TCP_EX)
+ RTE_ETH_RSS_IPV6_TCP_EX | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_IPV6_UDP_EX)
/* A list of pages registered with the device during setup and used by a queue
* as buffers
@@ -264,6 +267,7 @@ struct gve_priv {
uint32_t adminq_destroy_tx_queue_cnt;
uint32_t adminq_destroy_rx_queue_cnt;
uint32_t adminq_dcfg_device_resources_cnt;
+ uint32_t adminq_cfg_rss_cnt;
uint32_t adminq_set_driver_parameter_cnt;
uint32_t adminq_report_stats_cnt;
uint32_t adminq_report_link_speed_cnt;
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v3 3/7] net/gve: add gve_rss library for handling RSS-related behaviors
2024-01-24 0:14 ` Joshua Washington
2024-01-24 0:14 ` [PATCH v3 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
2024-01-24 0:14 ` [PATCH v3 2/7] net/gve: RSS adminq command changes Joshua Washington
@ 2024-01-24 0:14 ` Joshua Washington
2024-01-24 0:14 ` [PATCH v3 4/7] net/gve: RSS configuration update support Joshua Washington
` (4 subsequent siblings)
7 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:14 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This change includes a number of helper functions to facilitate RSS
configuration on the GVE DPDK driver. These methods are declared in
gve_rss.h.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/base/gve_adminq.h | 10 +-
drivers/net/gve/gve_ethdev.c | 2 +-
drivers/net/gve/gve_ethdev.h | 4 +-
| 206 ++++++++++++++++++++++++++++++
| 107 ++++++++++++++++
drivers/net/gve/meson.build | 1 +
6 files changed, 319 insertions(+), 11 deletions(-)
create mode 100644 drivers/net/gve/gve_rss.c
create mode 100644 drivers/net/gve/gve_rss.h
diff --git a/drivers/net/gve/base/gve_adminq.h b/drivers/net/gve/base/gve_adminq.h
index 95f4960561..24abd945cc 100644
--- a/drivers/net/gve/base/gve_adminq.h
+++ b/drivers/net/gve/base/gve_adminq.h
@@ -378,15 +378,6 @@ struct gve_adminq_get_ptype_map {
__be64 ptype_map_addr;
};
-#define GVE_RSS_HASH_IPV4 BIT(0)
-#define GVE_RSS_HASH_TCPV4 BIT(1)
-#define GVE_RSS_HASH_IPV6 BIT(2)
-#define GVE_RSS_HASH_IPV6_EX BIT(3)
-#define GVE_RSS_HASH_TCPV6 BIT(4)
-#define GVE_RSS_HASH_TCPV6_EX BIT(5)
-#define GVE_RSS_HASH_UDPV4 BIT(6)
-#define GVE_RSS_HASH_UDPV6 BIT(7)
-#define GVE_RSS_HASH_UDPV6_EX BIT(8)
/* RSS configuration command */
struct gve_adminq_configure_rss {
@@ -428,6 +419,7 @@ union gve_adminq_command {
GVE_CHECK_UNION_LEN(64, gve_adminq_command);
struct gve_priv;
+struct gve_rss_config;
struct gve_queue_page_list;
int gve_adminq_alloc(struct gve_priv *priv);
void gve_adminq_free(struct gve_priv *priv);
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 6acdb4e13b..936ca22cb9 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -442,7 +442,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.nb_align = 1,
};
- dev_info->flow_type_rss_offloads = GVE_RSS_OFFLOAD_ALL;
+ dev_info->flow_type_rss_offloads = GVE_RTE_RSS_OFFLOAD_ALL;
return 0;
}
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index f7635e829c..bc486cb941 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -33,7 +33,7 @@
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
-#define GVE_RSS_OFFLOAD_ALL ( \
+#define GVE_RTE_RSS_OFFLOAD_ALL ( \
RTE_ETH_RSS_IPV4 | \
RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
RTE_ETH_RSS_IPV6 | \
@@ -290,6 +290,8 @@ struct gve_priv {
const struct rte_memzone *stats_report_mem;
uint16_t stats_start_idx; /* start index of array of stats written by NIC */
uint16_t stats_end_idx; /* end index of array of stats written by NIC */
+
+ struct gve_rss_config rss_config;
};
static inline bool
--git a/drivers/net/gve/gve_rss.c b/drivers/net/gve/gve_rss.c
new file mode 100644
index 0000000000..931180f8f2
--- /dev/null
+++ b/drivers/net/gve/gve_rss.c
@@ -0,0 +1,206 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Google LLC
+ */
+
+#include "gve_rss.h"
+
+int
+gve_generate_rss_reta(struct rte_eth_dev *dev, struct gve_rss_config *config)
+{
+ int i;
+ if (!config || !config->indir)
+ return -EINVAL;
+
+ for (i = 0; i < config->indir_size; i++)
+ config->indir[i] = i % dev->data->nb_rx_queues;
+
+ return 0;
+}
+
+
+int
+gve_init_rss_config(struct gve_rss_config *gve_rss_conf,
+ uint16_t key_size, uint16_t indir_size)
+{
+ int err;
+
+ gve_rss_conf->alg = GVE_RSS_HASH_TOEPLITZ;
+
+ gve_rss_conf->key_size = key_size;
+ gve_rss_conf->key = rte_zmalloc("rss key",
+ key_size * sizeof(*gve_rss_conf->key),
+ RTE_CACHE_LINE_SIZE);
+ if (!gve_rss_conf->key)
+ return -ENOMEM;
+
+ gve_rss_conf->indir_size = indir_size;
+ gve_rss_conf->indir = rte_zmalloc("rss reta",
+ indir_size * sizeof(*gve_rss_conf->indir),
+ RTE_CACHE_LINE_SIZE);
+ if (!gve_rss_conf->indir) {
+ err = -ENOMEM;
+ goto err_with_key;
+ }
+
+ return 0;
+err_with_key:
+ rte_free(gve_rss_conf->key);
+ return err;
+}
+
+int
+gve_init_rss_config_from_priv(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf)
+{
+ int err = gve_init_rss_config(gve_rss_conf, priv->rss_config.key_size,
+ priv->rss_config.indir_size);
+ if (err)
+ return err;
+
+ gve_rss_conf->hash_types = priv->rss_config.hash_types;
+ gve_rss_conf->alg = priv->rss_config.alg;
+ memcpy(gve_rss_conf->key, priv->rss_config.key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ memcpy(gve_rss_conf->indir, priv->rss_config.indir,
+ gve_rss_conf->indir_size * sizeof(*gve_rss_conf->indir));
+
+ return 0;
+}
+
+void
+gve_free_rss_config(struct gve_rss_config *gve_rss_conf)
+{
+ rte_free(gve_rss_conf->indir);
+ gve_rss_conf->indir = NULL;
+ rte_free(gve_rss_conf->key);
+ gve_rss_conf->key = NULL;
+}
+
+int
+gve_update_priv_rss_config(struct gve_priv *priv, struct gve_rss_config *config)
+{
+ struct gve_rss_config *priv_config = &priv->rss_config;
+ int key_bytes, indir_bytes;
+
+ if (!config)
+ return -EINVAL;
+ if (config->key_size == 0 || !config->key)
+ return -EINVAL;
+ if (config->indir_size == 0 || !config->indir)
+ return -EINVAL;
+
+ priv_config->hash_types = config->hash_types;
+ priv_config->alg = config->alg;
+
+ priv_config->key_size = config->key_size;
+ key_bytes = priv_config->key_size * sizeof(*priv_config->key);
+ if (!priv_config->key)
+ priv_config->key = rte_zmalloc("priv rss key", key_bytes,
+ RTE_CACHE_LINE_SIZE);
+ else
+ priv_config->key = rte_realloc(priv_config->key, key_bytes,
+ RTE_CACHE_LINE_SIZE);
+ if (!priv_config->key)
+ return -ENOMEM;
+
+ priv_config->indir_size = config->indir_size;
+ indir_bytes = priv_config->indir_size * sizeof(*priv_config->indir);
+ if (!priv_config->indir)
+ priv_config->indir = rte_zmalloc("priv rss reta", indir_bytes,
+ RTE_CACHE_LINE_SIZE);
+ else
+ priv_config->indir = rte_realloc(priv_config->indir,
+ indir_bytes, RTE_CACHE_LINE_SIZE);
+
+ if (!priv_config->indir)
+ return -ENOMEM;
+
+ memcpy(priv_config->indir, config->indir,
+ config->indir_size * sizeof(*priv_config->indir));
+ memcpy(priv_config->key, config->key,
+ config->key_size * sizeof(*priv_config->key));
+
+ return 0;
+}
+
+int
+gve_update_rss_key(struct gve_priv *priv, struct gve_rss_config *gve_rss_conf,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ if (rss_conf->rss_key_len && rss_conf->rss_key) {
+ gve_rss_conf->key_size = rss_conf->rss_key_len;
+ memcpy(gve_rss_conf->key, rss_conf->rss_key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ } else if (priv->rss_config.key_size && priv->rss_config.key) {
+ gve_rss_conf->key_size = priv->rss_config.key_size;
+ memcpy(gve_rss_conf->key, priv->rss_config.key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ } else {
+ PMD_DRV_LOG(ERR, "RSS key must be set as part of initial RSS "
+ "configuration.");
+ return -EINVAL;
+ }
+ return 0;
+}
+
+int
+gve_update_rss_hash_types(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf, struct rte_eth_rss_conf *rss_conf)
+{
+ /* Initialize to 0 before modifying. */
+ gve_rss_conf->hash_types = 0;
+ if (rss_conf->rss_hf)
+ rte_to_gve_rss_hf(rss_conf->rss_hf, gve_rss_conf);
+ else if (priv->rss_config.key_size && priv->rss_config.key)
+ gve_rss_conf->hash_types = priv->rss_config.hash_types;
+ else
+ gve_rss_conf->hash_types = GVE_RSS_OFFLOAD_DEFAULT;
+ return 0;
+}
+
+void
+rte_to_gve_rss_hf(uint64_t rte_rss_hf, struct gve_rss_config *gve_rss_conf)
+{
+ if (rte_rss_hf & RTE_ETH_RSS_IPV4)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_TCP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV6_EX;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_TCP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_TCP_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV6_EX;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_UDP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_UDP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_UDP_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV6_EX;
+}
+
+void
+gve_to_rte_rss_hf(uint16_t gve_rss_types, struct rte_eth_rss_conf *rss_conf)
+{
+ if (gve_rss_types & GVE_RSS_HASH_IPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV4;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV4_TCP;
+ if (gve_rss_types & GVE_RSS_HASH_IPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6;
+ if (gve_rss_types & GVE_RSS_HASH_IPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_EX;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV6_TCP;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_TCP_EX;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV4_UDP;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV6_UDP;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_UDP_EX;
+}
+
--git a/drivers/net/gve/gve_rss.h b/drivers/net/gve/gve_rss.h
new file mode 100644
index 0000000000..0cc235a19a
--- /dev/null
+++ b/drivers/net/gve/gve_rss.h
@@ -0,0 +1,107 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Google LLC
+ */
+
+#include "gve_adminq.h"
+#include "gve_ethdev.h"
+
+#define GVE_RSS_HASH_IPV4 BIT(0)
+#define GVE_RSS_HASH_TCPV4 BIT(1)
+#define GVE_RSS_HASH_IPV6 BIT(2)
+#define GVE_RSS_HASH_IPV6_EX BIT(3)
+#define GVE_RSS_HASH_TCPV6 BIT(4)
+#define GVE_RSS_HASH_TCPV6_EX BIT(5)
+#define GVE_RSS_HASH_UDPV4 BIT(6)
+#define GVE_RSS_HASH_UDPV6 BIT(7)
+#define GVE_RSS_HASH_UDPV6_EX BIT(8)
+
+#define GVE_RSS_OFFLOAD_DEFAULT ( \
+ GVE_RSS_HASH_IPV4 | \
+ GVE_RSS_HASH_TCPV4 | \
+ GVE_RSS_HASH_IPV6 | \
+ GVE_RSS_HASH_IPV6_EX | \
+ GVE_RSS_HASH_TCPV6 | \
+ GVE_RSS_HASH_TCPV6_EX | \
+ GVE_RSS_HASH_UDPV4 | \
+ GVE_RSS_HASH_UDPV6 | \
+ GVE_RSS_HASH_UDPV6_EX)
+
+/**
+ * Generates default RSS redirection table based on the number of queues the
+ * device is configured with. This assigns hash values to queues in a
+ * round-robin manner.
+ */
+int
+gve_generate_rss_reta(struct rte_eth_dev *dev, struct gve_rss_config *config);
+
+/**
+ * Initializes `gve_rss_conf`, setting the fields to default values and
+ * allocating memory for the RSS key and redirection table.
+ */
+int
+gve_init_rss_config(struct gve_rss_config *gve_rss_conf,
+ uint16_t key_size, uint16_t indir_size);
+
+/**
+ * Initializes `gve_rss_conf` based on the RSS configuration stored in `priv`.
+ */
+int
+gve_init_rss_config_from_priv(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Frees RSS key and redriection table pointers stored in `gve_rss_conf`.
+ */
+void
+gve_free_rss_config(struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Updates the rss_config stored in `priv` with the contents of `config`.
+ */
+int
+gve_update_priv_rss_config(struct gve_priv *priv,
+ struct gve_rss_config *config);
+
+/**
+ * Updates the RSS key stored in `gve_rss_conf`. It is prioritized as follows:
+ * 1) retrieve from `rss_conf`, if non-null
+ * 2) retrieve from `priv`, if non-null
+ * If keys from both sources are unset, return -EINVAL.
+ */
+int
+gve_update_rss_key(struct gve_priv *priv, struct gve_rss_config *gve_rss_conf,
+ struct rte_eth_rss_conf *rss_conf);
+
+/**
+ * Updates the RSS hash types stored in `gve_rss_conf`. It is prioritized as
+ * follows:
+ * 1) retrieve from `rss_conf`, if set
+ * 2) retrieve from priv, if RSS has been configured
+ * 3) set default RSS offload
+ */
+int
+gve_update_rss_hash_types(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf, struct rte_eth_rss_conf *rss_conf);
+
+/**
+ * Ensures that only supported RSS hash fields are set in `rte_rss_hf`.
+ */
+static inline int
+gve_validate_rss_hf(uint64_t rte_rss_hf) {
+ return rte_rss_hf & ~GVE_RTE_RSS_OFFLOAD_ALL;
+}
+
+/**
+ * Converts RSS hash types from RTE values to GVE values, storing them in
+ * `gve_rss_conf`.
+ */
+void
+rte_to_gve_rss_hf(uint64_t rte_rss_hf, struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Converts RSS hash types from GVE values to RTE values, storing them in
+ * `rss_conf`.
+ */
+void
+gve_to_rte_rss_hf(uint16_t gve_rss_types, struct rte_eth_rss_conf *rss_conf);
+
diff --git a/drivers/net/gve/meson.build b/drivers/net/gve/meson.build
index 61d195009c..6da13a8406 100644
--- a/drivers/net/gve/meson.build
+++ b/drivers/net/gve/meson.build
@@ -15,5 +15,6 @@ sources = files(
'gve_tx_dqo.c',
'gve_ethdev.c',
'gve_version.c',
+ 'gve_rss.c',
)
includes += include_directories('base')
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v3 4/7] net/gve: RSS configuration update support
2024-01-24 0:14 ` Joshua Washington
` (2 preceding siblings ...)
2024-01-24 0:14 ` [PATCH v3 3/7] net/gve: add gve_rss library for handling RSS-related behaviors Joshua Washington
@ 2024-01-24 0:14 ` Joshua Washington
2024-01-24 0:14 ` [PATCH v3 5/7] net/gve: RSS redirection table " Joshua Washington
` (3 subsequent siblings)
7 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:14 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch adds support for updating the RSS hash key and hash fields
in the GVE PMD through the implementation of rss_hash_update and
rss_hash_conf_get.
The RSS hash key for gVNIC is required to be 40 bytes. On initial
configuration of the RSS hash key, the RSS redirection table will be
set to a static default, using a round-robin approach for all queues.
Note, however, that this patch does not include support for setting the
redirection table explicitly. In dev_configure, if the static
redirection table has been set, it will be updated to reflect the new
queue count, if it has changed.
The RSS key must be set before any other RSS configuration can happen.
As such, an attempt to set the hash types before the key is configured
will fail.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/gve_ethdev.c | 132 ++++++++++++++++++++++++++++++++++-
drivers/net/gve/gve_ethdev.h | 3 +
2 files changed, 133 insertions(+), 2 deletions(-)
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 936ca22cb9..2a68d31808 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -1,5 +1,6 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(C) 2022 Intel Corporation
+ * Copyright(C) 2022-2023 Intel Corporation
+ * Copyright(C) 2023 Google LLC
*/
#include "gve_ethdev.h"
@@ -8,6 +9,7 @@
#include "base/gve_osdep.h"
#include "gve_version.h"
#include "rte_ether.h"
+#include "gve_rss.h"
static void
gve_write_version(uint8_t *driver_version_register)
@@ -88,12 +90,31 @@ gve_dev_configure(struct rte_eth_dev *dev)
{
struct gve_priv *priv = dev->data->dev_private;
- if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) {
dev->data->dev_conf.rxmode.offloads |= RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ priv->rss_config.alg = GVE_RSS_HASH_TOEPLITZ;
+ }
if (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO)
priv->enable_rsc = 1;
+ /* Reset RSS RETA in case number of queues changed. */
+ if (priv->rss_config.indir) {
+ struct gve_rss_config update_reta_config;
+ gve_init_rss_config_from_priv(priv, &update_reta_config);
+ gve_generate_rss_reta(dev, &update_reta_config);
+
+ int err = gve_adminq_configure_rss(priv, &update_reta_config);
+ if (err)
+ PMD_DRV_LOG(ERR,
+ "Could not reconfigure RSS redirection table.");
+ else
+ gve_update_priv_rss_config(priv, &update_reta_config);
+
+ gve_free_rss_config(&update_reta_config);
+ return err;
+ }
+
return 0;
}
@@ -443,6 +464,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
};
dev_info->flow_type_rss_offloads = GVE_RTE_RSS_OFFLOAD_ALL;
+ dev_info->hash_key_size = GVE_RSS_HASH_KEY_SIZE;
+ dev_info->reta_size = GVE_RSS_INDIR_SIZE;
return 0;
}
@@ -646,6 +669,107 @@ gve_xstats_get_names(struct rte_eth_dev *dev,
return count;
}
+
+static int
+gve_rss_hash_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ struct gve_rss_config gve_rss_conf;
+ int rss_reta_size;
+ int err;
+
+ if (gve_validate_rss_hf(rss_conf->rss_hf)) {
+ PMD_DRV_LOG(ERR, "Unsupported hash function.");
+ return -EINVAL;
+ }
+
+ if (rss_conf->algorithm != RTE_ETH_HASH_FUNCTION_TOEPLITZ &&
+ rss_conf->algorithm != RTE_ETH_HASH_FUNCTION_DEFAULT) {
+ PMD_DRV_LOG(ERR, "Device only supports Toeplitz algorithm.");
+ return -EINVAL;
+ }
+
+ if (rss_conf->rss_key_len) {
+ if (rss_conf->rss_key_len != GVE_RSS_HASH_KEY_SIZE) {
+ PMD_DRV_LOG(ERR,
+ "Invalid hash key size. Only RSS hash key size "
+ "of %u supported", GVE_RSS_HASH_KEY_SIZE);
+ return -EINVAL;
+ }
+
+ if (!rss_conf->rss_key) {
+ PMD_DRV_LOG(ERR, "RSS key must be non-null.");
+ return -EINVAL;
+ }
+ } else {
+ if (!priv->rss_config.key_size) {
+ PMD_DRV_LOG(ERR, "RSS key must be initialized before "
+ "any other configuration.");
+ return -EINVAL;
+ }
+ rss_conf->rss_key_len = priv->rss_config.key_size;
+ }
+
+ rss_reta_size = priv->rss_config.indir ?
+ priv->rss_config.indir_size :
+ GVE_RSS_INDIR_SIZE;
+ err = gve_init_rss_config(&gve_rss_conf, rss_conf->rss_key_len,
+ rss_reta_size);
+ if (err)
+ return err;
+
+ gve_rss_conf.alg = GVE_RSS_HASH_TOEPLITZ;
+ err = gve_update_rss_hash_types(priv, &gve_rss_conf, rss_conf);
+ if (err)
+ goto err;
+ err = gve_update_rss_key(priv, &gve_rss_conf, rss_conf);
+ if (err)
+ goto err;
+
+ /* Set redirection table to default or preexisting. */
+ if (!priv->rss_config.indir)
+ gve_generate_rss_reta(dev, &gve_rss_conf);
+ else
+ memcpy(gve_rss_conf.indir, priv->rss_config.indir,
+ gve_rss_conf.indir_size * sizeof(*priv->rss_config.indir));
+
+ err = gve_adminq_configure_rss(priv, &gve_rss_conf);
+ if (!err)
+ gve_update_priv_rss_config(priv, &gve_rss_conf);
+
+err:
+ gve_free_rss_config(&gve_rss_conf);
+ return err;
+}
+
+static int
+gve_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+
+ if (!(dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)) {
+ PMD_DRV_LOG(ERR, "RSS not configured.");
+ return -ENOTSUP;
+ }
+
+
+ gve_to_rte_rss_hf(priv->rss_config.hash_types, rss_conf);
+ rss_conf->rss_key_len = priv->rss_config.key_size;
+ if (rss_conf->rss_key) {
+ if (!priv->rss_config.key) {
+ PMD_DRV_LOG(ERR, "Unable to retrieve default RSS hash key.");
+ return -ENOTSUP;
+ }
+ memcpy(rss_conf->rss_key, priv->rss_config.key,
+ rss_conf->rss_key_len * sizeof(*rss_conf->rss_key));
+ }
+
+ return 0;
+}
+
static const struct eth_dev_ops gve_eth_dev_ops = {
.dev_configure = gve_dev_configure,
.dev_start = gve_dev_start,
@@ -666,6 +790,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = {
.mtu_set = gve_dev_mtu_set,
.xstats_get = gve_xstats_get,
.xstats_get_names = gve_xstats_get_names,
+ .rss_hash_update = gve_rss_hash_update,
+ .rss_hash_conf_get = gve_rss_hash_conf_get,
};
static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -688,6 +814,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
.mtu_set = gve_dev_mtu_set,
.xstats_get = gve_xstats_get,
.xstats_get_names = gve_xstats_get_names,
+ .rss_hash_update = gve_rss_hash_update,
+ .rss_hash_conf_get = gve_rss_hash_conf_get,
};
static void
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index bc486cb941..d713657d10 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -29,6 +29,9 @@
#define GVE_RX_MIN_BUF_SIZE_GQI 2048
#define GVE_RX_MAX_BUF_SIZE_GQI 4096
+#define GVE_RSS_HASH_KEY_SIZE 40
+#define GVE_RSS_INDIR_SIZE 128
+
#define GVE_TX_CKSUM_OFFLOAD_MASK ( \
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v3 5/7] net/gve: RSS redirection table update support
2024-01-24 0:14 ` Joshua Washington
` (3 preceding siblings ...)
2024-01-24 0:14 ` [PATCH v3 4/7] net/gve: RSS configuration update support Joshua Washington
@ 2024-01-24 0:14 ` Joshua Washington
2024-01-24 0:14 ` [PATCH v3 6/7] net/gve: update gve.ini with RSS capabilities Joshua Washington
` (2 subsequent siblings)
7 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:14 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch introduces support for updating the RSS redirection table in
the GVE PMD through the implementation of rss_reta_update and
rss_reta_query.
Due to an infrastructure limitation, the RSS hash key must be manually
configured before the redirection table can be updated or queried. The
redirection table is expected to be exactly 128 bytes.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/gve_ethdev.c | 95 ++++++++++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 2a68d31808..3b8ec5872d 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -770,6 +770,97 @@ gve_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
}
+static int
+gve_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ struct gve_rss_config gve_rss_conf;
+ int table_id;
+ int err;
+ int i;
+
+ /* RSS key must be set before the redirection table can be set. */
+ if (!priv->rss_config.key || priv->rss_config.key_size == 0) {
+ PMD_DRV_LOG(ERR, "RSS hash key msut be set before the "
+ "redirection table can be updated.");
+ return -ENOTSUP;
+ }
+
+ if (reta_size != GVE_RSS_INDIR_SIZE) {
+ PMD_DRV_LOG(ERR, "Redirection table must have %hu elements",
+ (uint16_t)GVE_RSS_INDIR_SIZE);
+ return -EINVAL;
+ }
+
+ err = gve_init_rss_config_from_priv(priv, &gve_rss_conf);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Error allocating new RSS config.");
+ return err;
+ }
+
+ table_id = 0;
+ for (i = 0; i < priv->rss_config.indir_size; i++) {
+ int table_entry = i % RTE_ETH_RETA_GROUP_SIZE;
+ if (reta_conf[table_id].mask & (1ULL << table_entry))
+ gve_rss_conf.indir[i] =
+ reta_conf[table_id].reta[table_entry];
+
+ if (table_entry == RTE_ETH_RETA_GROUP_SIZE - 1)
+ table_id++;
+ }
+
+ err = gve_adminq_configure_rss(priv, &gve_rss_conf);
+ if (err)
+ PMD_DRV_LOG(ERR, "Problem configuring RSS with device.");
+ else
+ gve_update_priv_rss_config(priv, &gve_rss_conf);
+
+ gve_free_rss_config(&gve_rss_conf);
+ return err;
+}
+
+static int
+gve_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ int table_id;
+ int i;
+
+ if (!(dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)) {
+ PMD_DRV_LOG(ERR, "RSS not configured.");
+ return -ENOTSUP;
+ }
+
+ /* RSS key must be set before the redirection table can be queried. */
+ if (!priv->rss_config.key) {
+ PMD_DRV_LOG(ERR, "RSS hash key must be set before the "
+ "redirection table can be initialized.");
+ return -ENOTSUP;
+ }
+
+ if (reta_size != priv->rss_config.indir_size) {
+ PMD_DRV_LOG(ERR, "RSS redirection table must have %d entries.",
+ priv->rss_config.indir_size);
+ return -EINVAL;
+ }
+
+ table_id = 0;
+ for (i = 0; i < priv->rss_config.indir_size; i++) {
+ int table_entry = i % RTE_ETH_RETA_GROUP_SIZE;
+ if (reta_conf[table_id].mask & (1ULL << table_entry))
+ reta_conf[table_id].reta[table_entry] =
+ priv->rss_config.indir[i];
+
+ if (table_entry == RTE_ETH_RETA_GROUP_SIZE - 1)
+ table_id++;
+ }
+
+ return 0;
+}
+
static const struct eth_dev_ops gve_eth_dev_ops = {
.dev_configure = gve_dev_configure,
.dev_start = gve_dev_start,
@@ -792,6 +883,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = {
.xstats_get_names = gve_xstats_get_names,
.rss_hash_update = gve_rss_hash_update,
.rss_hash_conf_get = gve_rss_hash_conf_get,
+ .reta_update = gve_rss_reta_update,
+ .reta_query = gve_rss_reta_query,
};
static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -816,6 +909,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
.xstats_get_names = gve_xstats_get_names,
.rss_hash_update = gve_rss_hash_update,
.rss_hash_conf_get = gve_rss_hash_conf_get,
+ .reta_update = gve_rss_reta_update,
+ .reta_query = gve_rss_reta_query,
};
static void
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v3 6/7] net/gve: update gve.ini with RSS capabilities
2024-01-24 0:14 ` Joshua Washington
` (4 preceding siblings ...)
2024-01-24 0:14 ` [PATCH v3 5/7] net/gve: RSS redirection table " Joshua Washington
@ 2024-01-24 0:14 ` Joshua Washington
2024-01-24 0:15 ` [PATCH v3 7/7] net/gve: update GVE documentation with RSS support Joshua Washington
[not found] ` <20240126173317.2779230-1-joshwash@google.com>
7 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:14 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch updates the DPDK feature matrix to expose that the GVE driver
supports RSS hash, RSS key update, and RSS reta update.
Signed-off-by: Joshua Washington <joshwash@google.com>
---
doc/guides/nics/features/gve.ini | 3 +++
1 file changed, 3 insertions(+)
diff --git a/doc/guides/nics/features/gve.ini b/doc/guides/nics/features/gve.ini
index 838edd456a..bc08648dbc 100644
--- a/doc/guides/nics/features/gve.ini
+++ b/doc/guides/nics/features/gve.ini
@@ -15,3 +15,6 @@ Linux = Y
x86-32 = Y
x86-64 = Y
Usage doc = Y
+RSS hash = Y
+RSS key update = Y
+RSS reta update = Y
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v3 7/7] net/gve: update GVE documentation with RSS support
2024-01-24 0:14 ` Joshua Washington
` (5 preceding siblings ...)
2024-01-24 0:14 ` [PATCH v3 6/7] net/gve: update gve.ini with RSS capabilities Joshua Washington
@ 2024-01-24 0:15 ` Joshua Washington
[not found] ` <20240126173317.2779230-1-joshwash@google.com>
7 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-24 0:15 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch updates the GVE doc page to communicate that GVE now supports
RSS configuration and explains the limitations.
Signed-off-by: Joshua Washington <joshwash@google.com>
---
doc/guides/nics/gve.rst | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/doc/guides/nics/gve.rst b/doc/guides/nics/gve.rst
index 1c3eaf03ef..80991e70cf 100644
--- a/doc/guides/nics/gve.rst
+++ b/doc/guides/nics/gve.rst
@@ -70,6 +70,8 @@ Supported features of the GVE PMD are:
- Link state information
- Tx multi-segments (Scatter Tx)
- Tx UDP/TCP/SCTP Checksum
+- RSS hash configuration
+- RSS redirection table update and query
Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD.
Jumbo Frame is not supported in PMD for now.
@@ -77,10 +79,12 @@ It'll be added in a future DPDK release.
Also, only GQI_QPL queue format is in use on GCP
since GQI_RDA hasn't been released in production.
-Currently, setting MTU with value larger than 1460 is not supported.
+RSS
+^^^
-Currently, only "RSS hash" is force enabled
-so that the backend hardware device calculated hash values
-could be shared with applications.
-But for RSS, there is no such API to config RSS hash function or RETA table.
-So, limited RSS is supported only with default config/setting.
+GVE RSS can be enabled and configured using the standard interfaces. The driver
+does not support querying the initial RSS configuration.
+
+The RSS hash key must be exactly 40 bytes, and the redirection table must have
+128 entries. The RSS hash key must be configured before the redirection table
+can be set up.
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 1/7] net/gve: fully expose RSS offload support in dev_info
[not found] ` <20240126173317.2779230-1-joshwash@google.com>
@ 2024-01-26 17:33 ` Joshua Washington
2024-01-31 22:13 ` [PATCH v5 0/5] net/gve: RSS Support for GVE Driver Joshua Washington
2024-01-26 17:33 ` [PATCH v4 2/7] net/gve: RSS adminq command changes Joshua Washington
` (6 subsequent siblings)
7 siblings, 1 reply; 44+ messages in thread
From: Joshua Washington @ 2024-01-26 17:33 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch communicates that the GVE driver supports RSS, along with
the RSS offloads supported by the driver.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/gve_ethdev.c | 4 +++-
drivers/net/gve/gve_ethdev.h | 8 ++++++++
2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index d162fd3864..6acdb4e13b 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -405,7 +405,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = priv->max_mtu;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
- dev_info->rx_offload_capa = 0;
+ dev_info->rx_offload_capa = RTE_ETH_RX_OFFLOAD_RSS_HASH;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
@@ -442,6 +442,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.nb_align = 1,
};
+ dev_info->flow_type_rss_offloads = GVE_RSS_OFFLOAD_ALL;
+
return 0;
}
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 9893fcfee6..14c72ec91a 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -33,6 +33,14 @@
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
+#define GVE_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_IPV6_EX | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_IPV6_TCP_EX)
+
/* A list of pages registered with the device during setup and used by a queue
* as buffers
*/
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 2/7] net/gve: RSS adminq command changes
[not found] ` <20240126173317.2779230-1-joshwash@google.com>
2024-01-26 17:33 ` [PATCH v4 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
@ 2024-01-26 17:33 ` Joshua Washington
2024-01-26 17:33 ` [PATCH v4 3/7] net/gve: add gve_rss library for handling RSS-related behaviors Joshua Washington
` (5 subsequent siblings)
7 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-26 17:33 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This change introduces admin queue changes that enable the configuration
of RSS parameters for the GVE driver.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/base/gve.h | 15 ++++++++
drivers/net/gve/base/gve_adminq.c | 58 +++++++++++++++++++++++++++++++
drivers/net/gve/base/gve_adminq.h | 29 ++++++++++++++++
drivers/net/gve/gve_ethdev.h | 6 +++-
4 files changed, 107 insertions(+), 1 deletion(-)
diff --git a/drivers/net/gve/base/gve.h b/drivers/net/gve/base/gve.h
index f7b297e759..9c58fc4238 100644
--- a/drivers/net/gve/base/gve.h
+++ b/drivers/net/gve/base/gve.h
@@ -51,4 +51,19 @@ enum gve_state_flags_bit {
GVE_PRIV_FLAGS_NAPI_ENABLED = 4,
};
+enum gve_rss_hash_algorithm {
+ GVE_RSS_HASH_UNDEFINED = 0,
+ GVE_RSS_HASH_TOEPLITZ = 1,
+};
+
+struct gve_rss_config {
+ uint16_t hash_types;
+ enum gve_rss_hash_algorithm alg;
+ uint16_t key_size;
+ uint16_t indir_size;
+ uint8_t *key;
+ uint32_t *indir;
+};
+
+
#endif /* _GVE_H_ */
diff --git a/drivers/net/gve/base/gve_adminq.c b/drivers/net/gve/base/gve_adminq.c
index 343bd13d67..629d15cfbe 100644
--- a/drivers/net/gve/base/gve_adminq.c
+++ b/drivers/net/gve/base/gve_adminq.c
@@ -389,6 +389,9 @@ static int gve_adminq_issue_cmd(struct gve_priv *priv,
case GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES:
priv->adminq_dcfg_device_resources_cnt++;
break;
+ case GVE_ADMINQ_CONFIGURE_RSS:
+ priv->adminq_cfg_rss_cnt++;
+ break;
case GVE_ADMINQ_SET_DRIVER_PARAMETER:
priv->adminq_set_driver_parameter_cnt++;
break;
@@ -938,3 +941,58 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,
gve_free_dma_mem(&ptype_map_dma_mem);
return err;
}
+
+int gve_adminq_configure_rss(struct gve_priv *priv,
+ struct gve_rss_config *rss_config)
+{
+ struct gve_dma_mem indirection_table_dma_mem;
+ struct gve_dma_mem rss_key_dma_mem;
+ union gve_adminq_command cmd;
+ __be32 *indir = NULL;
+ u8 *key = NULL;
+ int err = 0;
+ int i;
+
+ if (!rss_config->indir_size || !rss_config->key_size)
+ return -EINVAL;
+
+ indir = gve_alloc_dma_mem(&indirection_table_dma_mem,
+ rss_config->indir_size *
+ sizeof(*rss_config->indir));
+ if (!indir) {
+ err = -ENOMEM;
+ goto out;
+ }
+ for (i = 0; i < rss_config->indir_size; i++)
+ indir[i] = cpu_to_be32(rss_config->indir[i]);
+
+ key = gve_alloc_dma_mem(&rss_key_dma_mem,
+ rss_config->key_size *
+ sizeof(*rss_config->key));
+ if (!key) {
+ err = -ENOMEM;
+ goto out;
+ }
+ memcpy(key, rss_config->key, rss_config->key_size);
+
+ memset(&cmd, 0, sizeof(cmd));
+ cmd.opcode = cpu_to_be32(GVE_ADMINQ_CONFIGURE_RSS);
+ cmd.configure_rss = (struct gve_adminq_configure_rss) {
+ .hash_types = cpu_to_be16(rss_config->hash_types),
+ .halg = rss_config->alg,
+ .hkey_len = cpu_to_be16(rss_config->key_size),
+ .indir_len = cpu_to_be16(rss_config->indir_size),
+ .hkey_addr = cpu_to_be64(rss_key_dma_mem.pa),
+ .indir_addr = cpu_to_be64(indirection_table_dma_mem.pa),
+ };
+
+ err = gve_adminq_execute_cmd(priv, &cmd);
+
+out:
+ if (indir)
+ gve_free_dma_mem(&indirection_table_dma_mem);
+ if (key)
+ gve_free_dma_mem(&rss_key_dma_mem);
+ return err;
+}
+
diff --git a/drivers/net/gve/base/gve_adminq.h b/drivers/net/gve/base/gve_adminq.h
index f05362f85f..95f4960561 100644
--- a/drivers/net/gve/base/gve_adminq.h
+++ b/drivers/net/gve/base/gve_adminq.h
@@ -19,6 +19,7 @@ enum gve_adminq_opcodes {
GVE_ADMINQ_DESTROY_TX_QUEUE = 0x7,
GVE_ADMINQ_DESTROY_RX_QUEUE = 0x8,
GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES = 0x9,
+ GVE_ADMINQ_CONFIGURE_RSS = 0xA,
GVE_ADMINQ_SET_DRIVER_PARAMETER = 0xB,
GVE_ADMINQ_REPORT_STATS = 0xC,
GVE_ADMINQ_REPORT_LINK_SPEED = 0xD,
@@ -377,6 +378,27 @@ struct gve_adminq_get_ptype_map {
__be64 ptype_map_addr;
};
+#define GVE_RSS_HASH_IPV4 BIT(0)
+#define GVE_RSS_HASH_TCPV4 BIT(1)
+#define GVE_RSS_HASH_IPV6 BIT(2)
+#define GVE_RSS_HASH_IPV6_EX BIT(3)
+#define GVE_RSS_HASH_TCPV6 BIT(4)
+#define GVE_RSS_HASH_TCPV6_EX BIT(5)
+#define GVE_RSS_HASH_UDPV4 BIT(6)
+#define GVE_RSS_HASH_UDPV6 BIT(7)
+#define GVE_RSS_HASH_UDPV6_EX BIT(8)
+
+/* RSS configuration command */
+struct gve_adminq_configure_rss {
+ __be16 hash_types;
+ u8 halg; /* hash algorithm */
+ u8 reserved;
+ __be16 hkey_len;
+ __be16 indir_len;
+ __be64 hkey_addr;
+ __be64 indir_addr;
+};
+
union gve_adminq_command {
struct {
__be32 opcode;
@@ -391,6 +413,7 @@ union gve_adminq_command {
struct gve_adminq_describe_device describe_device;
struct gve_adminq_register_page_list reg_page_list;
struct gve_adminq_unregister_page_list unreg_page_list;
+ struct gve_adminq_configure_rss configure_rss;
struct gve_adminq_set_driver_parameter set_driver_param;
struct gve_adminq_report_stats report_stats;
struct gve_adminq_report_link_speed report_link_speed;
@@ -404,6 +427,8 @@ union gve_adminq_command {
GVE_CHECK_UNION_LEN(64, gve_adminq_command);
+struct gve_priv;
+struct gve_queue_page_list;
int gve_adminq_alloc(struct gve_priv *priv);
void gve_adminq_free(struct gve_priv *priv);
void gve_adminq_release(struct gve_priv *priv);
@@ -433,4 +458,8 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,
int gve_adminq_verify_driver_compatibility(struct gve_priv *priv,
u64 driver_info_len,
dma_addr_t driver_info_addr);
+
+int gve_adminq_configure_rss(struct gve_priv *priv,
+ struct gve_rss_config *rss_config);
+
#endif /* _GVE_ADMINQ_H */
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 14c72ec91a..f7635e829c 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -39,7 +39,10 @@
RTE_ETH_RSS_IPV6 | \
RTE_ETH_RSS_IPV6_EX | \
RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
- RTE_ETH_RSS_IPV6_TCP_EX)
+ RTE_ETH_RSS_IPV6_TCP_EX | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_IPV6_UDP_EX)
/* A list of pages registered with the device during setup and used by a queue
* as buffers
@@ -264,6 +267,7 @@ struct gve_priv {
uint32_t adminq_destroy_tx_queue_cnt;
uint32_t adminq_destroy_rx_queue_cnt;
uint32_t adminq_dcfg_device_resources_cnt;
+ uint32_t adminq_cfg_rss_cnt;
uint32_t adminq_set_driver_parameter_cnt;
uint32_t adminq_report_stats_cnt;
uint32_t adminq_report_link_speed_cnt;
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 3/7] net/gve: add gve_rss library for handling RSS-related behaviors
[not found] ` <20240126173317.2779230-1-joshwash@google.com>
2024-01-26 17:33 ` [PATCH v4 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
2024-01-26 17:33 ` [PATCH v4 2/7] net/gve: RSS adminq command changes Joshua Washington
@ 2024-01-26 17:33 ` Joshua Washington
2024-01-26 17:33 ` [PATCH v4 4/7] net/gve: RSS configuration update support Joshua Washington
` (4 subsequent siblings)
7 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-26 17:33 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This change includes a number of helper functions to facilitate RSS
configuration on the GVE DPDK driver. These methods are declared in
gve_rss.h.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/base/gve_adminq.h | 10 +-
drivers/net/gve/gve_ethdev.c | 2 +-
drivers/net/gve/gve_ethdev.h | 4 +-
| 206 ++++++++++++++++++++++++++++++
| 107 ++++++++++++++++
drivers/net/gve/meson.build | 1 +
6 files changed, 319 insertions(+), 11 deletions(-)
create mode 100644 drivers/net/gve/gve_rss.c
create mode 100644 drivers/net/gve/gve_rss.h
diff --git a/drivers/net/gve/base/gve_adminq.h b/drivers/net/gve/base/gve_adminq.h
index 95f4960561..24abd945cc 100644
--- a/drivers/net/gve/base/gve_adminq.h
+++ b/drivers/net/gve/base/gve_adminq.h
@@ -378,15 +378,6 @@ struct gve_adminq_get_ptype_map {
__be64 ptype_map_addr;
};
-#define GVE_RSS_HASH_IPV4 BIT(0)
-#define GVE_RSS_HASH_TCPV4 BIT(1)
-#define GVE_RSS_HASH_IPV6 BIT(2)
-#define GVE_RSS_HASH_IPV6_EX BIT(3)
-#define GVE_RSS_HASH_TCPV6 BIT(4)
-#define GVE_RSS_HASH_TCPV6_EX BIT(5)
-#define GVE_RSS_HASH_UDPV4 BIT(6)
-#define GVE_RSS_HASH_UDPV6 BIT(7)
-#define GVE_RSS_HASH_UDPV6_EX BIT(8)
/* RSS configuration command */
struct gve_adminq_configure_rss {
@@ -428,6 +419,7 @@ union gve_adminq_command {
GVE_CHECK_UNION_LEN(64, gve_adminq_command);
struct gve_priv;
+struct gve_rss_config;
struct gve_queue_page_list;
int gve_adminq_alloc(struct gve_priv *priv);
void gve_adminq_free(struct gve_priv *priv);
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 6acdb4e13b..936ca22cb9 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -442,7 +442,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.nb_align = 1,
};
- dev_info->flow_type_rss_offloads = GVE_RSS_OFFLOAD_ALL;
+ dev_info->flow_type_rss_offloads = GVE_RTE_RSS_OFFLOAD_ALL;
return 0;
}
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index f7635e829c..bc486cb941 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -33,7 +33,7 @@
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
-#define GVE_RSS_OFFLOAD_ALL ( \
+#define GVE_RTE_RSS_OFFLOAD_ALL ( \
RTE_ETH_RSS_IPV4 | \
RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
RTE_ETH_RSS_IPV6 | \
@@ -290,6 +290,8 @@ struct gve_priv {
const struct rte_memzone *stats_report_mem;
uint16_t stats_start_idx; /* start index of array of stats written by NIC */
uint16_t stats_end_idx; /* end index of array of stats written by NIC */
+
+ struct gve_rss_config rss_config;
};
static inline bool
--git a/drivers/net/gve/gve_rss.c b/drivers/net/gve/gve_rss.c
new file mode 100644
index 0000000000..931180f8f2
--- /dev/null
+++ b/drivers/net/gve/gve_rss.c
@@ -0,0 +1,206 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Google LLC
+ */
+
+#include "gve_rss.h"
+
+int
+gve_generate_rss_reta(struct rte_eth_dev *dev, struct gve_rss_config *config)
+{
+ int i;
+ if (!config || !config->indir)
+ return -EINVAL;
+
+ for (i = 0; i < config->indir_size; i++)
+ config->indir[i] = i % dev->data->nb_rx_queues;
+
+ return 0;
+}
+
+
+int
+gve_init_rss_config(struct gve_rss_config *gve_rss_conf,
+ uint16_t key_size, uint16_t indir_size)
+{
+ int err;
+
+ gve_rss_conf->alg = GVE_RSS_HASH_TOEPLITZ;
+
+ gve_rss_conf->key_size = key_size;
+ gve_rss_conf->key = rte_zmalloc("rss key",
+ key_size * sizeof(*gve_rss_conf->key),
+ RTE_CACHE_LINE_SIZE);
+ if (!gve_rss_conf->key)
+ return -ENOMEM;
+
+ gve_rss_conf->indir_size = indir_size;
+ gve_rss_conf->indir = rte_zmalloc("rss reta",
+ indir_size * sizeof(*gve_rss_conf->indir),
+ RTE_CACHE_LINE_SIZE);
+ if (!gve_rss_conf->indir) {
+ err = -ENOMEM;
+ goto err_with_key;
+ }
+
+ return 0;
+err_with_key:
+ rte_free(gve_rss_conf->key);
+ return err;
+}
+
+int
+gve_init_rss_config_from_priv(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf)
+{
+ int err = gve_init_rss_config(gve_rss_conf, priv->rss_config.key_size,
+ priv->rss_config.indir_size);
+ if (err)
+ return err;
+
+ gve_rss_conf->hash_types = priv->rss_config.hash_types;
+ gve_rss_conf->alg = priv->rss_config.alg;
+ memcpy(gve_rss_conf->key, priv->rss_config.key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ memcpy(gve_rss_conf->indir, priv->rss_config.indir,
+ gve_rss_conf->indir_size * sizeof(*gve_rss_conf->indir));
+
+ return 0;
+}
+
+void
+gve_free_rss_config(struct gve_rss_config *gve_rss_conf)
+{
+ rte_free(gve_rss_conf->indir);
+ gve_rss_conf->indir = NULL;
+ rte_free(gve_rss_conf->key);
+ gve_rss_conf->key = NULL;
+}
+
+int
+gve_update_priv_rss_config(struct gve_priv *priv, struct gve_rss_config *config)
+{
+ struct gve_rss_config *priv_config = &priv->rss_config;
+ int key_bytes, indir_bytes;
+
+ if (!config)
+ return -EINVAL;
+ if (config->key_size == 0 || !config->key)
+ return -EINVAL;
+ if (config->indir_size == 0 || !config->indir)
+ return -EINVAL;
+
+ priv_config->hash_types = config->hash_types;
+ priv_config->alg = config->alg;
+
+ priv_config->key_size = config->key_size;
+ key_bytes = priv_config->key_size * sizeof(*priv_config->key);
+ if (!priv_config->key)
+ priv_config->key = rte_zmalloc("priv rss key", key_bytes,
+ RTE_CACHE_LINE_SIZE);
+ else
+ priv_config->key = rte_realloc(priv_config->key, key_bytes,
+ RTE_CACHE_LINE_SIZE);
+ if (!priv_config->key)
+ return -ENOMEM;
+
+ priv_config->indir_size = config->indir_size;
+ indir_bytes = priv_config->indir_size * sizeof(*priv_config->indir);
+ if (!priv_config->indir)
+ priv_config->indir = rte_zmalloc("priv rss reta", indir_bytes,
+ RTE_CACHE_LINE_SIZE);
+ else
+ priv_config->indir = rte_realloc(priv_config->indir,
+ indir_bytes, RTE_CACHE_LINE_SIZE);
+
+ if (!priv_config->indir)
+ return -ENOMEM;
+
+ memcpy(priv_config->indir, config->indir,
+ config->indir_size * sizeof(*priv_config->indir));
+ memcpy(priv_config->key, config->key,
+ config->key_size * sizeof(*priv_config->key));
+
+ return 0;
+}
+
+int
+gve_update_rss_key(struct gve_priv *priv, struct gve_rss_config *gve_rss_conf,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ if (rss_conf->rss_key_len && rss_conf->rss_key) {
+ gve_rss_conf->key_size = rss_conf->rss_key_len;
+ memcpy(gve_rss_conf->key, rss_conf->rss_key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ } else if (priv->rss_config.key_size && priv->rss_config.key) {
+ gve_rss_conf->key_size = priv->rss_config.key_size;
+ memcpy(gve_rss_conf->key, priv->rss_config.key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ } else {
+ PMD_DRV_LOG(ERR, "RSS key must be set as part of initial RSS "
+ "configuration.");
+ return -EINVAL;
+ }
+ return 0;
+}
+
+int
+gve_update_rss_hash_types(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf, struct rte_eth_rss_conf *rss_conf)
+{
+ /* Initialize to 0 before modifying. */
+ gve_rss_conf->hash_types = 0;
+ if (rss_conf->rss_hf)
+ rte_to_gve_rss_hf(rss_conf->rss_hf, gve_rss_conf);
+ else if (priv->rss_config.key_size && priv->rss_config.key)
+ gve_rss_conf->hash_types = priv->rss_config.hash_types;
+ else
+ gve_rss_conf->hash_types = GVE_RSS_OFFLOAD_DEFAULT;
+ return 0;
+}
+
+void
+rte_to_gve_rss_hf(uint64_t rte_rss_hf, struct gve_rss_config *gve_rss_conf)
+{
+ if (rte_rss_hf & RTE_ETH_RSS_IPV4)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_TCP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV6_EX;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_TCP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_TCP_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV6_EX;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_UDP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_UDP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_UDP_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV6_EX;
+}
+
+void
+gve_to_rte_rss_hf(uint16_t gve_rss_types, struct rte_eth_rss_conf *rss_conf)
+{
+ if (gve_rss_types & GVE_RSS_HASH_IPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV4;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV4_TCP;
+ if (gve_rss_types & GVE_RSS_HASH_IPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6;
+ if (gve_rss_types & GVE_RSS_HASH_IPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_EX;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV6_TCP;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_TCP_EX;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV4_UDP;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV6_UDP;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_UDP_EX;
+}
+
--git a/drivers/net/gve/gve_rss.h b/drivers/net/gve/gve_rss.h
new file mode 100644
index 0000000000..0cc235a19a
--- /dev/null
+++ b/drivers/net/gve/gve_rss.h
@@ -0,0 +1,107 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Google LLC
+ */
+
+#include "gve_adminq.h"
+#include "gve_ethdev.h"
+
+#define GVE_RSS_HASH_IPV4 BIT(0)
+#define GVE_RSS_HASH_TCPV4 BIT(1)
+#define GVE_RSS_HASH_IPV6 BIT(2)
+#define GVE_RSS_HASH_IPV6_EX BIT(3)
+#define GVE_RSS_HASH_TCPV6 BIT(4)
+#define GVE_RSS_HASH_TCPV6_EX BIT(5)
+#define GVE_RSS_HASH_UDPV4 BIT(6)
+#define GVE_RSS_HASH_UDPV6 BIT(7)
+#define GVE_RSS_HASH_UDPV6_EX BIT(8)
+
+#define GVE_RSS_OFFLOAD_DEFAULT ( \
+ GVE_RSS_HASH_IPV4 | \
+ GVE_RSS_HASH_TCPV4 | \
+ GVE_RSS_HASH_IPV6 | \
+ GVE_RSS_HASH_IPV6_EX | \
+ GVE_RSS_HASH_TCPV6 | \
+ GVE_RSS_HASH_TCPV6_EX | \
+ GVE_RSS_HASH_UDPV4 | \
+ GVE_RSS_HASH_UDPV6 | \
+ GVE_RSS_HASH_UDPV6_EX)
+
+/**
+ * Generates default RSS redirection table based on the number of queues the
+ * device is configured with. This assigns hash values to queues in a
+ * round-robin manner.
+ */
+int
+gve_generate_rss_reta(struct rte_eth_dev *dev, struct gve_rss_config *config);
+
+/**
+ * Initializes `gve_rss_conf`, setting the fields to default values and
+ * allocating memory for the RSS key and redirection table.
+ */
+int
+gve_init_rss_config(struct gve_rss_config *gve_rss_conf,
+ uint16_t key_size, uint16_t indir_size);
+
+/**
+ * Initializes `gve_rss_conf` based on the RSS configuration stored in `priv`.
+ */
+int
+gve_init_rss_config_from_priv(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Frees RSS key and redriection table pointers stored in `gve_rss_conf`.
+ */
+void
+gve_free_rss_config(struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Updates the rss_config stored in `priv` with the contents of `config`.
+ */
+int
+gve_update_priv_rss_config(struct gve_priv *priv,
+ struct gve_rss_config *config);
+
+/**
+ * Updates the RSS key stored in `gve_rss_conf`. It is prioritized as follows:
+ * 1) retrieve from `rss_conf`, if non-null
+ * 2) retrieve from `priv`, if non-null
+ * If keys from both sources are unset, return -EINVAL.
+ */
+int
+gve_update_rss_key(struct gve_priv *priv, struct gve_rss_config *gve_rss_conf,
+ struct rte_eth_rss_conf *rss_conf);
+
+/**
+ * Updates the RSS hash types stored in `gve_rss_conf`. It is prioritized as
+ * follows:
+ * 1) retrieve from `rss_conf`, if set
+ * 2) retrieve from priv, if RSS has been configured
+ * 3) set default RSS offload
+ */
+int
+gve_update_rss_hash_types(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf, struct rte_eth_rss_conf *rss_conf);
+
+/**
+ * Ensures that only supported RSS hash fields are set in `rte_rss_hf`.
+ */
+static inline int
+gve_validate_rss_hf(uint64_t rte_rss_hf) {
+ return rte_rss_hf & ~GVE_RTE_RSS_OFFLOAD_ALL;
+}
+
+/**
+ * Converts RSS hash types from RTE values to GVE values, storing them in
+ * `gve_rss_conf`.
+ */
+void
+rte_to_gve_rss_hf(uint64_t rte_rss_hf, struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Converts RSS hash types from GVE values to RTE values, storing them in
+ * `rss_conf`.
+ */
+void
+gve_to_rte_rss_hf(uint16_t gve_rss_types, struct rte_eth_rss_conf *rss_conf);
+
diff --git a/drivers/net/gve/meson.build b/drivers/net/gve/meson.build
index 61d195009c..6da13a8406 100644
--- a/drivers/net/gve/meson.build
+++ b/drivers/net/gve/meson.build
@@ -15,5 +15,6 @@ sources = files(
'gve_tx_dqo.c',
'gve_ethdev.c',
'gve_version.c',
+ 'gve_rss.c',
)
includes += include_directories('base')
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 4/7] net/gve: RSS configuration update support
[not found] ` <20240126173317.2779230-1-joshwash@google.com>
` (2 preceding siblings ...)
2024-01-26 17:33 ` [PATCH v4 3/7] net/gve: add gve_rss library for handling RSS-related behaviors Joshua Washington
@ 2024-01-26 17:33 ` Joshua Washington
2024-01-26 17:33 ` [PATCH v4 5/7] net/gve: RSS redirection table " Joshua Washington
` (3 subsequent siblings)
7 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-26 17:33 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch adds support for updating the RSS hash key and hash fields
in the GVE PMD through the implementation of rss_hash_update and
rss_hash_conf_get.
The RSS hash key for gVNIC is required to be 40 bytes. On initial
configuration of the RSS hash key, the RSS redirection table will be
set to a static default, using a round-robin approach for all queues.
Note, however, that this patch does not include support for setting the
redirection table explicitly. In dev_configure, if the static
redirection table has been set, it will be updated to reflect the new
queue count, if it has changed.
The RSS key must be set before any other RSS configuration can happen.
As such, an attempt to set the hash types before the key is configured
will fail.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/gve_ethdev.c | 132 ++++++++++++++++++++++++++++++++++-
drivers/net/gve/gve_ethdev.h | 3 +
2 files changed, 133 insertions(+), 2 deletions(-)
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 936ca22cb9..2a68d31808 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -1,5 +1,6 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(C) 2022 Intel Corporation
+ * Copyright(C) 2022-2023 Intel Corporation
+ * Copyright(C) 2023 Google LLC
*/
#include "gve_ethdev.h"
@@ -8,6 +9,7 @@
#include "base/gve_osdep.h"
#include "gve_version.h"
#include "rte_ether.h"
+#include "gve_rss.h"
static void
gve_write_version(uint8_t *driver_version_register)
@@ -88,12 +90,31 @@ gve_dev_configure(struct rte_eth_dev *dev)
{
struct gve_priv *priv = dev->data->dev_private;
- if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) {
dev->data->dev_conf.rxmode.offloads |= RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ priv->rss_config.alg = GVE_RSS_HASH_TOEPLITZ;
+ }
if (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO)
priv->enable_rsc = 1;
+ /* Reset RSS RETA in case number of queues changed. */
+ if (priv->rss_config.indir) {
+ struct gve_rss_config update_reta_config;
+ gve_init_rss_config_from_priv(priv, &update_reta_config);
+ gve_generate_rss_reta(dev, &update_reta_config);
+
+ int err = gve_adminq_configure_rss(priv, &update_reta_config);
+ if (err)
+ PMD_DRV_LOG(ERR,
+ "Could not reconfigure RSS redirection table.");
+ else
+ gve_update_priv_rss_config(priv, &update_reta_config);
+
+ gve_free_rss_config(&update_reta_config);
+ return err;
+ }
+
return 0;
}
@@ -443,6 +464,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
};
dev_info->flow_type_rss_offloads = GVE_RTE_RSS_OFFLOAD_ALL;
+ dev_info->hash_key_size = GVE_RSS_HASH_KEY_SIZE;
+ dev_info->reta_size = GVE_RSS_INDIR_SIZE;
return 0;
}
@@ -646,6 +669,107 @@ gve_xstats_get_names(struct rte_eth_dev *dev,
return count;
}
+
+static int
+gve_rss_hash_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ struct gve_rss_config gve_rss_conf;
+ int rss_reta_size;
+ int err;
+
+ if (gve_validate_rss_hf(rss_conf->rss_hf)) {
+ PMD_DRV_LOG(ERR, "Unsupported hash function.");
+ return -EINVAL;
+ }
+
+ if (rss_conf->algorithm != RTE_ETH_HASH_FUNCTION_TOEPLITZ &&
+ rss_conf->algorithm != RTE_ETH_HASH_FUNCTION_DEFAULT) {
+ PMD_DRV_LOG(ERR, "Device only supports Toeplitz algorithm.");
+ return -EINVAL;
+ }
+
+ if (rss_conf->rss_key_len) {
+ if (rss_conf->rss_key_len != GVE_RSS_HASH_KEY_SIZE) {
+ PMD_DRV_LOG(ERR,
+ "Invalid hash key size. Only RSS hash key size "
+ "of %u supported", GVE_RSS_HASH_KEY_SIZE);
+ return -EINVAL;
+ }
+
+ if (!rss_conf->rss_key) {
+ PMD_DRV_LOG(ERR, "RSS key must be non-null.");
+ return -EINVAL;
+ }
+ } else {
+ if (!priv->rss_config.key_size) {
+ PMD_DRV_LOG(ERR, "RSS key must be initialized before "
+ "any other configuration.");
+ return -EINVAL;
+ }
+ rss_conf->rss_key_len = priv->rss_config.key_size;
+ }
+
+ rss_reta_size = priv->rss_config.indir ?
+ priv->rss_config.indir_size :
+ GVE_RSS_INDIR_SIZE;
+ err = gve_init_rss_config(&gve_rss_conf, rss_conf->rss_key_len,
+ rss_reta_size);
+ if (err)
+ return err;
+
+ gve_rss_conf.alg = GVE_RSS_HASH_TOEPLITZ;
+ err = gve_update_rss_hash_types(priv, &gve_rss_conf, rss_conf);
+ if (err)
+ goto err;
+ err = gve_update_rss_key(priv, &gve_rss_conf, rss_conf);
+ if (err)
+ goto err;
+
+ /* Set redirection table to default or preexisting. */
+ if (!priv->rss_config.indir)
+ gve_generate_rss_reta(dev, &gve_rss_conf);
+ else
+ memcpy(gve_rss_conf.indir, priv->rss_config.indir,
+ gve_rss_conf.indir_size * sizeof(*priv->rss_config.indir));
+
+ err = gve_adminq_configure_rss(priv, &gve_rss_conf);
+ if (!err)
+ gve_update_priv_rss_config(priv, &gve_rss_conf);
+
+err:
+ gve_free_rss_config(&gve_rss_conf);
+ return err;
+}
+
+static int
+gve_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+
+ if (!(dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)) {
+ PMD_DRV_LOG(ERR, "RSS not configured.");
+ return -ENOTSUP;
+ }
+
+
+ gve_to_rte_rss_hf(priv->rss_config.hash_types, rss_conf);
+ rss_conf->rss_key_len = priv->rss_config.key_size;
+ if (rss_conf->rss_key) {
+ if (!priv->rss_config.key) {
+ PMD_DRV_LOG(ERR, "Unable to retrieve default RSS hash key.");
+ return -ENOTSUP;
+ }
+ memcpy(rss_conf->rss_key, priv->rss_config.key,
+ rss_conf->rss_key_len * sizeof(*rss_conf->rss_key));
+ }
+
+ return 0;
+}
+
static const struct eth_dev_ops gve_eth_dev_ops = {
.dev_configure = gve_dev_configure,
.dev_start = gve_dev_start,
@@ -666,6 +790,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = {
.mtu_set = gve_dev_mtu_set,
.xstats_get = gve_xstats_get,
.xstats_get_names = gve_xstats_get_names,
+ .rss_hash_update = gve_rss_hash_update,
+ .rss_hash_conf_get = gve_rss_hash_conf_get,
};
static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -688,6 +814,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
.mtu_set = gve_dev_mtu_set,
.xstats_get = gve_xstats_get,
.xstats_get_names = gve_xstats_get_names,
+ .rss_hash_update = gve_rss_hash_update,
+ .rss_hash_conf_get = gve_rss_hash_conf_get,
};
static void
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index bc486cb941..d713657d10 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -29,6 +29,9 @@
#define GVE_RX_MIN_BUF_SIZE_GQI 2048
#define GVE_RX_MAX_BUF_SIZE_GQI 4096
+#define GVE_RSS_HASH_KEY_SIZE 40
+#define GVE_RSS_INDIR_SIZE 128
+
#define GVE_TX_CKSUM_OFFLOAD_MASK ( \
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 5/7] net/gve: RSS redirection table update support
[not found] ` <20240126173317.2779230-1-joshwash@google.com>
` (3 preceding siblings ...)
2024-01-26 17:33 ` [PATCH v4 4/7] net/gve: RSS configuration update support Joshua Washington
@ 2024-01-26 17:33 ` Joshua Washington
2024-01-26 17:33 ` [PATCH v4 6/7] net/gve: update gve.ini with RSS capabilities Joshua Washington
` (2 subsequent siblings)
7 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-26 17:33 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch introduces support for updating the RSS redirection table in
the GVE PMD through the implementation of rss_reta_update and
rss_reta_query.
Due to an infrastructure limitation, the RSS hash key must be manually
configured before the redirection table can be updated or queried. The
redirection table is expected to be exactly 128 bytes.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/gve_ethdev.c | 95 ++++++++++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 2a68d31808..3b8ec5872d 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -770,6 +770,97 @@ gve_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
}
+static int
+gve_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ struct gve_rss_config gve_rss_conf;
+ int table_id;
+ int err;
+ int i;
+
+ /* RSS key must be set before the redirection table can be set. */
+ if (!priv->rss_config.key || priv->rss_config.key_size == 0) {
+ PMD_DRV_LOG(ERR, "RSS hash key msut be set before the "
+ "redirection table can be updated.");
+ return -ENOTSUP;
+ }
+
+ if (reta_size != GVE_RSS_INDIR_SIZE) {
+ PMD_DRV_LOG(ERR, "Redirection table must have %hu elements",
+ (uint16_t)GVE_RSS_INDIR_SIZE);
+ return -EINVAL;
+ }
+
+ err = gve_init_rss_config_from_priv(priv, &gve_rss_conf);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Error allocating new RSS config.");
+ return err;
+ }
+
+ table_id = 0;
+ for (i = 0; i < priv->rss_config.indir_size; i++) {
+ int table_entry = i % RTE_ETH_RETA_GROUP_SIZE;
+ if (reta_conf[table_id].mask & (1ULL << table_entry))
+ gve_rss_conf.indir[i] =
+ reta_conf[table_id].reta[table_entry];
+
+ if (table_entry == RTE_ETH_RETA_GROUP_SIZE - 1)
+ table_id++;
+ }
+
+ err = gve_adminq_configure_rss(priv, &gve_rss_conf);
+ if (err)
+ PMD_DRV_LOG(ERR, "Problem configuring RSS with device.");
+ else
+ gve_update_priv_rss_config(priv, &gve_rss_conf);
+
+ gve_free_rss_config(&gve_rss_conf);
+ return err;
+}
+
+static int
+gve_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ int table_id;
+ int i;
+
+ if (!(dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)) {
+ PMD_DRV_LOG(ERR, "RSS not configured.");
+ return -ENOTSUP;
+ }
+
+ /* RSS key must be set before the redirection table can be queried. */
+ if (!priv->rss_config.key) {
+ PMD_DRV_LOG(ERR, "RSS hash key must be set before the "
+ "redirection table can be initialized.");
+ return -ENOTSUP;
+ }
+
+ if (reta_size != priv->rss_config.indir_size) {
+ PMD_DRV_LOG(ERR, "RSS redirection table must have %d entries.",
+ priv->rss_config.indir_size);
+ return -EINVAL;
+ }
+
+ table_id = 0;
+ for (i = 0; i < priv->rss_config.indir_size; i++) {
+ int table_entry = i % RTE_ETH_RETA_GROUP_SIZE;
+ if (reta_conf[table_id].mask & (1ULL << table_entry))
+ reta_conf[table_id].reta[table_entry] =
+ priv->rss_config.indir[i];
+
+ if (table_entry == RTE_ETH_RETA_GROUP_SIZE - 1)
+ table_id++;
+ }
+
+ return 0;
+}
+
static const struct eth_dev_ops gve_eth_dev_ops = {
.dev_configure = gve_dev_configure,
.dev_start = gve_dev_start,
@@ -792,6 +883,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = {
.xstats_get_names = gve_xstats_get_names,
.rss_hash_update = gve_rss_hash_update,
.rss_hash_conf_get = gve_rss_hash_conf_get,
+ .reta_update = gve_rss_reta_update,
+ .reta_query = gve_rss_reta_query,
};
static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -816,6 +909,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
.xstats_get_names = gve_xstats_get_names,
.rss_hash_update = gve_rss_hash_update,
.rss_hash_conf_get = gve_rss_hash_conf_get,
+ .reta_update = gve_rss_reta_update,
+ .reta_query = gve_rss_reta_query,
};
static void
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 6/7] net/gve: update gve.ini with RSS capabilities
[not found] ` <20240126173317.2779230-1-joshwash@google.com>
` (4 preceding siblings ...)
2024-01-26 17:33 ` [PATCH v4 5/7] net/gve: RSS redirection table " Joshua Washington
@ 2024-01-26 17:33 ` Joshua Washington
2024-01-31 14:48 ` Ferruh Yigit
2024-01-26 17:33 ` [PATCH v4 7/7] net/gve: update GVE documentation with RSS support Joshua Washington
2024-01-31 14:58 ` Ferruh Yigit
7 siblings, 1 reply; 44+ messages in thread
From: Joshua Washington @ 2024-01-26 17:33 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch updates the DPDK feature matrix to expose that the GVE driver
supports RSS hash, RSS key update, and RSS reta update.
Signed-off-by: Joshua Washington <joshwash@google.com>
---
doc/guides/nics/features/gve.ini | 3 +++
1 file changed, 3 insertions(+)
diff --git a/doc/guides/nics/features/gve.ini b/doc/guides/nics/features/gve.ini
index 838edd456a..bc08648dbc 100644
--- a/doc/guides/nics/features/gve.ini
+++ b/doc/guides/nics/features/gve.ini
@@ -15,3 +15,6 @@ Linux = Y
x86-32 = Y
x86-64 = Y
Usage doc = Y
+RSS hash = Y
+RSS key update = Y
+RSS reta update = Y
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 7/7] net/gve: update GVE documentation with RSS support
[not found] ` <20240126173317.2779230-1-joshwash@google.com>
` (5 preceding siblings ...)
2024-01-26 17:33 ` [PATCH v4 6/7] net/gve: update gve.ini with RSS capabilities Joshua Washington
@ 2024-01-26 17:33 ` Joshua Washington
2024-01-31 14:50 ` Ferruh Yigit
2024-01-31 14:58 ` Ferruh Yigit
7 siblings, 1 reply; 44+ messages in thread
From: Joshua Washington @ 2024-01-26 17:33 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch updates the GVE doc page to communicate that GVE now supports
RSS configuration and explains the limitations.
Signed-off-by: Joshua Washington <joshwash@google.com>
---
doc/guides/nics/gve.rst | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/doc/guides/nics/gve.rst b/doc/guides/nics/gve.rst
index 1c3eaf03ef..80991e70cf 100644
--- a/doc/guides/nics/gve.rst
+++ b/doc/guides/nics/gve.rst
@@ -70,6 +70,8 @@ Supported features of the GVE PMD are:
- Link state information
- Tx multi-segments (Scatter Tx)
- Tx UDP/TCP/SCTP Checksum
+- RSS hash configuration
+- RSS redirection table update and query
Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD.
Jumbo Frame is not supported in PMD for now.
@@ -77,10 +79,12 @@ It'll be added in a future DPDK release.
Also, only GQI_QPL queue format is in use on GCP
since GQI_RDA hasn't been released in production.
-Currently, setting MTU with value larger than 1460 is not supported.
+RSS
+^^^
-Currently, only "RSS hash" is force enabled
-so that the backend hardware device calculated hash values
-could be shared with applications.
-But for RSS, there is no such API to config RSS hash function or RETA table.
-So, limited RSS is supported only with default config/setting.
+GVE RSS can be enabled and configured using the standard interfaces. The driver
+does not support querying the initial RSS configuration.
+
+The RSS hash key must be exactly 40 bytes, and the redirection table must have
+128 entries. The RSS hash key must be configured before the redirection table
+can be set up.
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* Re: [PATCH v4 6/7] net/gve: update gve.ini with RSS capabilities
2024-01-26 17:33 ` [PATCH v4 6/7] net/gve: update gve.ini with RSS capabilities Joshua Washington
@ 2024-01-31 14:48 ` Ferruh Yigit
2024-01-31 14:55 ` Ferruh Yigit
0 siblings, 1 reply; 44+ messages in thread
From: Ferruh Yigit @ 2024-01-31 14:48 UTC (permalink / raw)
To: Joshua Washington, Junfeng Guo, Jeroen de Borst, Rushil Gupta; +Cc: dev
On 1/26/2024 5:33 PM, Joshua Washington wrote:
> This patch updates the DPDK feature matrix to expose that the GVE driver
> supports RSS hash, RSS key update, and RSS reta update.
>
> Signed-off-by: Joshua Washington <joshwash@google.com>
> ---
> doc/guides/nics/features/gve.ini | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/doc/guides/nics/features/gve.ini b/doc/guides/nics/features/gve.ini
> index 838edd456a..bc08648dbc 100644
> --- a/doc/guides/nics/features/gve.ini
> +++ b/doc/guides/nics/features/gve.ini
> @@ -15,3 +15,6 @@ Linux = Y
> x86-32 = Y
> x86-64 = Y
> Usage doc = Y
> +RSS hash = Y
> +RSS key update = Y
> +RSS reta update = Y
Please distribute this patch to other patches, each update can go to the
patch that adds the support.
Please keep the order of capabilities above same as order in default.ini
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 7/7] net/gve: update GVE documentation with RSS support
2024-01-26 17:33 ` [PATCH v4 7/7] net/gve: update GVE documentation with RSS support Joshua Washington
@ 2024-01-31 14:50 ` Ferruh Yigit
0 siblings, 0 replies; 44+ messages in thread
From: Ferruh Yigit @ 2024-01-31 14:50 UTC (permalink / raw)
To: Joshua Washington, Junfeng Guo, Jeroen de Borst, Rushil Gupta; +Cc: dev
On 1/26/2024 5:33 PM, Joshua Washington wrote:
> This patch updates the GVE doc page to communicate that GVE now supports
> RSS configuration and explains the limitations.
>
> Signed-off-by: Joshua Washington <joshwash@google.com>
> ---
> doc/guides/nics/gve.rst | 16 ++++++++++------
> 1 file changed, 10 insertions(+), 6 deletions(-)
>
> diff --git a/doc/guides/nics/gve.rst b/doc/guides/nics/gve.rst
> index 1c3eaf03ef..80991e70cf 100644
> --- a/doc/guides/nics/gve.rst
> +++ b/doc/guides/nics/gve.rst
> @@ -70,6 +70,8 @@ Supported features of the GVE PMD are:
> - Link state information
> - Tx multi-segments (Scatter Tx)
> - Tx UDP/TCP/SCTP Checksum
> +- RSS hash configuration
> +- RSS redirection table update and query
>
> Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD.
> Jumbo Frame is not supported in PMD for now.
> @@ -77,10 +79,12 @@ It'll be added in a future DPDK release.
> Also, only GQI_QPL queue format is in use on GCP
> since GQI_RDA hasn't been released in production.
>
> -Currently, setting MTU with value larger than 1460 is not supported.
> +RSS
> +^^^
>
> -Currently, only "RSS hash" is force enabled
> -so that the backend hardware device calculated hash values
> -could be shared with applications.
> -But for RSS, there is no such API to config RSS hash function or RETA table.
> -So, limited RSS is supported only with default config/setting.
> +GVE RSS can be enabled and configured using the standard interfaces. The driver
> +does not support querying the initial RSS configuration.
> +
> +The RSS hash key must be exactly 40 bytes, and the redirection table must have
> +128 entries. The RSS hash key must be configured before the redirection table
> +can be set up.
>
Similarly can you please update documentation on each relevant patch,
instead of having a separate documentation patch.
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 6/7] net/gve: update gve.ini with RSS capabilities
2024-01-31 14:48 ` Ferruh Yigit
@ 2024-01-31 14:55 ` Ferruh Yigit
0 siblings, 0 replies; 44+ messages in thread
From: Ferruh Yigit @ 2024-01-31 14:55 UTC (permalink / raw)
To: Joshua Washington, Junfeng Guo, Jeroen de Borst, Rushil Gupta; +Cc: dev
On 1/31/2024 2:48 PM, Ferruh Yigit wrote:
> On 1/26/2024 5:33 PM, Joshua Washington wrote:
>> This patch updates the DPDK feature matrix to expose that the GVE driver
>> supports RSS hash, RSS key update, and RSS reta update.
>>
>> Signed-off-by: Joshua Washington <joshwash@google.com>
>> ---
>> doc/guides/nics/features/gve.ini | 3 +++
>> 1 file changed, 3 insertions(+)
>>
>> diff --git a/doc/guides/nics/features/gve.ini b/doc/guides/nics/features/gve.ini
>> index 838edd456a..bc08648dbc 100644
>> --- a/doc/guides/nics/features/gve.ini
>> +++ b/doc/guides/nics/features/gve.ini
>> @@ -15,3 +15,6 @@ Linux = Y
>> x86-32 = Y
>> x86-64 = Y
>> Usage doc = Y
>> +RSS hash = Y
>> +RSS key update = Y
>> +RSS reta update = Y
>
> Please distribute this patch to other patches, each update can go to the
> patch that adds the support.
>
> Please keep the order of capabilities above same as order in default.ini
>
btw, "RSS hash" seems duplicated.
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re:
[not found] ` <20240126173317.2779230-1-joshwash@google.com>
` (6 preceding siblings ...)
2024-01-26 17:33 ` [PATCH v4 7/7] net/gve: update GVE documentation with RSS support Joshua Washington
@ 2024-01-31 14:58 ` Ferruh Yigit
7 siblings, 0 replies; 44+ messages in thread
From: Ferruh Yigit @ 2024-01-31 14:58 UTC (permalink / raw)
To: Joshua Washington; +Cc: dev, Rushil Gupta
On 1/26/2024 5:33 PM, Joshua Washington wrote:
> Subject: [PATCH v4 0/7] net/gve: RSS Support for GVE Driver
>
> This patch series introduces RSS support for the GVE poll-mode driver.
> This series includes implementations of the following eth_dev_ops:
>
> 1) rss_hash_update
> 2) rss_hash_conf_get
> 3) reta_query
> 4) reta_update
>
> In rss_hash_update, the GVE driver supports the following RSS hash
> types:
>
> * RTE_ETH_RSS_IPV4
> * RTE_ETH_RSS_NONFRAG_IPV4_TCP
> * RTE_ETH_RSS_NONFRAG_IPV4_UDP
> * RTE_ETH_RSS_IPV6
> * RTE_ETH_RSS_IPV6_EX
> * RTE_ETH_RSS_NONFRAG_IPV6_TCP
> * RTE_ETH_RSS_NONFRAG_IPV6_UDP
> * RTE_ETH_RSS_IPV6_TCP_EX
> * RTE_ETH_RSS_IPV6_UDP_EX
>
> The hash key is 40B, and the lookup table has 128 entries. These values
> are not configurable in this implementation.
>
> In general, the DPDK driver expects the RSS hash configuration to be set
> with a key before the redriection table is set up. When the RSS hash is
> configured, a default redirection table is generated based on the number
> of queues. When the device is re-configured, the redirection table is
> reset to the default value based on the queue count.
>
> An important note is that the gVNIC device expects 32 bit integers for
> RSS redirection table entries, while the RTE API uses 16 bit integers.
> However, this is unlikely to be an issue, as these values represent
> receive queues, and the gVNIC device does not support anywhere near 64K
> queues.
>
> This series also updates the corresponding feature matrix ertries and
> documentation as it pertains to RSS support in the GVE driver.
>
> v2:
> Add commmit messages for patches with it missing, and other checkpatches
> fixes.
>
> Note: There is a warning about complex macros being parenthesized that
> does not seem to be well-founded.
>
> v3:
> Fix build warnings that come up on certain distros.
>
> v4:
> Fix formatting in gve_adminq.c
>
> Joshua Washington (7):
> net/gve: fully expose RSS offload support in dev_info
> net/gve: RSS adminq command changes
> net/gve: add gve_rss library for handling RSS-related behaviors
> net/gve: RSS configuration update support
> net/gve: RSS redirection table update support
> net/gve: update gve.ini with RSS capabilities
> net/gve: update GVE documentation with RSS support
>
>
'./devtools/check-git-log.sh' script is giving warnings [1]:
Expected patch title format is:
net/gve: <verb> <object>
<verb> should start with lowercase.
[1]
- check-git-log:
Wrong headline format:
net/gve: fully expose RSS offload support in dev_info
net/gve: add gve_rss library for handling RSS-related behaviors
Wrong headline uppercase:
net/gve: RSS adminq command changes
net/gve: RSS configuration update support
net/gve: RSS redirection table update support
Headline too long:
net/gve: add gve_rss library for handling RSS-related behaviors
^ permalink raw reply [flat|nested] 44+ messages in thread
* [PATCH v5 0/5] net/gve: RSS Support for GVE Driver
2024-01-26 17:33 ` [PATCH v4 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
@ 2024-01-31 22:13 ` Joshua Washington
2024-01-31 22:13 ` [PATCH v5 1/5] net/gve: expose RSS offload support in dev info Joshua Washington
` (5 more replies)
0 siblings, 6 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-31 22:13 UTC (permalink / raw)
Cc: dev, Ferruh Yigit, Rushil Gupta, Joshua Washington
This patch series introduces RSS support for the GVE poll-mode driver.
This series includes implementations of the following eth_dev_ops:
1) rss_hash_update
2) rss_hash_conf_get
3) reta_query
4) reta_update
In rss_hash_update, the GVE driver supports the following RSS hash
types:
* RTE_ETH_RSS_IPV4
* RTE_ETH_RSS_NONFRAG_IPV4_TCP
* RTE_ETH_RSS_NONFRAG_IPV4_UDP
* RTE_ETH_RSS_IPV6
* RTE_ETH_RSS_IPV6_EX
* RTE_ETH_RSS_NONFRAG_IPV6_TCP
* RTE_ETH_RSS_NONFRAG_IPV6_UDP
* RTE_ETH_RSS_IPV6_TCP_EX
* RTE_ETH_RSS_IPV6_UDP_EX
The hash key is 40B, and the lookup table has 128 entries. These values
are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set
with a key before the redriection table is set up. When the RSS hash is
configured, a default redirection table is generated based on the number
of queues. When the device is re-configured, the redirection table is
reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for
RSS redirection table entries, while the RTE API uses 16 bit integers.
However, this is unlikely to be an issue, as these values represent
receive queues, and the gVNIC device does not support anywhere near 64K
queues.
This series also updates the corresponding feature matrix ertries and
documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches
fixes.
Note: There is a warning about complex macros being parenthesized that
does not seem to be well-founded.
v3:
Fix build warnings that come up on certain distros.
v4:
Fix formatting in gve_adminq.c
v5:
Update commit messages to comply with check-git-log.
Fold feature matrix and documentation updates into relevant commits.
Joshua Washington (5):
net/gve: expose RSS offload support in dev info
net/gve: add RSS adminq command
net/gve: add gve RSS library
net/gve: add RSS configuration update support
net/gve: add RSS redirection table update support
doc/guides/nics/features/gve.ini | 2 +
doc/guides/nics/gve.rst | 28 +++-
drivers/net/gve/base/gve.h | 15 ++
drivers/net/gve/base/gve_adminq.c | 58 ++++++++
drivers/net/gve/base/gve_adminq.h | 21 +++
drivers/net/gve/gve_ethdev.c | 231 +++++++++++++++++++++++++++++-
drivers/net/gve/gve_ethdev.h | 17 +++
drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++
drivers/net/gve/gve_rss.h | 107 ++++++++++++++
drivers/net/gve/meson.build | 1 +
10 files changed, 677 insertions(+), 9 deletions(-)
create mode 100644 drivers/net/gve/gve_rss.c
create mode 100644 drivers/net/gve/gve_rss.h
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply [flat|nested] 44+ messages in thread
* [PATCH v5 1/5] net/gve: expose RSS offload support in dev info
2024-01-31 22:13 ` [PATCH v5 0/5] net/gve: RSS Support for GVE Driver Joshua Washington
@ 2024-01-31 22:13 ` Joshua Washington
2024-01-31 22:13 ` [PATCH v5 2/5] net/gve: add RSS adminq command Joshua Washington
` (4 subsequent siblings)
5 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-31 22:13 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch communicates that the GVE driver supports RSS, along with
the RSS offloads supported by the driver.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/gve_ethdev.c | 4 +++-
drivers/net/gve/gve_ethdev.h | 8 ++++++++
2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index d162fd3864..6acdb4e13b 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -405,7 +405,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = priv->max_mtu;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
- dev_info->rx_offload_capa = 0;
+ dev_info->rx_offload_capa = RTE_ETH_RX_OFFLOAD_RSS_HASH;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
@@ -442,6 +442,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.nb_align = 1,
};
+ dev_info->flow_type_rss_offloads = GVE_RSS_OFFLOAD_ALL;
+
return 0;
}
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 9893fcfee6..14c72ec91a 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -33,6 +33,14 @@
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
+#define GVE_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_IPV6_EX | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_IPV6_TCP_EX)
+
/* A list of pages registered with the device during setup and used by a queue
* as buffers
*/
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v5 2/5] net/gve: add RSS adminq command
2024-01-31 22:13 ` [PATCH v5 0/5] net/gve: RSS Support for GVE Driver Joshua Washington
2024-01-31 22:13 ` [PATCH v5 1/5] net/gve: expose RSS offload support in dev info Joshua Washington
@ 2024-01-31 22:13 ` Joshua Washington
2024-01-31 22:13 ` [PATCH v5 3/5] net/gve: add gve RSS library Joshua Washington
` (3 subsequent siblings)
5 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-31 22:13 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This change introduces admin queue changes that enable the configuration
of RSS parameters for the GVE driver.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/base/gve.h | 15 ++++++++
drivers/net/gve/base/gve_adminq.c | 58 +++++++++++++++++++++++++++++++
drivers/net/gve/base/gve_adminq.h | 29 ++++++++++++++++
drivers/net/gve/gve_ethdev.h | 6 +++-
4 files changed, 107 insertions(+), 1 deletion(-)
diff --git a/drivers/net/gve/base/gve.h b/drivers/net/gve/base/gve.h
index f7b297e759..9c58fc4238 100644
--- a/drivers/net/gve/base/gve.h
+++ b/drivers/net/gve/base/gve.h
@@ -51,4 +51,19 @@ enum gve_state_flags_bit {
GVE_PRIV_FLAGS_NAPI_ENABLED = 4,
};
+enum gve_rss_hash_algorithm {
+ GVE_RSS_HASH_UNDEFINED = 0,
+ GVE_RSS_HASH_TOEPLITZ = 1,
+};
+
+struct gve_rss_config {
+ uint16_t hash_types;
+ enum gve_rss_hash_algorithm alg;
+ uint16_t key_size;
+ uint16_t indir_size;
+ uint8_t *key;
+ uint32_t *indir;
+};
+
+
#endif /* _GVE_H_ */
diff --git a/drivers/net/gve/base/gve_adminq.c b/drivers/net/gve/base/gve_adminq.c
index 343bd13d67..629d15cfbe 100644
--- a/drivers/net/gve/base/gve_adminq.c
+++ b/drivers/net/gve/base/gve_adminq.c
@@ -389,6 +389,9 @@ static int gve_adminq_issue_cmd(struct gve_priv *priv,
case GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES:
priv->adminq_dcfg_device_resources_cnt++;
break;
+ case GVE_ADMINQ_CONFIGURE_RSS:
+ priv->adminq_cfg_rss_cnt++;
+ break;
case GVE_ADMINQ_SET_DRIVER_PARAMETER:
priv->adminq_set_driver_parameter_cnt++;
break;
@@ -938,3 +941,58 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,
gve_free_dma_mem(&ptype_map_dma_mem);
return err;
}
+
+int gve_adminq_configure_rss(struct gve_priv *priv,
+ struct gve_rss_config *rss_config)
+{
+ struct gve_dma_mem indirection_table_dma_mem;
+ struct gve_dma_mem rss_key_dma_mem;
+ union gve_adminq_command cmd;
+ __be32 *indir = NULL;
+ u8 *key = NULL;
+ int err = 0;
+ int i;
+
+ if (!rss_config->indir_size || !rss_config->key_size)
+ return -EINVAL;
+
+ indir = gve_alloc_dma_mem(&indirection_table_dma_mem,
+ rss_config->indir_size *
+ sizeof(*rss_config->indir));
+ if (!indir) {
+ err = -ENOMEM;
+ goto out;
+ }
+ for (i = 0; i < rss_config->indir_size; i++)
+ indir[i] = cpu_to_be32(rss_config->indir[i]);
+
+ key = gve_alloc_dma_mem(&rss_key_dma_mem,
+ rss_config->key_size *
+ sizeof(*rss_config->key));
+ if (!key) {
+ err = -ENOMEM;
+ goto out;
+ }
+ memcpy(key, rss_config->key, rss_config->key_size);
+
+ memset(&cmd, 0, sizeof(cmd));
+ cmd.opcode = cpu_to_be32(GVE_ADMINQ_CONFIGURE_RSS);
+ cmd.configure_rss = (struct gve_adminq_configure_rss) {
+ .hash_types = cpu_to_be16(rss_config->hash_types),
+ .halg = rss_config->alg,
+ .hkey_len = cpu_to_be16(rss_config->key_size),
+ .indir_len = cpu_to_be16(rss_config->indir_size),
+ .hkey_addr = cpu_to_be64(rss_key_dma_mem.pa),
+ .indir_addr = cpu_to_be64(indirection_table_dma_mem.pa),
+ };
+
+ err = gve_adminq_execute_cmd(priv, &cmd);
+
+out:
+ if (indir)
+ gve_free_dma_mem(&indirection_table_dma_mem);
+ if (key)
+ gve_free_dma_mem(&rss_key_dma_mem);
+ return err;
+}
+
diff --git a/drivers/net/gve/base/gve_adminq.h b/drivers/net/gve/base/gve_adminq.h
index f05362f85f..95f4960561 100644
--- a/drivers/net/gve/base/gve_adminq.h
+++ b/drivers/net/gve/base/gve_adminq.h
@@ -19,6 +19,7 @@ enum gve_adminq_opcodes {
GVE_ADMINQ_DESTROY_TX_QUEUE = 0x7,
GVE_ADMINQ_DESTROY_RX_QUEUE = 0x8,
GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES = 0x9,
+ GVE_ADMINQ_CONFIGURE_RSS = 0xA,
GVE_ADMINQ_SET_DRIVER_PARAMETER = 0xB,
GVE_ADMINQ_REPORT_STATS = 0xC,
GVE_ADMINQ_REPORT_LINK_SPEED = 0xD,
@@ -377,6 +378,27 @@ struct gve_adminq_get_ptype_map {
__be64 ptype_map_addr;
};
+#define GVE_RSS_HASH_IPV4 BIT(0)
+#define GVE_RSS_HASH_TCPV4 BIT(1)
+#define GVE_RSS_HASH_IPV6 BIT(2)
+#define GVE_RSS_HASH_IPV6_EX BIT(3)
+#define GVE_RSS_HASH_TCPV6 BIT(4)
+#define GVE_RSS_HASH_TCPV6_EX BIT(5)
+#define GVE_RSS_HASH_UDPV4 BIT(6)
+#define GVE_RSS_HASH_UDPV6 BIT(7)
+#define GVE_RSS_HASH_UDPV6_EX BIT(8)
+
+/* RSS configuration command */
+struct gve_adminq_configure_rss {
+ __be16 hash_types;
+ u8 halg; /* hash algorithm */
+ u8 reserved;
+ __be16 hkey_len;
+ __be16 indir_len;
+ __be64 hkey_addr;
+ __be64 indir_addr;
+};
+
union gve_adminq_command {
struct {
__be32 opcode;
@@ -391,6 +413,7 @@ union gve_adminq_command {
struct gve_adminq_describe_device describe_device;
struct gve_adminq_register_page_list reg_page_list;
struct gve_adminq_unregister_page_list unreg_page_list;
+ struct gve_adminq_configure_rss configure_rss;
struct gve_adminq_set_driver_parameter set_driver_param;
struct gve_adminq_report_stats report_stats;
struct gve_adminq_report_link_speed report_link_speed;
@@ -404,6 +427,8 @@ union gve_adminq_command {
GVE_CHECK_UNION_LEN(64, gve_adminq_command);
+struct gve_priv;
+struct gve_queue_page_list;
int gve_adminq_alloc(struct gve_priv *priv);
void gve_adminq_free(struct gve_priv *priv);
void gve_adminq_release(struct gve_priv *priv);
@@ -433,4 +458,8 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,
int gve_adminq_verify_driver_compatibility(struct gve_priv *priv,
u64 driver_info_len,
dma_addr_t driver_info_addr);
+
+int gve_adminq_configure_rss(struct gve_priv *priv,
+ struct gve_rss_config *rss_config);
+
#endif /* _GVE_ADMINQ_H */
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 14c72ec91a..f7635e829c 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -39,7 +39,10 @@
RTE_ETH_RSS_IPV6 | \
RTE_ETH_RSS_IPV6_EX | \
RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
- RTE_ETH_RSS_IPV6_TCP_EX)
+ RTE_ETH_RSS_IPV6_TCP_EX | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_IPV6_UDP_EX)
/* A list of pages registered with the device during setup and used by a queue
* as buffers
@@ -264,6 +267,7 @@ struct gve_priv {
uint32_t adminq_destroy_tx_queue_cnt;
uint32_t adminq_destroy_rx_queue_cnt;
uint32_t adminq_dcfg_device_resources_cnt;
+ uint32_t adminq_cfg_rss_cnt;
uint32_t adminq_set_driver_parameter_cnt;
uint32_t adminq_report_stats_cnt;
uint32_t adminq_report_link_speed_cnt;
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v5 3/5] net/gve: add gve RSS library
2024-01-31 22:13 ` [PATCH v5 0/5] net/gve: RSS Support for GVE Driver Joshua Washington
2024-01-31 22:13 ` [PATCH v5 1/5] net/gve: expose RSS offload support in dev info Joshua Washington
2024-01-31 22:13 ` [PATCH v5 2/5] net/gve: add RSS adminq command Joshua Washington
@ 2024-01-31 22:13 ` Joshua Washington
2024-01-31 22:13 ` [PATCH v5 4/5] net/gve: add RSS configuration update support Joshua Washington
` (2 subsequent siblings)
5 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-31 22:13 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This change includes a number of helper functions to facilitate RSS
configuration on the GVE DPDK driver. These methods are declared in
gve_rss.h.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
drivers/net/gve/base/gve_adminq.h | 10 +-
drivers/net/gve/gve_ethdev.c | 2 +-
drivers/net/gve/gve_ethdev.h | 4 +-
| 206 ++++++++++++++++++++++++++++++
| 107 ++++++++++++++++
drivers/net/gve/meson.build | 1 +
6 files changed, 319 insertions(+), 11 deletions(-)
create mode 100644 drivers/net/gve/gve_rss.c
create mode 100644 drivers/net/gve/gve_rss.h
diff --git a/drivers/net/gve/base/gve_adminq.h b/drivers/net/gve/base/gve_adminq.h
index 95f4960561..24abd945cc 100644
--- a/drivers/net/gve/base/gve_adminq.h
+++ b/drivers/net/gve/base/gve_adminq.h
@@ -378,15 +378,6 @@ struct gve_adminq_get_ptype_map {
__be64 ptype_map_addr;
};
-#define GVE_RSS_HASH_IPV4 BIT(0)
-#define GVE_RSS_HASH_TCPV4 BIT(1)
-#define GVE_RSS_HASH_IPV6 BIT(2)
-#define GVE_RSS_HASH_IPV6_EX BIT(3)
-#define GVE_RSS_HASH_TCPV6 BIT(4)
-#define GVE_RSS_HASH_TCPV6_EX BIT(5)
-#define GVE_RSS_HASH_UDPV4 BIT(6)
-#define GVE_RSS_HASH_UDPV6 BIT(7)
-#define GVE_RSS_HASH_UDPV6_EX BIT(8)
/* RSS configuration command */
struct gve_adminq_configure_rss {
@@ -428,6 +419,7 @@ union gve_adminq_command {
GVE_CHECK_UNION_LEN(64, gve_adminq_command);
struct gve_priv;
+struct gve_rss_config;
struct gve_queue_page_list;
int gve_adminq_alloc(struct gve_priv *priv);
void gve_adminq_free(struct gve_priv *priv);
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 6acdb4e13b..936ca22cb9 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -442,7 +442,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.nb_align = 1,
};
- dev_info->flow_type_rss_offloads = GVE_RSS_OFFLOAD_ALL;
+ dev_info->flow_type_rss_offloads = GVE_RTE_RSS_OFFLOAD_ALL;
return 0;
}
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index f7635e829c..bc486cb941 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -33,7 +33,7 @@
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
-#define GVE_RSS_OFFLOAD_ALL ( \
+#define GVE_RTE_RSS_OFFLOAD_ALL ( \
RTE_ETH_RSS_IPV4 | \
RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
RTE_ETH_RSS_IPV6 | \
@@ -290,6 +290,8 @@ struct gve_priv {
const struct rte_memzone *stats_report_mem;
uint16_t stats_start_idx; /* start index of array of stats written by NIC */
uint16_t stats_end_idx; /* end index of array of stats written by NIC */
+
+ struct gve_rss_config rss_config;
};
static inline bool
--git a/drivers/net/gve/gve_rss.c b/drivers/net/gve/gve_rss.c
new file mode 100644
index 0000000000..931180f8f2
--- /dev/null
+++ b/drivers/net/gve/gve_rss.c
@@ -0,0 +1,206 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Google LLC
+ */
+
+#include "gve_rss.h"
+
+int
+gve_generate_rss_reta(struct rte_eth_dev *dev, struct gve_rss_config *config)
+{
+ int i;
+ if (!config || !config->indir)
+ return -EINVAL;
+
+ for (i = 0; i < config->indir_size; i++)
+ config->indir[i] = i % dev->data->nb_rx_queues;
+
+ return 0;
+}
+
+
+int
+gve_init_rss_config(struct gve_rss_config *gve_rss_conf,
+ uint16_t key_size, uint16_t indir_size)
+{
+ int err;
+
+ gve_rss_conf->alg = GVE_RSS_HASH_TOEPLITZ;
+
+ gve_rss_conf->key_size = key_size;
+ gve_rss_conf->key = rte_zmalloc("rss key",
+ key_size * sizeof(*gve_rss_conf->key),
+ RTE_CACHE_LINE_SIZE);
+ if (!gve_rss_conf->key)
+ return -ENOMEM;
+
+ gve_rss_conf->indir_size = indir_size;
+ gve_rss_conf->indir = rte_zmalloc("rss reta",
+ indir_size * sizeof(*gve_rss_conf->indir),
+ RTE_CACHE_LINE_SIZE);
+ if (!gve_rss_conf->indir) {
+ err = -ENOMEM;
+ goto err_with_key;
+ }
+
+ return 0;
+err_with_key:
+ rte_free(gve_rss_conf->key);
+ return err;
+}
+
+int
+gve_init_rss_config_from_priv(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf)
+{
+ int err = gve_init_rss_config(gve_rss_conf, priv->rss_config.key_size,
+ priv->rss_config.indir_size);
+ if (err)
+ return err;
+
+ gve_rss_conf->hash_types = priv->rss_config.hash_types;
+ gve_rss_conf->alg = priv->rss_config.alg;
+ memcpy(gve_rss_conf->key, priv->rss_config.key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ memcpy(gve_rss_conf->indir, priv->rss_config.indir,
+ gve_rss_conf->indir_size * sizeof(*gve_rss_conf->indir));
+
+ return 0;
+}
+
+void
+gve_free_rss_config(struct gve_rss_config *gve_rss_conf)
+{
+ rte_free(gve_rss_conf->indir);
+ gve_rss_conf->indir = NULL;
+ rte_free(gve_rss_conf->key);
+ gve_rss_conf->key = NULL;
+}
+
+int
+gve_update_priv_rss_config(struct gve_priv *priv, struct gve_rss_config *config)
+{
+ struct gve_rss_config *priv_config = &priv->rss_config;
+ int key_bytes, indir_bytes;
+
+ if (!config)
+ return -EINVAL;
+ if (config->key_size == 0 || !config->key)
+ return -EINVAL;
+ if (config->indir_size == 0 || !config->indir)
+ return -EINVAL;
+
+ priv_config->hash_types = config->hash_types;
+ priv_config->alg = config->alg;
+
+ priv_config->key_size = config->key_size;
+ key_bytes = priv_config->key_size * sizeof(*priv_config->key);
+ if (!priv_config->key)
+ priv_config->key = rte_zmalloc("priv rss key", key_bytes,
+ RTE_CACHE_LINE_SIZE);
+ else
+ priv_config->key = rte_realloc(priv_config->key, key_bytes,
+ RTE_CACHE_LINE_SIZE);
+ if (!priv_config->key)
+ return -ENOMEM;
+
+ priv_config->indir_size = config->indir_size;
+ indir_bytes = priv_config->indir_size * sizeof(*priv_config->indir);
+ if (!priv_config->indir)
+ priv_config->indir = rte_zmalloc("priv rss reta", indir_bytes,
+ RTE_CACHE_LINE_SIZE);
+ else
+ priv_config->indir = rte_realloc(priv_config->indir,
+ indir_bytes, RTE_CACHE_LINE_SIZE);
+
+ if (!priv_config->indir)
+ return -ENOMEM;
+
+ memcpy(priv_config->indir, config->indir,
+ config->indir_size * sizeof(*priv_config->indir));
+ memcpy(priv_config->key, config->key,
+ config->key_size * sizeof(*priv_config->key));
+
+ return 0;
+}
+
+int
+gve_update_rss_key(struct gve_priv *priv, struct gve_rss_config *gve_rss_conf,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ if (rss_conf->rss_key_len && rss_conf->rss_key) {
+ gve_rss_conf->key_size = rss_conf->rss_key_len;
+ memcpy(gve_rss_conf->key, rss_conf->rss_key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ } else if (priv->rss_config.key_size && priv->rss_config.key) {
+ gve_rss_conf->key_size = priv->rss_config.key_size;
+ memcpy(gve_rss_conf->key, priv->rss_config.key,
+ gve_rss_conf->key_size * sizeof(*gve_rss_conf->key));
+ } else {
+ PMD_DRV_LOG(ERR, "RSS key must be set as part of initial RSS "
+ "configuration.");
+ return -EINVAL;
+ }
+ return 0;
+}
+
+int
+gve_update_rss_hash_types(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf, struct rte_eth_rss_conf *rss_conf)
+{
+ /* Initialize to 0 before modifying. */
+ gve_rss_conf->hash_types = 0;
+ if (rss_conf->rss_hf)
+ rte_to_gve_rss_hf(rss_conf->rss_hf, gve_rss_conf);
+ else if (priv->rss_config.key_size && priv->rss_config.key)
+ gve_rss_conf->hash_types = priv->rss_config.hash_types;
+ else
+ gve_rss_conf->hash_types = GVE_RSS_OFFLOAD_DEFAULT;
+ return 0;
+}
+
+void
+rte_to_gve_rss_hf(uint64_t rte_rss_hf, struct gve_rss_config *gve_rss_conf)
+{
+ if (rte_rss_hf & RTE_ETH_RSS_IPV4)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_TCP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_IPV6_EX;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_TCP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_TCP_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_TCPV6_EX;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_UDP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV4;
+ if (rte_rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_UDP)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV6;
+ if (rte_rss_hf & RTE_ETH_RSS_IPV6_UDP_EX)
+ gve_rss_conf->hash_types |= GVE_RSS_HASH_UDPV6_EX;
+}
+
+void
+gve_to_rte_rss_hf(uint16_t gve_rss_types, struct rte_eth_rss_conf *rss_conf)
+{
+ if (gve_rss_types & GVE_RSS_HASH_IPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV4;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV4_TCP;
+ if (gve_rss_types & GVE_RSS_HASH_IPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6;
+ if (gve_rss_types & GVE_RSS_HASH_IPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_EX;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV6_TCP;
+ if (gve_rss_types & GVE_RSS_HASH_TCPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_TCP_EX;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV4)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV4_UDP;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV6)
+ rss_conf->rss_hf |= RTE_ETH_RSS_NONFRAG_IPV6_UDP;
+ if (gve_rss_types & GVE_RSS_HASH_UDPV6_EX)
+ rss_conf->rss_hf |= RTE_ETH_RSS_IPV6_UDP_EX;
+}
+
--git a/drivers/net/gve/gve_rss.h b/drivers/net/gve/gve_rss.h
new file mode 100644
index 0000000000..0cc235a19a
--- /dev/null
+++ b/drivers/net/gve/gve_rss.h
@@ -0,0 +1,107 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Google LLC
+ */
+
+#include "gve_adminq.h"
+#include "gve_ethdev.h"
+
+#define GVE_RSS_HASH_IPV4 BIT(0)
+#define GVE_RSS_HASH_TCPV4 BIT(1)
+#define GVE_RSS_HASH_IPV6 BIT(2)
+#define GVE_RSS_HASH_IPV6_EX BIT(3)
+#define GVE_RSS_HASH_TCPV6 BIT(4)
+#define GVE_RSS_HASH_TCPV6_EX BIT(5)
+#define GVE_RSS_HASH_UDPV4 BIT(6)
+#define GVE_RSS_HASH_UDPV6 BIT(7)
+#define GVE_RSS_HASH_UDPV6_EX BIT(8)
+
+#define GVE_RSS_OFFLOAD_DEFAULT ( \
+ GVE_RSS_HASH_IPV4 | \
+ GVE_RSS_HASH_TCPV4 | \
+ GVE_RSS_HASH_IPV6 | \
+ GVE_RSS_HASH_IPV6_EX | \
+ GVE_RSS_HASH_TCPV6 | \
+ GVE_RSS_HASH_TCPV6_EX | \
+ GVE_RSS_HASH_UDPV4 | \
+ GVE_RSS_HASH_UDPV6 | \
+ GVE_RSS_HASH_UDPV6_EX)
+
+/**
+ * Generates default RSS redirection table based on the number of queues the
+ * device is configured with. This assigns hash values to queues in a
+ * round-robin manner.
+ */
+int
+gve_generate_rss_reta(struct rte_eth_dev *dev, struct gve_rss_config *config);
+
+/**
+ * Initializes `gve_rss_conf`, setting the fields to default values and
+ * allocating memory for the RSS key and redirection table.
+ */
+int
+gve_init_rss_config(struct gve_rss_config *gve_rss_conf,
+ uint16_t key_size, uint16_t indir_size);
+
+/**
+ * Initializes `gve_rss_conf` based on the RSS configuration stored in `priv`.
+ */
+int
+gve_init_rss_config_from_priv(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Frees RSS key and redriection table pointers stored in `gve_rss_conf`.
+ */
+void
+gve_free_rss_config(struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Updates the rss_config stored in `priv` with the contents of `config`.
+ */
+int
+gve_update_priv_rss_config(struct gve_priv *priv,
+ struct gve_rss_config *config);
+
+/**
+ * Updates the RSS key stored in `gve_rss_conf`. It is prioritized as follows:
+ * 1) retrieve from `rss_conf`, if non-null
+ * 2) retrieve from `priv`, if non-null
+ * If keys from both sources are unset, return -EINVAL.
+ */
+int
+gve_update_rss_key(struct gve_priv *priv, struct gve_rss_config *gve_rss_conf,
+ struct rte_eth_rss_conf *rss_conf);
+
+/**
+ * Updates the RSS hash types stored in `gve_rss_conf`. It is prioritized as
+ * follows:
+ * 1) retrieve from `rss_conf`, if set
+ * 2) retrieve from priv, if RSS has been configured
+ * 3) set default RSS offload
+ */
+int
+gve_update_rss_hash_types(struct gve_priv *priv,
+ struct gve_rss_config *gve_rss_conf, struct rte_eth_rss_conf *rss_conf);
+
+/**
+ * Ensures that only supported RSS hash fields are set in `rte_rss_hf`.
+ */
+static inline int
+gve_validate_rss_hf(uint64_t rte_rss_hf) {
+ return rte_rss_hf & ~GVE_RTE_RSS_OFFLOAD_ALL;
+}
+
+/**
+ * Converts RSS hash types from RTE values to GVE values, storing them in
+ * `gve_rss_conf`.
+ */
+void
+rte_to_gve_rss_hf(uint64_t rte_rss_hf, struct gve_rss_config *gve_rss_conf);
+
+/**
+ * Converts RSS hash types from GVE values to RTE values, storing them in
+ * `rss_conf`.
+ */
+void
+gve_to_rte_rss_hf(uint16_t gve_rss_types, struct rte_eth_rss_conf *rss_conf);
+
diff --git a/drivers/net/gve/meson.build b/drivers/net/gve/meson.build
index 61d195009c..6da13a8406 100644
--- a/drivers/net/gve/meson.build
+++ b/drivers/net/gve/meson.build
@@ -15,5 +15,6 @@ sources = files(
'gve_tx_dqo.c',
'gve_ethdev.c',
'gve_version.c',
+ 'gve_rss.c',
)
includes += include_directories('base')
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v5 4/5] net/gve: add RSS configuration update support
2024-01-31 22:13 ` [PATCH v5 0/5] net/gve: RSS Support for GVE Driver Joshua Washington
` (2 preceding siblings ...)
2024-01-31 22:13 ` [PATCH v5 3/5] net/gve: add gve RSS library Joshua Washington
@ 2024-01-31 22:13 ` Joshua Washington
2024-01-31 22:13 ` [PATCH v5 5/5] net/gve: add RSS redirection table " Joshua Washington
2024-02-01 13:28 ` [PATCH v5 0/5] net/gve: RSS Support for GVE Driver Ferruh Yigit
5 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-31 22:13 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch adds support for updating the RSS hash key and hash fields
in the GVE PMD through the implementation of rss_hash_update and
rss_hash_conf_get.
The RSS hash key for gVNIC is required to be 40 bytes. On initial
configuration of the RSS hash key, the RSS redirection table will be
set to a static default, using a round-robin approach for all queues.
Note, however, that this patch does not include support for setting the
redirection table explicitly. In dev_configure, if the static
redirection table has been set, it will be updated to reflect the new
queue count, if it has changed.
The RSS key must be set before any other RSS configuration can happen.
As such, an attempt to set the hash types before the key is configured
will fail.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
doc/guides/nics/features/gve.ini | 1 +
doc/guides/nics/gve.rst | 20 +++--
drivers/net/gve/gve_ethdev.c | 132 ++++++++++++++++++++++++++++++-
drivers/net/gve/gve_ethdev.h | 3 +
4 files changed, 148 insertions(+), 8 deletions(-)
diff --git a/doc/guides/nics/features/gve.ini b/doc/guides/nics/features/gve.ini
index 838edd456a..4381b1565f 100644
--- a/doc/guides/nics/features/gve.ini
+++ b/doc/guides/nics/features/gve.ini
@@ -9,6 +9,7 @@ Link status = Y
MTU update = Y
TSO = Y
RSS hash = Y
+RSS key update = Y
L4 checksum offload = Y
Basic stats = Y
Linux = Y
diff --git a/doc/guides/nics/gve.rst b/doc/guides/nics/gve.rst
index 1c3eaf03ef..908b2aab11 100644
--- a/doc/guides/nics/gve.rst
+++ b/doc/guides/nics/gve.rst
@@ -70,6 +70,7 @@ Supported features of the GVE PMD are:
- Link state information
- Tx multi-segments (Scatter Tx)
- Tx UDP/TCP/SCTP Checksum
+- RSS hash configuration
Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD.
Jumbo Frame is not supported in PMD for now.
@@ -77,10 +78,17 @@ It'll be added in a future DPDK release.
Also, only GQI_QPL queue format is in use on GCP
since GQI_RDA hasn't been released in production.
-Currently, setting MTU with value larger than 1460 is not supported.
+RSS
+^^^
+
+GVE RSS can be enabled and configured using the standard interfaces. The driver
+does not support querying the initial RSS configuration.
+
+The RSS hash key must be exactly 40 bytes. Upon RSS hash configuration, a
+default redirection table will be set using a round-robin assignment of hash
+values to queues. The default GVE redirection table has 128 entries.
+
+Note that the initial configuration requires a hash key to be provided if one
+had not been provided before. Attempting to set hash types alone without the
+existence of a set key will result in a failed request.
-Currently, only "RSS hash" is force enabled
-so that the backend hardware device calculated hash values
-could be shared with applications.
-But for RSS, there is no such API to config RSS hash function or RETA table.
-So, limited RSS is supported only with default config/setting.
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 936ca22cb9..2a68d31808 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -1,5 +1,6 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(C) 2022 Intel Corporation
+ * Copyright(C) 2022-2023 Intel Corporation
+ * Copyright(C) 2023 Google LLC
*/
#include "gve_ethdev.h"
@@ -8,6 +9,7 @@
#include "base/gve_osdep.h"
#include "gve_version.h"
#include "rte_ether.h"
+#include "gve_rss.h"
static void
gve_write_version(uint8_t *driver_version_register)
@@ -88,12 +90,31 @@ gve_dev_configure(struct rte_eth_dev *dev)
{
struct gve_priv *priv = dev->data->dev_private;
- if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) {
dev->data->dev_conf.rxmode.offloads |= RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ priv->rss_config.alg = GVE_RSS_HASH_TOEPLITZ;
+ }
if (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO)
priv->enable_rsc = 1;
+ /* Reset RSS RETA in case number of queues changed. */
+ if (priv->rss_config.indir) {
+ struct gve_rss_config update_reta_config;
+ gve_init_rss_config_from_priv(priv, &update_reta_config);
+ gve_generate_rss_reta(dev, &update_reta_config);
+
+ int err = gve_adminq_configure_rss(priv, &update_reta_config);
+ if (err)
+ PMD_DRV_LOG(ERR,
+ "Could not reconfigure RSS redirection table.");
+ else
+ gve_update_priv_rss_config(priv, &update_reta_config);
+
+ gve_free_rss_config(&update_reta_config);
+ return err;
+ }
+
return 0;
}
@@ -443,6 +464,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
};
dev_info->flow_type_rss_offloads = GVE_RTE_RSS_OFFLOAD_ALL;
+ dev_info->hash_key_size = GVE_RSS_HASH_KEY_SIZE;
+ dev_info->reta_size = GVE_RSS_INDIR_SIZE;
return 0;
}
@@ -646,6 +669,107 @@ gve_xstats_get_names(struct rte_eth_dev *dev,
return count;
}
+
+static int
+gve_rss_hash_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ struct gve_rss_config gve_rss_conf;
+ int rss_reta_size;
+ int err;
+
+ if (gve_validate_rss_hf(rss_conf->rss_hf)) {
+ PMD_DRV_LOG(ERR, "Unsupported hash function.");
+ return -EINVAL;
+ }
+
+ if (rss_conf->algorithm != RTE_ETH_HASH_FUNCTION_TOEPLITZ &&
+ rss_conf->algorithm != RTE_ETH_HASH_FUNCTION_DEFAULT) {
+ PMD_DRV_LOG(ERR, "Device only supports Toeplitz algorithm.");
+ return -EINVAL;
+ }
+
+ if (rss_conf->rss_key_len) {
+ if (rss_conf->rss_key_len != GVE_RSS_HASH_KEY_SIZE) {
+ PMD_DRV_LOG(ERR,
+ "Invalid hash key size. Only RSS hash key size "
+ "of %u supported", GVE_RSS_HASH_KEY_SIZE);
+ return -EINVAL;
+ }
+
+ if (!rss_conf->rss_key) {
+ PMD_DRV_LOG(ERR, "RSS key must be non-null.");
+ return -EINVAL;
+ }
+ } else {
+ if (!priv->rss_config.key_size) {
+ PMD_DRV_LOG(ERR, "RSS key must be initialized before "
+ "any other configuration.");
+ return -EINVAL;
+ }
+ rss_conf->rss_key_len = priv->rss_config.key_size;
+ }
+
+ rss_reta_size = priv->rss_config.indir ?
+ priv->rss_config.indir_size :
+ GVE_RSS_INDIR_SIZE;
+ err = gve_init_rss_config(&gve_rss_conf, rss_conf->rss_key_len,
+ rss_reta_size);
+ if (err)
+ return err;
+
+ gve_rss_conf.alg = GVE_RSS_HASH_TOEPLITZ;
+ err = gve_update_rss_hash_types(priv, &gve_rss_conf, rss_conf);
+ if (err)
+ goto err;
+ err = gve_update_rss_key(priv, &gve_rss_conf, rss_conf);
+ if (err)
+ goto err;
+
+ /* Set redirection table to default or preexisting. */
+ if (!priv->rss_config.indir)
+ gve_generate_rss_reta(dev, &gve_rss_conf);
+ else
+ memcpy(gve_rss_conf.indir, priv->rss_config.indir,
+ gve_rss_conf.indir_size * sizeof(*priv->rss_config.indir));
+
+ err = gve_adminq_configure_rss(priv, &gve_rss_conf);
+ if (!err)
+ gve_update_priv_rss_config(priv, &gve_rss_conf);
+
+err:
+ gve_free_rss_config(&gve_rss_conf);
+ return err;
+}
+
+static int
+gve_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+
+ if (!(dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)) {
+ PMD_DRV_LOG(ERR, "RSS not configured.");
+ return -ENOTSUP;
+ }
+
+
+ gve_to_rte_rss_hf(priv->rss_config.hash_types, rss_conf);
+ rss_conf->rss_key_len = priv->rss_config.key_size;
+ if (rss_conf->rss_key) {
+ if (!priv->rss_config.key) {
+ PMD_DRV_LOG(ERR, "Unable to retrieve default RSS hash key.");
+ return -ENOTSUP;
+ }
+ memcpy(rss_conf->rss_key, priv->rss_config.key,
+ rss_conf->rss_key_len * sizeof(*rss_conf->rss_key));
+ }
+
+ return 0;
+}
+
static const struct eth_dev_ops gve_eth_dev_ops = {
.dev_configure = gve_dev_configure,
.dev_start = gve_dev_start,
@@ -666,6 +790,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = {
.mtu_set = gve_dev_mtu_set,
.xstats_get = gve_xstats_get,
.xstats_get_names = gve_xstats_get_names,
+ .rss_hash_update = gve_rss_hash_update,
+ .rss_hash_conf_get = gve_rss_hash_conf_get,
};
static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -688,6 +814,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
.mtu_set = gve_dev_mtu_set,
.xstats_get = gve_xstats_get,
.xstats_get_names = gve_xstats_get_names,
+ .rss_hash_update = gve_rss_hash_update,
+ .rss_hash_conf_get = gve_rss_hash_conf_get,
};
static void
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index bc486cb941..d713657d10 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -29,6 +29,9 @@
#define GVE_RX_MIN_BUF_SIZE_GQI 2048
#define GVE_RX_MAX_BUF_SIZE_GQI 4096
+#define GVE_RSS_HASH_KEY_SIZE 40
+#define GVE_RSS_INDIR_SIZE 128
+
#define GVE_TX_CKSUM_OFFLOAD_MASK ( \
RTE_MBUF_F_TX_L4_MASK | \
RTE_MBUF_F_TX_TCP_SEG)
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v5 5/5] net/gve: add RSS redirection table update support
2024-01-31 22:13 ` [PATCH v5 0/5] net/gve: RSS Support for GVE Driver Joshua Washington
` (3 preceding siblings ...)
2024-01-31 22:13 ` [PATCH v5 4/5] net/gve: add RSS configuration update support Joshua Washington
@ 2024-01-31 22:13 ` Joshua Washington
2024-02-01 13:28 ` [PATCH v5 0/5] net/gve: RSS Support for GVE Driver Ferruh Yigit
5 siblings, 0 replies; 44+ messages in thread
From: Joshua Washington @ 2024-01-31 22:13 UTC (permalink / raw)
To: Junfeng Guo, Jeroen de Borst, Rushil Gupta, Joshua Washington
Cc: dev, Ferruh Yigit
This patch introduces support for updating the RSS redirection table in
the GVE PMD through the implementation of rss_reta_update and
rss_reta_query.
Due to an infrastructure limitation, the RSS hash key must be manually
configured before the redirection table can be updated or queried. The
redirection table is expected to be exactly 128 bytes.
Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
Reviewed-by: Jeroen de Borst <jeroendb@google.com>
---
doc/guides/nics/features/gve.ini | 1 +
doc/guides/nics/gve.rst | 8 +++
drivers/net/gve/gve_ethdev.c | 95 ++++++++++++++++++++++++++++++++
3 files changed, 104 insertions(+)
diff --git a/doc/guides/nics/features/gve.ini b/doc/guides/nics/features/gve.ini
index 4381b1565f..8dfa229bb0 100644
--- a/doc/guides/nics/features/gve.ini
+++ b/doc/guides/nics/features/gve.ini
@@ -10,6 +10,7 @@ MTU update = Y
TSO = Y
RSS hash = Y
RSS key update = Y
+RSS reta update = Y
L4 checksum offload = Y
Basic stats = Y
Linux = Y
diff --git a/doc/guides/nics/gve.rst b/doc/guides/nics/gve.rst
index 908b2aab11..69e4eaabdb 100644
--- a/doc/guides/nics/gve.rst
+++ b/doc/guides/nics/gve.rst
@@ -71,6 +71,7 @@ Supported features of the GVE PMD are:
- Tx multi-segments (Scatter Tx)
- Tx UDP/TCP/SCTP Checksum
- RSS hash configuration
+- RSS redirection table query and update
Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD.
Jumbo Frame is not supported in PMD for now.
@@ -92,3 +93,10 @@ Note that the initial configuration requires a hash key to be provided if one
had not been provided before. Attempting to set hash types alone without the
existence of a set key will result in a failed request.
+As stated above, the RSS redirection table has exactly 128 entries. The RSS hash
+must be configured before the redirection table can be updated using the standard
+interface. Because the initial RSS hash creates a default redirection table, the
+redirection table will be available for querying upon initial hash configuration.
+When performing redirection table updates, it is possible to update individual
+table entries.
+
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 2a68d31808..3b8ec5872d 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -770,6 +770,97 @@ gve_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
}
+static int
+gve_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ struct gve_rss_config gve_rss_conf;
+ int table_id;
+ int err;
+ int i;
+
+ /* RSS key must be set before the redirection table can be set. */
+ if (!priv->rss_config.key || priv->rss_config.key_size == 0) {
+ PMD_DRV_LOG(ERR, "RSS hash key msut be set before the "
+ "redirection table can be updated.");
+ return -ENOTSUP;
+ }
+
+ if (reta_size != GVE_RSS_INDIR_SIZE) {
+ PMD_DRV_LOG(ERR, "Redirection table must have %hu elements",
+ (uint16_t)GVE_RSS_INDIR_SIZE);
+ return -EINVAL;
+ }
+
+ err = gve_init_rss_config_from_priv(priv, &gve_rss_conf);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Error allocating new RSS config.");
+ return err;
+ }
+
+ table_id = 0;
+ for (i = 0; i < priv->rss_config.indir_size; i++) {
+ int table_entry = i % RTE_ETH_RETA_GROUP_SIZE;
+ if (reta_conf[table_id].mask & (1ULL << table_entry))
+ gve_rss_conf.indir[i] =
+ reta_conf[table_id].reta[table_entry];
+
+ if (table_entry == RTE_ETH_RETA_GROUP_SIZE - 1)
+ table_id++;
+ }
+
+ err = gve_adminq_configure_rss(priv, &gve_rss_conf);
+ if (err)
+ PMD_DRV_LOG(ERR, "Problem configuring RSS with device.");
+ else
+ gve_update_priv_rss_config(priv, &gve_rss_conf);
+
+ gve_free_rss_config(&gve_rss_conf);
+ return err;
+}
+
+static int
+gve_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ struct gve_priv *priv = dev->data->dev_private;
+ int table_id;
+ int i;
+
+ if (!(dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)) {
+ PMD_DRV_LOG(ERR, "RSS not configured.");
+ return -ENOTSUP;
+ }
+
+ /* RSS key must be set before the redirection table can be queried. */
+ if (!priv->rss_config.key) {
+ PMD_DRV_LOG(ERR, "RSS hash key must be set before the "
+ "redirection table can be initialized.");
+ return -ENOTSUP;
+ }
+
+ if (reta_size != priv->rss_config.indir_size) {
+ PMD_DRV_LOG(ERR, "RSS redirection table must have %d entries.",
+ priv->rss_config.indir_size);
+ return -EINVAL;
+ }
+
+ table_id = 0;
+ for (i = 0; i < priv->rss_config.indir_size; i++) {
+ int table_entry = i % RTE_ETH_RETA_GROUP_SIZE;
+ if (reta_conf[table_id].mask & (1ULL << table_entry))
+ reta_conf[table_id].reta[table_entry] =
+ priv->rss_config.indir[i];
+
+ if (table_entry == RTE_ETH_RETA_GROUP_SIZE - 1)
+ table_id++;
+ }
+
+ return 0;
+}
+
static const struct eth_dev_ops gve_eth_dev_ops = {
.dev_configure = gve_dev_configure,
.dev_start = gve_dev_start,
@@ -792,6 +883,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = {
.xstats_get_names = gve_xstats_get_names,
.rss_hash_update = gve_rss_hash_update,
.rss_hash_conf_get = gve_rss_hash_conf_get,
+ .reta_update = gve_rss_reta_update,
+ .reta_query = gve_rss_reta_query,
};
static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -816,6 +909,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
.xstats_get_names = gve_xstats_get_names,
.rss_hash_update = gve_rss_hash_update,
.rss_hash_conf_get = gve_rss_hash_conf_get,
+ .reta_update = gve_rss_reta_update,
+ .reta_query = gve_rss_reta_query,
};
static void
--
2.43.0.429.g432eaa2c6b-goog
^ permalink raw reply related [flat|nested] 44+ messages in thread
* Re: [PATCH v5 0/5] net/gve: RSS Support for GVE Driver
2024-01-31 22:13 ` [PATCH v5 0/5] net/gve: RSS Support for GVE Driver Joshua Washington
` (4 preceding siblings ...)
2024-01-31 22:13 ` [PATCH v5 5/5] net/gve: add RSS redirection table " Joshua Washington
@ 2024-02-01 13:28 ` Ferruh Yigit
5 siblings, 0 replies; 44+ messages in thread
From: Ferruh Yigit @ 2024-02-01 13:28 UTC (permalink / raw)
To: Joshua Washington; +Cc: dev, Rushil Gupta
On 1/31/2024 10:13 PM, Joshua Washington wrote:
> This patch series introduces RSS support for the GVE poll-mode driver.
> This series includes implementations of the following eth_dev_ops:
>
> 1) rss_hash_update
> 2) rss_hash_conf_get
> 3) reta_query
> 4) reta_update
>
> In rss_hash_update, the GVE driver supports the following RSS hash
> types:
>
> * RTE_ETH_RSS_IPV4
> * RTE_ETH_RSS_NONFRAG_IPV4_TCP
> * RTE_ETH_RSS_NONFRAG_IPV4_UDP
> * RTE_ETH_RSS_IPV6
> * RTE_ETH_RSS_IPV6_EX
> * RTE_ETH_RSS_NONFRAG_IPV6_TCP
> * RTE_ETH_RSS_NONFRAG_IPV6_UDP
> * RTE_ETH_RSS_IPV6_TCP_EX
> * RTE_ETH_RSS_IPV6_UDP_EX
>
> The hash key is 40B, and the lookup table has 128 entries. These values
> are not configurable in this implementation.
>
> In general, the DPDK driver expects the RSS hash configuration to be set
> with a key before the redriection table is set up. When the RSS hash is
> configured, a default redirection table is generated based on the number
> of queues. When the device is re-configured, the redirection table is
> reset to the default value based on the queue count.
>
> An important note is that the gVNIC device expects 32 bit integers for
> RSS redirection table entries, while the RTE API uses 16 bit integers.
> However, this is unlikely to be an issue, as these values represent
> receive queues, and the gVNIC device does not support anywhere near 64K
> queues.
>
> This series also updates the corresponding feature matrix ertries and
> documentation as it pertains to RSS support in the GVE driver.
>
> v2:
> Add commmit messages for patches with it missing, and other checkpatches
> fixes.
>
> Note: There is a warning about complex macros being parenthesized that
> does not seem to be well-founded.
>
> v3:
> Fix build warnings that come up on certain distros.
>
> v4:
> Fix formatting in gve_adminq.c
>
> v5:
> Update commit messages to comply with check-git-log.
> Fold feature matrix and documentation updates into relevant commits.
>
> Joshua Washington (5):
> net/gve: expose RSS offload support in dev info
> net/gve: add RSS adminq command
> net/gve: add gve RSS library
> net/gve: add RSS configuration update support
> net/gve: add RSS redirection table update support
Series applied to dpdk-next-net/main, thanks.
^ permalink raw reply [flat|nested] 44+ messages in thread
end of thread, other threads:[~2024-02-01 13:28 UTC | newest]
Thread overview: 44+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-01-23 17:58 [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
2024-01-23 17:58 ` [PATCH v2 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
2024-01-23 17:58 ` [PATCH v2 2/7] net/gve: RSS adminq command changes Joshua Washington
2024-01-23 17:58 ` [PATCH v2 3/7] net/gve: add gve_rss library for handling RSS-related behaviors Joshua Washington
2024-01-23 17:58 ` [PATCH v2 4/7] net/gve: RSS configuration update support Joshua Washington
2024-01-23 17:58 ` [PATCH v2 5/7] net/gve: RSS redirection table " Joshua Washington
2024-01-23 17:58 ` [PATCH v2 6/7] net/gve: update gve.ini with RSS capabilities Joshua Washington
2024-01-23 17:58 ` [PATCH v2 7/7] net/gve: update GVE documentation with RSS support Joshua Washington
2024-01-24 0:04 ` [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
2024-01-24 0:04 ` [PATCH v3 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
2024-01-24 0:04 ` [PATCH v3 2/7] net/gve: RSS adminq command changes Joshua Washington
2024-01-24 0:04 ` [PATCH v3 3/7] net/gve: add gve_rss library for handling RSS-related behaviors Joshua Washington
2024-01-24 0:04 ` [PATCH v3 4/7] net/gve: RSS configuration update support Joshua Washington
2024-01-24 0:04 ` [PATCH v3 5/7] net/gve: RSS redirection table " Joshua Washington
2024-01-24 0:04 ` [PATCH v3 6/7] net/gve: update gve.ini with RSS capabilities Joshua Washington
2024-01-24 0:04 ` [PATCH v3 7/7] net/gve: update GVE documentation with RSS support Joshua Washington
2024-01-24 0:09 ` [PATCH 0/7] net/gve: RSS Support for GVE Driver Joshua Washington
2024-01-24 0:13 ` [PATCH v3 " Joshua Washington
2024-01-24 0:14 ` Joshua Washington
2024-01-24 0:14 ` [PATCH v3 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
2024-01-24 0:14 ` [PATCH v3 2/7] net/gve: RSS adminq command changes Joshua Washington
2024-01-24 0:14 ` [PATCH v3 3/7] net/gve: add gve_rss library for handling RSS-related behaviors Joshua Washington
2024-01-24 0:14 ` [PATCH v3 4/7] net/gve: RSS configuration update support Joshua Washington
2024-01-24 0:14 ` [PATCH v3 5/7] net/gve: RSS redirection table " Joshua Washington
2024-01-24 0:14 ` [PATCH v3 6/7] net/gve: update gve.ini with RSS capabilities Joshua Washington
2024-01-24 0:15 ` [PATCH v3 7/7] net/gve: update GVE documentation with RSS support Joshua Washington
[not found] ` <20240126173317.2779230-1-joshwash@google.com>
2024-01-26 17:33 ` [PATCH v4 1/7] net/gve: fully expose RSS offload support in dev_info Joshua Washington
2024-01-31 22:13 ` [PATCH v5 0/5] net/gve: RSS Support for GVE Driver Joshua Washington
2024-01-31 22:13 ` [PATCH v5 1/5] net/gve: expose RSS offload support in dev info Joshua Washington
2024-01-31 22:13 ` [PATCH v5 2/5] net/gve: add RSS adminq command Joshua Washington
2024-01-31 22:13 ` [PATCH v5 3/5] net/gve: add gve RSS library Joshua Washington
2024-01-31 22:13 ` [PATCH v5 4/5] net/gve: add RSS configuration update support Joshua Washington
2024-01-31 22:13 ` [PATCH v5 5/5] net/gve: add RSS redirection table " Joshua Washington
2024-02-01 13:28 ` [PATCH v5 0/5] net/gve: RSS Support for GVE Driver Ferruh Yigit
2024-01-26 17:33 ` [PATCH v4 2/7] net/gve: RSS adminq command changes Joshua Washington
2024-01-26 17:33 ` [PATCH v4 3/7] net/gve: add gve_rss library for handling RSS-related behaviors Joshua Washington
2024-01-26 17:33 ` [PATCH v4 4/7] net/gve: RSS configuration update support Joshua Washington
2024-01-26 17:33 ` [PATCH v4 5/7] net/gve: RSS redirection table " Joshua Washington
2024-01-26 17:33 ` [PATCH v4 6/7] net/gve: update gve.ini with RSS capabilities Joshua Washington
2024-01-31 14:48 ` Ferruh Yigit
2024-01-31 14:55 ` Ferruh Yigit
2024-01-26 17:33 ` [PATCH v4 7/7] net/gve: update GVE documentation with RSS support Joshua Washington
2024-01-31 14:50 ` Ferruh Yigit
2024-01-31 14:58 ` Ferruh Yigit
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.