* [PATCH net-next 1/7] vmxnet3: prepare for version 6 changes
2021-07-16 22:36 [PATCH net-next 0/7] vmxnet3: upgrade to version 6 Ronak Doshi
@ 2021-07-16 22:36 ` Ronak Doshi
2021-07-16 22:36 ` [PATCH net-next 2/7] vmxnet3: add support for 32 Tx/Rx queues Ronak Doshi
` (6 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: Ronak Doshi @ 2021-07-16 22:36 UTC (permalink / raw)
To: netdev
Cc: Ronak Doshi, maintainer:VMWARE VMXNET3 ETHERNET DRIVER,
David S. Miller, Jakub Kicinski, open list
vmxnet3 is currently at version 4 and this patch initiates the
preparation to accommodate changes for upto version 6. Introduced
utility macros for vmxnet3 version 6 comparison and update Copyright
information.
Signed-off-by: Ronak Doshi <doshir@vmware.com>
Acked-by: Guolin Yang <gyang@vmware.com>
---
drivers/net/vmxnet3/Makefile | 2 +-
drivers/net/vmxnet3/upt1_defs.h | 2 +-
drivers/net/vmxnet3/vmxnet3_defs.h | 2 +-
drivers/net/vmxnet3/vmxnet3_drv.c | 2 +-
drivers/net/vmxnet3/vmxnet3_int.h | 8 +++++++-
5 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/drivers/net/vmxnet3/Makefile b/drivers/net/vmxnet3/Makefile
index c5a167a1c85c..7a38925f4165 100644
--- a/drivers/net/vmxnet3/Makefile
+++ b/drivers/net/vmxnet3/Makefile
@@ -2,7 +2,7 @@
#
# Linux driver for VMware's vmxnet3 ethernet NIC.
#
-# Copyright (C) 2007-2020, VMware, Inc. All Rights Reserved.
+# Copyright (C) 2007-2021, VMware, Inc. All Rights Reserved.
#
# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
diff --git a/drivers/net/vmxnet3/upt1_defs.h b/drivers/net/vmxnet3/upt1_defs.h
index 8c014c98471c..f9f3a23d1698 100644
--- a/drivers/net/vmxnet3/upt1_defs.h
+++ b/drivers/net/vmxnet3/upt1_defs.h
@@ -1,7 +1,7 @@
/*
* Linux driver for VMware's vmxnet3 ethernet NIC.
*
- * Copyright (C) 2008-2020, VMware, Inc. All Rights Reserved.
+ * Copyright (C) 2008-2021, VMware, Inc. All Rights Reserved.
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
diff --git a/drivers/net/vmxnet3/vmxnet3_defs.h b/drivers/net/vmxnet3/vmxnet3_defs.h
index a8d5ebd47c71..ce76ebc376da 100644
--- a/drivers/net/vmxnet3/vmxnet3_defs.h
+++ b/drivers/net/vmxnet3/vmxnet3_defs.h
@@ -1,7 +1,7 @@
/*
* Linux driver for VMware's vmxnet3 ethernet NIC.
*
- * Copyright (C) 2008-2020, VMware, Inc. All Rights Reserved.
+ * Copyright (C) 2008-2021, VMware, Inc. All Rights Reserved.
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
diff --git a/drivers/net/vmxnet3/vmxnet3_drv.c b/drivers/net/vmxnet3/vmxnet3_drv.c
index 6e87f1fc4874..1f072cfdff3d 100644
--- a/drivers/net/vmxnet3/vmxnet3_drv.c
+++ b/drivers/net/vmxnet3/vmxnet3_drv.c
@@ -1,7 +1,7 @@
/*
* Linux driver for VMware's vmxnet3 ethernet NIC.
*
- * Copyright (C) 2008-2020, VMware, Inc. All Rights Reserved.
+ * Copyright (C) 2008-2021, VMware, Inc. All Rights Reserved.
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
diff --git a/drivers/net/vmxnet3/vmxnet3_int.h b/drivers/net/vmxnet3/vmxnet3_int.h
index e910596b79cf..075c1f56aecc 100644
--- a/drivers/net/vmxnet3/vmxnet3_int.h
+++ b/drivers/net/vmxnet3/vmxnet3_int.h
@@ -1,7 +1,7 @@
/*
* Linux driver for VMware's vmxnet3 ethernet NIC.
*
- * Copyright (C) 2008-2020, VMware, Inc. All Rights Reserved.
+ * Copyright (C) 2008-2021, VMware, Inc. All Rights Reserved.
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
@@ -81,6 +81,8 @@
#define VMXNET3_RSS
#endif
+#define VMXNET3_REV_6 5 /* Vmxnet3 Rev. 6 */
+#define VMXNET3_REV_5 4 /* Vmxnet3 Rev. 5 */
#define VMXNET3_REV_4 3 /* Vmxnet3 Rev. 4 */
#define VMXNET3_REV_3 2 /* Vmxnet3 Rev. 3 */
#define VMXNET3_REV_2 1 /* Vmxnet3 Rev. 2 */
@@ -421,6 +423,10 @@ struct vmxnet3_adapter {
(adapter->version >= VMXNET3_REV_3 + 1)
#define VMXNET3_VERSION_GE_4(adapter) \
(adapter->version >= VMXNET3_REV_4 + 1)
+#define VMXNET3_VERSION_GE_5(adapter) \
+ (adapter->version >= VMXNET3_REV_5 + 1)
+#define VMXNET3_VERSION_GE_6(adapter) \
+ (adapter->version >= VMXNET3_REV_6 + 1)
/* must be a multiple of VMXNET3_RING_SIZE_ALIGN */
#define VMXNET3_DEF_TX_RING_SIZE 512
--
2.11.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH net-next 2/7] vmxnet3: add support for 32 Tx/Rx queues
2021-07-16 22:36 [PATCH net-next 0/7] vmxnet3: upgrade to version 6 Ronak Doshi
2021-07-16 22:36 ` [PATCH net-next 1/7] vmxnet3: prepare for version 6 changes Ronak Doshi
@ 2021-07-16 22:36 ` Ronak Doshi
2021-07-16 22:36 ` [PATCH net-next 3/7] vmxnet3: remove power of 2 limitation on the queues Ronak Doshi
` (5 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: Ronak Doshi @ 2021-07-16 22:36 UTC (permalink / raw)
To: netdev
Cc: Ronak Doshi, maintainer:VMWARE VMXNET3 ETHERNET DRIVER,
David S. Miller, Jakub Kicinski, open list
Currently, vmxnet3 supports maximum of 8 Tx/Rx queues. With increase
in number of vcpus on a VM, to achieve better performance and utilize
idle vcpus, we need to increase the max number of queues supported.
This patch enhances vmxnet3 to support maximum of 32 Tx/Rx queues.
Increasing the Rx queues also increases the probability of distrubuting
the traffic from different flows to different queues with RSS.
Signed-off-by: Ronak Doshi <doshir@vmware.com>
Acked-by: Guolin Yang <gyang@vmware.com>
---
drivers/net/vmxnet3/vmxnet3_defs.h | 31 ++++++-
drivers/net/vmxnet3/vmxnet3_drv.c | 170 +++++++++++++++++++++++++------------
drivers/net/vmxnet3/vmxnet3_int.h | 10 ++-
3 files changed, 151 insertions(+), 60 deletions(-)
diff --git a/drivers/net/vmxnet3/vmxnet3_defs.h b/drivers/net/vmxnet3/vmxnet3_defs.h
index ce76ebc376da..a9c108166a52 100644
--- a/drivers/net/vmxnet3/vmxnet3_defs.h
+++ b/drivers/net/vmxnet3/vmxnet3_defs.h
@@ -98,6 +98,9 @@ enum {
VMXNET3_CMD_GET_TXDATA_DESC_SIZE,
VMXNET3_CMD_GET_COALESCE,
VMXNET3_CMD_GET_RSS_FIELDS,
+ VMXNET3_CMD_GET_RESERVED2,
+ VMXNET3_CMD_GET_RESERVED3,
+ VMXNET3_CMD_GET_MAX_QUEUES_CONF,
};
/*
@@ -533,6 +536,13 @@ enum vmxnet3_intr_type {
/* addition 1 for events */
#define VMXNET3_MAX_INTRS 25
+/* Version 6 and later will use below macros */
+#define VMXNET3_EXT_MAX_TX_QUEUES 32
+#define VMXNET3_EXT_MAX_RX_QUEUES 32
+/* addition 1 for events */
+#define VMXNET3_EXT_MAX_INTRS 65
+#define VMXNET3_FIRST_SET_INTRS 64
+
/* value of intrCtrl */
#define VMXNET3_IC_DISABLE_ALL 0x1 /* bit 0 */
@@ -547,6 +557,19 @@ struct Vmxnet3_IntrConf {
__le32 reserved[2];
};
+struct Vmxnet3_IntrConfExt {
+ u8 autoMask;
+ u8 numIntrs; /* # of interrupts */
+ u8 eventIntrIdx;
+ u8 reserved;
+ __le32 intrCtrl;
+ __le32 reserved1;
+ u8 modLevels[VMXNET3_EXT_MAX_INTRS]; /* moderation level for
+ * each intr
+ */
+ u8 reserved2[3];
+};
+
/* one bit per VLAN ID, the size is in the units of u32 */
#define VMXNET3_VFT_SIZE (4096 / (sizeof(u32) * 8))
@@ -719,11 +742,16 @@ struct Vmxnet3_DSDevRead {
struct Vmxnet3_VariableLenConfDesc pluginConfDesc;
};
+struct Vmxnet3_DSDevReadExt {
+ /* read-only region for device, read by dev in response to a SET cmd */
+ struct Vmxnet3_IntrConfExt intrConfExt;
+};
+
/* All structures in DriverShared are padded to multiples of 8 bytes */
struct Vmxnet3_DriverShared {
__le32 magic;
/* make devRead start at 64bit boundaries */
- __le32 pad;
+ __le32 size; /* size of DriverShared */
struct Vmxnet3_DSDevRead devRead;
__le32 ecr;
__le32 reserved;
@@ -734,6 +762,7 @@ struct Vmxnet3_DriverShared {
* command
*/
} cu;
+ struct Vmxnet3_DSDevReadExt devReadExt;
};
diff --git a/drivers/net/vmxnet3/vmxnet3_drv.c b/drivers/net/vmxnet3/vmxnet3_drv.c
index 1f072cfdff3d..d32fa6f3ae57 100644
--- a/drivers/net/vmxnet3/vmxnet3_drv.c
+++ b/drivers/net/vmxnet3/vmxnet3_drv.c
@@ -2460,6 +2460,7 @@ vmxnet3_setup_driver_shared(struct vmxnet3_adapter *adapter)
{
struct Vmxnet3_DriverShared *shared = adapter->shared;
struct Vmxnet3_DSDevRead *devRead = &shared->devRead;
+ struct Vmxnet3_DSDevReadExt *devReadExt = &shared->devReadExt;
struct Vmxnet3_TxQueueConf *tqc;
struct Vmxnet3_RxQueueConf *rqc;
int i;
@@ -2572,14 +2573,26 @@ vmxnet3_setup_driver_shared(struct vmxnet3_adapter *adapter)
#endif /* VMXNET3_RSS */
/* intr settings */
- devRead->intrConf.autoMask = adapter->intr.mask_mode ==
- VMXNET3_IMM_AUTO;
- devRead->intrConf.numIntrs = adapter->intr.num_intrs;
- for (i = 0; i < adapter->intr.num_intrs; i++)
- devRead->intrConf.modLevels[i] = adapter->intr.mod_levels[i];
+ if (!VMXNET3_VERSION_GE_6(adapter) ||
+ !adapter->queuesExtEnabled) {
+ devRead->intrConf.autoMask = adapter->intr.mask_mode ==
+ VMXNET3_IMM_AUTO;
+ devRead->intrConf.numIntrs = adapter->intr.num_intrs;
+ for (i = 0; i < adapter->intr.num_intrs; i++)
+ devRead->intrConf.modLevels[i] = adapter->intr.mod_levels[i];
+
+ devRead->intrConf.eventIntrIdx = adapter->intr.event_intr_idx;
+ devRead->intrConf.intrCtrl |= cpu_to_le32(VMXNET3_IC_DISABLE_ALL);
+ } else {
+ devReadExt->intrConfExt.autoMask = adapter->intr.mask_mode ==
+ VMXNET3_IMM_AUTO;
+ devReadExt->intrConfExt.numIntrs = adapter->intr.num_intrs;
+ for (i = 0; i < adapter->intr.num_intrs; i++)
+ devReadExt->intrConfExt.modLevels[i] = adapter->intr.mod_levels[i];
- devRead->intrConf.eventIntrIdx = adapter->intr.event_intr_idx;
- devRead->intrConf.intrCtrl |= cpu_to_le32(VMXNET3_IC_DISABLE_ALL);
+ devReadExt->intrConfExt.eventIntrIdx = adapter->intr.event_intr_idx;
+ devReadExt->intrConfExt.intrCtrl |= cpu_to_le32(VMXNET3_IC_DISABLE_ALL);
+ }
/* rx filter settings */
devRead->rxFilterConf.rxMode = 0;
@@ -2717,6 +2730,7 @@ vmxnet3_activate_dev(struct vmxnet3_adapter *adapter)
* tx queue if the link is up.
*/
vmxnet3_check_link(adapter, true);
+ netif_tx_wake_all_queues(adapter->netdev);
for (i = 0; i < adapter->num_rx_queues; i++)
napi_enable(&adapter->rx_queue[i].napi);
vmxnet3_enable_all_intrs(adapter);
@@ -3372,6 +3386,8 @@ vmxnet3_probe_device(struct pci_dev *pdev,
int size;
int num_tx_queues;
int num_rx_queues;
+ int queues;
+ unsigned long flags;
if (!pci_msi_enabled())
enable_mq = 0;
@@ -3394,10 +3410,6 @@ vmxnet3_probe_device(struct pci_dev *pdev,
num_tx_queues = rounddown_pow_of_two(num_tx_queues);
netdev = alloc_etherdev_mq(sizeof(struct vmxnet3_adapter),
max(num_tx_queues, num_rx_queues));
- dev_info(&pdev->dev,
- "# of Tx queues : %d, # of Rx queues : %d\n",
- num_tx_queues, num_rx_queues);
-
if (!netdev)
return -ENOMEM;
@@ -3447,45 +3459,6 @@ vmxnet3_probe_device(struct pci_dev *pdev,
goto err_alloc_shared;
}
- adapter->num_rx_queues = num_rx_queues;
- adapter->num_tx_queues = num_tx_queues;
- adapter->rx_buf_per_pkt = 1;
-
- size = sizeof(struct Vmxnet3_TxQueueDesc) * adapter->num_tx_queues;
- size += sizeof(struct Vmxnet3_RxQueueDesc) * adapter->num_rx_queues;
- adapter->tqd_start = dma_alloc_coherent(&adapter->pdev->dev, size,
- &adapter->queue_desc_pa,
- GFP_KERNEL);
-
- if (!adapter->tqd_start) {
- dev_err(&pdev->dev, "Failed to allocate memory\n");
- err = -ENOMEM;
- goto err_alloc_queue_desc;
- }
- adapter->rqd_start = (struct Vmxnet3_RxQueueDesc *)(adapter->tqd_start +
- adapter->num_tx_queues);
-
- adapter->pm_conf = dma_alloc_coherent(&adapter->pdev->dev,
- sizeof(struct Vmxnet3_PMConf),
- &adapter->pm_conf_pa,
- GFP_KERNEL);
- if (adapter->pm_conf == NULL) {
- err = -ENOMEM;
- goto err_alloc_pm;
- }
-
-#ifdef VMXNET3_RSS
-
- adapter->rss_conf = dma_alloc_coherent(&adapter->pdev->dev,
- sizeof(struct UPT1_RSSConf),
- &adapter->rss_conf_pa,
- GFP_KERNEL);
- if (adapter->rss_conf == NULL) {
- err = -ENOMEM;
- goto err_alloc_rss;
- }
-#endif /* VMXNET3_RSS */
-
err = vmxnet3_alloc_pci_resources(adapter);
if (err < 0)
goto err_alloc_pci;
@@ -3529,6 +3502,75 @@ vmxnet3_probe_device(struct pci_dev *pdev,
goto err_ver;
}
+ if (VMXNET3_VERSION_GE_6(adapter)) {
+ spin_lock_irqsave(&adapter->cmd_lock, flags);
+ VMXNET3_WRITE_BAR1_REG(adapter, VMXNET3_REG_CMD,
+ VMXNET3_CMD_GET_MAX_QUEUES_CONF);
+ queues = VMXNET3_READ_BAR1_REG(adapter, VMXNET3_REG_CMD);
+ spin_unlock_irqrestore(&adapter->cmd_lock, flags);
+ if (queues > 0) {
+ adapter->num_rx_queues = min(num_rx_queues, ((queues >> 8) & 0xff));
+ adapter->num_tx_queues = min(num_tx_queues, (queues & 0xff));
+ } else {
+ adapter->num_rx_queues = min(num_rx_queues,
+ VMXNET3_DEVICE_DEFAULT_RX_QUEUES);
+ adapter->num_tx_queues = min(num_tx_queues,
+ VMXNET3_DEVICE_DEFAULT_TX_QUEUES);
+ }
+ if (adapter->num_rx_queues > VMXNET3_MAX_RX_QUEUES ||
+ adapter->num_tx_queues > VMXNET3_MAX_TX_QUEUES) {
+ adapter->queuesExtEnabled = true;
+ } else {
+ adapter->queuesExtEnabled = false;
+ }
+ } else {
+ adapter->queuesExtEnabled = false;
+ adapter->num_rx_queues = min(num_rx_queues,
+ VMXNET3_DEVICE_DEFAULT_RX_QUEUES);
+ adapter->num_tx_queues = min(num_tx_queues,
+ VMXNET3_DEVICE_DEFAULT_TX_QUEUES);
+ }
+ dev_info(&pdev->dev,
+ "# of Tx queues : %d, # of Rx queues : %d\n",
+ adapter->num_tx_queues, adapter->num_rx_queues);
+
+ adapter->rx_buf_per_pkt = 1;
+
+ size = sizeof(struct Vmxnet3_TxQueueDesc) * adapter->num_tx_queues;
+ size += sizeof(struct Vmxnet3_RxQueueDesc) * adapter->num_rx_queues;
+ adapter->tqd_start = dma_alloc_coherent(&adapter->pdev->dev, size,
+ &adapter->queue_desc_pa,
+ GFP_KERNEL);
+
+ if (!adapter->tqd_start) {
+ dev_err(&pdev->dev, "Failed to allocate memory\n");
+ err = -ENOMEM;
+ goto err_ver;
+ }
+ adapter->rqd_start = (struct Vmxnet3_RxQueueDesc *)(adapter->tqd_start +
+ adapter->num_tx_queues);
+
+ adapter->pm_conf = dma_alloc_coherent(&adapter->pdev->dev,
+ sizeof(struct Vmxnet3_PMConf),
+ &adapter->pm_conf_pa,
+ GFP_KERNEL);
+ if (adapter->pm_conf == NULL) {
+ err = -ENOMEM;
+ goto err_alloc_pm;
+ }
+
+#ifdef VMXNET3_RSS
+
+ adapter->rss_conf = dma_alloc_coherent(&adapter->pdev->dev,
+ sizeof(struct UPT1_RSSConf),
+ &adapter->rss_conf_pa,
+ GFP_KERNEL);
+ if (adapter->rss_conf == NULL) {
+ err = -ENOMEM;
+ goto err_alloc_rss;
+ }
+#endif /* VMXNET3_RSS */
+
if (VMXNET3_VERSION_GE_3(adapter)) {
adapter->coal_conf =
dma_alloc_coherent(&adapter->pdev->dev,
@@ -3538,7 +3580,7 @@ vmxnet3_probe_device(struct pci_dev *pdev,
GFP_KERNEL);
if (!adapter->coal_conf) {
err = -ENOMEM;
- goto err_ver;
+ goto err_coal_conf;
}
adapter->coal_conf->coalMode = VMXNET3_COALESCE_DISABLED;
adapter->default_coal_mode = true;
@@ -3621,9 +3663,7 @@ vmxnet3_probe_device(struct pci_dev *pdev,
adapter->coal_conf, adapter->coal_conf_pa);
}
vmxnet3_free_intr_resources(adapter);
-err_ver:
- vmxnet3_free_pci_resources(adapter);
-err_alloc_pci:
+err_coal_conf:
#ifdef VMXNET3_RSS
dma_free_coherent(&adapter->pdev->dev, sizeof(struct UPT1_RSSConf),
adapter->rss_conf, adapter->rss_conf_pa);
@@ -3634,7 +3674,9 @@ vmxnet3_probe_device(struct pci_dev *pdev,
err_alloc_pm:
dma_free_coherent(&adapter->pdev->dev, size, adapter->tqd_start,
adapter->queue_desc_pa);
-err_alloc_queue_desc:
+err_ver:
+ vmxnet3_free_pci_resources(adapter);
+err_alloc_pci:
dma_free_coherent(&adapter->pdev->dev,
sizeof(struct Vmxnet3_DriverShared),
adapter->shared, adapter->shared_pa);
@@ -3653,7 +3695,8 @@ vmxnet3_remove_device(struct pci_dev *pdev)
struct net_device *netdev = pci_get_drvdata(pdev);
struct vmxnet3_adapter *adapter = netdev_priv(netdev);
int size = 0;
- int num_rx_queues;
+ int num_rx_queues, rx_queues;
+ unsigned long flags;
#ifdef VMXNET3_RSS
if (enable_mq)
@@ -3663,6 +3706,21 @@ vmxnet3_remove_device(struct pci_dev *pdev)
#endif
num_rx_queues = 1;
num_rx_queues = rounddown_pow_of_two(num_rx_queues);
+ if (VMXNET3_VERSION_GE_6(adapter)) {
+ spin_lock_irqsave(&adapter->cmd_lock, flags);
+ VMXNET3_WRITE_BAR1_REG(adapter, VMXNET3_REG_CMD,
+ VMXNET3_CMD_GET_MAX_QUEUES_CONF);
+ rx_queues = VMXNET3_READ_BAR1_REG(adapter, VMXNET3_REG_CMD);
+ spin_unlock_irqrestore(&adapter->cmd_lock, flags);
+ if (rx_queues > 0)
+ rx_queues = (rx_queues >> 8) & 0xff;
+ else
+ rx_queues = min(num_rx_queues, VMXNET3_DEVICE_DEFAULT_RX_QUEUES);
+ num_rx_queues = min(num_rx_queues, rx_queues);
+ } else {
+ num_rx_queues = min(num_rx_queues,
+ VMXNET3_DEVICE_DEFAULT_RX_QUEUES);
+ }
cancel_work_sync(&adapter->work);
diff --git a/drivers/net/vmxnet3/vmxnet3_int.h b/drivers/net/vmxnet3/vmxnet3_int.h
index 075c1f56aecc..8675209070ea 100644
--- a/drivers/net/vmxnet3/vmxnet3_int.h
+++ b/drivers/net/vmxnet3/vmxnet3_int.h
@@ -303,15 +303,18 @@ struct vmxnet3_rx_queue {
struct vmxnet3_rq_driver_stats stats;
} __attribute__((__aligned__(SMP_CACHE_BYTES)));
-#define VMXNET3_DEVICE_MAX_TX_QUEUES 8
-#define VMXNET3_DEVICE_MAX_RX_QUEUES 8 /* Keep this value as a power of 2 */
+#define VMXNET3_DEVICE_MAX_TX_QUEUES 32
+#define VMXNET3_DEVICE_MAX_RX_QUEUES 32 /* Keep this value as a power of 2 */
+
+#define VMXNET3_DEVICE_DEFAULT_TX_QUEUES 8
+#define VMXNET3_DEVICE_DEFAULT_RX_QUEUES 8 /* Keep this value as a power of 2 */
/* Should be less than UPT1_RSS_MAX_IND_TABLE_SIZE */
#define VMXNET3_RSS_IND_TABLE_SIZE (VMXNET3_DEVICE_MAX_RX_QUEUES * 4)
#define VMXNET3_LINUX_MAX_MSIX_VECT (VMXNET3_DEVICE_MAX_TX_QUEUES + \
VMXNET3_DEVICE_MAX_RX_QUEUES + 1)
-#define VMXNET3_LINUX_MIN_MSIX_VECT 2 /* 1 for tx-rx pair and 1 for event */
+#define VMXNET3_LINUX_MIN_MSIX_VECT 3 /* 1 for tx, 1 for rx pair and 1 for event */
struct vmxnet3_intr {
@@ -398,6 +401,7 @@ struct vmxnet3_adapter {
dma_addr_t adapter_pa;
dma_addr_t pm_conf_pa;
dma_addr_t rss_conf_pa;
+ bool queuesExtEnabled;
};
#define VMXNET3_WRITE_BAR0_REG(adapter, reg, val) \
--
2.11.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH net-next 3/7] vmxnet3: remove power of 2 limitation on the queues
2021-07-16 22:36 [PATCH net-next 0/7] vmxnet3: upgrade to version 6 Ronak Doshi
2021-07-16 22:36 ` [PATCH net-next 1/7] vmxnet3: prepare for version 6 changes Ronak Doshi
2021-07-16 22:36 ` [PATCH net-next 2/7] vmxnet3: add support for 32 Tx/Rx queues Ronak Doshi
@ 2021-07-16 22:36 ` Ronak Doshi
2021-07-16 22:36 ` [PATCH net-next 4/7] vmxnet3: add support for ESP IPv6 RSS Ronak Doshi
` (4 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: Ronak Doshi @ 2021-07-16 22:36 UTC (permalink / raw)
To: netdev
Cc: Ronak Doshi, maintainer:VMWARE VMXNET3 ETHERNET DRIVER,
David S. Miller, Jakub Kicinski, open list
With version 6, vmxnet3 relaxes the restriction on queues to
be power of two. This is helpful in cases (Edge VM) where
vcpus are less than 8 and device requires more than 4 queues.
Signed-off-by: Ronak Doshi <doshir@vmware.com>
Acked-by: Guolin Yang <gyang@vmware.com>
---
drivers/net/vmxnet3/vmxnet3_drv.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/net/vmxnet3/vmxnet3_drv.c b/drivers/net/vmxnet3/vmxnet3_drv.c
index d32fa6f3ae57..41e694d13c92 100644
--- a/drivers/net/vmxnet3/vmxnet3_drv.c
+++ b/drivers/net/vmxnet3/vmxnet3_drv.c
@@ -3399,7 +3399,6 @@ vmxnet3_probe_device(struct pci_dev *pdev,
else
#endif
num_rx_queues = 1;
- num_rx_queues = rounddown_pow_of_two(num_rx_queues);
if (enable_mq)
num_tx_queues = min(VMXNET3_DEVICE_MAX_TX_QUEUES,
@@ -3407,7 +3406,6 @@ vmxnet3_probe_device(struct pci_dev *pdev,
else
num_tx_queues = 1;
- num_tx_queues = rounddown_pow_of_two(num_tx_queues);
netdev = alloc_etherdev_mq(sizeof(struct vmxnet3_adapter),
max(num_tx_queues, num_rx_queues));
if (!netdev)
@@ -3525,6 +3523,8 @@ vmxnet3_probe_device(struct pci_dev *pdev,
}
} else {
adapter->queuesExtEnabled = false;
+ num_rx_queues = rounddown_pow_of_two(num_rx_queues);
+ num_tx_queues = rounddown_pow_of_two(num_tx_queues);
adapter->num_rx_queues = min(num_rx_queues,
VMXNET3_DEVICE_DEFAULT_RX_QUEUES);
adapter->num_tx_queues = min(num_tx_queues,
@@ -3705,7 +3705,9 @@ vmxnet3_remove_device(struct pci_dev *pdev)
else
#endif
num_rx_queues = 1;
- num_rx_queues = rounddown_pow_of_two(num_rx_queues);
+ if (!VMXNET3_VERSION_GE_6(adapter)) {
+ num_rx_queues = rounddown_pow_of_two(num_rx_queues);
+ }
if (VMXNET3_VERSION_GE_6(adapter)) {
spin_lock_irqsave(&adapter->cmd_lock, flags);
VMXNET3_WRITE_BAR1_REG(adapter, VMXNET3_REG_CMD,
--
2.11.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH net-next 4/7] vmxnet3: add support for ESP IPv6 RSS
2021-07-16 22:36 [PATCH net-next 0/7] vmxnet3: upgrade to version 6 Ronak Doshi
` (2 preceding siblings ...)
2021-07-16 22:36 ` [PATCH net-next 3/7] vmxnet3: remove power of 2 limitation on the queues Ronak Doshi
@ 2021-07-16 22:36 ` Ronak Doshi
2021-07-16 22:36 ` [PATCH net-next 5/7] vmxnet3: set correct hash type based on rss information Ronak Doshi
` (3 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: Ronak Doshi @ 2021-07-16 22:36 UTC (permalink / raw)
To: netdev
Cc: Ronak Doshi, maintainer:VMWARE VMXNET3 ETHERNET DRIVER,
David S. Miller, Jakub Kicinski, open list
Vmxnet3 version 4 added support for ESP RSS. However, only IPv4 was
supported. With vmxnet3 version 6, this patch enables RSS for ESP
IPv6 packets as well.
Signed-off-by: Ronak Doshi <doshir@vmware.com>
Acked-by: Guolin Yang <gyang@vmware.com>
---
drivers/net/vmxnet3/vmxnet3_ethtool.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/drivers/net/vmxnet3/vmxnet3_ethtool.c b/drivers/net/vmxnet3/vmxnet3_ethtool.c
index 1b483cf2b1ca..a3e2f2ba68b5 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethtool.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethtool.c
@@ -787,6 +787,10 @@ vmxnet3_get_rss_hash_opts(struct vmxnet3_adapter *adapter,
case AH_ESP_V6_FLOW:
case AH_V6_FLOW:
case ESP_V6_FLOW:
+ if (VMXNET3_VERSION_GE_6(adapter) &&
+ (rss_fields & VMXNET3_RSS_FIELDS_ESPIP6))
+ info->data |= RXH_L4_B_0_1 | RXH_L4_B_2_3;
+ fallthrough;
case SCTP_V6_FLOW:
case IPV6_FLOW:
info->data |= RXH_IP_SRC | RXH_IP_DST;
@@ -871,6 +875,22 @@ vmxnet3_set_rss_hash_opt(struct net_device *netdev,
case ESP_V6_FLOW:
case AH_V6_FLOW:
case AH_ESP_V6_FLOW:
+ if (!VMXNET3_VERSION_GE_6(adapter))
+ return -EOPNOTSUPP;
+ if (!(nfc->data & RXH_IP_SRC) ||
+ !(nfc->data & RXH_IP_DST))
+ return -EINVAL;
+ switch (nfc->data & (RXH_L4_B_0_1 | RXH_L4_B_2_3)) {
+ case 0:
+ rss_fields &= ~VMXNET3_RSS_FIELDS_ESPIP6;
+ break;
+ case (RXH_L4_B_0_1 | RXH_L4_B_2_3):
+ rss_fields |= VMXNET3_RSS_FIELDS_ESPIP6;
+ break;
+ default:
+ return -EINVAL;
+ }
+ break;
case SCTP_V4_FLOW:
case SCTP_V6_FLOW:
if (!(nfc->data & RXH_IP_SRC) ||
--
2.11.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH net-next 5/7] vmxnet3: set correct hash type based on rss information
2021-07-16 22:36 [PATCH net-next 0/7] vmxnet3: upgrade to version 6 Ronak Doshi
` (3 preceding siblings ...)
2021-07-16 22:36 ` [PATCH net-next 4/7] vmxnet3: add support for ESP IPv6 RSS Ronak Doshi
@ 2021-07-16 22:36 ` Ronak Doshi
2021-07-16 22:36 ` [PATCH net-next 6/7] vmxnet3: increase maximum configurable mtu to 9190 Ronak Doshi
` (2 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: Ronak Doshi @ 2021-07-16 22:36 UTC (permalink / raw)
To: netdev
Cc: Ronak Doshi, maintainer:VMWARE VMXNET3 ETHERNET DRIVER,
David S. Miller, Jakub Kicinski, open list
As vmxnet3 supports IP/TCP/UDP RSS, this patch sets appropriate
hash type based on the type of RSS performed.
Signed-off-by: Ronak Doshi <doshir@vmware.com>
Acked-by: Guolin Yang <gyang@vmware.com>
---
drivers/net/vmxnet3/vmxnet3_defs.h | 16 +++++++++-------
drivers/net/vmxnet3/vmxnet3_drv.c | 22 ++++++++++++++++++++--
2 files changed, 29 insertions(+), 9 deletions(-)
diff --git a/drivers/net/vmxnet3/vmxnet3_defs.h b/drivers/net/vmxnet3/vmxnet3_defs.h
index a9c108166a52..bc82bbbcb1ab 100644
--- a/drivers/net/vmxnet3/vmxnet3_defs.h
+++ b/drivers/net/vmxnet3/vmxnet3_defs.h
@@ -344,13 +344,15 @@ struct Vmxnet3_RxCompDescExt {
#define VMXNET3_TXD_EOP_SIZE 1
/* value of RxCompDesc.rssType */
-enum {
- VMXNET3_RCD_RSS_TYPE_NONE = 0,
- VMXNET3_RCD_RSS_TYPE_IPV4 = 1,
- VMXNET3_RCD_RSS_TYPE_TCPIPV4 = 2,
- VMXNET3_RCD_RSS_TYPE_IPV6 = 3,
- VMXNET3_RCD_RSS_TYPE_TCPIPV6 = 4,
-};
+#define VMXNET3_RCD_RSS_TYPE_NONE 0
+#define VMXNET3_RCD_RSS_TYPE_IPV4 1
+#define VMXNET3_RCD_RSS_TYPE_TCPIPV4 2
+#define VMXNET3_RCD_RSS_TYPE_IPV6 3
+#define VMXNET3_RCD_RSS_TYPE_TCPIPV6 4
+#define VMXNET3_RCD_RSS_TYPE_UDPIPV4 5
+#define VMXNET3_RCD_RSS_TYPE_UDPIPV6 6
+#define VMXNET3_RCD_RSS_TYPE_ESPIPV4 7
+#define VMXNET3_RCD_RSS_TYPE_ESPIPV6 8
/* a union for accessing all cmd/completion descriptors */
diff --git a/drivers/net/vmxnet3/vmxnet3_drv.c b/drivers/net/vmxnet3/vmxnet3_drv.c
index 41e694d13c92..4fd6ce15a860 100644
--- a/drivers/net/vmxnet3/vmxnet3_drv.c
+++ b/drivers/net/vmxnet3/vmxnet3_drv.c
@@ -1478,10 +1478,28 @@ vmxnet3_rq_rx_complete(struct vmxnet3_rx_queue *rq,
#ifdef VMXNET3_RSS
if (rcd->rssType != VMXNET3_RCD_RSS_TYPE_NONE &&
- (adapter->netdev->features & NETIF_F_RXHASH))
+ (adapter->netdev->features & NETIF_F_RXHASH)) {
+ enum pkt_hash_types hash_type;
+
+ switch (rcd->rssType) {
+ case VMXNET3_RCD_RSS_TYPE_IPV4:
+ case VMXNET3_RCD_RSS_TYPE_IPV6:
+ hash_type = PKT_HASH_TYPE_L3;
+ break;
+ case VMXNET3_RCD_RSS_TYPE_TCPIPV4:
+ case VMXNET3_RCD_RSS_TYPE_TCPIPV6:
+ case VMXNET3_RCD_RSS_TYPE_UDPIPV4:
+ case VMXNET3_RCD_RSS_TYPE_UDPIPV6:
+ hash_type = PKT_HASH_TYPE_L4;
+ break;
+ default:
+ hash_type = PKT_HASH_TYPE_L3;
+ break;
+ }
skb_set_hash(ctx->skb,
le32_to_cpu(rcd->rssHash),
- PKT_HASH_TYPE_L3);
+ hash_type);
+ }
#endif
skb_put(ctx->skb, rcd->len);
--
2.11.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH net-next 6/7] vmxnet3: increase maximum configurable mtu to 9190
2021-07-16 22:36 [PATCH net-next 0/7] vmxnet3: upgrade to version 6 Ronak Doshi
` (4 preceding siblings ...)
2021-07-16 22:36 ` [PATCH net-next 5/7] vmxnet3: set correct hash type based on rss information Ronak Doshi
@ 2021-07-16 22:36 ` Ronak Doshi
2021-07-16 22:36 ` [PATCH net-next 7/7] vmxnet3: update to version 6 Ronak Doshi
2021-07-17 1:40 ` [PATCH net-next 0/7] vmxnet3: upgrade " patchwork-bot+netdevbpf
7 siblings, 0 replies; 10+ messages in thread
From: Ronak Doshi @ 2021-07-16 22:36 UTC (permalink / raw)
To: netdev
Cc: Ronak Doshi, maintainer:VMWARE VMXNET3 ETHERNET DRIVER,
David S. Miller, Jakub Kicinski, open list
This patch increases the maximum configurable mtu to 9190
to accommodate jumbo packets of overlay traffic.
Signed-off-by: Ronak Doshi <doshir@vmware.com>
Acked-by: Guolin Yang <gyang@vmware.com>
---
drivers/net/vmxnet3/vmxnet3_defs.h | 1 +
drivers/net/vmxnet3/vmxnet3_drv.c | 7 +++++--
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/net/vmxnet3/vmxnet3_defs.h b/drivers/net/vmxnet3/vmxnet3_defs.h
index bc82bbbcb1ab..74d4e8bc4abc 100644
--- a/drivers/net/vmxnet3/vmxnet3_defs.h
+++ b/drivers/net/vmxnet3/vmxnet3_defs.h
@@ -795,6 +795,7 @@ struct Vmxnet3_DriverShared {
((vfTable[vid >> 5] & (1 << (vid & 31))) != 0)
#define VMXNET3_MAX_MTU 9000
+#define VMXNET3_V6_MAX_MTU 9190
#define VMXNET3_MIN_MTU 60
#define VMXNET3_LINK_UP (10000 << 16 | 1) /* 10 Gbps, up */
diff --git a/drivers/net/vmxnet3/vmxnet3_drv.c b/drivers/net/vmxnet3/vmxnet3_drv.c
index 4fd6ce15a860..9f52f9c254f4 100644
--- a/drivers/net/vmxnet3/vmxnet3_drv.c
+++ b/drivers/net/vmxnet3/vmxnet3_drv.c
@@ -3641,9 +3641,12 @@ vmxnet3_probe_device(struct pci_dev *pdev,
vmxnet3_set_ethtool_ops(netdev);
netdev->watchdog_timeo = 5 * HZ;
- /* MTU range: 60 - 9000 */
+ /* MTU range: 60 - 9190 */
netdev->min_mtu = VMXNET3_MIN_MTU;
- netdev->max_mtu = VMXNET3_MAX_MTU;
+ if (VMXNET3_VERSION_GE_6(adapter))
+ netdev->max_mtu = VMXNET3_V6_MAX_MTU;
+ else
+ netdev->max_mtu = VMXNET3_MAX_MTU;
INIT_WORK(&adapter->work, vmxnet3_reset_work);
set_bit(VMXNET3_STATE_BIT_QUIESCED, &adapter->state);
--
2.11.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH net-next 7/7] vmxnet3: update to version 6
2021-07-16 22:36 [PATCH net-next 0/7] vmxnet3: upgrade to version 6 Ronak Doshi
` (5 preceding siblings ...)
2021-07-16 22:36 ` [PATCH net-next 6/7] vmxnet3: increase maximum configurable mtu to 9190 Ronak Doshi
@ 2021-07-16 22:36 ` Ronak Doshi
2021-07-17 1:40 ` [PATCH net-next 0/7] vmxnet3: upgrade " patchwork-bot+netdevbpf
7 siblings, 0 replies; 10+ messages in thread
From: Ronak Doshi @ 2021-07-16 22:36 UTC (permalink / raw)
To: netdev
Cc: Ronak Doshi, maintainer:VMWARE VMXNET3 ETHERNET DRIVER,
David S. Miller, Jakub Kicinski, open list
With all vmxnet3 version 6 changes incorporated in the vmxnet3 driver,
the driver can configure emulation to run at vmxnet3 version 6, provided
the emulation advertises support for version 6.
Signed-off-by: Ronak Doshi <doshir@vmware.com>
Acked-by: Guolin Yang <gyang@vmware.com>
---
drivers/net/vmxnet3/vmxnet3_drv.c | 12 +++++++++++-
drivers/net/vmxnet3/vmxnet3_int.h | 4 ++--
2 files changed, 13 insertions(+), 3 deletions(-)
diff --git a/drivers/net/vmxnet3/vmxnet3_drv.c b/drivers/net/vmxnet3/vmxnet3_drv.c
index 9f52f9c254f4..e3c6b7e3bfdd 100644
--- a/drivers/net/vmxnet3/vmxnet3_drv.c
+++ b/drivers/net/vmxnet3/vmxnet3_drv.c
@@ -3480,7 +3480,17 @@ vmxnet3_probe_device(struct pci_dev *pdev,
goto err_alloc_pci;
ver = VMXNET3_READ_BAR1_REG(adapter, VMXNET3_REG_VRRS);
- if (ver & (1 << VMXNET3_REV_4)) {
+ if (ver & (1 << VMXNET3_REV_6)) {
+ VMXNET3_WRITE_BAR1_REG(adapter,
+ VMXNET3_REG_VRRS,
+ 1 << VMXNET3_REV_6);
+ adapter->version = VMXNET3_REV_6 + 1;
+ } else if (ver & (1 << VMXNET3_REV_5)) {
+ VMXNET3_WRITE_BAR1_REG(adapter,
+ VMXNET3_REG_VRRS,
+ 1 << VMXNET3_REV_5);
+ adapter->version = VMXNET3_REV_5 + 1;
+ } else if (ver & (1 << VMXNET3_REV_4)) {
VMXNET3_WRITE_BAR1_REG(adapter,
VMXNET3_REG_VRRS,
1 << VMXNET3_REV_4);
diff --git a/drivers/net/vmxnet3/vmxnet3_int.h b/drivers/net/vmxnet3/vmxnet3_int.h
index 8675209070ea..7027ff483fa5 100644
--- a/drivers/net/vmxnet3/vmxnet3_int.h
+++ b/drivers/net/vmxnet3/vmxnet3_int.h
@@ -69,12 +69,12 @@
/*
* Version numbers
*/
-#define VMXNET3_DRIVER_VERSION_STRING "1.5.0.0-k"
+#define VMXNET3_DRIVER_VERSION_STRING "1.6.0.0-k"
/* Each byte of this 32-bit integer encodes a version number in
* VMXNET3_DRIVER_VERSION_STRING.
*/
-#define VMXNET3_DRIVER_VERSION_NUM 0x01050000
+#define VMXNET3_DRIVER_VERSION_NUM 0x01060000
#if defined(CONFIG_PCI_MSI)
/* RSS only makes sense if MSI-X is supported. */
--
2.11.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH net-next 0/7] vmxnet3: upgrade to version 6
2021-07-16 22:36 [PATCH net-next 0/7] vmxnet3: upgrade to version 6 Ronak Doshi
` (6 preceding siblings ...)
2021-07-16 22:36 ` [PATCH net-next 7/7] vmxnet3: update to version 6 Ronak Doshi
@ 2021-07-17 1:40 ` patchwork-bot+netdevbpf
7 siblings, 0 replies; 10+ messages in thread
From: patchwork-bot+netdevbpf @ 2021-07-17 1:40 UTC (permalink / raw)
To: Ronak Doshi; +Cc: netdev, pv-drivers, davem, kuba, linux-kernel
Hello:
This series was applied to netdev/net-next.git (refs/heads/master):
On Fri, 16 Jul 2021 15:36:19 -0700 you wrote:
> vmxnet3 emulation has recently added several new features which includes
> increase in queues supported, remove power of 2 limitation on queues,
> add RSS for ESP IPv6, etc. This patch series extends the vmxnet3 driver
> to leverage these new features.
>
> Compatibility is maintained using existing vmxnet3 versioning mechanism as
> follows:
> - new features added to vmxnet3 emulation are associated with new vmxnet3
> version viz. vmxnet3 version 6.
> - emulation advertises all the versions it supports to the driver.
> - during initialization, vmxnet3 driver picks the highest version number
> supported by both the emulation and the driver and configures emulation
> to run at that version.
>
> [...]
Here is the summary with links:
- [net-next,1/7] vmxnet3: prepare for version 6 changes
https://git.kernel.org/netdev/net-next/c/69dbef0d1c22
- [net-next,2/7] vmxnet3: add support for 32 Tx/Rx queues
https://git.kernel.org/netdev/net-next/c/39f9895a00f4
- [net-next,3/7] vmxnet3: remove power of 2 limitation on the queues
https://git.kernel.org/netdev/net-next/c/15ccf2f4b09c
- [net-next,4/7] vmxnet3: add support for ESP IPv6 RSS
https://git.kernel.org/netdev/net-next/c/79d124bb36c0
- [net-next,5/7] vmxnet3: set correct hash type based on rss information
https://git.kernel.org/netdev/net-next/c/b3973bb40041
- [net-next,6/7] vmxnet3: increase maximum configurable mtu to 9190
https://git.kernel.org/netdev/net-next/c/8c5663e461e6
- [net-next,7/7] vmxnet3: update to version 6
https://git.kernel.org/netdev/net-next/c/ce2639ad6921
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 10+ messages in thread