[PATCH 0/4] enic: enable MTU update callback

[PATCH 0/4] enic: enable MTU update callback

[John Daley]

This patchset determines the max egress packet size allowed on the
NIC and uses it to set an upper limit for MTU. An MTU update function
is added, but only works if Rx scatter is disabled. If Rx scatter is
enabled, -ENOSUP is returned. Another patch with Rx scatter support will
come later.

These patches should apply cleanly to dpdk-net-next rel_16_07 or on the
enic Rx scatter patch http://www.dpdk.org/dev/patchwork/patch/13933/

John Daley (4):
  enic: enable NIC max packet size discovery
  enic: set the max allowed MTU for the NIC
  enic: add an update MTU function for non-rx scatter mode
  doc: add MTU update to feature matrix for enic

 doc/guides/nics/overview.rst      |  2 +-
 drivers/net/enic/base/vnic_enet.h | 17 ++++++++++++++-
 drivers/net/enic/enic.h           |  2 ++
 drivers/net/enic/enic_ethdev.c    | 13 ++++++++++--
 drivers/net/enic/enic_main.c      | 44 +++++++++++++++++++++++++++++++++++++++
 drivers/net/enic/enic_res.c       | 25 +++++++++++++++-------
 drivers/net/enic/enic_res.h       |  4 +++-
 7 files changed, 94 insertions(+), 13 deletions(-)

-- 
2.7.0

[PATCH 1/4] enic: enable NIC max packet size discovery

[John Daley]

Pull in common VNIC code which enables querying for max egress
packet size.

Signed-off-by: John Daley <johndale@cisco.com>
---
There are some non-related fields and defines in this file because
it is shared with other drivers and interfaces to the VIC.

 drivers/net/enic/base/vnic_enet.h | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/drivers/net/enic/base/vnic_enet.h b/drivers/net/enic/base/vnic_enet.h
index cc34998..5062247 100644
--- a/drivers/net/enic/base/vnic_enet.h
+++ b/drivers/net/enic/base/vnic_enet.h
@@ -35,6 +35,10 @@
 #ifndef _VNIC_ENIC_H_
 #define _VNIC_ENIC_H_
 
+/* Hardware intr coalesce timer is in units of 1.5us */
+#define INTR_COALESCE_USEC_TO_HW(usec) ((usec) * 2 / 3)
+#define INTR_COALESCE_HW_TO_USEC(usec) ((usec) * 3 / 2)
+
 /* Device-specific region: enet configuration */
 struct vnic_enet_config {
 	u32 flags;
@@ -50,6 +54,12 @@ struct vnic_enet_config {
 	u16 vf_rq_count;
 	u16 num_arfs;
 	u64 mem_paddr;
+	u16 rdma_qp_id;
+	u16 rdma_qp_count;
+	u16 rdma_resgrp;
+	u32 rdma_mr_id;
+	u32 rdma_mr_count;
+	u32 max_pkt_size;
 };
 
 #define VENETF_TSO		0x1	/* TSO enabled */
@@ -64,9 +74,14 @@ struct vnic_enet_config {
 #define VENETF_RSSHASH_IPV6_EX	0x200	/* Hash on IPv6 extended fields */
 #define VENETF_RSSHASH_TCPIPV6_EX 0x400	/* Hash on TCP + IPv6 ext. fields */
 #define VENETF_LOOP		0x800	/* Loopback enabled */
-#define VENETF_VMQ		0x4000  /* using VMQ flag for VMware NETQ */
+#define VENETF_FAILOVER		0x1000	/* Fabric failover enabled */
+#define VENETF_USPACE_NIC       0x2000	/* vHPC enabled */
+#define VENETF_VMQ      0x4000 /* VMQ enabled */
+#define VENETF_ARFS		0x8000  /* ARFS enabled */
 #define VENETF_VXLAN    0x10000 /* VxLAN offload */
 #define VENETF_NVGRE    0x20000 /* NVGRE offload */
+#define VENETF_GRPINTR  0x40000 /* group interrupt */
+
 #define VENET_INTR_TYPE_MIN	0	/* Timer specs min interrupt spacing */
 #define VENET_INTR_TYPE_IDLE	1	/* Timer specs idle time before irq */
 
-- 
2.7.0

[PATCH 2/4] enic: set the max allowed MTU for the NIC

[John Daley]

The max MTU is set to the max egress packet size allowed by the VIC
minus the size of a an IPv4 L2 header with .1Q (18 bytes).

Signed-off-by: John Daley <johndale@cisco.com>
---
 drivers/net/enic/enic.h        |  1 +
 drivers/net/enic/enic_ethdev.c |  3 ++-
 drivers/net/enic/enic_res.c    | 25 +++++++++++++++++--------
 drivers/net/enic/enic_res.h    |  4 +++-
 4 files changed, 23 insertions(+), 10 deletions(-)

diff --git a/drivers/net/enic/enic.h b/drivers/net/enic/enic.h
index 1e6914e..78f7bd7 100644
--- a/drivers/net/enic/enic.h
+++ b/drivers/net/enic/enic.h
@@ -118,6 +118,7 @@ struct enic {
 	u8 ig_vlan_strip_en;
 	int link_status;
 	u8 hw_ip_checksum;
+	u16 max_mtu;
 
 	unsigned int flags;
 	unsigned int priv_flags;
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 697ff82..31d9600 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -435,7 +435,8 @@ static void enicpmd_dev_info_get(struct rte_eth_dev *eth_dev,
 	device_info->max_rx_queues = enic->rq_count;
 	device_info->max_tx_queues = enic->wq_count;
 	device_info->min_rx_bufsize = ENIC_MIN_MTU;
-	device_info->max_rx_pktlen = enic->config.mtu;
+	device_info->max_rx_pktlen = enic->rte_dev->data->mtu
+				   + ETHER_HDR_LEN + 4;
 	device_info->max_mac_addrs = 1;
 	device_info->rx_offload_capa =
 		DEV_RX_OFFLOAD_VLAN_STRIP |
diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c
index ebe379d..e82181f 100644
--- a/drivers/net/enic/enic_res.c
+++ b/drivers/net/enic/enic_res.c
@@ -83,6 +83,20 @@ int enic_get_vnic_config(struct enic *enic)
 	GET_CONFIG(intr_timer_usec);
 	GET_CONFIG(loop_tag);
 	GET_CONFIG(num_arfs);
+	GET_CONFIG(max_pkt_size);
+
+	/* max packet size is only defined in newer VIC firmware
+	 * and will be 0 for legacy firmware and VICs
+	 */
+	if (c->max_pkt_size > ENIC_DEFAULT_MAX_PKT_SIZE)
+		enic->max_mtu = c->max_pkt_size - (ETHER_HDR_LEN + 4);
+	else
+		enic->max_mtu = ENIC_DEFAULT_MAX_PKT_SIZE - (ETHER_HDR_LEN + 4);
+	if (c->mtu == 0)
+		c->mtu = 1500;
+
+	enic->rte_dev->data->mtu = min_t(u16, enic->max_mtu,
+					 max_t(u16, ENIC_MIN_MTU, c->mtu));
 
 	c->wq_desc_count =
 		min_t(u32, ENIC_MAX_WQ_DESCS,
@@ -96,21 +110,16 @@ int enic_get_vnic_config(struct enic *enic)
 		c->rq_desc_count));
 	c->rq_desc_count &= 0xffffffe0; /* must be aligned to groups of 32 */
 
-	if (c->mtu == 0)
-		c->mtu = 1500;
-	c->mtu = min_t(u16, ENIC_MAX_MTU,
-		max_t(u16, ENIC_MIN_MTU,
-		c->mtu));
-
 	c->intr_timer_usec = min_t(u32, c->intr_timer_usec,
 		vnic_dev_get_intr_coal_timer_max(enic->vdev));
 
 	dev_info(enic_get_dev(enic),
 		"vNIC MAC addr %02x:%02x:%02x:%02x:%02x:%02x "
-		"wq/rq %d/%d mtu %d\n",
+		"wq/rq %d/%d mtu %d, max mtu:%d\n",
 		enic->mac_addr[0], enic->mac_addr[1], enic->mac_addr[2],
 		enic->mac_addr[3], enic->mac_addr[4], enic->mac_addr[5],
-		c->wq_desc_count, c->rq_desc_count, c->mtu);
+		c->wq_desc_count, c->rq_desc_count,
+		enic->rte_dev->data->mtu, enic->max_mtu);
 	dev_info(enic_get_dev(enic), "vNIC csum tx/rx %s/%s "
 		"rss %s intr mode %s type %s timer %d usec "
 		"loopback tag 0x%04x\n",
diff --git a/drivers/net/enic/enic_res.h b/drivers/net/enic/enic_res.h
index 3c8e303..303530e 100644
--- a/drivers/net/enic/enic_res.h
+++ b/drivers/net/enic/enic_res.h
@@ -46,7 +46,9 @@
 #define ENIC_MAX_RQ_DESCS		4096
 
 #define ENIC_MIN_MTU			68
-#define ENIC_MAX_MTU			9000
+
+/* Does not include (possible) inserted VLAN tag and FCS */
+#define ENIC_DEFAULT_MAX_PKT_SIZE	9022
 
 #define ENIC_MULTICAST_PERFECT_FILTERS	32
 #define ENIC_UNICAST_PERFECT_FILTERS	32
-- 
2.7.0

[PATCH 3/4] enic: add an update MTU function for non-Rx scatter mode

[John Daley]

Provide an update MTU callbaack. The function returns -ENOTSUP
if Rx scatter is enabled. Updating the MTU to be greater than
the value configured via the Cisco CIMC/UCSM management interface
is allowed provided it is still less than the maximum egress packet
size allowed by the NIC.

Signed-off-by: John Daley <johndale@cisco.com>
---
 drivers/net/enic/enic.h        |  1 +
 drivers/net/enic/enic_ethdev.c | 10 +++++++++-
 drivers/net/enic/enic_main.c   | 44 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 54 insertions(+), 1 deletion(-)

diff --git a/drivers/net/enic/enic.h b/drivers/net/enic/enic.h
index 78f7bd7..8122358 100644
--- a/drivers/net/enic/enic.h
+++ b/drivers/net/enic/enic.h
@@ -245,4 +245,5 @@ uint16_t enic_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			uint16_t nb_pkts);
 uint16_t enic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 			       uint16_t nb_pkts);
+int enic_set_mtu(struct enic *enic, uint16_t new_mtu);
 #endif /* _ENIC_H_ */
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 31d9600..9a738c2 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -520,6 +520,14 @@ static void enicpmd_remove_mac_addr(struct rte_eth_dev *eth_dev, __rte_unused ui
 	enic_del_mac_address(enic);
 }
 
+static int enicpmd_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu)
+{
+	struct enic *enic = pmd_priv(eth_dev);
+
+	ENICPMD_FUNC_TRACE();
+	return enic_set_mtu(enic, mtu);
+}
+
 static const struct eth_dev_ops enicpmd_eth_dev_ops = {
 	.dev_configure        = enicpmd_dev_configure,
 	.dev_start            = enicpmd_dev_start,
@@ -537,7 +545,7 @@ static const struct eth_dev_ops enicpmd_eth_dev_ops = {
 	.queue_stats_mapping_set = NULL,
 	.dev_infos_get        = enicpmd_dev_info_get,
 	.dev_supported_ptypes_get = enicpmd_dev_supported_ptypes_get,
-	.mtu_set              = NULL,
+	.mtu_set              = enicpmd_mtu_set,
 	.vlan_filter_set      = enicpmd_vlan_filter_set,
 	.vlan_tpid_set        = NULL,
 	.vlan_offload_set     = enicpmd_vlan_offload_set,
diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index 32ecdae..c23938a 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -854,6 +854,50 @@ int enic_set_vnic_res(struct enic *enic)
 	return rc;
 }
 
+/* The Cisco NIC can send and receive packets up to a max packet size
+ * determined by the NIC type and firmware. There is also an MTU
+ * configured into the NIC via the CIMC/UCSM management interface
+ * which can be overridden by this function (up to the max packet size).
+ * Depending on the network setup, doing so may cause packet drops
+ * and unexpected behavior.
+ */
+int enic_set_mtu(struct enic *enic, uint16_t new_mtu)
+{
+	uint16_t old_mtu;	/* previous setting */
+	uint16_t config_mtu;	/* Value configured into NIC via CIMC/UCSM */
+	struct rte_eth_dev *eth_dev = enic->rte_dev;
+
+	old_mtu = eth_dev->data->mtu;
+	config_mtu = enic->config.mtu;
+
+	/* only works with Rx scatter disabled */
+	if (enic->rte_dev->data->dev_conf.rxmode.enable_scatter)
+		return -ENOTSUP;
+
+	if (new_mtu > enic->max_mtu) {
+		dev_err(enic,
+			"MTU not updated: requested (%u) greater than max (%u)\n",
+			new_mtu, enic->max_mtu);
+		return -EINVAL;
+	}
+	if (new_mtu < ENIC_MIN_MTU) {
+		dev_info(enic,
+			"MTU not updated: requested (%u) less than min (%u)\n",
+			new_mtu, ENIC_MIN_MTU);
+		return -EINVAL;
+	}
+	if (new_mtu > config_mtu)
+		dev_warning(enic,
+			"MTU (%u) is greater than value configured in NIC (%u)\n",
+			new_mtu, config_mtu);
+
+	/* update the mtu */
+	eth_dev->data->mtu = new_mtu;
+
+	dev_info(enic, "MTU changed from %u to %u\n",  old_mtu, new_mtu);
+	return 0;
+}
+
 static int enic_dev_init(struct enic *enic)
 {
 	int err;
-- 
2.7.0

[PATCH 4/4] doc: add MTU update to feature matrix for enic

[John Daley]

Signed-off-by: John Daley <johndale@cisco.com>
---
 doc/guides/nics/overview.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/guides/nics/overview.rst b/doc/guides/nics/overview.rst
index 29a6163..6b30085 100644
--- a/doc/guides/nics/overview.rst
+++ b/doc/guides/nics/overview.rst
@@ -92,7 +92,7 @@ Most of these differences are summarized below.
    Queue status event                                                                     Y
    Rx interrupt                     Y     Y Y Y Y Y Y Y Y Y Y Y Y Y Y
    Queue start/stop           Y   Y   Y Y Y Y Y Y     Y Y     Y Y Y Y Y Y               Y   Y Y
-   MTU update                     Y Y Y           Y   Y Y Y Y         Y Y
+   MTU update                     Y Y Y Y         Y   Y Y Y Y         Y Y
    Jumbo frame                    Y Y Y Y Y Y Y Y Y   Y Y Y Y Y Y Y Y Y Y       Y Y Y
    Scattered Rx                   Y Y Y   Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y               Y   Y
    LRO                                                Y Y Y Y
-- 
2.7.0

Re: [PATCH 1/4] enic: enable NIC max packet size discovery

[Bruce Richardson]

On Thu, Jun 16, 2016 at 10:22:46PM -0700, John Daley wrote:
> Pull in common VNIC code which enables querying for max egress
> packet size.
> 
With this patch applied is the user able to query the max packet size, or is it
just that the driver is able to do so for use by other functions?

/Bruce

Re: [PATCH 2/4] enic: set the max allowed MTU for the NIC

[Bruce Richardson]

On Thu, Jun 16, 2016 at 10:22:47PM -0700, John Daley wrote:
> The max MTU is set to the max egress packet size allowed by the VIC
> minus the size of a an IPv4 L2 header with .1Q (18 bytes).
> 

I think a bit more detail might be needed here. For example:

* What was the MTU set to by default before this patch is applied? Was it just
set to 1518 or something else?
* What happens, if anything, if buffers bigger than the MTU size are sent down?

/Bruce

Re: [PATCH 2/4] enic: set the max allowed MTU for the NIC

[Bruce Richardson]

On Fri, Jun 24, 2016 at 11:59:19AM +0100, Bruce Richardson wrote:
> On Thu, Jun 16, 2016 at 10:22:47PM -0700, John Daley wrote:
> > The max MTU is set to the max egress packet size allowed by the VIC
> > minus the size of a an IPv4 L2 header with .1Q (18 bytes).
> > 
> 
> I think a bit more detail might be needed here. For example:
> 
> * What was the MTU set to by default before this patch is applied? Was it just
> set to 1518 or something else?
> * What happens, if anything, if buffers bigger than the MTU size are sent down?
This is obviously referring to buffers bigger than MTU on TX. There is also the
question of what happens if buffer sizes smaller than MTU are provided on RX.

> 
> /Bruce

Re: [PATCH 4/4] doc: add MTU update to feature matrix for enic

[Bruce Richardson]

On Thu, Jun 16, 2016 at 10:22:49PM -0700, John Daley wrote:
> Signed-off-by: John Daley <johndale@cisco.com>
> ---

This patch should be squashed into the previous one.

/Bruce

[PATCH v2 1/2] enic: determine max egress packet size and max MTU

[John Daley]

Pull in common VNIC code which enables querying for max egress
packet size with newer firmware via a device command. If the
field is non-zero, it is the max egress packet size. If it is
0, the default value (9022) can safely be assumed. The value
for 1300 series VICS using firmware versions >= 3.1.2 for blade
series and >= 2.0.13 for rack series servers is 9208.

Tx buffers can be emitted only if they are less than the max egress
packet size regardless of the MTU setting (the MTU is advisory).
The max egress packet size can used to determine the upper limit
of the MTU since the enic can also receive packets of size greater
than max egress packet size. A max_mtu variable is added with
a value of max egress packet size minus L2 header size.

The default MTU is set via the CIMC/UCSM management interface and
currently allows value up to 9000. If the value is changed, the
host must be reboot. To avoid the reboot and allow MTU values
up to the max capability of the NIC, MTU update capability will
be added with a max value capped by max_mtu.

Signed-off-by: John Daley <johndale@cisco.com>
---
v2: Squished patch 1/4 and 2/4 into one. Tried to do a little
better explanaiton of the intent of the patch in the commit
message.

 drivers/net/enic/base/vnic_enet.h | 17 ++++++++++++++++-
 drivers/net/enic/enic.h           |  1 +
 drivers/net/enic/enic_ethdev.c    |  3 ++-
 drivers/net/enic/enic_res.c       | 25 +++++++++++++++++--------
 drivers/net/enic/enic_res.h       |  4 +++-
 5 files changed, 39 insertions(+), 11 deletions(-)

diff --git a/drivers/net/enic/base/vnic_enet.h b/drivers/net/enic/base/vnic_enet.h
index cc34998..5062247 100644
--- a/drivers/net/enic/base/vnic_enet.h
+++ b/drivers/net/enic/base/vnic_enet.h
@@ -35,6 +35,10 @@
 #ifndef _VNIC_ENIC_H_
 #define _VNIC_ENIC_H_
 
+/* Hardware intr coalesce timer is in units of 1.5us */
+#define INTR_COALESCE_USEC_TO_HW(usec) ((usec) * 2 / 3)
+#define INTR_COALESCE_HW_TO_USEC(usec) ((usec) * 3 / 2)
+
 /* Device-specific region: enet configuration */
 struct vnic_enet_config {
 	u32 flags;
@@ -50,6 +54,12 @@ struct vnic_enet_config {
 	u16 vf_rq_count;
 	u16 num_arfs;
 	u64 mem_paddr;
+	u16 rdma_qp_id;
+	u16 rdma_qp_count;
+	u16 rdma_resgrp;
+	u32 rdma_mr_id;
+	u32 rdma_mr_count;
+	u32 max_pkt_size;
 };
 
 #define VENETF_TSO		0x1	/* TSO enabled */
@@ -64,9 +74,14 @@ struct vnic_enet_config {
 #define VENETF_RSSHASH_IPV6_EX	0x200	/* Hash on IPv6 extended fields */
 #define VENETF_RSSHASH_TCPIPV6_EX 0x400	/* Hash on TCP + IPv6 ext. fields */
 #define VENETF_LOOP		0x800	/* Loopback enabled */
-#define VENETF_VMQ		0x4000  /* using VMQ flag for VMware NETQ */
+#define VENETF_FAILOVER		0x1000	/* Fabric failover enabled */
+#define VENETF_USPACE_NIC       0x2000	/* vHPC enabled */
+#define VENETF_VMQ      0x4000 /* VMQ enabled */
+#define VENETF_ARFS		0x8000  /* ARFS enabled */
 #define VENETF_VXLAN    0x10000 /* VxLAN offload */
 #define VENETF_NVGRE    0x20000 /* NVGRE offload */
+#define VENETF_GRPINTR  0x40000 /* group interrupt */
+
 #define VENET_INTR_TYPE_MIN	0	/* Timer specs min interrupt spacing */
 #define VENET_INTR_TYPE_IDLE	1	/* Timer specs idle time before irq */
 
diff --git a/drivers/net/enic/enic.h b/drivers/net/enic/enic.h
index df302ff..b557e12 100644
--- a/drivers/net/enic/enic.h
+++ b/drivers/net/enic/enic.h
@@ -121,6 +121,7 @@ struct enic {
 	u8 ig_vlan_strip_en;
 	int link_status;
 	u8 hw_ip_checksum;
+	u16 max_mtu;
 
 	unsigned int flags;
 	unsigned int priv_flags;
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 83048d8..6fa54b2 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -439,7 +439,8 @@ static void enicpmd_dev_info_get(struct rte_eth_dev *eth_dev,
 	device_info->max_rx_queues = enic->rq_count;
 	device_info->max_tx_queues = enic->wq_count;
 	device_info->min_rx_bufsize = ENIC_MIN_MTU;
-	device_info->max_rx_pktlen = enic->config.mtu;
+	device_info->max_rx_pktlen = enic->rte_dev->data->mtu
+				   + ETHER_HDR_LEN + 4;
 	device_info->max_mac_addrs = 1;
 	device_info->rx_offload_capa =
 		DEV_RX_OFFLOAD_VLAN_STRIP |
diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c
index 42edd84..b271d34 100644
--- a/drivers/net/enic/enic_res.c
+++ b/drivers/net/enic/enic_res.c
@@ -83,6 +83,20 @@ int enic_get_vnic_config(struct enic *enic)
 	GET_CONFIG(intr_timer_usec);
 	GET_CONFIG(loop_tag);
 	GET_CONFIG(num_arfs);
+	GET_CONFIG(max_pkt_size);
+
+	/* max packet size is only defined in newer VIC firmware
+	 * and will be 0 for legacy firmware and VICs
+	 */
+	if (c->max_pkt_size > ENIC_DEFAULT_MAX_PKT_SIZE)
+		enic->max_mtu = c->max_pkt_size - (ETHER_HDR_LEN + 4);
+	else
+		enic->max_mtu = ENIC_DEFAULT_MAX_PKT_SIZE - (ETHER_HDR_LEN + 4);
+	if (c->mtu == 0)
+		c->mtu = 1500;
+
+	enic->rte_dev->data->mtu = min_t(u16, enic->max_mtu,
+					 max_t(u16, ENIC_MIN_MTU, c->mtu));
 
 	c->wq_desc_count =
 		min_t(u32, ENIC_MAX_WQ_DESCS,
@@ -96,21 +110,16 @@ int enic_get_vnic_config(struct enic *enic)
 		c->rq_desc_count));
 	c->rq_desc_count &= 0xffffffe0; /* must be aligned to groups of 32 */
 
-	if (c->mtu == 0)
-		c->mtu = 1500;
-	c->mtu = min_t(u16, ENIC_MAX_MTU,
-		max_t(u16, ENIC_MIN_MTU,
-		c->mtu));
-
 	c->intr_timer_usec = min_t(u32, c->intr_timer_usec,
 		vnic_dev_get_intr_coal_timer_max(enic->vdev));
 
 	dev_info(enic_get_dev(enic),
 		"vNIC MAC addr %02x:%02x:%02x:%02x:%02x:%02x "
-		"wq/rq %d/%d mtu %d\n",
+		"wq/rq %d/%d mtu %d, max mtu:%d\n",
 		enic->mac_addr[0], enic->mac_addr[1], enic->mac_addr[2],
 		enic->mac_addr[3], enic->mac_addr[4], enic->mac_addr[5],
-		c->wq_desc_count, c->rq_desc_count, c->mtu);
+		c->wq_desc_count, c->rq_desc_count,
+		enic->rte_dev->data->mtu, enic->max_mtu);
 	dev_info(enic_get_dev(enic), "vNIC csum tx/rx %s/%s "
 		"rss %s intr mode %s type %s timer %d usec "
 		"loopback tag 0x%04x\n",
diff --git a/drivers/net/enic/enic_res.h b/drivers/net/enic/enic_res.h
index 3c8e303..303530e 100644
--- a/drivers/net/enic/enic_res.h
+++ b/drivers/net/enic/enic_res.h
@@ -46,7 +46,9 @@
 #define ENIC_MAX_RQ_DESCS		4096
 
 #define ENIC_MIN_MTU			68
-#define ENIC_MAX_MTU			9000
+
+/* Does not include (possible) inserted VLAN tag and FCS */
+#define ENIC_DEFAULT_MAX_PKT_SIZE	9022
 
 #define ENIC_MULTICAST_PERFECT_FILTERS	32
 #define ENIC_UNICAST_PERFECT_FILTERS	32
-- 
2.7.0

[PATCH v2 2/2] enic: add an update MTU function for non-Rx scatter mode

[John Daley]

Provide an update MTU callback. The function returns -ENOTSUP
if Rx scatter is enabled. Updating the MTU to be greater than
the value configured via the Cisco CIMC/UCSM management interface
is allowed provided it is still less than the maximum egress packet
size allowed by the NIC minus the size of the L2 header.

Signed-off-by: John Daley <johndale@cisco.com>
---
v2: Squished 3/4 and 4/4 into 1 patch. Slight change of wording
and fixed typo in commit message.

 doc/guides/nics/overview.rst   |  2 +-
 drivers/net/enic/enic.h        |  1 +
 drivers/net/enic/enic_ethdev.c | 10 +++++++++-
 drivers/net/enic/enic_main.c   | 44 ++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 55 insertions(+), 2 deletions(-)

diff --git a/doc/guides/nics/overview.rst b/doc/guides/nics/overview.rst
index f94f6a2..872392b 100644
--- a/doc/guides/nics/overview.rst
+++ b/doc/guides/nics/overview.rst
@@ -92,7 +92,7 @@ Most of these differences are summarized below.
    Queue status event                                                                       Y
    Rx interrupt                     Y     Y Y Y Y Y Y Y Y Y Y Y Y Y Y
    Queue start/stop           Y   Y   Y Y Y Y Y Y     Y Y     Y Y Y Y Y Y               Y Y   Y Y
-   MTU update                     Y Y Y           Y   Y Y Y Y         Y Y         Y Y     Y
+   MTU update                     Y Y Y Y         Y   Y Y Y Y         Y Y         Y Y     Y
    Jumbo frame                    Y Y Y Y Y Y Y Y Y   Y Y Y Y Y Y Y Y Y Y       Y Y Y     Y
    Scattered Rx                   Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y               Y Y   Y
    LRO                                                Y Y Y Y
diff --git a/drivers/net/enic/enic.h b/drivers/net/enic/enic.h
index b557e12..9f5740d 100644
--- a/drivers/net/enic/enic.h
+++ b/drivers/net/enic/enic.h
@@ -263,4 +263,5 @@ uint16_t enic_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			uint16_t nb_pkts);
 uint16_t enic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 			       uint16_t nb_pkts);
+int enic_set_mtu(struct enic *enic, uint16_t new_mtu);
 #endif /* _ENIC_H_ */
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 6fa54b2..a7ce064 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -528,6 +528,14 @@ static void enicpmd_remove_mac_addr(struct rte_eth_dev *eth_dev, __rte_unused ui
 	enic_del_mac_address(enic);
 }
 
+static int enicpmd_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu)
+{
+	struct enic *enic = pmd_priv(eth_dev);
+
+	ENICPMD_FUNC_TRACE();
+	return enic_set_mtu(enic, mtu);
+}
+
 static const struct eth_dev_ops enicpmd_eth_dev_ops = {
 	.dev_configure        = enicpmd_dev_configure,
 	.dev_start            = enicpmd_dev_start,
@@ -545,7 +553,7 @@ static const struct eth_dev_ops enicpmd_eth_dev_ops = {
 	.queue_stats_mapping_set = NULL,
 	.dev_infos_get        = enicpmd_dev_info_get,
 	.dev_supported_ptypes_get = enicpmd_dev_supported_ptypes_get,
-	.mtu_set              = NULL,
+	.mtu_set              = enicpmd_mtu_set,
 	.vlan_filter_set      = enicpmd_vlan_filter_set,
 	.vlan_tpid_set        = NULL,
 	.vlan_offload_set     = enicpmd_vlan_offload_set,
diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index 91883f8..1d16f0e 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -1005,6 +1005,50 @@ int enic_set_vnic_res(struct enic *enic)
 	return rc;
 }
 
+/* The Cisco NIC can send and receive packets up to a max packet size
+ * determined by the NIC type and firmware. There is also an MTU
+ * configured into the NIC via the CIMC/UCSM management interface
+ * which can be overridden by this function (up to the max packet size).
+ * Depending on the network setup, doing so may cause packet drops
+ * and unexpected behavior.
+ */
+int enic_set_mtu(struct enic *enic, uint16_t new_mtu)
+{
+	uint16_t old_mtu;	/* previous setting */
+	uint16_t config_mtu;	/* Value configured into NIC via CIMC/UCSM */
+	struct rte_eth_dev *eth_dev = enic->rte_dev;
+
+	old_mtu = eth_dev->data->mtu;
+	config_mtu = enic->config.mtu;
+
+	/* only works with Rx scatter disabled */
+	if (enic->rte_dev->data->dev_conf.rxmode.enable_scatter)
+		return -ENOTSUP;
+
+	if (new_mtu > enic->max_mtu) {
+		dev_err(enic,
+			"MTU not updated: requested (%u) greater than max (%u)\n",
+			new_mtu, enic->max_mtu);
+		return -EINVAL;
+	}
+	if (new_mtu < ENIC_MIN_MTU) {
+		dev_info(enic,
+			"MTU not updated: requested (%u) less than min (%u)\n",
+			new_mtu, ENIC_MIN_MTU);
+		return -EINVAL;
+	}
+	if (new_mtu > config_mtu)
+		dev_warning(enic,
+			"MTU (%u) is greater than value configured in NIC (%u)\n",
+			new_mtu, config_mtu);
+
+	/* update the mtu */
+	eth_dev->data->mtu = new_mtu;
+
+	dev_info(enic, "MTU changed from %u to %u\n",  old_mtu, new_mtu);
+	return 0;
+}
+
 static int enic_dev_init(struct enic *enic)
 {
 	int err;
-- 
2.7.0

Re: [PATCH 2/4] enic: set the max allowed MTU for the NIC

[John Daley (johndale)]

Hi Bruce,

> > * What was the MTU set to by default before this patch is applied? Was
> > it just set to 1518 or something else?
> > * What happens, if anything, if buffers bigger than the MTU size are sent
> down?
> This is obviously referring to buffers bigger than MTU on TX. There is also the
> question of what happens if buffer sizes smaller than MTU are provided on
> RX.

I think I answered all your questions in the revised commit messages of the v2 patchset (and then some) except this last one. Enic doesn't do any checking on Rx that buffers are greater than the MTU since it would affect performance. However if a packet is bigger than a buffer and Rx scatter is disabled, the packet will be dropped and 'imissed' incremented.

Thanks,
Johnd

Re: [PATCH v2 1/2] enic: determine max egress packet size and max MTU

[Bruce Richardson]

On Fri, Jun 24, 2016 at 03:29:27PM -0700, John Daley wrote:
> Pull in common VNIC code which enables querying for max egress
> packet size with newer firmware via a device command. If the
> field is non-zero, it is the max egress packet size. If it is
> 0, the default value (9022) can safely be assumed. The value
> for 1300 series VICS using firmware versions >= 3.1.2 for blade
> series and >= 2.0.13 for rack series servers is 9208.
> 
> Tx buffers can be emitted only if they are less than the max egress
> packet size regardless of the MTU setting (the MTU is advisory).
> The max egress packet size can used to determine the upper limit
> of the MTU since the enic can also receive packets of size greater
> than max egress packet size. A max_mtu variable is added with
> a value of max egress packet size minus L2 header size.
> 
> The default MTU is set via the CIMC/UCSM management interface and
> currently allows value up to 9000. If the value is changed, the
> host must be reboot. To avoid the reboot and allow MTU values
> up to the max capability of the NIC, MTU update capability will
> be added with a max value capped by max_mtu.
> 
> Signed-off-by: John Daley <johndale@cisco.com>
> ---
> v2: Squished patch 1/4 and 2/4 into one. Tried to do a little
> better explanaiton of the intent of the patch in the commit
> message.
> 

Patchset applied to dpdk-next-net/rel_16_07

/Bruce