[Intel-wired-lan] [next PATCH S70 01/12] i40e/i40evf: Add tracepoints

[Intel-wired-lan] [next PATCH S70 01/12] i40e/i40evf: Add tracepoints

[Alice Michael]

From: "Peterson, Scott D" <scott.d.peterson@intel.com>

This patch adds tracepoints to the i40e and i40evf drivers to which
BPF programs can be attached for feature testing and verification.
It's expected that an attached BPF program will identify and count or
log some interesting subset of traffic. The bcc-tools package is
helpful there for containing all the BPF arcana in a handy Python
wrapper. Though you can make these tracepoints log trace messages, the
messages themselves probably won't be very useful (other to verify the
tracepoint is being called while you're debugging your BPF program).

The idea here is that tracepoints have such low performance cost when
disabled that we can leave these in the upstream drivers. This may
eventually enable the instrumentation of unmodified customer systems
should the need arise to verify a NIC feature is working as expected.
In general this enables one set of feature verification tools to be
used on these drivers whether they're built with the kernel or
separately.

Users are advised against using these tracepoints for anything other
than a diagnostic tool. They have a performance impact when enabled,
and their exact placement and form may change as we see how well they
work in practice for the purposes above.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
Change-ID: Id6014a7322c0e6d08068114dd20bd156f2f6435e
---
Testing Hints:

See that performance is unaffected while tracepoints are present
but disabled (default state). Enable them all and see they appear in
/sys/kernel/debug/tracing/trace when exercised.

When disabled, a tracepoint reduces to a 5-byte no-op.
When enabled, that code is patched to jump to the tracepoint clause,
which is located somewhere nearby.  See include/linux/jump_label.h for
info in static keys, and samples/trace-events/trace-events-samples.[ch]
for info on the trace events.

To test tracepoints:
insmod i40e and bring up a link
cd /sys/kernel/debug/tracing
echo > trace
echo 1 > events/i40e/enable
send something over i40e
more trace
echo 0 > events/i40e/enable

i40evf trace events are in events/i40evf, and enabled separately.

A sample BPF program (CORE/example_tp_bpf.py) is available, but won't
be submitted upstream and so is not included in this patch.

It demonstrates how to access the structures pointed to by the tracepoint
args from BPF. See the comments in that script for details and usage. As
is it will attach itself to one of these tracepoints and emit mostly
useless messages when the i40e_clean_rx_irq tracepoint is invoked. 
 drivers/net/ethernet/intel/i40e/Makefile        |   3 +
 drivers/net/ethernet/intel/i40e/i40e_main.c     |   6 +
 drivers/net/ethernet/intel/i40e/i40e_trace.h    | 229 ++++++++++++++++++++++++
 drivers/net/ethernet/intel/i40e/i40e_txrx.c     |   9 +
 drivers/net/ethernet/intel/i40evf/Makefile      |   3 +
 drivers/net/ethernet/intel/i40evf/i40e_trace.h  | 229 ++++++++++++++++++++++++
 drivers/net/ethernet/intel/i40evf/i40e_txrx.c   |   9 +
 drivers/net/ethernet/intel/i40evf/i40evf_main.c |   7 +
 8 files changed, 495 insertions(+)
 create mode 100644 drivers/net/ethernet/intel/i40e/i40e_trace.h
 create mode 100644 drivers/net/ethernet/intel/i40evf/i40e_trace.h

diff --git a/drivers/net/ethernet/intel/i40e/Makefile b/drivers/net/ethernet/intel/i40e/Makefile
index 4f454d3..3da482c 100644
--- a/drivers/net/ethernet/intel/i40e/Makefile
+++ b/drivers/net/ethernet/intel/i40e/Makefile
@@ -28,6 +28,9 @@
 # Makefile for the Intel(R) Ethernet Connection XL710 (i40e.ko) driver
 #
 
+ccflags-y += -I$(src)
+subdir-ccflags-y += -I$(src)
+
 obj-$(CONFIG_I40E) += i40e.o
 
 i40e-objs := i40e_main.o \
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 0c32304..4b507ce 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -32,6 +32,12 @@
 #include "i40e.h"
 #include "i40e_diag.h"
 #include <net/udp_tunnel.h>
+/* All i40e tracepoints are defined by the include below, which
+ * must be included exactly once across the whole kernel with
+ * CREATE_TRACE_POINTS defined
+ */
+#define CREATE_TRACE_POINTS
+#include "i40e_trace.h"
 
 const char i40e_driver_name[] = "i40e";
 static const char i40e_driver_string[] =
diff --git a/drivers/net/ethernet/intel/i40e/i40e_trace.h b/drivers/net/ethernet/intel/i40e/i40e_trace.h
new file mode 100644
index 0000000..d3e55f5
--- /dev/null
+++ b/drivers/net/ethernet/intel/i40e/i40e_trace.h
@@ -0,0 +1,229 @@
+/*******************************************************************************
+ *
+ * Intel(R) 40-10 Gigabit Ethernet Connection Network Driver
+ * Copyright(c) 2013 - 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * Contact Information:
+ * e1000-devel Mailing List <e1000-devel@lists.sourceforge.net>
+ * Intel Corporation, 5200 N.E. Elam Young Parkway, Hillsboro, OR 97124-6497
+ *
+ ******************************************************************************/
+
+/* Modeled on trace-events-sample.h */
+
+/* The trace subsystem name for i40e will be "i40e".
+ *
+ * This file is named i40e_trace.h.
+ *
+ * Since this include file's name is different from the trace
+ * subsystem name, we'll have to define TRACE_INCLUDE_FILE at the end
+ * of this file.
+ */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM i40e
+
+/* See trace-events-sample.h for a detailed description of why this
+ * guard clause is different from most normal include files.
+ */
+#if !defined(_I40E_TRACE_H_) || defined(TRACE_HEADER_MULTI_READ)
+#define _I40E_TRACE_H_
+
+#include <linux/tracepoint.h>
+
+/**
+ * i40e_trace() macro enables shared code to refer to trace points
+ * like:
+ *
+ * trace_i40e{,vf}_example(args...)
+ *
+ * ... as:
+ *
+ * i40e_trace(example, args...)
+ *
+ * ... to resolve to the PF or VF version of the tracepoint without
+ * ifdefs, and to allow tracepoints to be disabled entirely at build
+ * time.
+ *
+ * Trace point should always be referred to in the driver via this
+ * macro.
+ *
+ * Similarly, i40e_trace_enabled(trace_name) wraps references to
+ * trace_i40e{,vf}_<trace_name>_enabled() functions.
+ */
+#define _I40E_TRACE_NAME(trace_name) (trace_ ## i40e ## _ ## trace_name)
+#define I40E_TRACE_NAME(trace_name) _I40E_TRACE_NAME(trace_name)
+
+#define i40e_trace(trace_name, args...) I40E_TRACE_NAME(trace_name)(args)
+
+#define i40e_trace_enabled(trace_name) I40E_TRACE_NAME(trace_name##_enabled)()
+
+/* Events common to PF and VF. Corresponding versions will be defined
+ * for both, named trace_i40e_* and trace_i40evf_*. The i40e_trace()
+ * macro above will select the right trace point name for the driver
+ * being built from shared code.
+ */
+
+/* Events related to a vsi & ring */
+DECLARE_EVENT_CLASS(
+	i40e_tx_template,
+
+	TP_PROTO(struct i40e_ring *ring,
+		 struct i40e_tx_desc *desc,
+		 struct i40e_tx_buffer *buf),
+
+	TP_ARGS(ring, desc, buf),
+
+	/* The convention here is to make the first fields in the
+	 * TP_STRUCT match the TP_PROTO exactly. This enables the use
+	 * of the args struct generated by the tplist tool (from the
+	 * bcc-tools package) to be used for those fields. To access
+	 * fields other than the tracepoint args will require the
+	 * tplist output to be adjusted.
+	 */
+	TP_STRUCT__entry(
+		__field(void*, ring)
+		__field(void*, desc)
+		__field(void*, buf)
+		__string(devname, ring->netdev->name)
+	),
+
+	TP_fast_assign(
+		__entry->ring = ring;
+		__entry->desc = desc;
+		__entry->buf = buf;
+		__assign_str(devname, ring->netdev->name);
+	),
+
+	TP_printk(
+		"netdev: %s ring: %p desc: %p buf %p",
+		__get_str(devname), __entry->ring,
+		__entry->desc, __entry->buf)
+);
+
+DEFINE_EVENT(
+	i40e_tx_template, i40e_clean_tx_irq,
+	TP_PROTO(struct i40e_ring *ring,
+		 struct i40e_tx_desc *desc,
+		 struct i40e_tx_buffer *buf),
+
+	TP_ARGS(ring, desc, buf));
+
+DEFINE_EVENT(
+	i40e_tx_template, i40e_clean_tx_irq_unmap,
+	TP_PROTO(struct i40e_ring *ring,
+		 struct i40e_tx_desc *desc,
+		 struct i40e_tx_buffer *buf),
+
+	TP_ARGS(ring, desc, buf));
+
+DECLARE_EVENT_CLASS(
+	i40e_rx_template,
+
+	TP_PROTO(struct i40e_ring *ring,
+		 union i40e_32byte_rx_desc *desc,
+		 struct sk_buff *skb),
+
+	TP_ARGS(ring, desc, skb),
+
+	TP_STRUCT__entry(
+		__field(void*, ring)
+		__field(void*, desc)
+		__field(void*, skb)
+		__string(devname, ring->netdev->name)
+	),
+
+	TP_fast_assign(
+		__entry->ring = ring;
+		__entry->desc = desc;
+		__entry->skb = skb;
+		__assign_str(devname, ring->netdev->name);
+	),
+
+	TP_printk(
+		"netdev: %s ring: %p desc: %p skb %p",
+		__get_str(devname), __entry->ring,
+		__entry->desc, __entry->skb)
+);
+
+DEFINE_EVENT(
+	i40e_rx_template, i40e_clean_rx_irq,
+	TP_PROTO(struct i40e_ring *ring,
+		 union i40e_32byte_rx_desc *desc,
+		 struct sk_buff *skb),
+
+	TP_ARGS(ring, desc, skb));
+
+DEFINE_EVENT(
+	i40e_rx_template, i40e_clean_rx_irq_rx,
+	TP_PROTO(struct i40e_ring *ring,
+		 union i40e_32byte_rx_desc *desc,
+		 struct sk_buff *skb),
+
+	TP_ARGS(ring, desc, skb));
+
+DECLARE_EVENT_CLASS(
+	i40e_xmit_template,
+
+	TP_PROTO(struct sk_buff *skb,
+		 struct i40e_ring *ring),
+
+	TP_ARGS(skb, ring),
+
+	TP_STRUCT__entry(
+		__field(void*, skb)
+		__field(void*, ring)
+		__string(devname, ring->netdev->name)
+	),
+
+	TP_fast_assign(
+		__entry->skb = skb;
+		__entry->ring = ring;
+		__assign_str(devname, ring->netdev->name);
+	),
+
+	TP_printk(
+		"netdev: %s skb: %p ring: %p",
+		__get_str(devname), __entry->skb,
+		__entry->ring)
+);
+
+DEFINE_EVENT(
+	i40e_xmit_template, i40e_xmit_frame_ring,
+	TP_PROTO(struct sk_buff *skb,
+		 struct i40e_ring *ring),
+
+	TP_ARGS(skb, ring));
+
+DEFINE_EVENT(
+	i40e_xmit_template, i40e_xmit_frame_ring_drop,
+	TP_PROTO(struct sk_buff *skb,
+		 struct i40e_ring *ring),
+
+	TP_ARGS(skb, ring));
+
+/* Events unique to the PF. */
+
+#endif /* _I40E_TRACE_H_ */
+/* This must be outside ifdef _I40E_TRACE_H */
+
+/* This trace include file is not located in the .../include/trace
+ * with the kernel tracepoint definitions, because we're a loadable
+ * module.
+ */
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE i40e_trace
+#include <trace/define_trace.h>
diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
index 5f04929..3288572 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
@@ -27,6 +27,7 @@
 #include <linux/prefetch.h>
 #include <net/busy_poll.h>
 #include "i40e.h"
+#include "i40e_trace.h"
 #include "i40e_prototype.h"
 
 static inline __le64 build_ctob(u32 td_cmd, u32 td_offset, unsigned int size,
@@ -765,6 +766,7 @@ static bool i40e_clean_tx_irq(struct i40e_vsi *vsi,
 		/* prevent any other reads prior to eop_desc */
 		read_barrier_depends();
 
+		i40e_trace(clean_tx_irq, tx_ring, tx_desc, tx_buf);
 		/* we have caught up to head, no work left to do */
 		if (tx_head == tx_desc)
 			break;
@@ -791,6 +793,8 @@ static bool i40e_clean_tx_irq(struct i40e_vsi *vsi,
 
 		/* unmap remaining buffers */
 		while (tx_desc != eop_desc) {
+			i40e_trace(clean_tx_irq_unmap,
+				   tx_ring, tx_desc, tx_buf);
 
 			tx_buf++;
 			tx_desc++;
@@ -2062,6 +2066,7 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)
 		if (!size)
 			break;
 
+		i40e_trace(clean_rx_irq, rx_ring, rx_desc, skb);
 		rx_buffer = i40e_get_rx_buffer(rx_ring, size);
 
 		/* retrieve a buffer from the ring */
@@ -2114,6 +2119,7 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)
 		vlan_tag = (qword & BIT(I40E_RX_DESC_STATUS_L2TAG1P_SHIFT)) ?
 			   le16_to_cpu(rx_desc->wb.qword0.lo_dword.l2tag1) : 0;
 
+		i40e_trace(clean_rx_irq_rx, rx_ring, rx_desc, skb);
 		i40e_receive_skb(rx_ring, skb, vlan_tag, lpbk);
 		skb = NULL;
 
@@ -3247,6 +3253,8 @@ static netdev_tx_t i40e_xmit_frame_ring(struct sk_buff *skb,
 	/* prefetch the data, we'll need it later */
 	prefetch(skb->data);
 
+	i40e_trace(xmit_frame_ring, skb, tx_ring);
+
 	count = i40e_xmit_descriptor_count(skb);
 	if (i40e_chk_linearize(skb, count)) {
 		if (__skb_linearize(skb)) {
@@ -3327,6 +3335,7 @@ static netdev_tx_t i40e_xmit_frame_ring(struct sk_buff *skb,
 	return NETDEV_TX_OK;
 
 out_drop:
+	i40e_trace(xmit_frame_ring_drop, first->skb, tx_ring);
 	dev_kfree_skb_any(first->skb);
 	first->skb = NULL;
 	return NETDEV_TX_OK;
diff --git a/drivers/net/ethernet/intel/i40evf/Makefile b/drivers/net/ethernet/intel/i40evf/Makefile
index 827c7a6..a393f4a 100644
--- a/drivers/net/ethernet/intel/i40evf/Makefile
+++ b/drivers/net/ethernet/intel/i40evf/Makefile
@@ -29,6 +29,9 @@
 #
 #
 
+ccflags-y += -I$(src)
+subdir-ccflags-y += -I$(src)
+
 obj-$(CONFIG_I40EVF) += i40evf.o
 
 i40evf-objs :=	i40evf_main.o i40evf_ethtool.o i40evf_virtchnl.o \
diff --git a/drivers/net/ethernet/intel/i40evf/i40e_trace.h b/drivers/net/ethernet/intel/i40evf/i40e_trace.h
new file mode 100644
index 0000000..9a5100b
--- /dev/null
+++ b/drivers/net/ethernet/intel/i40evf/i40e_trace.h
@@ -0,0 +1,229 @@
+/*******************************************************************************
+ *
+ * Intel(R) 40-10 Gigabit Ethernet Virtual Function Driver
+ * Copyright(c) 2013 - 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * Contact Information:
+ * e1000-devel Mailing List <e1000-devel@lists.sourceforge.net>
+ * Intel Corporation, 5200 N.E. Elam Young Parkway, Hillsboro, OR 97124-6497
+ *
+ ******************************************************************************/
+
+/* Modeled on trace-events-sample.h */
+
+/* The trace subsystem name for i40evf will be "i40evf".
+ *
+ * This file is named i40e_trace.h.
+ *
+ * Since this include file's name is different from the trace
+ * subsystem name, we'll have to define TRACE_INCLUDE_FILE at the end
+ * of this file.
+ */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM i40evf
+
+/* See trace-events-sample.h for a detailed description of why this
+ * guard clause is different from most normal include files.
+ */
+#if !defined(_I40E_TRACE_H_) || defined(TRACE_HEADER_MULTI_READ)
+#define _I40E_TRACE_H_
+
+#include <linux/tracepoint.h>
+
+/**
+ * i40e_trace() macro enables shared code to refer to trace points
+ * like:
+ *
+ * trace_i40e{,vf}_example(args...)
+ *
+ * ... as:
+ *
+ * i40e_trace(example, args...)
+ *
+ * ... to resolve to the PF or VF version of the tracepoint without
+ * ifdefs, and to allow tracepoints to be disabled entirely at build
+ * time.
+ *
+ * Trace point should always be referred to in the driver via this
+ * macro.
+ *
+ * Similarly, i40e_trace_enabled(trace_name) wraps references to
+ * trace_i40e{,vf}_<trace_name>_enabled() functions.
+ */
+#define _I40E_TRACE_NAME(trace_name) (trace_ ## i40evf ## _ ## trace_name)
+#define I40E_TRACE_NAME(trace_name) _I40E_TRACE_NAME(trace_name)
+
+#define i40e_trace(trace_name, args...) I40E_TRACE_NAME(trace_name)(args)
+
+#define i40e_trace_enabled(trace_name) I40E_TRACE_NAME(trace_name##_enabled)()
+
+/* Events common to PF and VF. Corresponding versions will be defined
+ * for both, named trace_i40e_* and trace_i40evf_*. The i40e_trace()
+ * macro above will select the right trace point name for the driver
+ * being built from shared code.
+ */
+
+/* Events related to a vsi & ring */
+DECLARE_EVENT_CLASS(
+	i40evf_tx_template,
+
+	TP_PROTO(struct i40e_ring *ring,
+		 struct i40e_tx_desc *desc,
+		 struct i40e_tx_buffer *buf),
+
+	TP_ARGS(ring, desc, buf),
+
+	/* The convention here is to make the first fields in the
+	 * TP_STRUCT match the TP_PROTO exactly. This enables the use
+	 * of the args struct generated by the tplist tool (from the
+	 * bcc-tools package) to be used for those fields. To access
+	 * fields other than the tracepoint args will require the
+	 * tplist output to be adjusted.
+	 */
+	TP_STRUCT__entry(
+		__field(void*, ring)
+		__field(void*, desc)
+		__field(void*, buf)
+		__string(devname, ring->netdev->name)
+	),
+
+	TP_fast_assign(
+		__entry->ring = ring;
+		__entry->desc = desc;
+		__entry->buf = buf;
+		__assign_str(devname, ring->netdev->name);
+	),
+
+	TP_printk(
+		"netdev: %s ring: %p desc: %p buf %p",
+		__get_str(devname), __entry->ring,
+		__entry->desc, __entry->buf)
+);
+
+DEFINE_EVENT(
+	i40evf_tx_template, i40evf_clean_tx_irq,
+	TP_PROTO(struct i40e_ring *ring,
+		 struct i40e_tx_desc *desc,
+		 struct i40e_tx_buffer *buf),
+
+	TP_ARGS(ring, desc, buf));
+
+DEFINE_EVENT(
+	i40evf_tx_template, i40evf_clean_tx_irq_unmap,
+	TP_PROTO(struct i40e_ring *ring,
+		 struct i40e_tx_desc *desc,
+		 struct i40e_tx_buffer *buf),
+
+	TP_ARGS(ring, desc, buf));
+
+DECLARE_EVENT_CLASS(
+	i40evf_rx_template,
+
+	TP_PROTO(struct i40e_ring *ring,
+		 union i40e_32byte_rx_desc *desc,
+		 struct sk_buff *skb),
+
+	TP_ARGS(ring, desc, skb),
+
+	TP_STRUCT__entry(
+		__field(void*, ring)
+		__field(void*, desc)
+		__field(void*, skb)
+		__string(devname, ring->netdev->name)
+	),
+
+	TP_fast_assign(
+		__entry->ring = ring;
+		__entry->desc = desc;
+		__entry->skb = skb;
+		__assign_str(devname, ring->netdev->name);
+	),
+
+	TP_printk(
+		"netdev: %s ring: %p desc: %p skb %p",
+		__get_str(devname), __entry->ring,
+		__entry->desc, __entry->skb)
+);
+
+DEFINE_EVENT(
+	i40evf_rx_template, i40evf_clean_rx_irq,
+	TP_PROTO(struct i40e_ring *ring,
+		 union i40e_32byte_rx_desc *desc,
+		 struct sk_buff *skb),
+
+	TP_ARGS(ring, desc, skb));
+
+DEFINE_EVENT(
+	i40evf_rx_template, i40evf_clean_rx_irq_rx,
+	TP_PROTO(struct i40e_ring *ring,
+		 union i40e_32byte_rx_desc *desc,
+		 struct sk_buff *skb),
+
+	TP_ARGS(ring, desc, skb));
+
+DECLARE_EVENT_CLASS(
+	i40evf_xmit_template,
+
+	TP_PROTO(struct sk_buff *skb,
+		 struct i40e_ring *ring),
+
+	TP_ARGS(skb, ring),
+
+	TP_STRUCT__entry(
+		__field(void*, skb)
+		__field(void*, ring)
+		__string(devname, ring->netdev->name)
+	),
+
+	TP_fast_assign(
+		__entry->skb = skb;
+		__entry->ring = ring;
+		__assign_str(devname, ring->netdev->name);
+	),
+
+	TP_printk(
+		"netdev: %s skb: %p ring: %p",
+		__get_str(devname), __entry->skb,
+		__entry->ring)
+);
+
+DEFINE_EVENT(
+	i40evf_xmit_template, i40evf_xmit_frame_ring,
+	TP_PROTO(struct sk_buff *skb,
+		 struct i40e_ring *ring),
+
+	TP_ARGS(skb, ring));
+
+DEFINE_EVENT(
+	i40evf_xmit_template, i40evf_xmit_frame_ring_drop,
+	TP_PROTO(struct sk_buff *skb,
+		 struct i40e_ring *ring),
+
+	TP_ARGS(skb, ring));
+
+/* Events unique to the VF. */
+
+#endif /* _I40E_TRACE_H_ */
+/* This must be outside ifdef _I40E_TRACE_H */
+
+/* This trace include file is not located in the .../include/trace
+ * with the kernel tracepoint definitions, because we're a loadable
+ * module.
+ */
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE i40e_trace
+#include <trace/define_trace.h>
diff --git a/drivers/net/ethernet/intel/i40evf/i40e_txrx.c b/drivers/net/ethernet/intel/i40evf/i40e_txrx.c
index 80931e3..34e96d9 100644
--- a/drivers/net/ethernet/intel/i40evf/i40e_txrx.c
+++ b/drivers/net/ethernet/intel/i40evf/i40e_txrx.c
@@ -28,6 +28,7 @@
 #include <net/busy_poll.h>
 
 #include "i40evf.h"
+#include "i40e_trace.h"
 #include "i40e_prototype.h"
 
 static inline __le64 build_ctob(u32 td_cmd, u32 td_offset, unsigned int size,
@@ -180,6 +181,7 @@ static bool i40e_clean_tx_irq(struct i40e_vsi *vsi,
 		/* prevent any other reads prior to eop_desc */
 		read_barrier_depends();
 
+		i40e_trace(clean_tx_irq, tx_ring, tx_desc, tx_buf);
 		/* if the descriptor isn't done, no work yet to do */
 		if (!(eop_desc->cmd_type_offset_bsz &
 		      cpu_to_le64(I40E_TX_DESC_DTYPE_DESC_DONE)))
@@ -207,6 +209,8 @@ static bool i40e_clean_tx_irq(struct i40e_vsi *vsi,
 
 		/* unmap remaining buffers */
 		while (tx_desc != eop_desc) {
+			i40e_trace(clean_tx_irq_unmap,
+				   tx_ring, tx_desc, tx_buf);
 
 			tx_buf++;
 			tx_desc++;
@@ -1329,6 +1333,7 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)
 		if (!size)
 			break;
 
+		i40e_trace(clean_rx_irq, rx_ring, rx_desc, skb);
 		rx_buffer = i40e_get_rx_buffer(rx_ring, size);
 
 		/* retrieve a buffer from the ring */
@@ -1382,6 +1387,7 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)
 		vlan_tag = (qword & BIT(I40E_RX_DESC_STATUS_L2TAG1P_SHIFT)) ?
 			   le16_to_cpu(rx_desc->wb.qword0.lo_dword.l2tag1) : 0;
 
+		i40e_trace(clean_rx_irq_rx, rx_ring, rx_desc, skb);
 		i40e_receive_skb(rx_ring, skb, vlan_tag);
 		skb = NULL;
 
@@ -2223,6 +2229,8 @@ static netdev_tx_t i40e_xmit_frame_ring(struct sk_buff *skb,
 	/* prefetch the data, we'll need it later */
 	prefetch(skb->data);
 
+	i40e_trace(xmit_frame_ring, skb, tx_ring);
+
 	count = i40e_xmit_descriptor_count(skb);
 	if (i40e_chk_linearize(skb, count)) {
 		if (__skb_linearize(skb)) {
@@ -2290,6 +2298,7 @@ static netdev_tx_t i40e_xmit_frame_ring(struct sk_buff *skb,
 	return NETDEV_TX_OK;
 
 out_drop:
+	i40e_trace(xmit_frame_ring_drop, first->skb, tx_ring);
 	dev_kfree_skb_any(first->skb);
 	first->skb = NULL;
 	return NETDEV_TX_OK;
diff --git a/drivers/net/ethernet/intel/i40evf/i40evf_main.c b/drivers/net/ethernet/intel/i40evf/i40evf_main.c
index 671913c..5915273 100644
--- a/drivers/net/ethernet/intel/i40evf/i40evf_main.c
+++ b/drivers/net/ethernet/intel/i40evf/i40evf_main.c
@@ -27,6 +27,13 @@
 #include "i40evf.h"
 #include "i40e_prototype.h"
 #include "i40evf_client.h"
+/* All i40evf tracepoints are defined by the include below, which must
+ * be included exactly once across the whole kernel with
+ * CREATE_TRACE_POINTS defined
+ */
+#define CREATE_TRACE_POINTS
+#include "i40e_trace.h"
+
 static int i40evf_setup_all_tx_resources(struct i40evf_adapter *adapter);
 static int i40evf_setup_all_rx_resources(struct i40evf_adapter *adapter);
 static int i40evf_close(struct net_device *netdev);
-- 
2.9.3

[Intel-wired-lan] [next PATCH S70 02/12] i40e: DCR 287 new AQ commands

[Alice Michael]

From: Jingjing Wu <jingjing.wu@intel.com>

Add admin queue functions for Pipeline Personalization Profile AQ
commands defined in DCR 287:
 - Write Recipe Command buffer (Opcode: 0x0270)
 - Get Applied Profiles list (Opcode: 0x0271)

Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Change-ID: I558b4145364140f624013af48d4bbf79d21ebb0d
---
 drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h  |  34 ++++
 drivers/net/ethernet/intel/i40e/i40e_common.c      | 212 +++++++++++++++++++++
 drivers/net/ethernet/intel/i40e/i40e_prototype.h   |  17 ++
 drivers/net/ethernet/intel/i40e/i40e_type.h        |  80 ++++++++
 .../net/ethernet/intel/i40evf/i40e_adminq_cmd.h    |  34 ++++
 drivers/net/ethernet/intel/i40evf/i40e_common.c    | 212 +++++++++++++++++++++
 drivers/net/ethernet/intel/i40evf/i40e_prototype.h |  17 ++
 drivers/net/ethernet/intel/i40evf/i40e_type.h      |  80 ++++++++
 8 files changed, 686 insertions(+)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h b/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h
index 251074c..5eb0411 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h
@@ -190,6 +190,10 @@ enum i40e_admin_queue_opc {
 	i40e_aqc_opc_add_mirror_rule	= 0x0260,
 	i40e_aqc_opc_delete_mirror_rule	= 0x0261,
 
+	/* Pipeline Personalization Profile */
+	i40e_aqc_opc_write_personalization_profile	= 0x0270,
+	i40e_aqc_opc_get_personalization_profile_list	= 0x0271,
+
 	/* DCB commands */
 	i40e_aqc_opc_dcb_ignore_pfc	= 0x0301,
 	i40e_aqc_opc_dcb_updated	= 0x0302,
@@ -1431,6 +1435,36 @@ struct i40e_aqc_add_delete_mirror_rule_completion {
 
 I40E_CHECK_CMD_LENGTH(i40e_aqc_add_delete_mirror_rule_completion);
 
+/* Pipeline Personalization Profile */
+struct i40e_aqc_write_personalization_profile {
+	u8      flags;
+	u8      reserved[3];
+	__le32  profile_track_id;
+	__le32  addr_high;
+	__le32  addr_low;
+};
+
+I40E_CHECK_CMD_LENGTH(i40e_aqc_write_personalization_profile);
+
+struct i40e_aqc_write_ppp_resp {
+	__le32 error_offset;
+	__le32 error_info;
+	__le32 addr_high;
+	__le32 addr_low;
+};
+
+struct i40e_aqc_get_applied_profiles {
+	u8      flags;
+#define I40E_AQC_GET_PPP_GET_CONF	0x1
+#define I40E_AQC_GET_PPP_GET_RDPU_CONF	0x2
+	u8      rsv[3];
+	__le32  reserved;
+	__le32  addr_high;
+	__le32  addr_low;
+};
+
+I40E_CHECK_CMD_LENGTH(i40e_aqc_get_applied_profiles);
+
 /* DCB 0x03xx*/
 
 /* PFC Ignore (direct 0x0301)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_common.c b/drivers/net/ethernet/intel/i40e/i40e_common.c
index f9db95a..72d1793 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_common.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_common.c
@@ -5042,3 +5042,215 @@ void i40e_write_rx_ctl(struct i40e_hw *hw, u32 reg_addr, u32 reg_val)
 	if (status || use_register)
 		wr32(hw, reg_addr, reg_val);
 }
+
+/**
+ * i40e_aq_write_ppp - Write pipeline personalization profile (ppp)
+ * @hw: pointer to the hw struct
+ * @buff: command buffer (size in bytes = buff_size)
+ * @buff_size: buffer size in bytes
+ * @track_id: package tracking id
+ * @error_offset: returns error offset
+ * @error_info: returns error information
+ * @cmd_details: pointer to command details structure or NULL
+ **/
+enum
+i40e_status_code i40e_aq_write_ppp(struct i40e_hw *hw, void *buff,
+				   u16 buff_size, u32 track_id,
+				   u32 *error_offset, u32 *error_info,
+				   struct i40e_asq_cmd_details *cmd_details)
+{
+	struct i40e_aq_desc desc;
+	struct i40e_aqc_write_personalization_profile *cmd =
+		(struct i40e_aqc_write_personalization_profile *)
+		&desc.params.raw;
+	struct i40e_aqc_write_ppp_resp *resp;
+	i40e_status status;
+
+	i40e_fill_default_direct_cmd_desc(&desc,
+					  i40e_aqc_opc_write_personalization_profile);
+
+	desc.flags |= cpu_to_le16(I40E_AQ_FLAG_BUF | I40E_AQ_FLAG_RD);
+	if (buff_size > I40E_AQ_LARGE_BUF)
+		desc.flags |= cpu_to_le16((u16)I40E_AQ_FLAG_LB);
+
+	desc.datalen = cpu_to_le16(buff_size);
+
+	cmd->profile_track_id = cpu_to_le32(track_id);
+
+	status = i40e_asq_send_command(hw, &desc, buff, buff_size, cmd_details);
+	if (!status) {
+		resp = (struct i40e_aqc_write_ppp_resp *)&desc.params.raw;
+		if (error_offset)
+			*error_offset = le32_to_cpu(resp->error_offset);
+		if (error_info)
+			*error_info = le32_to_cpu(resp->error_info);
+	}
+
+	return status;
+}
+
+/**
+ * i40e_aq_get_ppp_list - Read pipeline personalization profile (ppp)
+ * @hw: pointer to the hw struct
+ * @buff: command buffer (size in bytes = buff_size)
+ * @buff_size: buffer size in bytes
+ * @cmd_details: pointer to command details structure or NULL
+ **/
+enum
+i40e_status_code i40e_aq_get_ppp_list(struct i40e_hw *hw, void *buff,
+				      u16 buff_size, u8 flags,
+				       struct i40e_asq_cmd_details *cmd_details)
+{
+	struct i40e_aq_desc desc;
+	struct i40e_aqc_get_applied_profiles *cmd =
+		(struct i40e_aqc_get_applied_profiles *)&desc.params.raw;
+	i40e_status status;
+
+	i40e_fill_default_direct_cmd_desc(&desc,
+					  i40e_aqc_opc_get_personalization_profile_list);
+
+	desc.flags |= cpu_to_le16((u16)I40E_AQ_FLAG_BUF);
+	if (buff_size > I40E_AQ_LARGE_BUF)
+		desc.flags |= cpu_to_le16((u16)I40E_AQ_FLAG_LB);
+	desc.datalen = cpu_to_le16(buff_size);
+
+	cmd->flags = flags;
+
+	status = i40e_asq_send_command(hw, &desc, buff, buff_size, cmd_details);
+
+	return status;
+}
+
+/**
+ * i40e_find_segment_in_package
+ * @segment_type: the segment type to search for (i.e., SEGMENT_TYPE_I40E)
+ * @pkg_hdr: pointer to the package header to be searched
+ *
+ * This function searches a package file for a particular segment type. On
+ * success it returns a pointer to the segment header, otherwise it will
+ * return NULL.
+ **/
+struct i40e_generic_seg_header *
+i40e_find_segment_in_package(u32 segment_type,
+			     struct i40e_package_header *pkg_hdr)
+{
+	struct i40e_generic_seg_header *segment;
+	u32 i;
+
+	/* Search all package segments for the requested segment type */
+	for (i = 0; i < pkg_hdr->segment_count; i++) {
+		segment =
+			(struct i40e_generic_seg_header *)((u8 *)pkg_hdr +
+			 pkg_hdr->segment_offset[i]);
+
+		if (segment->type == segment_type)
+			return segment;
+	}
+
+	return NULL;
+}
+
+/**
+ * i40e_write_profile
+ * @hw: pointer to the hardware structure
+ * @profile: pointer to the profile segment of the package to be downloaded
+ * @track_id: package tracking id
+ *
+ * Handles the download of a complete package.
+ */
+enum i40e_status_code
+i40e_write_profile(struct i40e_hw *hw, struct i40e_profile_segment *profile,
+		   u32 track_id)
+{
+	i40e_status status = 0;
+	struct i40e_section_table *sec_tbl;
+	struct i40e_profile_section_header *sec = NULL;
+	u32 dev_cnt;
+	u32 vendor_dev_id;
+	u32 *nvm;
+	u32 section_size = 0;
+	u32 offset = 0, info = 0;
+	u32 i;
+
+	if (!track_id) {
+		i40e_debug(hw, I40E_DEBUG_PACKAGE, "Track_id can't be 0.");
+		return I40E_NOT_SUPPORTED;
+	}
+
+	dev_cnt = profile->device_table_count;
+
+	for (i = 0; i < dev_cnt; i++) {
+		vendor_dev_id = profile->device_table[i].vendor_dev_id;
+		if ((vendor_dev_id >> 16) == PCI_VENDOR_ID_INTEL)
+			if (hw->device_id == (vendor_dev_id & 0xFFFF))
+				break;
+	}
+	if (i == dev_cnt) {
+		i40e_debug(hw, I40E_DEBUG_PACKAGE, "Device doesn't support PPP");
+		return I40E_ERR_DEVICE_NOT_SUPPORTED;
+	}
+
+	nvm = (u32 *)&profile->device_table[dev_cnt];
+	sec_tbl = (struct i40e_section_table *)&nvm[nvm[0] + 1];
+
+	for (i = 0; i < sec_tbl->section_count; i++) {
+		sec = (struct i40e_profile_section_header *)((u8 *)profile +
+					     sec_tbl->section_offset[i]);
+
+		/* Skip 'AQ', 'note' and 'name' sections */
+		if (sec->section.type != SECTION_TYPE_MMIO)
+			continue;
+
+		section_size = sec->section.size +
+			sizeof(struct i40e_profile_section_header);
+
+		/* Write profile */
+		status = i40e_aq_write_ppp(hw, (void *)sec, (u16)section_size,
+					   track_id, &offset, &info, NULL);
+		if (status) {
+			i40e_debug(hw, I40E_DEBUG_PACKAGE,
+				   "Failed to write profile: offset %d, info %d",
+				   offset, info);
+			break;
+		}
+	}
+	return status;
+}
+
+/**
+ * i40e_add_pinfo_to_list
+ * @hw: pointer to the hardware structure
+ * @profile: pointer to the profile segment of the package
+ * @profile_info_sec: buffer for information section
+ * @track_id: package tracking id
+ *
+ * Register a profile to the list of loaded profiles.
+ */
+enum i40e_status_code
+i40e_add_pinfo_to_list(struct i40e_hw *hw,
+		       struct i40e_profile_segment *profile,
+		       u8 *profile_info_sec, u32 track_id)
+{
+	i40e_status status = 0;
+	struct i40e_profile_section_header *sec = NULL;
+	struct i40e_profile_info *pinfo;
+	u32 offset = 0, info = 0;
+
+	sec = (struct i40e_profile_section_header *)profile_info_sec;
+	sec->tbl_size = 1;
+	sec->data_end = sizeof(struct i40e_profile_section_header) +
+			sizeof(struct i40e_profile_info);
+	sec->section.type = SECTION_TYPE_INFO;
+	sec->section.offset = sizeof(struct i40e_profile_section_header);
+	sec->section.size = sizeof(struct i40e_profile_info);
+	pinfo = (struct i40e_profile_info *)(profile_info_sec +
+					     sec->section.offset);
+	pinfo->track_id = track_id;
+	pinfo->version = profile->version;
+	pinfo->op = I40E_PPP_ADD_TRACKID;
+	memcpy(pinfo->name, profile->name, I40E_PPP_NAME_SIZE);
+
+	status = i40e_aq_write_ppp(hw, (void *)sec, sec->data_end,
+				   track_id, &offset, &info, NULL);
+	return status;
+}
diff --git a/drivers/net/ethernet/intel/i40e/i40e_prototype.h b/drivers/net/ethernet/intel/i40e/i40e_prototype.h
index dfc5e59..611bbc7 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_prototype.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_prototype.h
@@ -377,4 +377,21 @@ i40e_status i40e_write_phy_register(struct i40e_hw *hw, u8 page, u16 reg,
 u8 i40e_get_phy_address(struct i40e_hw *hw, u8 dev_num);
 i40e_status i40e_blink_phy_link_led(struct i40e_hw *hw,
 				    u32 time, u32 interval);
+i40e_status i40e_aq_write_ppp(struct i40e_hw *hw, void *buff,
+			      u16 buff_size, u32 track_id,
+				u32 *error_offset, u32 *error_info,
+				struct i40e_asq_cmd_details *cmd_details);
+i40e_status i40e_aq_get_ppp_list(struct i40e_hw *hw, void *buff,
+				 u16 buff_size, u8 flags,
+				   struct i40e_asq_cmd_details *cmd_details);
+struct i40e_generic_seg_header *
+i40e_find_segment_in_package(u32 segment_type,
+			     struct i40e_package_header *pkg_header);
+enum i40e_status_code
+i40e_write_profile(struct i40e_hw *hw, struct i40e_profile_segment *i40e_seg,
+		   u32 track_id);
+enum i40e_status_code
+i40e_add_pinfo_to_list(struct i40e_hw *hw,
+		       struct i40e_profile_segment *profile,
+		       u8 *profile_info_sec, u32 track_id);
 #endif /* _I40E_PROTOTYPE_H_ */
diff --git a/drivers/net/ethernet/intel/i40e/i40e_type.h b/drivers/net/ethernet/intel/i40e/i40e_type.h
index 08364a4..f235899 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_type.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_type.h
@@ -78,6 +78,7 @@ enum i40e_debug_mask {
 	I40E_DEBUG_DCB			= 0x00000400,
 	I40E_DEBUG_DIAG			= 0x00000800,
 	I40E_DEBUG_FD			= 0x00001000,
+	I40E_DEBUG_PACKAGE		= 0x00002000,
 	I40E_DEBUG_IWARP		= 0x00F00000,
 	I40E_DEBUG_AQ_MESSAGE		= 0x01000000,
 	I40E_DEBUG_AQ_DESCRIPTOR	= 0x02000000,
@@ -1465,4 +1466,83 @@ struct i40e_lldp_variables {
 #define I40E_FLEX_56_MASK		(0x1ULL << I40E_FLEX_56_SHIFT)
 #define I40E_FLEX_57_SHIFT		6
 #define I40E_FLEX_57_MASK		(0x1ULL << I40E_FLEX_57_SHIFT)
+
+/* Version format for PPP */
+struct i40e_ppp_version {
+	u8 major;
+	u8 minor;
+	u8 update;
+	u8 draft;
+};
+
+#define I40E_PPP_NAME_SIZE	32
+
+/* Package header */
+struct i40e_package_header {
+	struct i40e_ppp_version version;
+	u32 segment_count;
+	u32 segment_offset[1];
+};
+
+/* Generic segment header */
+struct i40e_generic_seg_header {
+#define SEGMENT_TYPE_METADATA	0x00000001
+#define SEGMENT_TYPE_NOTES	0x00000002
+#define SEGMENT_TYPE_I40E	0x00000011
+#define SEGMENT_TYPE_X722	0x00000012
+	u32 type;
+	struct i40e_ppp_version version;
+	u32 size;
+	char name[I40E_PPP_NAME_SIZE];
+};
+
+struct i40e_metadata_segment {
+	struct i40e_generic_seg_header header;
+	struct i40e_ppp_version version;
+	u32 track_id;
+	char     name[I40E_PPP_NAME_SIZE];
+};
+
+struct i40e_device_id_entry {
+	u32 vendor_dev_id;
+	u32 sub_vendor_dev_id;
+};
+
+struct i40e_profile_segment {
+	struct i40e_generic_seg_header header;
+	struct i40e_ppp_version version;
+	char name[I40E_PPP_NAME_SIZE];
+	u32 device_table_count;
+	struct i40e_device_id_entry device_table[1];
+};
+
+struct i40e_section_table {
+	u32 section_count;
+	u32 section_offset[1];
+};
+
+struct i40e_profile_section_header {
+	u16 tbl_size;
+	u16 data_end;
+	struct {
+#define SECTION_TYPE_INFO	0x00000010
+#define SECTION_TYPE_MMIO	0x00000800
+#define SECTION_TYPE_AQ		0x00000801
+#define SECTION_TYPE_NOTE	0x80000000
+#define SECTION_TYPE_NAME	0x80000001
+		u32 type;
+		u32 offset;
+		u32 size;
+	} section;
+};
+
+struct i40e_profile_info {
+	u32 track_id;
+	struct i40e_ppp_version version;
+	u8 op;
+#define I40E_PPP_ADD_TRACKID		0x01
+#define I40E_PPP_REMOVE_TRACKID	0x02
+	u8 reserved[7];
+	u8 name[I40E_PPP_NAME_SIZE];
+};
 #endif /* _I40E_TYPE_H_ */
diff --git a/drivers/net/ethernet/intel/i40evf/i40e_adminq_cmd.h b/drivers/net/ethernet/intel/i40evf/i40e_adminq_cmd.h
index c28cb8f..91d8786 100644
--- a/drivers/net/ethernet/intel/i40evf/i40e_adminq_cmd.h
+++ b/drivers/net/ethernet/intel/i40evf/i40e_adminq_cmd.h
@@ -190,6 +190,10 @@ enum i40e_admin_queue_opc {
 	i40e_aqc_opc_add_mirror_rule	= 0x0260,
 	i40e_aqc_opc_delete_mirror_rule	= 0x0261,
 
+	/* Pipeline Personalization Profile */
+	i40e_aqc_opc_write_personalization_profile	= 0x0270,
+	i40e_aqc_opc_get_personalization_profile_list	= 0x0271,
+
 	/* DCB commands */
 	i40e_aqc_opc_dcb_ignore_pfc	= 0x0301,
 	i40e_aqc_opc_dcb_updated	= 0x0302,
@@ -1426,6 +1430,36 @@ struct i40e_aqc_add_delete_mirror_rule_completion {
 
 I40E_CHECK_CMD_LENGTH(i40e_aqc_add_delete_mirror_rule_completion);
 
+/* Pipeline Personalization Profile */
+struct i40e_aqc_write_personalization_profile {
+	u8      flags;
+	u8      reserved[3];
+	__le32  profile_track_id;
+	__le32  addr_high;
+	__le32  addr_low;
+};
+
+I40E_CHECK_CMD_LENGTH(i40e_aqc_write_personalization_profile);
+
+struct i40e_aqc_write_ppp_resp {
+	__le32 error_offset;
+	__le32 error_info;
+	__le32 addr_high;
+	__le32 addr_low;
+};
+
+struct i40e_aqc_get_applied_profiles {
+	u8      flags;
+#define I40E_AQC_GET_PPP_GET_CONF	0x1
+#define I40E_AQC_GET_PPP_GET_RDPU_CONF	0x2
+	u8      rsv[3];
+	__le32  reserved;
+	__le32  addr_high;
+	__le32  addr_low;
+};
+
+I40E_CHECK_CMD_LENGTH(i40e_aqc_get_applied_profiles);
+
 /* DCB 0x03xx*/
 
 /* PFC Ignore (direct 0x0301)
diff --git a/drivers/net/ethernet/intel/i40evf/i40e_common.c b/drivers/net/ethernet/intel/i40evf/i40e_common.c
index 626fbf1..606f5f1 100644
--- a/drivers/net/ethernet/intel/i40evf/i40e_common.c
+++ b/drivers/net/ethernet/intel/i40evf/i40e_common.c
@@ -1131,3 +1131,215 @@ i40e_status i40e_vf_reset(struct i40e_hw *hw)
 	return i40e_aq_send_msg_to_pf(hw, I40E_VIRTCHNL_OP_RESET_VF,
 				      0, NULL, 0, NULL);
 }
+
+/**
+ * i40evf_aq_write_ppp - Write pipeline personalization profile (ppp)
+ * @hw: pointer to the hw struct
+ * @buff: command buffer (size in bytes = buff_size)
+ * @buff_size: buffer size in bytes
+ * @track_id: package tracking id
+ * @error_offset: returns error offset
+ * @error_info: returns error information
+ * @cmd_details: pointer to command details structure or NULL
+ **/
+enum
+i40e_status_code i40evf_aq_write_ppp(struct i40e_hw *hw, void *buff,
+				     u16 buff_size, u32 track_id,
+				   u32 *error_offset, u32 *error_info,
+				   struct i40e_asq_cmd_details *cmd_details)
+{
+	struct i40e_aq_desc desc;
+	struct i40e_aqc_write_personalization_profile *cmd =
+		(struct i40e_aqc_write_personalization_profile *)
+		&desc.params.raw;
+	struct i40e_aqc_write_ppp_resp *resp;
+	i40e_status status;
+
+	i40evf_fill_default_direct_cmd_desc(&desc,
+					    i40e_aqc_opc_write_personalization_profile);
+
+	desc.flags |= cpu_to_le16(I40E_AQ_FLAG_BUF | I40E_AQ_FLAG_RD);
+	if (buff_size > I40E_AQ_LARGE_BUF)
+		desc.flags |= cpu_to_le16((u16)I40E_AQ_FLAG_LB);
+
+	desc.datalen = cpu_to_le16(buff_size);
+
+	cmd->profile_track_id = cpu_to_le32(track_id);
+
+	status = i40evf_asq_send_command(hw, &desc, buff, buff_size, cmd_details);
+	if (!status) {
+		resp = (struct i40e_aqc_write_ppp_resp *)&desc.params.raw;
+		if (error_offset)
+			*error_offset = le32_to_cpu(resp->error_offset);
+		if (error_info)
+			*error_info = le32_to_cpu(resp->error_info);
+	}
+
+	return status;
+}
+
+/**
+ * i40evf_aq_get_ppp_list - Read pipeline personalization profile (ppp)
+ * @hw: pointer to the hw struct
+ * @buff: command buffer (size in bytes = buff_size)
+ * @buff_size: buffer size in bytes
+ * @cmd_details: pointer to command details structure or NULL
+ **/
+enum
+i40e_status_code i40evf_aq_get_ppp_list(struct i40e_hw *hw, void *buff,
+					u16 buff_size, u8 flags,
+				      struct i40e_asq_cmd_details *cmd_details)
+{
+	struct i40e_aq_desc desc;
+	struct i40e_aqc_get_applied_profiles *cmd =
+		(struct i40e_aqc_get_applied_profiles *)&desc.params.raw;
+	i40e_status status;
+
+	i40evf_fill_default_direct_cmd_desc(&desc,
+					    i40e_aqc_opc_get_personalization_profile_list);
+
+	desc.flags |= cpu_to_le16((u16)I40E_AQ_FLAG_BUF);
+	if (buff_size > I40E_AQ_LARGE_BUF)
+		desc.flags |= cpu_to_le16((u16)I40E_AQ_FLAG_LB);
+	desc.datalen = cpu_to_le16(buff_size);
+
+	cmd->flags = flags;
+
+	status = i40evf_asq_send_command(hw, &desc, buff, buff_size, cmd_details);
+
+	return status;
+}
+
+/**
+ * i40evf_find_segment_in_package
+ * @segment_type: the segment type to search for (i.e., SEGMENT_TYPE_I40E)
+ * @pkg_hdr: pointer to the package header to be searched
+ *
+ * This function searches a package file for a particular segment type. On
+ * success it returns a pointer to the segment header, otherwise it will
+ * return NULL.
+ **/
+struct i40e_generic_seg_header *
+i40evf_find_segment_in_package(u32 segment_type,
+			       struct i40e_package_header *pkg_hdr)
+{
+	struct i40e_generic_seg_header *segment;
+	u32 i;
+
+	/* Search all package segments for the requested segment type */
+	for (i = 0; i < pkg_hdr->segment_count; i++) {
+		segment =
+			(struct i40e_generic_seg_header *)((u8 *)pkg_hdr +
+			 pkg_hdr->segment_offset[i]);
+
+		if (segment->type == segment_type)
+			return segment;
+	}
+
+	return NULL;
+}
+
+/**
+ * i40evf_write_profile
+ * @hw: pointer to the hardware structure
+ * @profile: pointer to the profile segment of the package to be downloaded
+ * @track_id: package tracking id
+ *
+ * Handles the download of a complete package.
+ */
+enum i40e_status_code
+i40evf_write_profile(struct i40e_hw *hw, struct i40e_profile_segment *profile,
+		     u32 track_id)
+{
+	i40e_status status = 0;
+	struct i40e_section_table *sec_tbl;
+	struct i40e_profile_section_header *sec = NULL;
+	u32 dev_cnt;
+	u32 vendor_dev_id;
+	u32 *nvm;
+	u32 section_size = 0;
+	u32 offset = 0, info = 0;
+	u32 i;
+
+	if (!track_id) {
+		i40e_debug(hw, I40E_DEBUG_PACKAGE, "Track_id can't be 0.");
+		return I40E_NOT_SUPPORTED;
+	}
+
+	dev_cnt = profile->device_table_count;
+
+	for (i = 0; i < dev_cnt; i++) {
+		vendor_dev_id = profile->device_table[i].vendor_dev_id;
+		if ((vendor_dev_id >> 16) == PCI_VENDOR_ID_INTEL)
+			if (hw->device_id == (vendor_dev_id & 0xFFFF))
+				break;
+	}
+	if (i == dev_cnt) {
+		i40e_debug(hw, I40E_DEBUG_PACKAGE, "Device doesn't support PPP");
+		return I40E_ERR_DEVICE_NOT_SUPPORTED;
+	}
+
+	nvm = (u32 *)&profile->device_table[dev_cnt];
+	sec_tbl = (struct i40e_section_table *)&nvm[nvm[0] + 1];
+
+	for (i = 0; i < sec_tbl->section_count; i++) {
+		sec = (struct i40e_profile_section_header *)((u8 *)profile +
+					     sec_tbl->section_offset[i]);
+
+		/* Skip 'AQ', 'note' and 'name' sections */
+		if (sec->section.type != SECTION_TYPE_MMIO)
+			continue;
+
+		section_size = sec->section.size +
+			sizeof(struct i40e_profile_section_header);
+
+		/* Write profile */
+		status = i40evf_aq_write_ppp(hw, (void *)sec, (u16)section_size,
+					     track_id, &offset, &info, NULL);
+		if (status) {
+			i40e_debug(hw, I40E_DEBUG_PACKAGE,
+				   "Failed to write profile: offset %d, info %d",
+				   offset, info);
+			break;
+		}
+	}
+	return status;
+}
+
+/**
+ * i40evf_add_pinfo_to_list
+ * @hw: pointer to the hardware structure
+ * @profile: pointer to the profile segment of the package
+ * @profile_info_sec: buffer for information section
+ * @track_id: package tracking id
+ *
+ * Register a profile to the list of loaded profiles.
+ */
+enum i40e_status_code
+i40evf_add_pinfo_to_list(struct i40e_hw *hw,
+			 struct i40e_profile_segment *profile,
+			  u8 *profile_info_sec, u32 track_id)
+{
+	i40e_status status = 0;
+	struct i40e_profile_section_header *sec = NULL;
+	struct i40e_profile_info *pinfo;
+	u32 offset = 0, info = 0;
+
+	sec = (struct i40e_profile_section_header *)profile_info_sec;
+	sec->tbl_size = 1;
+	sec->data_end = sizeof(struct i40e_profile_section_header) +
+			sizeof(struct i40e_profile_info);
+	sec->section.type = SECTION_TYPE_INFO;
+	sec->section.offset = sizeof(struct i40e_profile_section_header);
+	sec->section.size = sizeof(struct i40e_profile_info);
+	pinfo = (struct i40e_profile_info *)(profile_info_sec +
+					     sec->section.offset);
+	pinfo->track_id = track_id;
+	pinfo->version = profile->version;
+	pinfo->op = I40E_PPP_ADD_TRACKID;
+	memcpy(pinfo->name, profile->name, I40E_PPP_NAME_SIZE);
+
+	status = i40evf_aq_write_ppp(hw, (void *)sec, sec->data_end,
+				     track_id, &offset, &info, NULL);
+	return status;
+}
diff --git a/drivers/net/ethernet/intel/i40evf/i40e_prototype.h b/drivers/net/ethernet/intel/i40evf/i40e_prototype.h
index ba6c6bd..329ce2b 100644
--- a/drivers/net/ethernet/intel/i40evf/i40e_prototype.h
+++ b/drivers/net/ethernet/intel/i40evf/i40e_prototype.h
@@ -122,4 +122,21 @@ i40e_status i40e_write_phy_register(struct i40e_hw *hw, u8 page, u16 reg,
 u8 i40e_get_phy_address(struct i40e_hw *hw, u8 dev_num);
 i40e_status i40e_blink_phy_link_led(struct i40e_hw *hw,
 				    u32 time, u32 interval);
+i40e_status i40evf_aq_write_ppp(struct i40e_hw *hw, void *buff,
+				u16 buff_size, u32 track_id,
+				u32 *error_offset, u32 *error_info,
+				struct i40e_asq_cmd_details *cmd_details);
+i40e_status i40evf_aq_get_ppp_list(struct i40e_hw *hw, void *buff,
+				   u16 buff_size, u8 flags,
+				   struct i40e_asq_cmd_details *cmd_details);
+struct i40e_generic_seg_header *
+i40evf_find_segment_in_package(u32 segment_type,
+			       struct i40e_package_header *pkg_header);
+enum i40e_status_code
+i40evf_write_profile(struct i40e_hw *hw, struct i40e_profile_segment *i40e_seg,
+		     u32 track_id);
+enum i40e_status_code
+i40evf_add_pinfo_to_list(struct i40e_hw *hw,
+			 struct i40e_profile_segment *profile,
+			  u8 *profile_info_sec, u32 track_id);
 #endif /* _I40E_PROTOTYPE_H_ */
diff --git a/drivers/net/ethernet/intel/i40evf/i40e_type.h b/drivers/net/ethernet/intel/i40evf/i40e_type.h
index 16bb880..44bb9f0 100644
--- a/drivers/net/ethernet/intel/i40evf/i40e_type.h
+++ b/drivers/net/ethernet/intel/i40evf/i40e_type.h
@@ -78,6 +78,7 @@ enum i40e_debug_mask {
 	I40E_DEBUG_DCB			= 0x00000400,
 	I40E_DEBUG_DIAG			= 0x00000800,
 	I40E_DEBUG_FD			= 0x00001000,
+	I40E_DEBUG_PACKAGE		= 0x00002000,
 
 	I40E_DEBUG_AQ_MESSAGE		= 0x01000000,
 	I40E_DEBUG_AQ_DESCRIPTOR	= 0x02000000,
@@ -1396,4 +1397,83 @@ enum i40e_reset_type {
 #define I40E_FD_INSET_FLEX_WORD57_SHIFT		10
 #define I40E_FD_INSET_FLEX_WORD57_MASK		(0x1ULL << \
 					I40E_FD_INSET_FLEX_WORD57_SHIFT)
+
+/* Version format for PPP */
+struct i40e_ppp_version {
+	u8 major;
+	u8 minor;
+	u8 update;
+	u8 draft;
+};
+
+#define I40E_PPP_NAME_SIZE	32
+
+/* Package header */
+struct i40e_package_header {
+	struct i40e_ppp_version version;
+	u32 segment_count;
+	u32 segment_offset[1];
+};
+
+/* Generic segment header */
+struct i40e_generic_seg_header {
+#define SEGMENT_TYPE_METADATA	0x00000001
+#define SEGMENT_TYPE_NOTES	0x00000002
+#define SEGMENT_TYPE_I40E	0x00000011
+#define SEGMENT_TYPE_X722	0x00000012
+	u32 type;
+	struct i40e_ppp_version version;
+	u32 size;
+	char name[I40E_PPP_NAME_SIZE];
+};
+
+struct i40e_metadata_segment {
+	struct i40e_generic_seg_header header;
+	struct i40e_ppp_version version;
+	u32 track_id;
+	char     name[I40E_PPP_NAME_SIZE];
+};
+
+struct i40e_device_id_entry {
+	u32 vendor_dev_id;
+	u32 sub_vendor_dev_id;
+};
+
+struct i40e_profile_segment {
+	struct i40e_generic_seg_header header;
+	struct i40e_ppp_version version;
+	char name[I40E_PPP_NAME_SIZE];
+	u32 device_table_count;
+	struct i40e_device_id_entry device_table[1];
+};
+
+struct i40e_section_table {
+	u32 section_count;
+	u32 section_offset[1];
+};
+
+struct i40e_profile_section_header {
+	u16 tbl_size;
+	u16 data_end;
+	struct {
+#define SECTION_TYPE_INFO	0x00000010
+#define SECTION_TYPE_MMIO	0x00000800
+#define SECTION_TYPE_AQ		0x00000801
+#define SECTION_TYPE_NOTE	0x80000000
+#define SECTION_TYPE_NAME	0x80000001
+		u32 type;
+		u32 offset;
+		u32 size;
+	} section;
+};
+
+struct i40e_profile_info {
+	u32 track_id;
+	struct i40e_ppp_version version;
+	u8 op;
+#define I40E_PPP_ADD_TRACKID		0x01
+#define I40E_PPP_REMOVE_TRACKID	0x02
+	u8 reserved[7];
+	u8 name[I40E_PPP_NAME_SIZE];
+};
 #endif /* _I40E_TYPE_H_ */
-- 
2.9.3

[Intel-wired-lan] [next PATCH S70 03/12] i40e: don't hold RTNL lock while waiting for VF reset to finish

[Alice Michael]

From: Jacob Keller <jacob.e.keller@intel.com>

We made some effort to reduce the RTNL lock scope when resetting and
rebuilding the PF. Unfortunately we still held the RTNL lock during the
VF reset operation, which meant that multiple PFs could not reset in
parallel due to the global lock. For now, further reduce the scope by
not holding the RTNL lock while resetting VFs. This allows multiple PFs
to reset in a timely manner.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Change-ID: I2fbf823a0063f24dff67676cad09f0bbf83ee4ce
---
 drivers/net/ethernet/intel/i40e/i40e_main.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 4b507ce..1c06693 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -7151,6 +7151,10 @@ static void i40e_rebuild(struct i40e_pf *pf, bool reinit, bool lock_acquired)
 	/* restart the VSIs that were rebuilt and running before the reset */
 	i40e_pf_unquiesce_all_vsi(pf);
 
+	/* Release the RTNL lock before we start resetting VFs */
+	if (!lock_acquired)
+		rtnl_unlock();
+
 	if (pf->num_alloc_vfs) {
 		for (v = 0; v < pf->num_alloc_vfs; v++)
 			i40e_reset_vf(&pf->vf[v], true);
@@ -7159,9 +7163,12 @@ static void i40e_rebuild(struct i40e_pf *pf, bool reinit, bool lock_acquired)
 	/* tell the firmware that we're starting */
 	i40e_send_version(pf);
 
+	/* We've already released the lock, so don't do it again */
+	goto end_core_reset;
+
 end_unlock:
-if (!lock_acquired)
-	rtnl_unlock();
+	if (!lock_acquired)
+		rtnl_unlock();
 end_core_reset:
 	clear_bit(__I40E_RESET_FAILED, &pf->state);
 clear_recovery:
-- 
2.9.3

[Intel-wired-lan] [next PATCH S70 04/12] i40e: factor out queue control from i40e_vsi_control_(tx|rx)

[Alice Michael]

From: Jacob Keller <jacob.e.keller@intel.com>

A future patch will need to be able to handle controlling queues without
waiting until all VSIs are handled. Factor out the direct queue
modification so that we can easily re-use this code. The result is also
a bit easier to read since we don't embed multiple single-letter loop
counters.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Change-ID: Id923cbfa43127b1c24d8ed4f809b1012c736d9ac
---
 drivers/net/ethernet/intel/i40e/i40e_main.c | 139 ++++++++++++++++++----------
 1 file changed, 89 insertions(+), 50 deletions(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 1c06693..ebf2ad4 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -3919,6 +3919,8 @@ static void i40e_netpoll(struct net_device *netdev)
 }
 #endif
 
+#define I40E_QTX_ENA_WAIT_COUNT 50
+
 /**
  * i40e_pf_txq_wait - Wait for a PF's Tx queue to be enabled or disabled
  * @pf: the PF being configured
@@ -3949,6 +3951,50 @@ static int i40e_pf_txq_wait(struct i40e_pf *pf, int pf_q, bool enable)
 }
 
 /**
+ * i40e_control_tx_q - Start or stop a particular Tx queue
+ * @pf: the PF structure
+ * @pf_q: the PF queue to configure
+ * @enable: start or stop the queue
+ *
+ * This function enables or disables a single queue. Note that any delay
+ * required after the operation is expected to be handled by the caller of
+ * this function.
+ **/
+static void i40e_control_tx_q(struct i40e_pf *pf, int pf_q, bool enable)
+{
+	struct i40e_hw *hw = &pf->hw;
+	u32 tx_reg;
+	int i;
+
+	/* warn the TX unit of coming changes */
+	i40e_pre_tx_queue_cfg(&pf->hw, pf_q, enable);
+	if (!enable)
+		usleep_range(10, 20);
+
+	for (i = 0; i < I40E_QTX_ENA_WAIT_COUNT; i++) {
+		tx_reg = rd32(hw, I40E_QTX_ENA(pf_q));
+		if (((tx_reg >> I40E_QTX_ENA_QENA_REQ_SHIFT) & 1) ==
+		    ((tx_reg >> I40E_QTX_ENA_QENA_STAT_SHIFT) & 1))
+			break;
+		usleep_range(1000, 2000);
+	}
+
+	/* Skip if the queue is already in the requested state */
+	if (enable == !!(tx_reg & I40E_QTX_ENA_QENA_STAT_MASK))
+		return;
+
+	/* turn on/off the queue */
+	if (enable) {
+		wr32(hw, I40E_QTX_HEAD(pf_q), 0);
+		tx_reg |= I40E_QTX_ENA_QENA_REQ_MASK;
+	} else {
+		tx_reg &= ~I40E_QTX_ENA_QENA_REQ_MASK;
+	}
+
+	wr32(hw, I40E_QTX_ENA(pf_q), tx_reg);
+}
+
+/**
  * i40e_vsi_control_tx - Start or stop a VSI's rings
  * @vsi: the VSI being configured
  * @enable: start or stop the rings
@@ -3956,39 +4002,13 @@ static int i40e_pf_txq_wait(struct i40e_pf *pf, int pf_q, bool enable)
 static int i40e_vsi_control_tx(struct i40e_vsi *vsi, bool enable)
 {
 	struct i40e_pf *pf = vsi->back;
-	struct i40e_hw *hw = &pf->hw;
-	int i, j, pf_q, ret = 0;
-	u32 tx_reg;
+	int i, pf_q, ret = 0;
 
 	pf_q = vsi->base_queue;
 	for (i = 0; i < vsi->num_queue_pairs; i++, pf_q++) {
+		i40e_control_tx_q(pf, pf_q, enable);
 
-		/* warn the TX unit of coming changes */
-		i40e_pre_tx_queue_cfg(&pf->hw, pf_q, enable);
-		if (!enable)
-			usleep_range(10, 20);
-
-		for (j = 0; j < 50; j++) {
-			tx_reg = rd32(hw, I40E_QTX_ENA(pf_q));
-			if (((tx_reg >> I40E_QTX_ENA_QENA_REQ_SHIFT) & 1) ==
-			    ((tx_reg >> I40E_QTX_ENA_QENA_STAT_SHIFT) & 1))
-				break;
-			usleep_range(1000, 2000);
-		}
-		/* Skip if the queue is already in the requested state */
-		if (enable == !!(tx_reg & I40E_QTX_ENA_QENA_STAT_MASK))
-			continue;
-
-		/* turn on/off the queue */
-		if (enable) {
-			wr32(hw, I40E_QTX_HEAD(pf_q), 0);
-			tx_reg |= I40E_QTX_ENA_QENA_REQ_MASK;
-		} else {
-			tx_reg &= ~I40E_QTX_ENA_QENA_REQ_MASK;
-		}
-
-		wr32(hw, I40E_QTX_ENA(pf_q), tx_reg);
-		/* No waiting for the Tx queue to disable */
+		/* Don't wait to disable when port Tx is suspended */
 		if (!enable && test_bit(__I40E_PORT_TX_SUSPENDED, &pf->state))
 			continue;
 
@@ -4035,6 +4055,43 @@ static int i40e_pf_rxq_wait(struct i40e_pf *pf, int pf_q, bool enable)
 }
 
 /**
+ * i40e_control_rx_q - Start or stop a particular Rx queue
+ * @pf: the PF structure
+ * @pf_q: the PF queue to configure
+ * @enable: start or stop the queue
+ *
+ * This function enables or disables a single queue. Note that any delay
+ * required after the operation is expected to be handled by the caller of
+ * this function.
+ **/
+static void i40e_control_rx_q(struct i40e_pf *pf, int pf_q, bool enable)
+{
+	struct i40e_hw *hw = &pf->hw;
+	u32 rx_reg;
+	int i;
+
+	for (i = 0; i < I40E_QTX_ENA_WAIT_COUNT; i++) {
+		rx_reg = rd32(hw, I40E_QRX_ENA(pf_q));
+		if (((rx_reg >> I40E_QRX_ENA_QENA_REQ_SHIFT) & 1) ==
+		    ((rx_reg >> I40E_QRX_ENA_QENA_STAT_SHIFT) & 1))
+			break;
+		usleep_range(1000, 2000);
+	}
+
+	/* Skip if the queue is already in the requested state */
+	if (enable == !!(rx_reg & I40E_QRX_ENA_QENA_STAT_MASK))
+		return;
+
+	/* turn on/off the queue */
+	if (enable)
+		rx_reg |= I40E_QRX_ENA_QENA_REQ_MASK;
+	else
+		rx_reg &= ~I40E_QRX_ENA_QENA_REQ_MASK;
+
+	wr32(hw, I40E_QRX_ENA(pf_q), rx_reg);
+}
+
+/**
  * i40e_vsi_control_rx - Start or stop a VSI's rings
  * @vsi: the VSI being configured
  * @enable: start or stop the rings
@@ -4042,31 +4099,13 @@ static int i40e_pf_rxq_wait(struct i40e_pf *pf, int pf_q, bool enable)
 static int i40e_vsi_control_rx(struct i40e_vsi *vsi, bool enable)
 {
 	struct i40e_pf *pf = vsi->back;
-	struct i40e_hw *hw = &pf->hw;
-	int i, j, pf_q, ret = 0;
-	u32 rx_reg;
+	int i, pf_q, ret = 0;
 
 	pf_q = vsi->base_queue;
 	for (i = 0; i < vsi->num_queue_pairs; i++, pf_q++) {
-		for (j = 0; j < 50; j++) {
-			rx_reg = rd32(hw, I40E_QRX_ENA(pf_q));
-			if (((rx_reg >> I40E_QRX_ENA_QENA_REQ_SHIFT) & 1) ==
-			    ((rx_reg >> I40E_QRX_ENA_QENA_STAT_SHIFT) & 1))
-				break;
-			usleep_range(1000, 2000);
-		}
+		i40e_control_rx_q(pf, pf_q, enable);
 
-		/* Skip if the queue is already in the requested state */
-		if (enable == !!(rx_reg & I40E_QRX_ENA_QENA_STAT_MASK))
-			continue;
-
-		/* turn on/off the queue */
-		if (enable)
-			rx_reg |= I40E_QRX_ENA_QENA_REQ_MASK;
-		else
-			rx_reg &= ~I40E_QRX_ENA_QENA_REQ_MASK;
-		wr32(hw, I40E_QRX_ENA(pf_q), rx_reg);
-		/* No waiting for the Tx queue to disable */
+		/* Don't wait to disable when port Tx is suspended */
 		if (!enable && test_bit(__I40E_PORT_TX_SUSPENDED, &pf->state))
 			continue;
 
-- 
2.9.3

[Intel-wired-lan] [next PATCH S70 05/12] i40e: fix CONFIG_BUSY checks in i40e_set_settings function

[Alice Michael]

From: Jacob Keller <jacob.e.keller@intel.com>

The check for I40E_CONFIG_BUSY state bit in the i40e_set_link_ksettings
function is fishy. First we can notice a few things about the check here.

First a similar check was introduced by commit
'c7d05ca89f8e ("i40e: driver ethtool core")'

Later a commit introducing the link settings was added by commit
'bf9c71417f72 ("i40e: Implement set_settings for ethtool")'

However, this second check was against vsi->state instead of pf->state,
and also failed to set the bit, it only checks. That indicates the locking
was not quite correct. The only other place that the state bit
in vsi->state gets used is to protect the filter list.

Since this code does not care about the mac filter list,  and seems
clear the original code should have set the pf->state bit. Fix these
issues by using pf->state correctly, and by actually setting the bit
so that we properly lock as expected.

Since these checks occur while holding the rtnl_lock(), lets also add a
timeout so that we don't potentially softlock the system.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 38 ++++++++++++++++++++------
 1 file changed, 29 insertions(+), 9 deletions(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
index 10325b5..08035c4 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
@@ -698,6 +698,7 @@ static int i40e_set_link_ksettings(struct net_device *netdev,
 	struct ethtool_link_ksettings copy_cmd;
 	i40e_status status = 0;
 	bool change = false;
+	int timeout = 50;
 	int err = 0;
 	u32 autoneg;
 	u32 advertise;
@@ -756,14 +757,20 @@ static int i40e_set_link_ksettings(struct net_device *netdev,
 	if (memcmp(&copy_cmd, &safe_cmd, sizeof(struct ethtool_link_ksettings)))
 		return -EOPNOTSUPP;
 
-	while (test_bit(__I40E_CONFIG_BUSY, &vsi->state))
+	while (test_and_set_bit(__I40E_CONFIG_BUSY, &pf->state)) {
+		timeout--;
+		if (!timeout)
+			return -EBUSY;
 		usleep_range(1000, 2000);
+	}
 
 	/* Get the current phy config */
 	status = i40e_aq_get_phy_capabilities(hw, false, false, &abilities,
 					      NULL);
-	if (status)
-		return -EAGAIN;
+	if (status) {
+		err = -EAGAIN;
+		goto done;
+	}
 
 	/* Copy abilities to config in case autoneg is not
 	 * set below
@@ -779,7 +786,8 @@ static int i40e_set_link_ksettings(struct net_device *netdev,
 			if (!ethtool_link_ksettings_test_link_mode(
 				    &safe_cmd, supported, Autoneg)) {
 				netdev_info(netdev, "Autoneg not supported on this phy\n");
-				return -EINVAL;
+				err = -EINVAL;
+				goto done;
 			}
 			/* Autoneg is allowed to change */
 			config.abilities = abilities.abilities |
@@ -797,7 +805,8 @@ static int i40e_set_link_ksettings(struct net_device *netdev,
 			    hw->phy.link_info.phy_type !=
 			    I40E_PHY_TYPE_10GBASE_T) {
 				netdev_info(netdev, "Autoneg cannot be disabled on this phy\n");
-				return -EINVAL;
+				err = -EINVAL;
+				goto done;
 			}
 			/* Autoneg is allowed to change */
 			config.abilities = abilities.abilities &
@@ -808,8 +817,10 @@ static int i40e_set_link_ksettings(struct net_device *netdev,
 
 	ethtool_convert_link_mode_to_legacy_u32(&tmp,
 						safe_cmd.link_modes.supported);
-	if (advertise & ~tmp)
-		return -EINVAL;
+	if (advertise & ~tmp) {
+		err = -EINVAL;
+		goto done;
+	}
 
 	if (advertise & ADVERTISED_100baseT_Full)
 		config.link_speed |= I40E_LINK_SPEED_100MB;
@@ -865,7 +876,8 @@ static int i40e_set_link_ksettings(struct net_device *netdev,
 			netdev_info(netdev, "Set phy config failed, err %s aq_err %s\n",
 				    i40e_stat_str(hw, status),
 				    i40e_aq_str(hw, hw->aq.asq_last_status));
-			return -EAGAIN;
+			err = -EAGAIN;
+			goto done;
 		}
 
 		status = i40e_update_link_info(hw);
@@ -878,6 +890,9 @@ static int i40e_set_link_ksettings(struct net_device *netdev,
 		netdev_info(netdev, "Nothing changed, exiting without setting anything.\n");
 	}
 
+done:
+	clear_bit(__I40E_CONFIG_BUSY, &pf->state);
+
 	return err;
 }
 
@@ -1292,6 +1307,7 @@ static int i40e_set_ringparam(struct net_device *netdev,
 	struct i40e_vsi *vsi = np->vsi;
 	struct i40e_pf *pf = vsi->back;
 	u32 new_rx_count, new_tx_count;
+	int timeout = 50;
 	int i, err = 0;
 
 	if ((ring->rx_mini_pending) || (ring->rx_jumbo_pending))
@@ -1316,8 +1332,12 @@ static int i40e_set_ringparam(struct net_device *netdev,
 	    (new_rx_count == vsi->rx_rings[0]->count))
 		return 0;
 
-	while (test_and_set_bit(__I40E_CONFIG_BUSY, &pf->state))
+	while (test_and_set_bit(__I40E_CONFIG_BUSY, &pf->state)) {
+		timeout--;
+		if (!timeout)
+			return -EBUSY;
 		usleep_range(1000, 2000);
+	}
 
 	if (!netif_running(vsi->netdev)) {
 		/* simple case - set for the next time the netdev is started */
-- 
2.9.3

[Intel-wired-lan] [next PATCH S70 06/12] i40e: reduce wait time for adminq command completion

[Alice Michael]

From: Jacob Keller <jacob.e.keller@intel.com>

When sending an adminq command, we wait for the command to complete in
a loop. This loop waits for an entire millisecond, when in practice the
adminq command is processed often much faster.

Change the loop to use i40e_usec_delay instead, and wait for 50 usecs
each time instead. This appears to be about the minimum time required,
based on some manual observation and testing.

The primary benefit of this change is reducing latency of various
operations in the PF driver, especially when related to having a large
number of VFs enabled.

For example, on Linux, when instantiating 128 VFs, the time to finish
the operation dropped from about 9 seconds down to under 6 seconds.
Additionally, the time it takes to finish a PF reset with 128 VFs
dropped from 5.1 seconds down to 0.7 seconds.

As the examples above show, a significant portion of the delay is wasted
waiting for admiqn operations which have already finished.

This patch shouldn't cause impact to functionality, as we still check
and keep waiting until the command does get processed. The only expected
change is an increase in CPU utilization as we now check for completion
far more times. However, in practice the commands appear to generally be
complete within the first delay window anyways.

Previously for NDIS we used i40e_msec_stall since i40e_msec_delay would
cause the CPU to be yielded. We can now just use i40e_usec_delay instead
because this function will behave correctly for both NDIS and non-NDIS
code. Thus, we can simply drop the #ifdef logic entirely.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Change-ID: If8af8388e100da0a14eaf9e1af3afadf73a958cf
---
 drivers/net/ethernet/intel/i40e/i40e_adminq.c   | 4 ++--
 drivers/net/ethernet/intel/i40e/i40e_adminq.h   | 2 +-
 drivers/net/ethernet/intel/i40evf/i40e_adminq.c | 4 ++--
 drivers/net/ethernet/intel/i40evf/i40e_adminq.h | 2 +-
 4 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_adminq.c b/drivers/net/ethernet/intel/i40e/i40e_adminq.c
index 56fb272..ba04988 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_adminq.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_adminq.c
@@ -850,8 +850,8 @@ i40e_status i40e_asq_send_command(struct i40e_hw *hw,
 			 */
 			if (i40e_asq_done(hw))
 				break;
-			usleep_range(1000, 2000);
-			total_delay++;
+			udelay(50);
+			total_delay += 50;
 		} while (total_delay < hw->aq.asq_cmd_timeout);
 	}
 
diff --git a/drivers/net/ethernet/intel/i40e/i40e_adminq.h b/drivers/net/ethernet/intel/i40e/i40e_adminq.h
index d92aad3..2349fbe 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_adminq.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_adminq.h
@@ -151,7 +151,7 @@ static inline int i40e_aq_rc_to_posix(int aq_ret, int aq_rc)
 
 /* general information */
 #define I40E_AQ_LARGE_BUF	512
-#define I40E_ASQ_CMD_TIMEOUT	250  /* msecs */
+#define I40E_ASQ_CMD_TIMEOUT	250000  /* usecs */
 
 void i40e_fill_default_direct_cmd_desc(struct i40e_aq_desc *desc,
 				       u16 opcode);
diff --git a/drivers/net/ethernet/intel/i40evf/i40e_adminq.c b/drivers/net/ethernet/intel/i40evf/i40e_adminq.c
index 9638515..8b0d4b2 100644
--- a/drivers/net/ethernet/intel/i40evf/i40e_adminq.c
+++ b/drivers/net/ethernet/intel/i40evf/i40e_adminq.c
@@ -797,8 +797,8 @@ i40e_status i40evf_asq_send_command(struct i40e_hw *hw,
 			 */
 			if (i40evf_asq_done(hw))
 				break;
-			usleep_range(1000, 2000);
-			total_delay++;
+			udelay(50);
+			total_delay += 50;
 		} while (total_delay < hw->aq.asq_cmd_timeout);
 	}
 
diff --git a/drivers/net/ethernet/intel/i40evf/i40e_adminq.h b/drivers/net/ethernet/intel/i40evf/i40e_adminq.h
index 1f9b3b5..e0bfaa3 100644
--- a/drivers/net/ethernet/intel/i40evf/i40e_adminq.h
+++ b/drivers/net/ethernet/intel/i40evf/i40e_adminq.h
@@ -151,7 +151,7 @@ static inline int i40e_aq_rc_to_posix(int aq_ret, int aq_rc)
 
 /* general information */
 #define I40E_AQ_LARGE_BUF	512
-#define I40E_ASQ_CMD_TIMEOUT	250  /* msecs */
+#define I40E_ASQ_CMD_TIMEOUT	250000  /* usecs */
 
 void i40evf_fill_default_direct_cmd_desc(struct i40e_aq_desc *desc,
 				       u16 opcode);
-- 
2.9.3

[Intel-wired-lan] [next PATCH S70 07/12] i40e: remove I40E_FLAG_IN_NETPOLL entirely

[Alice Michael]

From: Jacob Keller <jacob.e.keller@intel.com>

This flag was originally intended to be used to let some
driver code know when we were running from netpoll.
Ultimately this was not necessary and we never used it.
Let's remove it

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Change-ID: I43b72483d91c1638071d2a7f389ab171ec5b796a
---
 drivers/net/ethernet/intel/i40evf/i40evf.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/drivers/net/ethernet/intel/i40evf/i40evf.h b/drivers/net/ethernet/intel/i40evf/i40evf.h
index 7bbd11a..40f56e2 100644
--- a/drivers/net/ethernet/intel/i40evf/i40evf.h
+++ b/drivers/net/ethernet/intel/i40evf/i40evf.h
@@ -202,7 +202,6 @@ struct i40evf_adapter {
 
 	u32 flags;
 #define I40EVF_FLAG_RX_CSUM_ENABLED		BIT(0)
-#define I40EVF_FLAG_IN_NETPOLL			BIT(4)
 #define I40EVF_FLAG_IMIR_ENABLED		BIT(5)
 #define I40EVF_FLAG_MQ_CAPABLE			BIT(6)
 #define I40EVF_FLAG_PF_COMMS_FAILED		BIT(8)
@@ -221,7 +220,6 @@ struct i40evf_adapter {
 /* duplicates for common code */
 #define I40E_FLAG_FDIR_ATR_ENABLED		0
 #define I40E_FLAG_DCB_ENABLED			0
-#define I40E_FLAG_IN_NETPOLL			I40EVF_FLAG_IN_NETPOLL
 #define I40E_FLAG_RX_CSUM_ENABLED		I40EVF_FLAG_RX_CSUM_ENABLED
 #define I40E_FLAG_WB_ON_ITR_CAPABLE		I40EVF_FLAG_WB_ON_ITR_CAPABLE
 #define I40E_FLAG_OUTER_UDP_CSUM_CAPABLE	I40EVF_FLAG_OUTER_UDP_CSUM_CAPABLE
-- 
2.9.3

[Intel-wired-lan] [next PATCH S70 08/12] i40e: split some code in i40e_reset_vf into helpers

[Alice Michael]

From: Jacob Keller <jacob.e.keller@intel.com>

A future patch is going to want to re-use some of the code in
i40e_reset_vf, so lets break up the beginning and ending parts into
their own helper functions. The first function will be used to
initialize the reset on a VF, while the second function will be used to
finalize the reset and restore functionality.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Change-ID: I48df808b8bf09de3c2ed8c521f57b3f0ab9e5907
---
 drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c | 109 ++++++++++++++-------
 1 file changed, 72 insertions(+), 37 deletions(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
index f8d25cb..a09c2b3 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
@@ -923,22 +923,19 @@ static int i40e_quiesce_vf_pci(struct i40e_vf *vf)
 }
 
 /**
- * i40e_reset_vf
+ * i40e_trigger_vf_reset
  * @vf: pointer to the VF structure
  * @flr: VFLR was issued or not
  *
- * reset the VF
+ * Trigger hardware to start a reset for a particular VF. Expects the caller
+ * to wait the proper amount of time to allow hardware to reset the VF before
+ * it cleans up and restores VF functionality.
  **/
-void i40e_reset_vf(struct i40e_vf *vf, bool flr)
+static void i40e_trigger_vf_reset(struct i40e_vf *vf, bool flr)
 {
 	struct i40e_pf *pf = vf->pf;
 	struct i40e_hw *hw = &pf->hw;
 	u32 reg, reg_idx, bit_idx;
-	bool rsd = false;
-	int i;
-
-	if (test_and_set_bit(__I40E_VF_DISABLE, &pf->state))
-		return;
 
 	/* warn the VF */
 	clear_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states);
@@ -970,37 +967,22 @@ void i40e_reset_vf(struct i40e_vf *vf, bool flr)
 	if (i40e_quiesce_vf_pci(vf))
 		dev_err(&pf->pdev->dev, "VF %d PCI transactions stuck\n",
 			vf->vf_id);
+}
 
-	/* poll VPGEN_VFRSTAT reg to make sure
-	 * that reset is complete
-	 */
-	for (i = 0; i < 10; i++) {
-		/* VF reset requires driver to first reset the VF and then
-		 * poll the status register to make sure that the reset
-		 * completed successfully. Due to internal HW FIFO flushes,
-		 * we must wait 10ms before the register will be valid.
-		 */
-		usleep_range(10000, 20000);
-		reg = rd32(hw, I40E_VPGEN_VFRSTAT(vf->vf_id));
-		if (reg & I40E_VPGEN_VFRSTAT_VFRD_MASK) {
-			rsd = true;
-			break;
-		}
-	}
-
-	if (flr)
-		usleep_range(10000, 20000);
-
-	if (!rsd)
-		dev_err(&pf->pdev->dev, "VF reset check timeout on VF %d\n",
-			vf->vf_id);
-
-	/* On initial reset, we won't have any queues */
-	if (vf->lan_vsi_idx == 0)
-		goto complete_reset;
+/**
+ * i40e_cleanup_reset_vf
+ * @vf: pointer to the VF structure
+ *
+ * Cleanup a VF after the hardware reset is finished. Expects the caller to
+ * have verified whether the reset is finished properly, and ensure the
+ * minimum amount of wait time has passed.
+ **/
+static void i40e_cleanup_reset_vf(struct i40e_vf *vf)
+{
+	struct i40e_pf *pf = vf->pf;
+	struct i40e_hw *hw = &pf->hw;
+	u32 reg;
 
-	i40e_vsi_stop_rings(pf->vsi[vf->lan_vsi_idx]);
-complete_reset:
 	/* free VF resources to begin resetting the VSI state */
 	i40e_free_vf_res(vf);
 
@@ -1035,6 +1017,59 @@ void i40e_reset_vf(struct i40e_vf *vf, bool flr)
 	 * request resources immediately after setting this flag.
 	 */
 	wr32(hw, I40E_VFGEN_RSTAT1(vf->vf_id), I40E_VFR_VFACTIVE);
+}
+
+/**
+ * i40e_reset_vf
+ * @vf: pointer to the VF structure
+ * @flr: VFLR was issued or not
+ *
+ * reset the VF
+ **/
+void i40e_reset_vf(struct i40e_vf *vf, bool flr)
+{
+	struct i40e_pf *pf = vf->pf;
+	struct i40e_hw *hw = &pf->hw;
+	bool rsd = false;
+	u32 reg;
+	int i;
+
+	/* If VFs have been disabled, there is no need to reset */
+	if (test_and_set_bit(__I40E_VF_DISABLE, &pf->state))
+		return;
+
+	i40e_trigger_vf_reset(vf, flr);
+
+	/* poll VPGEN_VFRSTAT reg to make sure
+	 * that reset is complete
+	 */
+	for (i = 0; i < 10; i++) {
+		/* VF reset requires driver to first reset the VF and then
+		 * poll the status register to make sure that the reset
+		 * completed successfully. Due to internal HW FIFO flushes,
+		 * we must wait 10ms before the register will be valid.
+		 */
+		usleep_range(10000, 20000);
+		reg = rd32(hw, I40E_VPGEN_VFRSTAT(vf->vf_id));
+		if (reg & I40E_VPGEN_VFRSTAT_VFRD_MASK) {
+			rsd = true;
+			break;
+		}
+	}
+
+	if (flr)
+		usleep_range(10000, 20000);
+
+	if (!rsd)
+		dev_err(&pf->pdev->dev, "VF reset check timeout on VF %d\n",
+			vf->vf_id);
+	usleep_range(10000, 20000);
+
+	/* On initial reset, we don't have any queues to disable */
+	if (vf->lan_vsi_idx != 0)
+		i40e_vsi_stop_rings(pf->vsi[vf->lan_vsi_idx]);
+
+	i40e_cleanup_reset_vf(vf);
 
 	i40e_flush(hw);
 	clear_bit(__I40E_VF_DISABLE, &pf->state);
-- 
2.9.3

[Intel-wired-lan] [next PATCH S70 09/12] i40e: reset all VFs in parallel when rebuilding PF

[Alice Michael]

From: Jacob Keller <jacob.e.keller@intel.com>

When there are a lot of active VFs, it can take multiple seconds to
finish resetting all of them during certain flows., which can cause some
VFs to fail to wait long enough for the reset to occur. The user might
see messages like "Never saw reset" or "Reset never finished" and the VF
driver will stop functioning properly.

The naive solution would be to simply increase the wait timer. We can
get much more clever. Notice that i40e_reset_vf is run in a serialized
fashion, and includes lots of delays.

There are two prominent delays which take most of the time. First, when
we begin resetting VFs, we have multiple 10ms delays which accrue
because we reset each VF in a serial fashion. These delays accumulate to
almost 4 seconds when handling the maximum number of VFs (128).

Secondly, there is a massive 50ms delay for each time we disable queues
on a VSI. This delay is necessary to allow HW to finish disabling queues
before we restore functionality. However, just like with the first case,
we are paying the cost for each VF, rather than disabling all VFs and
waiting once.

Both of these can be fixed, but required some previous refactoring to
handle the special case. First, we will need the
i40e_vsi_wait_queues_disabled function which was previously DCB
specific. Second, we will need to implement our own
i40e_vsi_stop_rings_no_wait function which will handle the stopping of
rings without the delays.

Finally, implement an i40e_reset_all_vfs function, which will first
start the reset of all VFs, and pay the wait cost all at once, rather
than serially waiting for each VF before we start processing then next
one. After the VF has been reset, we'll disable all the VF queues, and
then wait for them to disable. Again, we'll organize the flow such that
we pay the wait cost only once.

Finally, after we've disabled queues we'll go ahead and begin restoring
VF functionality. The result is reducing the wait time by a large factor
and ensuring that VFs do not timeout when waiting in the VF driver.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Change-ID: Ia6e8cf8d98131b78aec89db78afb8d905c9b12be
---
Testing-hints:
  "time echo 1 > /sys/class/net/<enpX>/device/reset" will trigger
  a request for a reset in the driver. I used the "time" command to
  measure how long it took for the reset to finish.

 drivers/net/ethernet/intel/i40e/i40e.h             |   2 +
 drivers/net/ethernet/intel/i40e/i40e_main.c        |  32 +++++--
 drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c | 100 +++++++++++++++++++++
 drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h |   1 +
 4 files changed, 129 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h
index a33a733..d35fec3 100644
--- a/drivers/net/ethernet/intel/i40e/i40e.h
+++ b/drivers/net/ethernet/intel/i40e/i40e.h
@@ -903,6 +903,8 @@ void i40e_notify_client_of_vf_msg(struct i40e_vsi *vsi, u32 vf_id,
 
 int i40e_vsi_start_rings(struct i40e_vsi *vsi);
 void i40e_vsi_stop_rings(struct i40e_vsi *vsi);
+void i40e_vsi_stop_rings_no_wait(struct  i40e_vsi *vsi);
+int i40e_vsi_wait_queues_disabled(struct i40e_vsi *vsi);
 int i40e_reconfig_rss_queues(struct i40e_pf *pf, int queue_count);
 struct i40e_veb *i40e_veb_setup(struct i40e_pf *pf, u16 flags, u16 uplink_seid,
 				u16 downlink_seid, u8 enabled_tc);
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index ebf2ad4..7ceb256 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -4159,6 +4159,29 @@ void i40e_vsi_stop_rings(struct i40e_vsi *vsi)
 }
 
 /**
+ * i40e_vsi_stop_rings_no_wait - Stop a VSI's rings and do not delay
+ * @vsi: the VSI being shutdown
+ *
+ * This function stops all the rings for a VSI but does not delay to verify
+ * that rings have been disabled. It is expected that the caller is shutting
+ * down multiple VSIs at once and will delay together for all the VSIs after
+ * initiating the shutdown. This is particularly useful for shutting down lots
+ * of VFs together. Otherwise, a large delay can be incurred while configuring
+ * each VSI in serial.
+ **/
+void i40e_vsi_stop_rings_no_wait(struct i40e_vsi *vsi)
+{
+	struct i40e_pf *pf = vsi->back;
+	int i, pf_q;
+
+	pf_q = vsi->base_queue;
+	for (i = 0; i < vsi->num_queue_pairs; i++, pf_q++) {
+		i40e_control_tx_q(pf, pf_q, false);
+		i40e_control_rx_q(pf, pf_q, false);
+	}
+}
+
+/**
  * i40e_vsi_free_irq - Free the irq association with the OS
  * @vsi: the VSI being configured
  **/
@@ -4488,14 +4511,13 @@ static void i40e_pf_unquiesce_all_vsi(struct i40e_pf *pf)
 	}
 }
 
-#ifdef CONFIG_I40E_DCB
 /**
  * i40e_vsi_wait_queues_disabled - Wait for VSI's queues to be disabled
  * @vsi: the VSI being configured
  *
  * Wait until all queues on a given VSI have been disabled.
  **/
-static int i40e_vsi_wait_queues_disabled(struct i40e_vsi *vsi)
+int i40e_vsi_wait_queues_disabled(struct i40e_vsi *vsi)
 {
 	struct i40e_pf *pf = vsi->back;
 	int i, pf_q, ret;
@@ -4523,6 +4545,7 @@ static int i40e_vsi_wait_queues_disabled(struct i40e_vsi *vsi)
 	return 0;
 }
 
+#ifdef CONFIG_I40E_DCB
 /**
  * i40e_pf_wait_queues_disabled - Wait for all queues of PF VSIs to be disabled
  * @pf: the PF
@@ -7194,10 +7217,7 @@ static void i40e_rebuild(struct i40e_pf *pf, bool reinit, bool lock_acquired)
 	if (!lock_acquired)
 		rtnl_unlock();
 
-	if (pf->num_alloc_vfs) {
-		for (v = 0; v < pf->num_alloc_vfs; v++)
-			i40e_reset_vf(&pf->vf[v], true);
-	}
+	i40e_reset_all_vfs(pf, true);
 
 	/* tell the firmware that we're starting */
 	i40e_send_version(pf);
diff --git a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
index a09c2b3..423068a 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
@@ -1076,6 +1076,106 @@ void i40e_reset_vf(struct i40e_vf *vf, bool flr)
 }
 
 /**
+ * i40e_reset_all_vfs
+ * @pf: pointer to the PF structure
+ * @flr: VFLR was issued or not
+ *
+ * Reset all allocated VFs in one go. First, tell the hardware to reset each
+ * VF, then do all the waiting in one chunk, and finally finish restoring each
+ * VF after the wait. This is useful during PF routines which need to reset
+ * all VFs, as otherwise it must perform these resets in a serialized fashion.
+ **/
+void i40e_reset_all_vfs(struct i40e_pf *pf, bool flr)
+{
+	struct i40e_hw *hw = &pf->hw;
+	struct i40e_vf *vf;
+	int i, v;
+	u32 reg;
+
+	/* If we don't have any VFs, then there is nothing to reset */
+	if (!pf->num_alloc_vfs)
+		return;
+
+	/* If VFs have been disabled, there is no need to reset */
+	if (test_and_set_bit(__I40E_VF_DISABLE, &pf->state))
+		return;
+
+	/* Begin reset on all VFs at once */
+	for (v = 0; v < pf->num_alloc_vfs; v++)
+		i40e_trigger_vf_reset(&pf->vf[v], flr);
+
+	/* HW requires some time to make sure it can flush the FIFO for a VF
+	 * when it resets it. Poll the VPGEN_VFRSTAT register for each VF in
+	 * sequence to make sure that it has completed. We'll keep track of
+	 * the VFs using a simple iterator that increments once that VF has
+	 * finished resetting.
+	 */
+	for (i = 0, v = 0; i < 10 && v < pf->num_alloc_vfs; i++) {
+		usleep_range(10000, 20000);
+
+		/* Check each VF in sequence, beginning with the VF to fail
+		 * the previous check.
+		 */
+		while (v < pf->num_alloc_vfs) {
+			vf = &pf->vf[v];
+			reg = rd32(hw, I40E_VPGEN_VFRSTAT(vf->vf_id));
+			if (!(reg & I40E_VPGEN_VFRSTAT_VFRD_MASK))
+				break;
+
+			/* If the current VF has finished resetting, move on
+			 * to the next VF in sequence.
+			 */
+			v++;
+		}
+	}
+
+	if (flr)
+		usleep_range(10000, 20000);
+
+	/* Display a warning if at least one VF didn't manage to reset in
+	 * time, but continue on with the operation.
+	 */
+	if (v < pf->num_alloc_vfs)
+		dev_err(&pf->pdev->dev, "VF reset check timeout on VF %d\n",
+			pf->vf[v].vf_id);
+	usleep_range(10000, 20000);
+
+	/* Begin disabling all the rings associated with VFs, but do not wait
+	 * between each VF.
+	 */
+	for (v = 0; v < pf->num_alloc_vfs; v++) {
+		/* On initial reset, we don't have any queues to disable */
+		if (pf->vf[v].lan_vsi_idx == 0)
+			continue;
+
+		i40e_vsi_stop_rings_no_wait(pf->vsi[pf->vf[v].lan_vsi_idx]);
+	}
+
+	/* Now that we've notified HW to disable all of the VF rings, wait
+	 * until they finish.
+	 */
+	for (v = 0; v < pf->num_alloc_vfs; v++) {
+		/* On initial reset, we don't have any queues to disable */
+		if (pf->vf[v].lan_vsi_idx == 0)
+			continue;
+
+		i40e_vsi_wait_queues_disabled(pf->vsi[pf->vf[v].lan_vsi_idx]);
+	}
+
+	/* Hw may need up to 50ms to finish disabling the RX queues. We
+	 * minimize the wait by delaying only once for all VFs.
+	 */
+	mdelay(50);
+
+	/* Finish the reset on each VF */
+	for (v = 0; v < pf->num_alloc_vfs; v++)
+		i40e_cleanup_reset_vf(&pf->vf[v]);
+
+	i40e_flush(hw);
+	clear_bit(__I40E_VF_DISABLE, &pf->state);
+}
+
+/**
  * i40e_free_vfs
  * @pf: pointer to the PF structure
  *
diff --git a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h
index 3e1f8f6..f76d234 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h
@@ -130,6 +130,7 @@ int i40e_vc_process_vf_msg(struct i40e_pf *pf, s16 vf_id, u32 v_opcode,
 			   u32 v_retval, u8 *msg, u16 msglen);
 int i40e_vc_process_vflr_event(struct i40e_pf *pf);
 void i40e_reset_vf(struct i40e_vf *vf, bool flr);
+void i40e_reset_all_vfs(struct i40e_pf *pf, bool flr);
 void i40e_vc_notify_vf_reset(struct i40e_vf *vf);
 
 /* VF configuration related iplink handlers */
-- 
2.9.3

[Intel-wired-lan] [next PATCH S70 10/12] i40e: use i40e_stop_rings_no_wait to implement PORT_SUSPENDED state

[Alice Michael]

From: Jacob Keller <jacob.e.keller@intel.com>

This state bit was added as a way for DCB to avoid having to wait for
the queues to disable when handling LLDP events. The logic for this was
burried deep within stop Tx and stop Rx queue code. First, let's rename
it so that it does not appear to only affect Tx when infact it modifies
both Tx and Rx flow. Second we can move it up into the i40e_stop_rings()
function, and we can simply re-use the i40e_stop_rings_no_wait() so that
we don't have to bury the implementation as deep into the call stack.

An alternative might be to remove the state bit and instead attempt to
shut down everything directly in DCP flow. This, however, is not ideal
because it creates yet another separate shutdown routine that we'd have
to maintain. In the current implementation any changes will be made to
both flows.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Change-ID: I68e1ccb901af320862bca395e9c9746f08e8b17c
---
 drivers/net/ethernet/intel/i40e/i40e.h      |  2 +-
 drivers/net/ethernet/intel/i40e/i40e_main.c | 16 ++++++----------
 2 files changed, 7 insertions(+), 11 deletions(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h
index d35fec3..76bc404 100644
--- a/drivers/net/ethernet/intel/i40e/i40e.h
+++ b/drivers/net/ethernet/intel/i40e/i40e.h
@@ -149,7 +149,7 @@ enum i40e_state_t {
 	__I40E_DOWN_REQUESTED,
 	__I40E_FD_FLUSH_REQUESTED,
 	__I40E_RESET_FAILED,
-	__I40E_PORT_TX_SUSPENDED,
+	__I40E_PORT_SUSPENDED,
 	__I40E_VF_DISABLE,
 };
 
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 7ceb256..b9d7798 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -4008,10 +4008,6 @@ static int i40e_vsi_control_tx(struct i40e_vsi *vsi, bool enable)
 	for (i = 0; i < vsi->num_queue_pairs; i++, pf_q++) {
 		i40e_control_tx_q(pf, pf_q, enable);
 
-		/* Don't wait to disable when port Tx is suspended */
-		if (!enable && test_bit(__I40E_PORT_TX_SUSPENDED, &pf->state))
-			continue;
-
 		/* wait for the change to finish */
 		ret = i40e_pf_txq_wait(pf, pf_q, enable);
 		if (ret) {
@@ -4105,10 +4101,6 @@ static int i40e_vsi_control_rx(struct i40e_vsi *vsi, bool enable)
 	for (i = 0; i < vsi->num_queue_pairs; i++, pf_q++) {
 		i40e_control_rx_q(pf, pf_q, enable);
 
-		/* Don't wait to disable when port Tx is suspended */
-		if (!enable && test_bit(__I40E_PORT_TX_SUSPENDED, &pf->state))
-			continue;
-
 		/* wait for the change to finish */
 		ret = i40e_pf_rxq_wait(pf, pf_q, enable);
 		if (ret) {
@@ -4151,6 +4143,10 @@ int i40e_vsi_start_rings(struct i40e_vsi *vsi)
  **/
 void i40e_vsi_stop_rings(struct i40e_vsi *vsi)
 {
+	/* When port TX is suspended, don't wait */
+	if (test_bit(__I40E_PORT_SUSPENDED, &vsi->back->state))
+		return i40e_vsi_stop_rings_no_wait(vsi);
+
 	/* do rx first for enable and last for disable
 	 * Ignore return value, we need to shutdown whatever we can
 	 */
@@ -5986,7 +5982,7 @@ static int i40e_handle_lldp_event(struct i40e_pf *pf,
 	else
 		pf->flags &= ~I40E_FLAG_DCB_ENABLED;
 
-	set_bit(__I40E_PORT_TX_SUSPENDED, &pf->state);
+	set_bit(__I40E_PORT_SUSPENDED, &pf->state);
 	/* Reconfiguration needed quiesce all VSIs */
 	i40e_pf_quiesce_all_vsi(pf);
 
@@ -5995,7 +5991,7 @@ static int i40e_handle_lldp_event(struct i40e_pf *pf,
 
 	ret = i40e_resume_port_tx(pf);
 
-	clear_bit(__I40E_PORT_TX_SUSPENDED, &pf->state);
+	clear_bit(__I40E_PORT_SUSPENDED, &pf->state);
 	/* In case of error no point in resuming VSIs */
 	if (ret)
 		goto exit;
-- 
2.9.3

[Intel-wired-lan] [next PATCH S70 11/12] i40e: properly spell I40E_VF_STATE_* flags

[Alice Michael]

From: Jacob Keller <jacob.e.keller@intel.com>

These flags represent the state of the VF at various times. Do not
spell them as _STAT_ which can be confusing to readers who may think
these refer to statistics.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Change-ID: I6bc092cd472e8276896a1fd7498aced2084312df
---
 drivers/net/ethernet/intel/i40e/i40e_ethtool.c     |  2 +-
 drivers/net/ethernet/intel/i40e/i40e_main.c        |  2 +-
 drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c | 98 +++++++++++-----------
 drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h | 14 ++--
 4 files changed, 58 insertions(+), 58 deletions(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
index 08035c4..523dd81 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
@@ -1826,7 +1826,7 @@ static inline bool i40e_active_vfs(struct i40e_pf *pf)
 	int i;
 
 	for (i = 0; i < pf->num_alloc_vfs; i++)
-		if (test_bit(I40E_VF_STAT_ACTIVE, &vfs[i].vf_states))
+		if (test_bit(I40E_VF_STATE_ACTIVE, &vfs[i].vf_states))
 			return true;
 	return false;
 }
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index b9d7798..5d17a52 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -7361,7 +7361,7 @@ static void i40e_handle_mdd_event(struct i40e_pf *pf)
 				 "Too many MDD events on VF %d, disabled\n", i);
 			dev_info(&pf->pdev->dev,
 				 "Use PF Control I/F to re-enable the VF\n");
-			set_bit(I40E_VF_STAT_DISABLED, &vf->vf_states);
+			set_bit(I40E_VF_STATE_DISABLED, &vf->vf_states);
 		}
 	}
 
diff --git a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
index 423068a..b97f9f2 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
@@ -50,8 +50,8 @@ static void i40e_vc_vf_broadcast(struct i40e_pf *pf,
 	for (i = 0; i < pf->num_alloc_vfs; i++, vf++) {
 		int abs_vf_id = vf->vf_id + (int)hw->func_caps.vf_base_id;
 		/* Not all vfs are enabled so skip the ones that are not */
-		if (!test_bit(I40E_VF_STAT_INIT, &vf->vf_states) &&
-		    !test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states))
+		if (!test_bit(I40E_VF_STATE_INIT, &vf->vf_states) &&
+		    !test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states))
 			continue;
 
 		/* Ignore return value on purpose - a given VF may fail, but
@@ -137,8 +137,8 @@ void i40e_vc_notify_vf_reset(struct i40e_vf *vf)
 		return;
 
 	/* verify if the VF is in either init or active before proceeding */
-	if (!test_bit(I40E_VF_STAT_INIT, &vf->vf_states) &&
-	    !test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states))
+	if (!test_bit(I40E_VF_STATE_INIT, &vf->vf_states) &&
+	    !test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states))
 		return;
 
 	abs_vf_id = vf->vf_id + (int)vf->pf->hw.func_caps.vf_base_id;
@@ -812,7 +812,7 @@ static void i40e_free_vf_res(struct i40e_vf *vf)
 	/* Start by disabling VF's configuration API to prevent the OS from
 	 * accessing the VF's VSI after it's freed / invalidated.
 	 */
-	clear_bit(I40E_VF_STAT_INIT, &vf->vf_states);
+	clear_bit(I40E_VF_STATE_INIT, &vf->vf_states);
 
 	/* free vsi & disconnect it from the parent uplink */
 	if (vf->lan_vsi_idx) {
@@ -884,7 +884,7 @@ static int i40e_alloc_vf_res(struct i40e_vf *vf)
 	vf->num_queue_pairs = total_queue_pairs;
 
 	/* VF is now completely initialized */
-	set_bit(I40E_VF_STAT_INIT, &vf->vf_states);
+	set_bit(I40E_VF_STATE_INIT, &vf->vf_states);
 
 error_alloc:
 	if (ret)
@@ -938,7 +938,7 @@ static void i40e_trigger_vf_reset(struct i40e_vf *vf, bool flr)
 	u32 reg, reg_idx, bit_idx;
 
 	/* warn the VF */
-	clear_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states);
+	clear_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states);
 
 	/* Disable VF's configuration API during reset. The flag is re-enabled
 	 * in i40e_alloc_vf_res(), when it's safe again to access VF's VSI.
@@ -946,7 +946,7 @@ static void i40e_trigger_vf_reset(struct i40e_vf *vf, bool flr)
 	 * to do it earlier to give some time to finish to any VF config
 	 * functions that may still be running at this point.
 	 */
-	clear_bit(I40E_VF_STAT_INIT, &vf->vf_states);
+	clear_bit(I40E_VF_STATE_INIT, &vf->vf_states);
 
 	/* In the case of a VFLR, the HW has already reset the VF and we
 	 * just need to clean up, so don't hit the VFRTRIG register.
@@ -1004,8 +1004,8 @@ static void i40e_cleanup_reset_vf(struct i40e_vf *vf)
 	if (!i40e_alloc_vf_res(vf)) {
 		int abs_vf_id = vf->vf_id + hw->func_caps.vf_base_id;
 		i40e_enable_vf_mappings(vf);
-		set_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states);
-		clear_bit(I40E_VF_STAT_DISABLED, &vf->vf_states);
+		set_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states);
+		clear_bit(I40E_VF_STATE_DISABLED, &vf->vf_states);
 		/* Do not notify the client during VF init */
 		if (vf->pf->num_alloc_vfs)
 			i40e_notify_client_of_vf_reset(pf, abs_vf_id);
@@ -1194,7 +1194,7 @@ void i40e_free_vfs(struct i40e_pf *pf)
 
 	i40e_notify_client_of_vf_enable(pf, 0);
 	for (i = 0; i < pf->num_alloc_vfs; i++)
-		if (test_bit(I40E_VF_STAT_INIT, &pf->vf[i].vf_states))
+		if (test_bit(I40E_VF_STATE_INIT, &pf->vf[i].vf_states))
 			i40e_vsi_stop_rings(pf->vsi[pf->vf[i].lan_vsi_idx]);
 
 	/* Disable IOV before freeing resources. This lets any VF drivers
@@ -1212,7 +1212,7 @@ void i40e_free_vfs(struct i40e_pf *pf)
 	tmp = pf->num_alloc_vfs;
 	pf->num_alloc_vfs = 0;
 	for (i = 0; i < tmp; i++) {
-		if (test_bit(I40E_VF_STAT_INIT, &pf->vf[i].vf_states))
+		if (test_bit(I40E_VF_STATE_INIT, &pf->vf[i].vf_states))
 			i40e_free_vf_res(&pf->vf[i]);
 		/* disable qp mappings */
 		i40e_disable_vf_mappings(&pf->vf[i]);
@@ -1427,7 +1427,7 @@ static int i40e_vc_send_msg_to_vf(struct i40e_vf *vf, u32 v_opcode,
 				"Number of invalid messages exceeded for VF %d\n",
 				vf->vf_id);
 			dev_err(&pf->pdev->dev, "Use PF Control I/F to enable the VF\n");
-			set_bit(I40E_VF_STAT_DISABLED, &vf->vf_states);
+			set_bit(I40E_VF_STATE_DISABLED, &vf->vf_states);
 		}
 	} else {
 		vf->num_valid_msgs++;
@@ -1502,7 +1502,7 @@ static int i40e_vc_get_vf_resources_msg(struct i40e_vf *vf, u8 *msg)
 	int len = 0;
 	int ret;
 
-	if (!test_bit(I40E_VF_STAT_INIT, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_INIT, &vf->vf_states)) {
 		aq_ret = I40E_ERR_PARAM;
 		goto err;
 	}
@@ -1531,7 +1531,7 @@ static int i40e_vc_get_vf_resources_msg(struct i40e_vf *vf, u8 *msg)
 	if (i40e_vf_client_capable(pf, vf->vf_id) &&
 	    (vf->driver_caps & I40E_VIRTCHNL_VF_OFFLOAD_IWARP)) {
 		vfres->vf_offload_flags |= I40E_VIRTCHNL_VF_OFFLOAD_IWARP;
-		set_bit(I40E_VF_STAT_IWARPENA, &vf->vf_states);
+		set_bit(I40E_VF_STATE_IWARPENA, &vf->vf_states);
 	}
 
 	if (vf->driver_caps & I40E_VIRTCHNL_VF_OFFLOAD_RSS_PF) {
@@ -1592,7 +1592,7 @@ static int i40e_vc_get_vf_resources_msg(struct i40e_vf *vf, u8 *msg)
 		ether_addr_copy(vfres->vsi_res[0].default_mac_addr,
 				vf->default_lan_addr.addr);
 	}
-	set_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states);
+	set_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states);
 
 err:
 	/* send the response back to the VF */
@@ -1615,7 +1615,7 @@ static int i40e_vc_get_vf_resources_msg(struct i40e_vf *vf, u8 *msg)
  **/
 static void i40e_vc_reset_vf_msg(struct i40e_vf *vf)
 {
-	if (test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states))
+	if (test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states))
 		i40e_reset_vf(vf, false);
 }
 
@@ -1663,7 +1663,7 @@ static int i40e_vc_config_promiscuous_mode_msg(struct i40e_vf *vf,
 	int bkt;
 
 	vsi = i40e_find_vsi_from_id(pf, info->vsi_id);
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states) ||
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states) ||
 	    !i40e_vc_isvalid_vsi_id(vf, info->vsi_id) ||
 	    !vsi) {
 		aq_ret = I40E_ERR_PARAM;
@@ -1724,9 +1724,9 @@ static int i40e_vc_config_promiscuous_mode_msg(struct i40e_vf *vf,
 			 "VF %d successfully set multicast promiscuous mode\n",
 			 vf->vf_id);
 		if (allmulti)
-			set_bit(I40E_VF_STAT_MC_PROMISC, &vf->vf_states);
+			set_bit(I40E_VF_STATE_MC_PROMISC, &vf->vf_states);
 		else
-			clear_bit(I40E_VF_STAT_MC_PROMISC, &vf->vf_states);
+			clear_bit(I40E_VF_STATE_MC_PROMISC, &vf->vf_states);
 	}
 
 	if (info->flags & I40E_FLAG_VF_UNICAST_PROMISC)
@@ -1775,9 +1775,9 @@ static int i40e_vc_config_promiscuous_mode_msg(struct i40e_vf *vf,
 			 "VF %d successfully set unicast promiscuous mode\n",
 			 vf->vf_id);
 		if (alluni)
-			set_bit(I40E_VF_STAT_UC_PROMISC, &vf->vf_states);
+			set_bit(I40E_VF_STATE_UC_PROMISC, &vf->vf_states);
 		else
-			clear_bit(I40E_VF_STAT_UC_PROMISC, &vf->vf_states);
+			clear_bit(I40E_VF_STATE_UC_PROMISC, &vf->vf_states);
 	}
 
 error_param:
@@ -1806,7 +1806,7 @@ static int i40e_vc_config_queues_msg(struct i40e_vf *vf, u8 *msg, u16 msglen)
 	i40e_status aq_ret = 0;
 	int i;
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states)) {
 		aq_ret = I40E_ERR_PARAM;
 		goto error_param;
 	}
@@ -1863,7 +1863,7 @@ static int i40e_vc_config_irq_map_msg(struct i40e_vf *vf, u8 *msg, u16 msglen)
 	unsigned long tempmap;
 	int i;
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states)) {
 		aq_ret = I40E_ERR_PARAM;
 		goto error_param;
 	}
@@ -1923,7 +1923,7 @@ static int i40e_vc_enable_queues_msg(struct i40e_vf *vf, u8 *msg, u16 msglen)
 	u16 vsi_id = vqs->vsi_id;
 	i40e_status aq_ret = 0;
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states)) {
 		aq_ret = I40E_ERR_PARAM;
 		goto error_param;
 	}
@@ -1968,7 +1968,7 @@ static int i40e_vc_disable_queues_msg(struct i40e_vf *vf, u8 *msg, u16 msglen)
 	struct i40e_pf *pf = vf->pf;
 	i40e_status aq_ret = 0;
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states)) {
 		aq_ret = I40E_ERR_PARAM;
 		goto error_param;
 	}
@@ -2015,7 +2015,7 @@ static int i40e_vc_get_stats_msg(struct i40e_vf *vf, u8 *msg, u16 msglen)
 
 	memset(&stats, 0, sizeof(struct i40e_eth_stats));
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states)) {
 		aq_ret = I40E_ERR_PARAM;
 		goto error_param;
 	}
@@ -2102,7 +2102,7 @@ static int i40e_vc_add_mac_addr_msg(struct i40e_vf *vf, u8 *msg, u16 msglen)
 	i40e_status ret = 0;
 	int i;
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states) ||
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states) ||
 	    !i40e_vc_isvalid_vsi_id(vf, vsi_id)) {
 		ret = I40E_ERR_PARAM;
 		goto error_param;
@@ -2171,7 +2171,7 @@ static int i40e_vc_del_mac_addr_msg(struct i40e_vf *vf, u8 *msg, u16 msglen)
 	i40e_status ret = 0;
 	int i;
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states) ||
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states) ||
 	    !i40e_vc_isvalid_vsi_id(vf, vsi_id)) {
 		ret = I40E_ERR_PARAM;
 		goto error_param;
@@ -2237,7 +2237,7 @@ static int i40e_vc_add_vlan_msg(struct i40e_vf *vf, u8 *msg, u16 msglen)
 			"VF is not trusted, switch the VF to trusted to add more VLAN addresses\n");
 		goto error_param;
 	}
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states) ||
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states) ||
 	    !i40e_vc_isvalid_vsi_id(vf, vsi_id)) {
 		aq_ret = I40E_ERR_PARAM;
 		goto error_param;
@@ -2264,12 +2264,12 @@ static int i40e_vc_add_vlan_msg(struct i40e_vf *vf, u8 *msg, u16 msglen)
 		if (!ret)
 			vf->num_vlan++;
 
-		if (test_bit(I40E_VF_STAT_UC_PROMISC, &vf->vf_states))
+		if (test_bit(I40E_VF_STATE_UC_PROMISC, &vf->vf_states))
 			i40e_aq_set_vsi_uc_promisc_on_vlan(&pf->hw, vsi->seid,
 							   true,
 							   vfl->vlan_id[i],
 							   NULL);
-		if (test_bit(I40E_VF_STAT_MC_PROMISC, &vf->vf_states))
+		if (test_bit(I40E_VF_STATE_MC_PROMISC, &vf->vf_states))
 			i40e_aq_set_vsi_mc_promisc_on_vlan(&pf->hw, vsi->seid,
 							   true,
 							   vfl->vlan_id[i],
@@ -2304,7 +2304,7 @@ static int i40e_vc_remove_vlan_msg(struct i40e_vf *vf, u8 *msg, u16 msglen)
 	i40e_status aq_ret = 0;
 	int i;
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states) ||
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states) ||
 	    !i40e_vc_isvalid_vsi_id(vf, vsi_id)) {
 		aq_ret = I40E_ERR_PARAM;
 		goto error_param;
@@ -2327,12 +2327,12 @@ static int i40e_vc_remove_vlan_msg(struct i40e_vf *vf, u8 *msg, u16 msglen)
 		i40e_vsi_kill_vlan(vsi, vfl->vlan_id[i]);
 		vf->num_vlan--;
 
-		if (test_bit(I40E_VF_STAT_UC_PROMISC, &vf->vf_states))
+		if (test_bit(I40E_VF_STATE_UC_PROMISC, &vf->vf_states))
 			i40e_aq_set_vsi_uc_promisc_on_vlan(&pf->hw, vsi->seid,
 							   false,
 							   vfl->vlan_id[i],
 							   NULL);
-		if (test_bit(I40E_VF_STAT_MC_PROMISC, &vf->vf_states))
+		if (test_bit(I40E_VF_STATE_MC_PROMISC, &vf->vf_states))
 			i40e_aq_set_vsi_mc_promisc_on_vlan(&pf->hw, vsi->seid,
 							   false,
 							   vfl->vlan_id[i],
@@ -2358,8 +2358,8 @@ static int i40e_vc_iwarp_msg(struct i40e_vf *vf, u8 *msg, u16 msglen)
 	int abs_vf_id = vf->vf_id + pf->hw.func_caps.vf_base_id;
 	i40e_status aq_ret = 0;
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states) ||
-	    !test_bit(I40E_VF_STAT_IWARPENA, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states) ||
+	    !test_bit(I40E_VF_STATE_IWARPENA, &vf->vf_states)) {
 		aq_ret = I40E_ERR_PARAM;
 		goto error_param;
 	}
@@ -2389,8 +2389,8 @@ static int i40e_vc_iwarp_qvmap_msg(struct i40e_vf *vf, u8 *msg, u16 msglen,
 				(struct i40e_virtchnl_iwarp_qvlist_info *)msg;
 	i40e_status aq_ret = 0;
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states) ||
-	    !test_bit(I40E_VF_STAT_IWARPENA, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states) ||
+	    !test_bit(I40E_VF_STATE_IWARPENA, &vf->vf_states)) {
 		aq_ret = I40E_ERR_PARAM;
 		goto error_param;
 	}
@@ -2427,7 +2427,7 @@ static int i40e_vc_config_rss_key(struct i40e_vf *vf, u8 *msg, u16 msglen)
 	u16 vsi_id = vrk->vsi_id;
 	i40e_status aq_ret = 0;
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states) ||
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states) ||
 	    !i40e_vc_isvalid_vsi_id(vf, vsi_id) ||
 	    (vrk->key_len != I40E_HKEY_ARRAY_SIZE)) {
 		aq_ret = I40E_ERR_PARAM;
@@ -2459,7 +2459,7 @@ static int i40e_vc_config_rss_lut(struct i40e_vf *vf, u8 *msg, u16 msglen)
 	u16 vsi_id = vrl->vsi_id;
 	i40e_status aq_ret = 0;
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states) ||
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states) ||
 	    !i40e_vc_isvalid_vsi_id(vf, vsi_id) ||
 	    (vrl->lut_entries != I40E_VF_HLUT_ARRAY_SIZE)) {
 		aq_ret = I40E_ERR_PARAM;
@@ -2489,7 +2489,7 @@ static int i40e_vc_get_rss_hena(struct i40e_vf *vf, u8 *msg, u16 msglen)
 	i40e_status aq_ret = 0;
 	int len = 0;
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states)) {
 		aq_ret = I40E_ERR_PARAM;
 		goto err;
 	}
@@ -2526,7 +2526,7 @@ static int i40e_vc_set_rss_hena(struct i40e_vf *vf, u8 *msg, u16 msglen)
 	struct i40e_hw *hw = &pf->hw;
 	i40e_status aq_ret = 0;
 
-	if (!test_bit(I40E_VF_STAT_ACTIVE, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states)) {
 		aq_ret = I40E_ERR_PARAM;
 		goto err;
 	}
@@ -2556,7 +2556,7 @@ static int i40e_vc_validate_vf_msg(struct i40e_vf *vf, u32 v_opcode,
 	int valid_len = 0;
 
 	/* Check if VF is disabled. */
-	if (test_bit(I40E_VF_STAT_DISABLED, &vf->vf_states))
+	if (test_bit(I40E_VF_STATE_DISABLED, &vf->vf_states))
 		return I40E_ERR_PARAM;
 
 	/* Validate message length. */
@@ -2880,7 +2880,7 @@ int i40e_ndo_set_vf_mac(struct net_device *netdev, int vf_id, u8 *mac)
 
 	vf = &(pf->vf[vf_id]);
 	vsi = pf->vsi[vf->lan_vsi_idx];
-	if (!test_bit(I40E_VF_STAT_INIT, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_INIT, &vf->vf_states)) {
 		dev_err(&pf->pdev->dev, "VF %d still in reset. Try again.\n",
 			vf_id);
 		ret = -EAGAIN;
@@ -2969,7 +2969,7 @@ int i40e_ndo_set_vf_port_vlan(struct net_device *netdev, int vf_id,
 
 	vf = &(pf->vf[vf_id]);
 	vsi = pf->vsi[vf->lan_vsi_idx];
-	if (!test_bit(I40E_VF_STAT_INIT, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_INIT, &vf->vf_states)) {
 		dev_err(&pf->pdev->dev, "VF %d still in reset. Try again.\n",
 			vf_id);
 		ret = -EAGAIN;
@@ -3101,7 +3101,7 @@ int i40e_ndo_set_vf_bw(struct net_device *netdev, int vf_id, int min_tx_rate,
 
 	vf = &(pf->vf[vf_id]);
 	vsi = pf->vsi[vf->lan_vsi_idx];
-	if (!test_bit(I40E_VF_STAT_INIT, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_INIT, &vf->vf_states)) {
 		dev_err(&pf->pdev->dev, "VF %d still in reset. Try again.\n",
 			vf_id);
 		ret = -EAGAIN;
@@ -3182,7 +3182,7 @@ int i40e_ndo_get_vf_config(struct net_device *netdev,
 	vf = &(pf->vf[vf_id]);
 	/* first vsi is always the LAN vsi */
 	vsi = pf->vsi[vf->lan_vsi_idx];
-	if (!test_bit(I40E_VF_STAT_INIT, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_INIT, &vf->vf_states)) {
 		dev_err(&pf->pdev->dev, "VF %d still in reset. Try again.\n",
 			vf_id);
 		ret = -EAGAIN;
@@ -3313,7 +3313,7 @@ int i40e_ndo_set_vf_spoofchk(struct net_device *netdev, int vf_id, bool enable)
 	}
 
 	vf = &(pf->vf[vf_id]);
-	if (!test_bit(I40E_VF_STAT_INIT, &vf->vf_states)) {
+	if (!test_bit(I40E_VF_STATE_INIT, &vf->vf_states)) {
 		dev_err(&pf->pdev->dev, "VF %d still in reset. Try again.\n",
 			vf_id);
 		ret = -EAGAIN;
diff --git a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h
index f76d234..7ff45df 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h
@@ -56,13 +56,13 @@ enum i40e_queue_ctrl {
 
 /* VF states */
 enum i40e_vf_states {
-	I40E_VF_STAT_INIT = 0,
-	I40E_VF_STAT_ACTIVE,
-	I40E_VF_STAT_IWARPENA,
-	I40E_VF_STAT_FCOEENA,
-	I40E_VF_STAT_DISABLED,
-	I40E_VF_STAT_MC_PROMISC,
-	I40E_VF_STAT_UC_PROMISC,
+	I40E_VF_STATE_INIT = 0,
+	I40E_VF_STATE_ACTIVE,
+	I40E_VF_STATE_IWARPENA,
+	I40E_VF_STATE_FCOEENA,
+	I40E_VF_STATE_DISABLED,
+	I40E_VF_STATE_MC_PROMISC,
+	I40E_VF_STATE_UC_PROMISC,
 };
 
 /* VF capabilities */
-- 
2.9.3

[Intel-wired-lan] [next PATCH S70 12/12] i40e: make use of i40e_reset_all_vfs when initializing new VFs

[Alice Michael]

From: Jacob Keller <jacob.e.keller@intel.com>

When allocating a large number of VFs, the driver previously used
i40e_reset_vf in a sequence. Just as when performing a normal reset,
this accumulates a large amount of delay for handling all of the VFs in
sequence. This delay is mainly due to a hardware requirement to wait
after initiating a reset on the VF.

We recently added a new function, i40e_reset_all_vfs() which can be used
to amortize the delay time, by first triggering all VF resets, then
waiting once, and finally cleaning up and allocating the VFs. This is
almost as good as truly running the resets in parallel.

In order to avoid sending a spurious reset message to a client
interface, we have a check to see whether we've assigned
pf->num_alloc_vfs yet. This was originally intended as a way to
distinquish the "initialization" case from the regular reset case.

Unfortunately, this means that we can't directly use i40e_reset_all_vfs
yet. Lets avoid this check of pf->num_alloc_vfs by replacing it with
a proper VSI state bit which we can use instead. This makes the
intention much clearer and allows us to re-use the i40e_reset_all_vfs
function directly.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Williams, Mitch A <mitch.a.williams@intel.com>
Change-ID: I694279b37eb6b5a91b6670182d0c15d10244fd6e
---
 drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c | 10 +++++++---
 drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h |  1 +
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
index b97f9f2..355821b 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
@@ -1007,7 +1007,8 @@ static void i40e_cleanup_reset_vf(struct i40e_vf *vf)
 		set_bit(I40E_VF_STATE_ACTIVE, &vf->vf_states);
 		clear_bit(I40E_VF_STATE_DISABLED, &vf->vf_states);
 		/* Do not notify the client during VF init */
-		if (vf->pf->num_alloc_vfs)
+		if (test_and_clear_bit(I40E_VF_STATE_PRE_ENABLE,
+				       &vf->vf_states))
 			i40e_notify_client_of_vf_reset(pf, abs_vf_id);
 		vf->num_vlan = 0;
 	}
@@ -1283,8 +1284,8 @@ int i40e_alloc_vfs(struct i40e_pf *pf, u16 num_alloc_vfs)
 		/* assign default capabilities */
 		set_bit(I40E_VIRTCHNL_VF_CAP_L2, &vfs[i].vf_caps);
 		vfs[i].spoofchk = true;
-		/* VF resources get allocated during reset */
-		i40e_reset_vf(&vfs[i], false);
+
+		set_bit(I40E_VF_STATE_PRE_ENABLE, &vfs[i].vf_states);
 
 		if (pf->eswitch_mode == DEVLINK_ESWITCH_MODE_SWITCHDEV) {
 			ret = i40e_alloc_port_netdev(&vfs[i],
@@ -1295,6 +1296,9 @@ int i40e_alloc_vfs(struct i40e_pf *pf, u16 num_alloc_vfs)
 	}
 	pf->num_alloc_vfs = num_alloc_vfs;
 
+	/* VF resources get allocated during reset */
+	i40e_reset_all_vfs(pf, false);
+
 	i40e_notify_client_of_vf_enable(pf, num_alloc_vfs);
 
 err_alloc:
diff --git a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h
index 7ff45df..3b122ac 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h
@@ -63,6 +63,7 @@ enum i40e_vf_states {
 	I40E_VF_STATE_DISABLED,
 	I40E_VF_STATE_MC_PROMISC,
 	I40E_VF_STATE_UC_PROMISC,
+	I40E_VF_STATE_PRE_ENABLE,
 };
 
 /* VF capabilities */
-- 
2.9.3

[Intel-wired-lan] [next PATCH S70 03/12] i40e: don't hold RTNL lock while waiting for VF reset to finish

[Keller, Jacob E]

ACK

Thanks,
Jake

> -----Original Message-----
> From: Michael, Alice
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-lan at lists.osuosl.org
> Cc: Keller, Jacob E <jacob.e.keller@intel.com>
> Subject: [next PATCH S70 03/12] i40e: don't hold RTNL lock while waiting for VF
> reset to finish
> 
> From: Jacob Keller <jacob.e.keller@intel.com>
> 
> We made some effort to reduce the RTNL lock scope when resetting and
> rebuilding the PF. Unfortunately we still held the RTNL lock during the
> VF reset operation, which meant that multiple PFs could not reset in
> parallel due to the global lock. For now, further reduce the scope by
> not holding the RTNL lock while resetting VFs. This allows multiple PFs
> to reset in a timely manner.
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Change-ID: I2fbf823a0063f24dff67676cad09f0bbf83ee4ce
> ---
>  drivers/net/ethernet/intel/i40e/i40e_main.c | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c
> b/drivers/net/ethernet/intel/i40e/i40e_main.c
> index 4b507ce..1c06693 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_main.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
> @@ -7151,6 +7151,10 @@ static void i40e_rebuild(struct i40e_pf *pf, bool reinit,
> bool lock_acquired)
>  	/* restart the VSIs that were rebuilt and running before the reset */
>  	i40e_pf_unquiesce_all_vsi(pf);
> 
> +	/* Release the RTNL lock before we start resetting VFs */
> +	if (!lock_acquired)
> +		rtnl_unlock();
> +
>  	if (pf->num_alloc_vfs) {
>  		for (v = 0; v < pf->num_alloc_vfs; v++)
>  			i40e_reset_vf(&pf->vf[v], true);
> @@ -7159,9 +7163,12 @@ static void i40e_rebuild(struct i40e_pf *pf, bool reinit,
> bool lock_acquired)
>  	/* tell the firmware that we're starting */
>  	i40e_send_version(pf);
> 
> +	/* We've already released the lock, so don't do it again */
> +	goto end_core_reset;
> +
>  end_unlock:
> -if (!lock_acquired)
> -	rtnl_unlock();
> +	if (!lock_acquired)
> +		rtnl_unlock();
>  end_core_reset:
>  	clear_bit(__I40E_RESET_FAILED, &pf->state);
>  clear_recovery:
> --
> 2.9.3

[Intel-wired-lan] [next PATCH S70 05/12] i40e: fix CONFIG_BUSY checks in i40e_set_settings function

[Keller, Jacob E]

You probably need to fixup or remove the commit hashes for the mentioned commits, unless you already did that?

Thanks,
Jake

> -----Original Message-----
> From: Michael, Alice
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-lan at lists.osuosl.org
> Cc: Keller, Jacob E <jacob.e.keller@intel.com>
> Subject: [next PATCH S70 05/12] i40e: fix CONFIG_BUSY checks in
> i40e_set_settings function
> 
> From: Jacob Keller <jacob.e.keller@intel.com>
> 
> The check for I40E_CONFIG_BUSY state bit in the i40e_set_link_ksettings
> function is fishy. First we can notice a few things about the check here.
> 
> First a similar check was introduced by commit
> 'c7d05ca89f8e ("i40e: driver ethtool core")'
> 
> Later a commit introducing the link settings was added by commit
> 'bf9c71417f72 ("i40e: Implement set_settings for ethtool")'
> 
> However, this second check was against vsi->state instead of pf->state,
> and also failed to set the bit, it only checks. That indicates the locking
> was not quite correct. The only other place that the state bit
> in vsi->state gets used is to protect the filter list.
> 
> Since this code does not care about the mac filter list,  and seems
> clear the original code should have set the pf->state bit. Fix these
> issues by using pf->state correctly, and by actually setting the bit
> so that we properly lock as expected.
> 
> Since these checks occur while holding the rtnl_lock(), lets also add a
> timeout so that we don't potentially softlock the system.
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> ---
>  drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 38 ++++++++++++++++++++---
> ---
>  1 file changed, 29 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
> b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
> index 10325b5..08035c4 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
> @@ -698,6 +698,7 @@ static int i40e_set_link_ksettings(struct net_device
> *netdev,
>  	struct ethtool_link_ksettings copy_cmd;
>  	i40e_status status = 0;
>  	bool change = false;
> +	int timeout = 50;
>  	int err = 0;
>  	u32 autoneg;
>  	u32 advertise;
> @@ -756,14 +757,20 @@ static int i40e_set_link_ksettings(struct net_device
> *netdev,
>  	if (memcmp(&copy_cmd, &safe_cmd, sizeof(struct
> ethtool_link_ksettings)))
>  		return -EOPNOTSUPP;
> 
> -	while (test_bit(__I40E_CONFIG_BUSY, &vsi->state))
> +	while (test_and_set_bit(__I40E_CONFIG_BUSY, &pf->state)) {
> +		timeout--;
> +		if (!timeout)
> +			return -EBUSY;
>  		usleep_range(1000, 2000);
> +	}
> 
>  	/* Get the current phy config */
>  	status = i40e_aq_get_phy_capabilities(hw, false, false, &abilities,
>  					      NULL);
> -	if (status)
> -		return -EAGAIN;
> +	if (status) {
> +		err = -EAGAIN;
> +		goto done;
> +	}
> 
>  	/* Copy abilities to config in case autoneg is not
>  	 * set below
> @@ -779,7 +786,8 @@ static int i40e_set_link_ksettings(struct net_device
> *netdev,
>  			if (!ethtool_link_ksettings_test_link_mode(
>  				    &safe_cmd, supported, Autoneg)) {
>  				netdev_info(netdev, "Autoneg not supported on
> this phy\n");
> -				return -EINVAL;
> +				err = -EINVAL;
> +				goto done;
>  			}
>  			/* Autoneg is allowed to change */
>  			config.abilities = abilities.abilities |
> @@ -797,7 +805,8 @@ static int i40e_set_link_ksettings(struct net_device
> *netdev,
>  			    hw->phy.link_info.phy_type !=
>  			    I40E_PHY_TYPE_10GBASE_T) {
>  				netdev_info(netdev, "Autoneg cannot be
> disabled on this phy\n");
> -				return -EINVAL;
> +				err = -EINVAL;
> +				goto done;
>  			}
>  			/* Autoneg is allowed to change */
>  			config.abilities = abilities.abilities &
> @@ -808,8 +817,10 @@ static int i40e_set_link_ksettings(struct net_device
> *netdev,
> 
>  	ethtool_convert_link_mode_to_legacy_u32(&tmp,
> 
> 	safe_cmd.link_modes.supported);
> -	if (advertise & ~tmp)
> -		return -EINVAL;
> +	if (advertise & ~tmp) {
> +		err = -EINVAL;
> +		goto done;
> +	}
> 
>  	if (advertise & ADVERTISED_100baseT_Full)
>  		config.link_speed |= I40E_LINK_SPEED_100MB;
> @@ -865,7 +876,8 @@ static int i40e_set_link_ksettings(struct net_device
> *netdev,
>  			netdev_info(netdev, "Set phy config failed, err %s aq_err
> %s\n",
>  				    i40e_stat_str(hw, status),
>  				    i40e_aq_str(hw, hw->aq.asq_last_status));
> -			return -EAGAIN;
> +			err = -EAGAIN;
> +			goto done;
>  		}
> 
>  		status = i40e_update_link_info(hw);
> @@ -878,6 +890,9 @@ static int i40e_set_link_ksettings(struct net_device
> *netdev,
>  		netdev_info(netdev, "Nothing changed, exiting without setting
> anything.\n");
>  	}
> 
> +done:
> +	clear_bit(__I40E_CONFIG_BUSY, &pf->state);
> +
>  	return err;
>  }
> 
> @@ -1292,6 +1307,7 @@ static int i40e_set_ringparam(struct net_device
> *netdev,
>  	struct i40e_vsi *vsi = np->vsi;
>  	struct i40e_pf *pf = vsi->back;
>  	u32 new_rx_count, new_tx_count;
> +	int timeout = 50;
>  	int i, err = 0;
> 
>  	if ((ring->rx_mini_pending) || (ring->rx_jumbo_pending))
> @@ -1316,8 +1332,12 @@ static int i40e_set_ringparam(struct net_device
> *netdev,
>  	    (new_rx_count == vsi->rx_rings[0]->count))
>  		return 0;
> 
> -	while (test_and_set_bit(__I40E_CONFIG_BUSY, &pf->state))
> +	while (test_and_set_bit(__I40E_CONFIG_BUSY, &pf->state)) {
> +		timeout--;
> +		if (!timeout)
> +			return -EBUSY;
>  		usleep_range(1000, 2000);
> +	}
> 
>  	if (!netif_running(vsi->netdev)) {
>  		/* simple case - set for the next time the netdev is started */
> --
> 2.9.3

[Intel-wired-lan] [next PATCH S70 04/12] i40e: factor out queue control from i40e_vsi_control_(tx|rx)

[Keller, Jacob E]

ACK

Thanks,
Jake

> -----Original Message-----
> From: Michael, Alice
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-lan at lists.osuosl.org
> Cc: Keller, Jacob E <jacob.e.keller@intel.com>
> Subject: [next PATCH S70 04/12] i40e: factor out queue control from
> i40e_vsi_control_(tx|rx)
> 
> From: Jacob Keller <jacob.e.keller@intel.com>
> 
> A future patch will need to be able to handle controlling queues without
> waiting until all VSIs are handled. Factor out the direct queue
> modification so that we can easily re-use this code. The result is also
> a bit easier to read since we don't embed multiple single-letter loop
> counters.
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Change-ID: Id923cbfa43127b1c24d8ed4f809b1012c736d9ac
> ---
>  drivers/net/ethernet/intel/i40e/i40e_main.c | 139 ++++++++++++++++++-------
> ---
>  1 file changed, 89 insertions(+), 50 deletions(-)
> 
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c
> b/drivers/net/ethernet/intel/i40e/i40e_main.c
> index 1c06693..ebf2ad4 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_main.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
> @@ -3919,6 +3919,8 @@ static void i40e_netpoll(struct net_device *netdev)
>  }
>  #endif
> 
> +#define I40E_QTX_ENA_WAIT_COUNT 50
> +
>  /**
>   * i40e_pf_txq_wait - Wait for a PF's Tx queue to be enabled or disabled
>   * @pf: the PF being configured
> @@ -3949,6 +3951,50 @@ static int i40e_pf_txq_wait(struct i40e_pf *pf, int pf_q,
> bool enable)
>  }
> 
>  /**
> + * i40e_control_tx_q - Start or stop a particular Tx queue
> + * @pf: the PF structure
> + * @pf_q: the PF queue to configure
> + * @enable: start or stop the queue
> + *
> + * This function enables or disables a single queue. Note that any delay
> + * required after the operation is expected to be handled by the caller of
> + * this function.
> + **/
> +static void i40e_control_tx_q(struct i40e_pf *pf, int pf_q, bool enable)
> +{
> +	struct i40e_hw *hw = &pf->hw;
> +	u32 tx_reg;
> +	int i;
> +
> +	/* warn the TX unit of coming changes */
> +	i40e_pre_tx_queue_cfg(&pf->hw, pf_q, enable);
> +	if (!enable)
> +		usleep_range(10, 20);
> +
> +	for (i = 0; i < I40E_QTX_ENA_WAIT_COUNT; i++) {
> +		tx_reg = rd32(hw, I40E_QTX_ENA(pf_q));
> +		if (((tx_reg >> I40E_QTX_ENA_QENA_REQ_SHIFT) & 1) ==
> +		    ((tx_reg >> I40E_QTX_ENA_QENA_STAT_SHIFT) & 1))
> +			break;
> +		usleep_range(1000, 2000);
> +	}
> +
> +	/* Skip if the queue is already in the requested state */
> +	if (enable == !!(tx_reg & I40E_QTX_ENA_QENA_STAT_MASK))
> +		return;
> +
> +	/* turn on/off the queue */
> +	if (enable) {
> +		wr32(hw, I40E_QTX_HEAD(pf_q), 0);
> +		tx_reg |= I40E_QTX_ENA_QENA_REQ_MASK;
> +	} else {
> +		tx_reg &= ~I40E_QTX_ENA_QENA_REQ_MASK;
> +	}
> +
> +	wr32(hw, I40E_QTX_ENA(pf_q), tx_reg);
> +}
> +
> +/**
>   * i40e_vsi_control_tx - Start or stop a VSI's rings
>   * @vsi: the VSI being configured
>   * @enable: start or stop the rings
> @@ -3956,39 +4002,13 @@ static int i40e_pf_txq_wait(struct i40e_pf *pf, int
> pf_q, bool enable)
>  static int i40e_vsi_control_tx(struct i40e_vsi *vsi, bool enable)
>  {
>  	struct i40e_pf *pf = vsi->back;
> -	struct i40e_hw *hw = &pf->hw;
> -	int i, j, pf_q, ret = 0;
> -	u32 tx_reg;
> +	int i, pf_q, ret = 0;
> 
>  	pf_q = vsi->base_queue;
>  	for (i = 0; i < vsi->num_queue_pairs; i++, pf_q++) {
> +		i40e_control_tx_q(pf, pf_q, enable);
> 
> -		/* warn the TX unit of coming changes */
> -		i40e_pre_tx_queue_cfg(&pf->hw, pf_q, enable);
> -		if (!enable)
> -			usleep_range(10, 20);
> -
> -		for (j = 0; j < 50; j++) {
> -			tx_reg = rd32(hw, I40E_QTX_ENA(pf_q));
> -			if (((tx_reg >> I40E_QTX_ENA_QENA_REQ_SHIFT) & 1) ==
> -			    ((tx_reg >> I40E_QTX_ENA_QENA_STAT_SHIFT) & 1))
> -				break;
> -			usleep_range(1000, 2000);
> -		}
> -		/* Skip if the queue is already in the requested state */
> -		if (enable == !!(tx_reg & I40E_QTX_ENA_QENA_STAT_MASK))
> -			continue;
> -
> -		/* turn on/off the queue */
> -		if (enable) {
> -			wr32(hw, I40E_QTX_HEAD(pf_q), 0);
> -			tx_reg |= I40E_QTX_ENA_QENA_REQ_MASK;
> -		} else {
> -			tx_reg &= ~I40E_QTX_ENA_QENA_REQ_MASK;
> -		}
> -
> -		wr32(hw, I40E_QTX_ENA(pf_q), tx_reg);
> -		/* No waiting for the Tx queue to disable */
> +		/* Don't wait to disable when port Tx is suspended */
>  		if (!enable && test_bit(__I40E_PORT_TX_SUSPENDED, &pf-
> >state))
>  			continue;
> 
> @@ -4035,6 +4055,43 @@ static int i40e_pf_rxq_wait(struct i40e_pf *pf, int pf_q,
> bool enable)
>  }
> 
>  /**
> + * i40e_control_rx_q - Start or stop a particular Rx queue
> + * @pf: the PF structure
> + * @pf_q: the PF queue to configure
> + * @enable: start or stop the queue
> + *
> + * This function enables or disables a single queue. Note that any delay
> + * required after the operation is expected to be handled by the caller of
> + * this function.
> + **/
> +static void i40e_control_rx_q(struct i40e_pf *pf, int pf_q, bool enable)
> +{
> +	struct i40e_hw *hw = &pf->hw;
> +	u32 rx_reg;
> +	int i;
> +
> +	for (i = 0; i < I40E_QTX_ENA_WAIT_COUNT; i++) {
> +		rx_reg = rd32(hw, I40E_QRX_ENA(pf_q));
> +		if (((rx_reg >> I40E_QRX_ENA_QENA_REQ_SHIFT) & 1) ==
> +		    ((rx_reg >> I40E_QRX_ENA_QENA_STAT_SHIFT) & 1))
> +			break;
> +		usleep_range(1000, 2000);
> +	}
> +
> +	/* Skip if the queue is already in the requested state */
> +	if (enable == !!(rx_reg & I40E_QRX_ENA_QENA_STAT_MASK))
> +		return;
> +
> +	/* turn on/off the queue */
> +	if (enable)
> +		rx_reg |= I40E_QRX_ENA_QENA_REQ_MASK;
> +	else
> +		rx_reg &= ~I40E_QRX_ENA_QENA_REQ_MASK;
> +
> +	wr32(hw, I40E_QRX_ENA(pf_q), rx_reg);
> +}
> +
> +/**
>   * i40e_vsi_control_rx - Start or stop a VSI's rings
>   * @vsi: the VSI being configured
>   * @enable: start or stop the rings
> @@ -4042,31 +4099,13 @@ static int i40e_pf_rxq_wait(struct i40e_pf *pf, int
> pf_q, bool enable)
>  static int i40e_vsi_control_rx(struct i40e_vsi *vsi, bool enable)
>  {
>  	struct i40e_pf *pf = vsi->back;
> -	struct i40e_hw *hw = &pf->hw;
> -	int i, j, pf_q, ret = 0;
> -	u32 rx_reg;
> +	int i, pf_q, ret = 0;
> 
>  	pf_q = vsi->base_queue;
>  	for (i = 0; i < vsi->num_queue_pairs; i++, pf_q++) {
> -		for (j = 0; j < 50; j++) {
> -			rx_reg = rd32(hw, I40E_QRX_ENA(pf_q));
> -			if (((rx_reg >> I40E_QRX_ENA_QENA_REQ_SHIFT) & 1) ==
> -			    ((rx_reg >> I40E_QRX_ENA_QENA_STAT_SHIFT) & 1))
> -				break;
> -			usleep_range(1000, 2000);
> -		}
> +		i40e_control_rx_q(pf, pf_q, enable);
> 
> -		/* Skip if the queue is already in the requested state */
> -		if (enable == !!(rx_reg & I40E_QRX_ENA_QENA_STAT_MASK))
> -			continue;
> -
> -		/* turn on/off the queue */
> -		if (enable)
> -			rx_reg |= I40E_QRX_ENA_QENA_REQ_MASK;
> -		else
> -			rx_reg &= ~I40E_QRX_ENA_QENA_REQ_MASK;
> -		wr32(hw, I40E_QRX_ENA(pf_q), rx_reg);
> -		/* No waiting for the Tx queue to disable */
> +		/* Don't wait to disable when port Tx is suspended */
>  		if (!enable && test_bit(__I40E_PORT_TX_SUSPENDED, &pf-
> >state))
>  			continue;
> 
> --
> 2.9.3

[Intel-wired-lan] [next PATCH S70 01/12] i40e/i40evf: Add tracepoints

[Peterson, Scott D]

ACK

On Thu, 2017-04-13 at 04:45 -0400, Alice Michael wrote:
> From: "Peterson, Scott D" <scott.d.peterson@intel.com>
> 
> This patch adds tracepoints to the i40e and i40evf drivers to which
> BPF programs can be attached for feature testing and verification.
> It's expected that an attached BPF program will identify and count or
> log some interesting subset of traffic. The bcc-tools package is
> helpful there for containing all the BPF arcana in a handy Python
> wrapper. Though you can make these tracepoints log trace messages, the
> messages themselves probably won't be very useful (other to verify the
> tracepoint is being called while you're debugging your BPF program).
> 
> The idea here is that tracepoints have such low performance cost when
> disabled that we can leave these in the upstream drivers. This may
> eventually enable the instrumentation of unmodified customer systems
> should the need arise to verify a NIC feature is working as expected.
> In general this enables one set of feature verification tools to be
> used on these drivers whether they're built with the kernel or
> separately.
> 
> Users are advised against using these tracepoints for anything other
> than a diagnostic tool. They have a performance impact when enabled,
> and their exact placement and form may change as we see how well they
> work in practice for the purposes above.
> 
> Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
> Change-ID: Id6014a7322c0e6d08068114dd20bd156f2f6435e
> ---
> Testing Hints:
> 
> See that performance is unaffected while tracepoints are present
> but disabled (default state). Enable them all and see they appear in
> /sys/kernel/debug/tracing/trace when exercised.
> 
> When disabled, a tracepoint reduces to a 5-byte no-op.
> When enabled, that code is patched to jump to the tracepoint clause,
> which is located somewhere nearby.??See include/linux/jump_label.h for
> info in static keys, and samples/trace-events/trace-events-samples.[ch]
> for info on the trace events.
> 
> To test tracepoints:
> insmod i40e and bring up a link
> cd /sys/kernel/debug/tracing
> echo > trace
> echo 1 > events/i40e/enable
> send something over i40e
> more trace
> echo 0 > events/i40e/enable
> 
> i40evf trace events are in events/i40evf, and enabled separately.
> 
> A sample BPF program (CORE/example_tp_bpf.py) is available, but won't
> be submitted upstream and so is not included in this patch.
> 
> It demonstrates how to access the structures pointed to by the tracepoint
> args from BPF. See the comments in that script for details and usage. As
> is it will attach itself to one of these tracepoints and emit mostly
> useless messages when the i40e_clean_rx_irq tracepoint is invoked.?
> ?drivers/net/ethernet/intel/i40e/Makefile????????|???3 +
> ?drivers/net/ethernet/intel/i40e/i40e_main.c?????|???6 +
> ?drivers/net/ethernet/intel/i40e/i40e_trace.h????| 229
> ++++++++++++++++++++++++
> ?drivers/net/ethernet/intel/i40e/i40e_txrx.c?????|???9 +
> ?drivers/net/ethernet/intel/i40evf/Makefile??????|???3 +
> ?drivers/net/ethernet/intel/i40evf/i40e_trace.h??| 229
> ++++++++++++++++++++++++
> ?drivers/net/ethernet/intel/i40evf/i40e_txrx.c???|???9 +
> ?drivers/net/ethernet/intel/i40evf/i40evf_main.c |???7 +
> ?8 files changed, 495 insertions(+)
> ?create mode 100644 drivers/net/ethernet/intel/i40e/i40e_trace.h
> ?create mode 100644 drivers/net/ethernet/intel/i40evf/i40e_trace.h
> 
> diff --git a/drivers/net/ethernet/intel/i40e/Makefile
> b/drivers/net/ethernet/intel/i40e/Makefile
> index 4f454d3..3da482c 100644
> --- a/drivers/net/ethernet/intel/i40e/Makefile
> +++ b/drivers/net/ethernet/intel/i40e/Makefile
> @@ -28,6 +28,9 @@
> ?# Makefile for the Intel(R) Ethernet Connection XL710 (i40e.ko) driver
> ?#
> ?
> +ccflags-y += -I$(src)
> +subdir-ccflags-y += -I$(src)
> +
> ?obj-$(CONFIG_I40E) += i40e.o
> ?
> ?i40e-objs := i40e_main.o \
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c
> b/drivers/net/ethernet/intel/i40e/i40e_main.c
> index 0c32304..4b507ce 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_main.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
> @@ -32,6 +32,12 @@
> ?#include "i40e.h"
> ?#include "i40e_diag.h"
> ?#include <net/udp_tunnel.h>
> +/* All i40e tracepoints are defined by the include below, which
> + * must be included exactly once across the whole kernel with
> + * CREATE_TRACE_POINTS defined
> + */
> +#define CREATE_TRACE_POINTS
> +#include "i40e_trace.h"
> ?
> ?const char i40e_driver_name[] = "i40e";
> ?static const char i40e_driver_string[] =
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_trace.h
> b/drivers/net/ethernet/intel/i40e/i40e_trace.h
> new file mode 100644
> index 0000000..d3e55f5
> --- /dev/null
> +++ b/drivers/net/ethernet/intel/i40e/i40e_trace.h
> @@ -0,0 +1,229 @@
> +/**************************************************************************
> *****
> + *
> + * Intel(R) 40-10 Gigabit Ethernet Connection Network Driver
> + * Copyright(c) 2013 - 2017 Intel Corporation.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms and conditions of the GNU General Public License,
> + * version 2, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE.??See the GNU General Public License
> for
> + * more details.
> + *
> + * The full GNU General Public License is included in this distribution in
> + * the file called "COPYING".
> + *
> + * Contact Information:
> + * e1000-devel Mailing List <e1000-devel@lists.sourceforge.net>
> + * Intel Corporation, 5200 N.E. Elam Young Parkway, Hillsboro, OR 97124-
> 6497
> + *
> +
> ****************************************************************************
> **/
> +
> +/* Modeled on trace-events-sample.h */
> +
> +/* The trace subsystem name for i40e will be "i40e".
> + *
> + * This file is named i40e_trace.h.
> + *
> + * Since this include file's name is different from the trace
> + * subsystem name, we'll have to define TRACE_INCLUDE_FILE at the end
> + * of this file.
> + */
> +#undef TRACE_SYSTEM
> +#define TRACE_SYSTEM i40e
> +
> +/* See trace-events-sample.h for a detailed description of why this
> + * guard clause is different from most normal include files.
> + */
> +#if !defined(_I40E_TRACE_H_) || defined(TRACE_HEADER_MULTI_READ)
> +#define _I40E_TRACE_H_
> +
> +#include <linux/tracepoint.h>
> +
> +/**
> + * i40e_trace() macro enables shared code to refer to trace points
> + * like:
> + *
> + * trace_i40e{,vf}_example(args...)
> + *
> + * ... as:
> + *
> + * i40e_trace(example, args...)
> + *
> + * ... to resolve to the PF or VF version of the tracepoint without
> + * ifdefs, and to allow tracepoints to be disabled entirely at build
> + * time.
> + *
> + * Trace point should always be referred to in the driver via this
> + * macro.
> + *
> + * Similarly, i40e_trace_enabled(trace_name) wraps references to
> + * trace_i40e{,vf}_<trace_name>_enabled() functions.
> + */
> +#define _I40E_TRACE_NAME(trace_name) (trace_ ## i40e ## _ ## trace_name)
> +#define I40E_TRACE_NAME(trace_name) _I40E_TRACE_NAME(trace_name)
> +
> +#define i40e_trace(trace_name, args...) I40E_TRACE_NAME(trace_name)(args)
> +
> +#define i40e_trace_enabled(trace_name)
> I40E_TRACE_NAME(trace_name##_enabled)()
> +
> +/* Events common to PF and VF. Corresponding versions will be defined
> + * for both, named trace_i40e_* and trace_i40evf_*. The i40e_trace()
> + * macro above will select the right trace point name for the driver
> + * being built from shared code.
> + */
> +
> +/* Events related to a vsi & ring */
> +DECLARE_EVENT_CLASS(
> +	i40e_tx_template,
> +
> +	TP_PROTO(struct i40e_ring *ring,
> +		?struct i40e_tx_desc *desc,
> +		?struct i40e_tx_buffer *buf),
> +
> +	TP_ARGS(ring, desc, buf),
> +
> +	/* The convention here is to make the first fields in the
> +	?* TP_STRUCT match the TP_PROTO exactly. This enables the use
> +	?* of the args struct generated by the tplist tool (from the
> +	?* bcc-tools package) to be used for those fields. To access
> +	?* fields other than the tracepoint args will require the
> +	?* tplist output to be adjusted.
> +	?*/
> +	TP_STRUCT__entry(
> +		__field(void*, ring)
> +		__field(void*, desc)
> +		__field(void*, buf)
> +		__string(devname, ring->netdev->name)
> +	),
> +
> +	TP_fast_assign(
> +		__entry->ring = ring;
> +		__entry->desc = desc;
> +		__entry->buf = buf;
> +		__assign_str(devname, ring->netdev->name);
> +	),
> +
> +	TP_printk(
> +		"netdev: %s ring: %p desc: %p buf %p",
> +		__get_str(devname), __entry->ring,
> +		__entry->desc, __entry->buf)
> +);
> +
> +DEFINE_EVENT(
> +	i40e_tx_template, i40e_clean_tx_irq,
> +	TP_PROTO(struct i40e_ring *ring,
> +		?struct i40e_tx_desc *desc,
> +		?struct i40e_tx_buffer *buf),
> +
> +	TP_ARGS(ring, desc, buf));
> +
> +DEFINE_EVENT(
> +	i40e_tx_template, i40e_clean_tx_irq_unmap,
> +	TP_PROTO(struct i40e_ring *ring,
> +		?struct i40e_tx_desc *desc,
> +		?struct i40e_tx_buffer *buf),
> +
> +	TP_ARGS(ring, desc, buf));
> +
> +DECLARE_EVENT_CLASS(
> +	i40e_rx_template,
> +
> +	TP_PROTO(struct i40e_ring *ring,
> +		?union i40e_32byte_rx_desc *desc,
> +		?struct sk_buff *skb),
> +
> +	TP_ARGS(ring, desc, skb),
> +
> +	TP_STRUCT__entry(
> +		__field(void*, ring)
> +		__field(void*, desc)
> +		__field(void*, skb)
> +		__string(devname, ring->netdev->name)
> +	),
> +
> +	TP_fast_assign(
> +		__entry->ring = ring;
> +		__entry->desc = desc;
> +		__entry->skb = skb;
> +		__assign_str(devname, ring->netdev->name);
> +	),
> +
> +	TP_printk(
> +		"netdev: %s ring: %p desc: %p skb %p",
> +		__get_str(devname), __entry->ring,
> +		__entry->desc, __entry->skb)
> +);
> +
> +DEFINE_EVENT(
> +	i40e_rx_template, i40e_clean_rx_irq,
> +	TP_PROTO(struct i40e_ring *ring,
> +		?union i40e_32byte_rx_desc *desc,
> +		?struct sk_buff *skb),
> +
> +	TP_ARGS(ring, desc, skb));
> +
> +DEFINE_EVENT(
> +	i40e_rx_template, i40e_clean_rx_irq_rx,
> +	TP_PROTO(struct i40e_ring *ring,
> +		?union i40e_32byte_rx_desc *desc,
> +		?struct sk_buff *skb),
> +
> +	TP_ARGS(ring, desc, skb));
> +
> +DECLARE_EVENT_CLASS(
> +	i40e_xmit_template,
> +
> +	TP_PROTO(struct sk_buff *skb,
> +		?struct i40e_ring *ring),
> +
> +	TP_ARGS(skb, ring),
> +
> +	TP_STRUCT__entry(
> +		__field(void*, skb)
> +		__field(void*, ring)
> +		__string(devname, ring->netdev->name)
> +	),
> +
> +	TP_fast_assign(
> +		__entry->skb = skb;
> +		__entry->ring = ring;
> +		__assign_str(devname, ring->netdev->name);
> +	),
> +
> +	TP_printk(
> +		"netdev: %s skb: %p ring: %p",
> +		__get_str(devname), __entry->skb,
> +		__entry->ring)
> +);
> +
> +DEFINE_EVENT(
> +	i40e_xmit_template, i40e_xmit_frame_ring,
> +	TP_PROTO(struct sk_buff *skb,
> +		?struct i40e_ring *ring),
> +
> +	TP_ARGS(skb, ring));
> +
> +DEFINE_EVENT(
> +	i40e_xmit_template, i40e_xmit_frame_ring_drop,
> +	TP_PROTO(struct sk_buff *skb,
> +		?struct i40e_ring *ring),
> +
> +	TP_ARGS(skb, ring));
> +
> +/* Events unique to the PF. */
> +
> +#endif /* _I40E_TRACE_H_ */
> +/* This must be outside ifdef _I40E_TRACE_H */
> +
> +/* This trace include file is not located in the .../include/trace
> + * with the kernel tracepoint definitions, because we're a loadable
> + * module.
> + */
> +#undef TRACE_INCLUDE_PATH
> +#define TRACE_INCLUDE_PATH .
> +#undef TRACE_INCLUDE_FILE
> +#define TRACE_INCLUDE_FILE i40e_trace
> +#include <trace/define_trace.h>
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> index 5f04929..3288572 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> @@ -27,6 +27,7 @@
> ?#include <linux/prefetch.h>
> ?#include <net/busy_poll.h>
> ?#include "i40e.h"
> +#include "i40e_trace.h"
> ?#include "i40e_prototype.h"
> ?
> ?static inline __le64 build_ctob(u32 td_cmd, u32 td_offset, unsigned int
> size,
> @@ -765,6 +766,7 @@ static bool i40e_clean_tx_irq(struct i40e_vsi *vsi,
> ?		/* prevent any other reads prior to eop_desc */
> ?		read_barrier_depends();
> ?
> +		i40e_trace(clean_tx_irq, tx_ring, tx_desc, tx_buf);
> ?		/* we have caught up to head, no work left to do */
> ?		if (tx_head == tx_desc)
> ?			break;
> @@ -791,6 +793,8 @@ static bool i40e_clean_tx_irq(struct i40e_vsi *vsi,
> ?
> ?		/* unmap remaining buffers */
> ?		while (tx_desc != eop_desc) {
> +			i40e_trace(clean_tx_irq_unmap,
> +				???tx_ring, tx_desc, tx_buf);
> ?
> ?			tx_buf++;
> ?			tx_desc++;
> @@ -2062,6 +2066,7 @@ static int i40e_clean_rx_irq(struct i40e_ring
> *rx_ring, int budget)
> ?		if (!size)
> ?			break;
> ?
> +		i40e_trace(clean_rx_irq, rx_ring, rx_desc, skb);
> ?		rx_buffer = i40e_get_rx_buffer(rx_ring, size);
> ?
> ?		/* retrieve a buffer from the ring */
> @@ -2114,6 +2119,7 @@ static int i40e_clean_rx_irq(struct i40e_ring
> *rx_ring, int budget)
> ?		vlan_tag = (qword & BIT(I40E_RX_DESC_STATUS_L2TAG1P_SHIFT))
> ?
> ?			???le16_to_cpu(rx_desc->wb.qword0.lo_dword.l2tag1)
> : 0;
> ?
> +		i40e_trace(clean_rx_irq_rx, rx_ring, rx_desc, skb);
> ?		i40e_receive_skb(rx_ring, skb, vlan_tag, lpbk);
> ?		skb = NULL;
> ?
> @@ -3247,6 +3253,8 @@ static netdev_tx_t i40e_xmit_frame_ring(struct sk_buff
> *skb,
> ?	/* prefetch the data, we'll need it later */
> ?	prefetch(skb->data);
> ?
> +	i40e_trace(xmit_frame_ring, skb, tx_ring);
> +
> ?	count = i40e_xmit_descriptor_count(skb);
> ?	if (i40e_chk_linearize(skb, count)) {
> ?		if (__skb_linearize(skb)) {
> @@ -3327,6 +3335,7 @@ static netdev_tx_t i40e_xmit_frame_ring(struct sk_buff
> *skb,
> ?	return NETDEV_TX_OK;
> ?
> ?out_drop:
> +	i40e_trace(xmit_frame_ring_drop, first->skb, tx_ring);
> ?	dev_kfree_skb_any(first->skb);
> ?	first->skb = NULL;
> ?	return NETDEV_TX_OK;
> diff --git a/drivers/net/ethernet/intel/i40evf/Makefile
> b/drivers/net/ethernet/intel/i40evf/Makefile
> index 827c7a6..a393f4a 100644
> --- a/drivers/net/ethernet/intel/i40evf/Makefile
> +++ b/drivers/net/ethernet/intel/i40evf/Makefile
> @@ -29,6 +29,9 @@
> ?#
> ?#
> ?
> +ccflags-y += -I$(src)
> +subdir-ccflags-y += -I$(src)
> +
> ?obj-$(CONFIG_I40EVF) += i40evf.o
> ?
> ?i40evf-objs :=	i40evf_main.o i40evf_ethtool.o i40evf_virtchnl.o \
> diff --git a/drivers/net/ethernet/intel/i40evf/i40e_trace.h
> b/drivers/net/ethernet/intel/i40evf/i40e_trace.h
> new file mode 100644
> index 0000000..9a5100b
> --- /dev/null
> +++ b/drivers/net/ethernet/intel/i40evf/i40e_trace.h
> @@ -0,0 +1,229 @@
> +/**************************************************************************
> *****
> + *
> + * Intel(R) 40-10 Gigabit Ethernet Virtual Function Driver
> + * Copyright(c) 2013 - 2017 Intel Corporation.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms and conditions of the GNU General Public License,
> + * version 2, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE.??See the GNU General Public License
> for
> + * more details.
> + *
> + * The full GNU General Public License is included in this distribution in
> + * the file called "COPYING".
> + *
> + * Contact Information:
> + * e1000-devel Mailing List <e1000-devel@lists.sourceforge.net>
> + * Intel Corporation, 5200 N.E. Elam Young Parkway, Hillsboro, OR 97124-
> 6497
> + *
> +
> ****************************************************************************
> **/
> +
> +/* Modeled on trace-events-sample.h */
> +
> +/* The trace subsystem name for i40evf will be "i40evf".
> + *
> + * This file is named i40e_trace.h.
> + *
> + * Since this include file's name is different from the trace
> + * subsystem name, we'll have to define TRACE_INCLUDE_FILE at the end
> + * of this file.
> + */
> +#undef TRACE_SYSTEM
> +#define TRACE_SYSTEM i40evf
> +
> +/* See trace-events-sample.h for a detailed description of why this
> + * guard clause is different from most normal include files.
> + */
> +#if !defined(_I40E_TRACE_H_) || defined(TRACE_HEADER_MULTI_READ)
> +#define _I40E_TRACE_H_
> +
> +#include <linux/tracepoint.h>
> +
> +/**
> + * i40e_trace() macro enables shared code to refer to trace points
> + * like:
> + *
> + * trace_i40e{,vf}_example(args...)
> + *
> + * ... as:
> + *
> + * i40e_trace(example, args...)
> + *
> + * ... to resolve to the PF or VF version of the tracepoint without
> + * ifdefs, and to allow tracepoints to be disabled entirely at build
> + * time.
> + *
> + * Trace point should always be referred to in the driver via this
> + * macro.
> + *
> + * Similarly, i40e_trace_enabled(trace_name) wraps references to
> + * trace_i40e{,vf}_<trace_name>_enabled() functions.
> + */
> +#define _I40E_TRACE_NAME(trace_name) (trace_ ## i40evf ## _ ## trace_name)
> +#define I40E_TRACE_NAME(trace_name) _I40E_TRACE_NAME(trace_name)
> +
> +#define i40e_trace(trace_name, args...) I40E_TRACE_NAME(trace_name)(args)
> +
> +#define i40e_trace_enabled(trace_name)
> I40E_TRACE_NAME(trace_name##_enabled)()
> +
> +/* Events common to PF and VF. Corresponding versions will be defined
> + * for both, named trace_i40e_* and trace_i40evf_*. The i40e_trace()
> + * macro above will select the right trace point name for the driver
> + * being built from shared code.
> + */
> +
> +/* Events related to a vsi & ring */
> +DECLARE_EVENT_CLASS(
> +	i40evf_tx_template,
> +
> +	TP_PROTO(struct i40e_ring *ring,
> +		?struct i40e_tx_desc *desc,
> +		?struct i40e_tx_buffer *buf),
> +
> +	TP_ARGS(ring, desc, buf),
> +
> +	/* The convention here is to make the first fields in the
> +	?* TP_STRUCT match the TP_PROTO exactly. This enables the use
> +	?* of the args struct generated by the tplist tool (from the
> +	?* bcc-tools package) to be used for those fields. To access
> +	?* fields other than the tracepoint args will require the
> +	?* tplist output to be adjusted.
> +	?*/
> +	TP_STRUCT__entry(
> +		__field(void*, ring)
> +		__field(void*, desc)
> +		__field(void*, buf)
> +		__string(devname, ring->netdev->name)
> +	),
> +
> +	TP_fast_assign(
> +		__entry->ring = ring;
> +		__entry->desc = desc;
> +		__entry->buf = buf;
> +		__assign_str(devname, ring->netdev->name);
> +	),
> +
> +	TP_printk(
> +		"netdev: %s ring: %p desc: %p buf %p",
> +		__get_str(devname), __entry->ring,
> +		__entry->desc, __entry->buf)
> +);
> +
> +DEFINE_EVENT(
> +	i40evf_tx_template, i40evf_clean_tx_irq,
> +	TP_PROTO(struct i40e_ring *ring,
> +		?struct i40e_tx_desc *desc,
> +		?struct i40e_tx_buffer *buf),
> +
> +	TP_ARGS(ring, desc, buf));
> +
> +DEFINE_EVENT(
> +	i40evf_tx_template, i40evf_clean_tx_irq_unmap,
> +	TP_PROTO(struct i40e_ring *ring,
> +		?struct i40e_tx_desc *desc,
> +		?struct i40e_tx_buffer *buf),
> +
> +	TP_ARGS(ring, desc, buf));
> +
> +DECLARE_EVENT_CLASS(
> +	i40evf_rx_template,
> +
> +	TP_PROTO(struct i40e_ring *ring,
> +		?union i40e_32byte_rx_desc *desc,
> +		?struct sk_buff *skb),
> +
> +	TP_ARGS(ring, desc, skb),
> +
> +	TP_STRUCT__entry(
> +		__field(void*, ring)
> +		__field(void*, desc)
> +		__field(void*, skb)
> +		__string(devname, ring->netdev->name)
> +	),
> +
> +	TP_fast_assign(
> +		__entry->ring = ring;
> +		__entry->desc = desc;
> +		__entry->skb = skb;
> +		__assign_str(devname, ring->netdev->name);
> +	),
> +
> +	TP_printk(
> +		"netdev: %s ring: %p desc: %p skb %p",
> +		__get_str(devname), __entry->ring,
> +		__entry->desc, __entry->skb)
> +);
> +
> +DEFINE_EVENT(
> +	i40evf_rx_template, i40evf_clean_rx_irq,
> +	TP_PROTO(struct i40e_ring *ring,
> +		?union i40e_32byte_rx_desc *desc,
> +		?struct sk_buff *skb),
> +
> +	TP_ARGS(ring, desc, skb));
> +
> +DEFINE_EVENT(
> +	i40evf_rx_template, i40evf_clean_rx_irq_rx,
> +	TP_PROTO(struct i40e_ring *ring,
> +		?union i40e_32byte_rx_desc *desc,
> +		?struct sk_buff *skb),
> +
> +	TP_ARGS(ring, desc, skb));
> +
> +DECLARE_EVENT_CLASS(
> +	i40evf_xmit_template,
> +
> +	TP_PROTO(struct sk_buff *skb,
> +		?struct i40e_ring *ring),
> +
> +	TP_ARGS(skb, ring),
> +
> +	TP_STRUCT__entry(
> +		__field(void*, skb)
> +		__field(void*, ring)
> +		__string(devname, ring->netdev->name)
> +	),
> +
> +	TP_fast_assign(
> +		__entry->skb = skb;
> +		__entry->ring = ring;
> +		__assign_str(devname, ring->netdev->name);
> +	),
> +
> +	TP_printk(
> +		"netdev: %s skb: %p ring: %p",
> +		__get_str(devname), __entry->skb,
> +		__entry->ring)
> +);
> +
> +DEFINE_EVENT(
> +	i40evf_xmit_template, i40evf_xmit_frame_ring,
> +	TP_PROTO(struct sk_buff *skb,
> +		?struct i40e_ring *ring),
> +
> +	TP_ARGS(skb, ring));
> +
> +DEFINE_EVENT(
> +	i40evf_xmit_template, i40evf_xmit_frame_ring_drop,
> +	TP_PROTO(struct sk_buff *skb,
> +		?struct i40e_ring *ring),
> +
> +	TP_ARGS(skb, ring));
> +
> +/* Events unique to the VF. */
> +
> +#endif /* _I40E_TRACE_H_ */
> +/* This must be outside ifdef _I40E_TRACE_H */
> +
> +/* This trace include file is not located in the .../include/trace
> + * with the kernel tracepoint definitions, because we're a loadable
> + * module.
> + */
> +#undef TRACE_INCLUDE_PATH
> +#define TRACE_INCLUDE_PATH .
> +#undef TRACE_INCLUDE_FILE
> +#define TRACE_INCLUDE_FILE i40e_trace
> +#include <trace/define_trace.h>
> diff --git a/drivers/net/ethernet/intel/i40evf/i40e_txrx.c
> b/drivers/net/ethernet/intel/i40evf/i40e_txrx.c
> index 80931e3..34e96d9 100644
> --- a/drivers/net/ethernet/intel/i40evf/i40e_txrx.c
> +++ b/drivers/net/ethernet/intel/i40evf/i40e_txrx.c
> @@ -28,6 +28,7 @@
> ?#include <net/busy_poll.h>
> ?
> ?#include "i40evf.h"
> +#include "i40e_trace.h"
> ?#include "i40e_prototype.h"
> ?
> ?static inline __le64 build_ctob(u32 td_cmd, u32 td_offset, unsigned int
> size,
> @@ -180,6 +181,7 @@ static bool i40e_clean_tx_irq(struct i40e_vsi *vsi,
> ?		/* prevent any other reads prior to eop_desc */
> ?		read_barrier_depends();
> ?
> +		i40e_trace(clean_tx_irq, tx_ring, tx_desc, tx_buf);
> ?		/* if the descriptor isn't done, no work yet to do */
> ?		if (!(eop_desc->cmd_type_offset_bsz &
> ?		??????cpu_to_le64(I40E_TX_DESC_DTYPE_DESC_DONE)))
> @@ -207,6 +209,8 @@ static bool i40e_clean_tx_irq(struct i40e_vsi *vsi,
> ?
> ?		/* unmap remaining buffers */
> ?		while (tx_desc != eop_desc) {
> +			i40e_trace(clean_tx_irq_unmap,
> +				???tx_ring, tx_desc, tx_buf);
> ?
> ?			tx_buf++;
> ?			tx_desc++;
> @@ -1329,6 +1333,7 @@ static int i40e_clean_rx_irq(struct i40e_ring
> *rx_ring, int budget)
> ?		if (!size)
> ?			break;
> ?
> +		i40e_trace(clean_rx_irq, rx_ring, rx_desc, skb);
> ?		rx_buffer = i40e_get_rx_buffer(rx_ring, size);
> ?
> ?		/* retrieve a buffer from the ring */
> @@ -1382,6 +1387,7 @@ static int i40e_clean_rx_irq(struct i40e_ring
> *rx_ring, int budget)
> ?		vlan_tag = (qword & BIT(I40E_RX_DESC_STATUS_L2TAG1P_SHIFT))
> ?
> ?			???le16_to_cpu(rx_desc->wb.qword0.lo_dword.l2tag1)
> : 0;
> ?
> +		i40e_trace(clean_rx_irq_rx, rx_ring, rx_desc, skb);
> ?		i40e_receive_skb(rx_ring, skb, vlan_tag);
> ?		skb = NULL;
> ?
> @@ -2223,6 +2229,8 @@ static netdev_tx_t i40e_xmit_frame_ring(struct sk_buff
> *skb,
> ?	/* prefetch the data, we'll need it later */
> ?	prefetch(skb->data);
> ?
> +	i40e_trace(xmit_frame_ring, skb, tx_ring);
> +
> ?	count = i40e_xmit_descriptor_count(skb);
> ?	if (i40e_chk_linearize(skb, count)) {
> ?		if (__skb_linearize(skb)) {
> @@ -2290,6 +2298,7 @@ static netdev_tx_t i40e_xmit_frame_ring(struct sk_buff
> *skb,
> ?	return NETDEV_TX_OK;
> ?
> ?out_drop:
> +	i40e_trace(xmit_frame_ring_drop, first->skb, tx_ring);
> ?	dev_kfree_skb_any(first->skb);
> ?	first->skb = NULL;
> ?	return NETDEV_TX_OK;
> diff --git a/drivers/net/ethernet/intel/i40evf/i40evf_main.c
> b/drivers/net/ethernet/intel/i40evf/i40evf_main.c
> index 671913c..5915273 100644
> --- a/drivers/net/ethernet/intel/i40evf/i40evf_main.c
> +++ b/drivers/net/ethernet/intel/i40evf/i40evf_main.c
> @@ -27,6 +27,13 @@
> ?#include "i40evf.h"
> ?#include "i40e_prototype.h"
> ?#include "i40evf_client.h"
> +/* All i40evf tracepoints are defined by the include below, which must
> + * be included exactly once across the whole kernel with
> + * CREATE_TRACE_POINTS defined
> + */
> +#define CREATE_TRACE_POINTS
> +#include "i40e_trace.h"
> +
> ?static int i40evf_setup_all_tx_resources(struct i40evf_adapter *adapter);
> ?static int i40evf_setup_all_rx_resources(struct i40evf_adapter *adapter);
> ?static int i40evf_close(struct net_device *netdev);

[Intel-wired-lan] [next PATCH S70 06/12] i40e: reduce wait time for adminq command completion

[Shannon Nelson]

On 4/13/2017 1:45 AM, Alice Michael wrote:
> From: Jacob Keller <jacob.e.keller@intel.com>
>
> When sending an adminq command, we wait for the command to complete in
> a loop. This loop waits for an entire millisecond, when in practice the
> adminq command is processed often much faster.
>
> Change the loop to use i40e_usec_delay instead, and wait for 50 usecs
> each time instead. This appears to be about the minimum time required,
> based on some manual observation and testing.
>
> The primary benefit of this change is reducing latency of various
> operations in the PF driver, especially when related to having a large
> number of VFs enabled.
>
> For example, on Linux, when instantiating 128 VFs, the time to finish
> the operation dropped from about 9 seconds down to under 6 seconds.
> Additionally, the time it takes to finish a PF reset with 128 VFs
> dropped from 5.1 seconds down to 0.7 seconds.
>
> As the examples above show, a significant portion of the delay is wasted
> waiting for admiqn operations which have already finished.
>
> This patch shouldn't cause impact to functionality, as we still check
> and keep waiting until the command does get processed. The only expected
> change is an increase in CPU utilization as we now check for completion
> far more times. However, in practice the commands appear to generally be
> complete within the first delay window anyways.
>
> Previously for NDIS we used i40e_msec_stall since i40e_msec_delay would
> cause the CPU to be yielded. We can now just use i40e_usec_delay instead
> because this function will behave correctly for both NDIS and non-NDIS
> code. Thus, we can simply drop the #ifdef logic entirely.

There should be no need to discuss NDIS here.
sln


>
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Change-ID: If8af8388e100da0a14eaf9e1af3afadf73a958cf
> ---
>  drivers/net/ethernet/intel/i40e/i40e_adminq.c   | 4 ++--
>  drivers/net/ethernet/intel/i40e/i40e_adminq.h   | 2 +-
>  drivers/net/ethernet/intel/i40evf/i40e_adminq.c | 4 ++--
>  drivers/net/ethernet/intel/i40evf/i40e_adminq.h | 2 +-
>  4 files changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_adminq.c b/drivers/net/ethernet/intel/i40e/i40e_adminq.c
> index 56fb272..ba04988 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_adminq.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_adminq.c
> @@ -850,8 +850,8 @@ i40e_status i40e_asq_send_command(struct i40e_hw *hw,
>  			 */
>  			if (i40e_asq_done(hw))
>  				break;
> -			usleep_range(1000, 2000);
> -			total_delay++;
> +			udelay(50);
> +			total_delay += 50;
>  		} while (total_delay < hw->aq.asq_cmd_timeout);
>  	}
>
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_adminq.h b/drivers/net/ethernet/intel/i40e/i40e_adminq.h
> index d92aad3..2349fbe 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_adminq.h
> +++ b/drivers/net/ethernet/intel/i40e/i40e_adminq.h
> @@ -151,7 +151,7 @@ static inline int i40e_aq_rc_to_posix(int aq_ret, int aq_rc)
>
>  /* general information */
>  #define I40E_AQ_LARGE_BUF	512
> -#define I40E_ASQ_CMD_TIMEOUT	250  /* msecs */
> +#define I40E_ASQ_CMD_TIMEOUT	250000  /* usecs */
>
>  void i40e_fill_default_direct_cmd_desc(struct i40e_aq_desc *desc,
>  				       u16 opcode);
> diff --git a/drivers/net/ethernet/intel/i40evf/i40e_adminq.c b/drivers/net/ethernet/intel/i40evf/i40e_adminq.c
> index 9638515..8b0d4b2 100644
> --- a/drivers/net/ethernet/intel/i40evf/i40e_adminq.c
> +++ b/drivers/net/ethernet/intel/i40evf/i40e_adminq.c
> @@ -797,8 +797,8 @@ i40e_status i40evf_asq_send_command(struct i40e_hw *hw,
>  			 */
>  			if (i40evf_asq_done(hw))
>  				break;
> -			usleep_range(1000, 2000);
> -			total_delay++;
> +			udelay(50);
> +			total_delay += 50;
>  		} while (total_delay < hw->aq.asq_cmd_timeout);
>  	}
>
> diff --git a/drivers/net/ethernet/intel/i40evf/i40e_adminq.h b/drivers/net/ethernet/intel/i40evf/i40e_adminq.h
> index 1f9b3b5..e0bfaa3 100644
> --- a/drivers/net/ethernet/intel/i40evf/i40e_adminq.h
> +++ b/drivers/net/ethernet/intel/i40evf/i40e_adminq.h
> @@ -151,7 +151,7 @@ static inline int i40e_aq_rc_to_posix(int aq_ret, int aq_rc)
>
>  /* general information */
>  #define I40E_AQ_LARGE_BUF	512
> -#define I40E_ASQ_CMD_TIMEOUT	250  /* msecs */
> +#define I40E_ASQ_CMD_TIMEOUT	250000  /* usecs */
>
>  void i40evf_fill_default_direct_cmd_desc(struct i40e_aq_desc *desc,
>  				       u16 opcode);
>

[Intel-wired-lan] [next PATCH S70 02/12] i40e: DCR 287 new AQ commands

[Shannon Nelson]

On 4/13/2017 1:45 AM, Alice Michael wrote:
> From: Jingjing Wu <jingjing.wu@intel.com>
>

The DCR is an internal numbering and means nothing to the rest of the 
world in the Subject line or the commit message.

More information would be good to describe this new feature - what is 
it, how does the typical user get to it.

> Add admin queue functions for Pipeline Personalization Profile AQ
> commands defined in DCR 287:
>  - Write Recipe Command buffer (Opcode: 0x0270)
>  - Get Applied Profiles list (Opcode: 0x0271)
>
> Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
> Change-ID: I558b4145364140f624013af48d4bbf79d21ebb0d
> ---
>  drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h  |  34 ++++
>  drivers/net/ethernet/intel/i40e/i40e_common.c      | 212 +++++++++++++++++++++
>  drivers/net/ethernet/intel/i40e/i40e_prototype.h   |  17 ++
>  drivers/net/ethernet/intel/i40e/i40e_type.h        |  80 ++++++++
>  .../net/ethernet/intel/i40evf/i40e_adminq_cmd.h    |  34 ++++
>  drivers/net/ethernet/intel/i40evf/i40e_common.c    | 212 +++++++++++++++++++++
>  drivers/net/ethernet/intel/i40evf/i40e_prototype.h |  17 ++
>  drivers/net/ethernet/intel/i40evf/i40e_type.h      |  80 ++++++++
>  8 files changed, 686 insertions(+)
>
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h b/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h
> index 251074c..5eb0411 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h
> +++ b/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h
> @@ -190,6 +190,10 @@ enum i40e_admin_queue_opc {
>  	i40e_aqc_opc_add_mirror_rule	= 0x0260,
>  	i40e_aqc_opc_delete_mirror_rule	= 0x0261,
>
> +	/* Pipeline Personalization Profile */
> +	i40e_aqc_opc_write_personalization_profile	= 0x0270,
> +	i40e_aqc_opc_get_personalization_profile_list	= 0x0271,
> +

No version bump for the new API?

>  	/* DCB commands */
>  	i40e_aqc_opc_dcb_ignore_pfc	= 0x0301,
>  	i40e_aqc_opc_dcb_updated	= 0x0302,
> @@ -1431,6 +1435,36 @@ struct i40e_aqc_add_delete_mirror_rule_completion {
>
>  I40E_CHECK_CMD_LENGTH(i40e_aqc_add_delete_mirror_rule_completion);
>
> +/* Pipeline Personalization Profile */
> +struct i40e_aqc_write_personalization_profile {
> +	u8      flags;
> +	u8      reserved[3];
> +	__le32  profile_track_id;
> +	__le32  addr_high;
> +	__le32  addr_low;
> +};
> +
> +I40E_CHECK_CMD_LENGTH(i40e_aqc_write_personalization_profile);
> +
> +struct i40e_aqc_write_ppp_resp {
> +	__le32 error_offset;
> +	__le32 error_info;
> +	__le32 addr_high;
> +	__le32 addr_low;
> +};
> +
> +struct i40e_aqc_get_applied_profiles {
> +	u8      flags;
> +#define I40E_AQC_GET_PPP_GET_CONF	0x1
> +#define I40E_AQC_GET_PPP_GET_RDPU_CONF	0x2
> +	u8      rsv[3];
> +	__le32  reserved;
> +	__le32  addr_high;
> +	__le32  addr_low;
> +};
> +
> +I40E_CHECK_CMD_LENGTH(i40e_aqc_get_applied_profiles);
> +
>  /* DCB 0x03xx*/
>
>  /* PFC Ignore (direct 0x0301)
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_common.c b/drivers/net/ethernet/intel/i40e/i40e_common.c
> index f9db95a..72d1793 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_common.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_common.c
> @@ -5042,3 +5042,215 @@ void i40e_write_rx_ctl(struct i40e_hw *hw, u32 reg_addr, u32 reg_val)
>  	if (status || use_register)
>  		wr32(hw, reg_addr, reg_val);
>  }
> +
> +/**
> + * i40e_aq_write_ppp - Write pipeline personalization profile (ppp)
> + * @hw: pointer to the hw struct
> + * @buff: command buffer (size in bytes = buff_size)
> + * @buff_size: buffer size in bytes
> + * @track_id: package tracking id
> + * @error_offset: returns error offset
> + * @error_info: returns error information
> + * @cmd_details: pointer to command details structure or NULL
> + **/
> +enum
> +i40e_status_code i40e_aq_write_ppp(struct i40e_hw *hw, void *buff,
> +				   u16 buff_size, u32 track_id,
> +				   u32 *error_offset, u32 *error_info,
> +				   struct i40e_asq_cmd_details *cmd_details)
> +{
> +	struct i40e_aq_desc desc;
> +	struct i40e_aqc_write_personalization_profile *cmd =
> +		(struct i40e_aqc_write_personalization_profile *)
> +		&desc.params.raw;
> +	struct i40e_aqc_write_ppp_resp *resp;
> +	i40e_status status;
> +
> +	i40e_fill_default_direct_cmd_desc(&desc,
> +					  i40e_aqc_opc_write_personalization_profile);
> +
> +	desc.flags |= cpu_to_le16(I40E_AQ_FLAG_BUF | I40E_AQ_FLAG_RD);
> +	if (buff_size > I40E_AQ_LARGE_BUF)
> +		desc.flags |= cpu_to_le16((u16)I40E_AQ_FLAG_LB);
> +
> +	desc.datalen = cpu_to_le16(buff_size);
> +
> +	cmd->profile_track_id = cpu_to_le32(track_id);
> +
> +	status = i40e_asq_send_command(hw, &desc, buff, buff_size, cmd_details);
> +	if (!status) {
> +		resp = (struct i40e_aqc_write_ppp_resp *)&desc.params.raw;
> +		if (error_offset)
> +			*error_offset = le32_to_cpu(resp->error_offset);
> +		if (error_info)
> +			*error_info = le32_to_cpu(resp->error_info);
> +	}
> +
> +	return status;
> +}
> +
> +/**
> + * i40e_aq_get_ppp_list - Read pipeline personalization profile (ppp)
> + * @hw: pointer to the hw struct
> + * @buff: command buffer (size in bytes = buff_size)
> + * @buff_size: buffer size in bytes
> + * @cmd_details: pointer to command details structure or NULL
> + **/
> +enum
> +i40e_status_code i40e_aq_get_ppp_list(struct i40e_hw *hw, void *buff,
> +				      u16 buff_size, u8 flags,
> +				       struct i40e_asq_cmd_details *cmd_details)
> +{
> +	struct i40e_aq_desc desc;
> +	struct i40e_aqc_get_applied_profiles *cmd =
> +		(struct i40e_aqc_get_applied_profiles *)&desc.params.raw;
> +	i40e_status status;
> +
> +	i40e_fill_default_direct_cmd_desc(&desc,
> +					  i40e_aqc_opc_get_personalization_profile_list);
> +
> +	desc.flags |= cpu_to_le16((u16)I40E_AQ_FLAG_BUF);
> +	if (buff_size > I40E_AQ_LARGE_BUF)
> +		desc.flags |= cpu_to_le16((u16)I40E_AQ_FLAG_LB);
> +	desc.datalen = cpu_to_le16(buff_size);
> +
> +	cmd->flags = flags;
> +
> +	status = i40e_asq_send_command(hw, &desc, buff, buff_size, cmd_details);
> +
> +	return status;
> +}
> +
> +/**
> + * i40e_find_segment_in_package
> + * @segment_type: the segment type to search for (i.e., SEGMENT_TYPE_I40E)
> + * @pkg_hdr: pointer to the package header to be searched
> + *
> + * This function searches a package file for a particular segment type. On
> + * success it returns a pointer to the segment header, otherwise it will
> + * return NULL.
> + **/
> +struct i40e_generic_seg_header *
> +i40e_find_segment_in_package(u32 segment_type,
> +			     struct i40e_package_header *pkg_hdr)
> +{
> +	struct i40e_generic_seg_header *segment;
> +	u32 i;
> +
> +	/* Search all package segments for the requested segment type */
> +	for (i = 0; i < pkg_hdr->segment_count; i++) {
> +		segment =
> +			(struct i40e_generic_seg_header *)((u8 *)pkg_hdr +
> +			 pkg_hdr->segment_offset[i]);
> +
> +		if (segment->type == segment_type)
> +			return segment;
> +	}
> +
> +	return NULL;
> +}
> +
> +/**
> + * i40e_write_profile
> + * @hw: pointer to the hardware structure
> + * @profile: pointer to the profile segment of the package to be downloaded
> + * @track_id: package tracking id
> + *
> + * Handles the download of a complete package.
> + */
> +enum i40e_status_code
> +i40e_write_profile(struct i40e_hw *hw, struct i40e_profile_segment *profile,
> +		   u32 track_id)
> +{
> +	i40e_status status = 0;
> +	struct i40e_section_table *sec_tbl;
> +	struct i40e_profile_section_header *sec = NULL;
> +	u32 dev_cnt;
> +	u32 vendor_dev_id;
> +	u32 *nvm;
> +	u32 section_size = 0;
> +	u32 offset = 0, info = 0;
> +	u32 i;
> +
> +	if (!track_id) {
> +		i40e_debug(hw, I40E_DEBUG_PACKAGE, "Track_id can't be 0.");

I believe there should be newlines in all the i40e_debug messages.

> +		return I40E_NOT_SUPPORTED;
> +	}
> +
> +	dev_cnt = profile->device_table_count;
> +
> +	for (i = 0; i < dev_cnt; i++) {
> +		vendor_dev_id = profile->device_table[i].vendor_dev_id;
> +		if ((vendor_dev_id >> 16) == PCI_VENDOR_ID_INTEL)
> +			if (hw->device_id == (vendor_dev_id & 0xFFFF))
> +				break;
> +	}
> +	if (i == dev_cnt) {
> +		i40e_debug(hw, I40E_DEBUG_PACKAGE, "Device doesn't support PPP");
> +		return I40E_ERR_DEVICE_NOT_SUPPORTED;
> +	}
> +
> +	nvm = (u32 *)&profile->device_table[dev_cnt];
> +	sec_tbl = (struct i40e_section_table *)&nvm[nvm[0] + 1];
> +
> +	for (i = 0; i < sec_tbl->section_count; i++) {
> +		sec = (struct i40e_profile_section_header *)((u8 *)profile +
> +					     sec_tbl->section_offset[i]);
> +
> +		/* Skip 'AQ', 'note' and 'name' sections */
> +		if (sec->section.type != SECTION_TYPE_MMIO)
> +			continue;
> +
> +		section_size = sec->section.size +
> +			sizeof(struct i40e_profile_section_header);
> +
> +		/* Write profile */
> +		status = i40e_aq_write_ppp(hw, (void *)sec, (u16)section_size,
> +					   track_id, &offset, &info, NULL);
> +		if (status) {
> +			i40e_debug(hw, I40E_DEBUG_PACKAGE,
> +				   "Failed to write profile: offset %d, info %d",
> +				   offset, info);
> +			break;
> +		}
> +	}
> +	return status;
> +}
> +
> +/**
> + * i40e_add_pinfo_to_list
> + * @hw: pointer to the hardware structure
> + * @profile: pointer to the profile segment of the package
> + * @profile_info_sec: buffer for information section
> + * @track_id: package tracking id
> + *
> + * Register a profile to the list of loaded profiles.
> + */
> +enum i40e_status_code
> +i40e_add_pinfo_to_list(struct i40e_hw *hw,
> +		       struct i40e_profile_segment *profile,
> +		       u8 *profile_info_sec, u32 track_id)
> +{
> +	i40e_status status = 0;
> +	struct i40e_profile_section_header *sec = NULL;
> +	struct i40e_profile_info *pinfo;
> +	u32 offset = 0, info = 0;
> +
> +	sec = (struct i40e_profile_section_header *)profile_info_sec;
> +	sec->tbl_size = 1;
> +	sec->data_end = sizeof(struct i40e_profile_section_header) +
> +			sizeof(struct i40e_profile_info);
> +	sec->section.type = SECTION_TYPE_INFO;
> +	sec->section.offset = sizeof(struct i40e_profile_section_header);
> +	sec->section.size = sizeof(struct i40e_profile_info);
> +	pinfo = (struct i40e_profile_info *)(profile_info_sec +
> +					     sec->section.offset);
> +	pinfo->track_id = track_id;
> +	pinfo->version = profile->version;
> +	pinfo->op = I40E_PPP_ADD_TRACKID;
> +	memcpy(pinfo->name, profile->name, I40E_PPP_NAME_SIZE);
> +
> +	status = i40e_aq_write_ppp(hw, (void *)sec, sec->data_end,
> +				   track_id, &offset, &info, NULL);
> +	return status;
> +}
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_prototype.h b/drivers/net/ethernet/intel/i40e/i40e_prototype.h
> index dfc5e59..611bbc7 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_prototype.h
> +++ b/drivers/net/ethernet/intel/i40e/i40e_prototype.h
> @@ -377,4 +377,21 @@ i40e_status i40e_write_phy_register(struct i40e_hw *hw, u8 page, u16 reg,
>  u8 i40e_get_phy_address(struct i40e_hw *hw, u8 dev_num);
>  i40e_status i40e_blink_phy_link_led(struct i40e_hw *hw,
>  				    u32 time, u32 interval);
> +i40e_status i40e_aq_write_ppp(struct i40e_hw *hw, void *buff,
> +			      u16 buff_size, u32 track_id,
> +				u32 *error_offset, u32 *error_info,
> +				struct i40e_asq_cmd_details *cmd_details);
> +i40e_status i40e_aq_get_ppp_list(struct i40e_hw *hw, void *buff,
> +				 u16 buff_size, u8 flags,
> +				   struct i40e_asq_cmd_details *cmd_details);
> +struct i40e_generic_seg_header *
> +i40e_find_segment_in_package(u32 segment_type,
> +			     struct i40e_package_header *pkg_header);
> +enum i40e_status_code
> +i40e_write_profile(struct i40e_hw *hw, struct i40e_profile_segment *i40e_seg,
> +		   u32 track_id);
> +enum i40e_status_code
> +i40e_add_pinfo_to_list(struct i40e_hw *hw,
> +		       struct i40e_profile_segment *profile,
> +		       u8 *profile_info_sec, u32 track_id);
>  #endif /* _I40E_PROTOTYPE_H_ */
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_type.h b/drivers/net/ethernet/intel/i40e/i40e_type.h
> index 08364a4..f235899 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_type.h
> +++ b/drivers/net/ethernet/intel/i40e/i40e_type.h
> @@ -78,6 +78,7 @@ enum i40e_debug_mask {
>  	I40E_DEBUG_DCB			= 0x00000400,
>  	I40E_DEBUG_DIAG			= 0x00000800,
>  	I40E_DEBUG_FD			= 0x00001000,
> +	I40E_DEBUG_PACKAGE		= 0x00002000,
>  	I40E_DEBUG_IWARP		= 0x00F00000,
>  	I40E_DEBUG_AQ_MESSAGE		= 0x01000000,
>  	I40E_DEBUG_AQ_DESCRIPTOR	= 0x02000000,
> @@ -1465,4 +1466,83 @@ struct i40e_lldp_variables {
>  #define I40E_FLEX_56_MASK		(0x1ULL << I40E_FLEX_56_SHIFT)
>  #define I40E_FLEX_57_SHIFT		6
>  #define I40E_FLEX_57_MASK		(0x1ULL << I40E_FLEX_57_SHIFT)
> +
> +/* Version format for PPP */
> +struct i40e_ppp_version {
> +	u8 major;
> +	u8 minor;
> +	u8 update;
> +	u8 draft;
> +};
> +
> +#define I40E_PPP_NAME_SIZE	32
> +
> +/* Package header */
> +struct i40e_package_header {
> +	struct i40e_ppp_version version;
> +	u32 segment_count;
> +	u32 segment_offset[1];
> +};
> +
> +/* Generic segment header */
> +struct i40e_generic_seg_header {
> +#define SEGMENT_TYPE_METADATA	0x00000001
> +#define SEGMENT_TYPE_NOTES	0x00000002
> +#define SEGMENT_TYPE_I40E	0x00000011
> +#define SEGMENT_TYPE_X722	0x00000012
> +	u32 type;
> +	struct i40e_ppp_version version;
> +	u32 size;
> +	char name[I40E_PPP_NAME_SIZE];
> +};
> +
> +struct i40e_metadata_segment {
> +	struct i40e_generic_seg_header header;
> +	struct i40e_ppp_version version;
> +	u32 track_id;
> +	char     name[I40E_PPP_NAME_SIZE];
> +};
> +
> +struct i40e_device_id_entry {
> +	u32 vendor_dev_id;
> +	u32 sub_vendor_dev_id;
> +};
> +
> +struct i40e_profile_segment {
> +	struct i40e_generic_seg_header header;
> +	struct i40e_ppp_version version;
> +	char name[I40E_PPP_NAME_SIZE];
> +	u32 device_table_count;
> +	struct i40e_device_id_entry device_table[1];
> +};
> +
> +struct i40e_section_table {
> +	u32 section_count;
> +	u32 section_offset[1];
> +};
> +
> +struct i40e_profile_section_header {
> +	u16 tbl_size;
> +	u16 data_end;
> +	struct {
> +#define SECTION_TYPE_INFO	0x00000010
> +#define SECTION_TYPE_MMIO	0x00000800
> +#define SECTION_TYPE_AQ		0x00000801
> +#define SECTION_TYPE_NOTE	0x80000000
> +#define SECTION_TYPE_NAME	0x80000001
> +		u32 type;
> +		u32 offset;
> +		u32 size;
> +	} section;
> +};
> +
> +struct i40e_profile_info {
> +	u32 track_id;
> +	struct i40e_ppp_version version;
> +	u8 op;
> +#define I40E_PPP_ADD_TRACKID		0x01
> +#define I40E_PPP_REMOVE_TRACKID	0x02
> +	u8 reserved[7];
> +	u8 name[I40E_PPP_NAME_SIZE];
> +};
>  #endif /* _I40E_TYPE_H_ */
> diff --git a/drivers/net/ethernet/intel/i40evf/i40e_adminq_cmd.h b/drivers/net/ethernet/intel/i40evf/i40e_adminq_cmd.h
> index c28cb8f..91d8786 100644
> --- a/drivers/net/ethernet/intel/i40evf/i40e_adminq_cmd.h
> +++ b/drivers/net/ethernet/intel/i40evf/i40e_adminq_cmd.h
> @@ -190,6 +190,10 @@ enum i40e_admin_queue_opc {
>  	i40e_aqc_opc_add_mirror_rule	= 0x0260,
>  	i40e_aqc_opc_delete_mirror_rule	= 0x0261,
>
> +	/* Pipeline Personalization Profile */
> +	i40e_aqc_opc_write_personalization_profile	= 0x0270,
> +	i40e_aqc_opc_get_personalization_profile_list	= 0x0271,
> +
>  	/* DCB commands */
>  	i40e_aqc_opc_dcb_ignore_pfc	= 0x0301,
>  	i40e_aqc_opc_dcb_updated	= 0x0302,
> @@ -1426,6 +1430,36 @@ struct i40e_aqc_add_delete_mirror_rule_completion {
>
>  I40E_CHECK_CMD_LENGTH(i40e_aqc_add_delete_mirror_rule_completion);
>
> +/* Pipeline Personalization Profile */
> +struct i40e_aqc_write_personalization_profile {
> +	u8      flags;
> +	u8      reserved[3];
> +	__le32  profile_track_id;
> +	__le32  addr_high;
> +	__le32  addr_low;
> +};
> +
> +I40E_CHECK_CMD_LENGTH(i40e_aqc_write_personalization_profile);
> +
> +struct i40e_aqc_write_ppp_resp {
> +	__le32 error_offset;
> +	__le32 error_info;
> +	__le32 addr_high;
> +	__le32 addr_low;
> +};
> +
> +struct i40e_aqc_get_applied_profiles {
> +	u8      flags;
> +#define I40E_AQC_GET_PPP_GET_CONF	0x1
> +#define I40E_AQC_GET_PPP_GET_RDPU_CONF	0x2
> +	u8      rsv[3];
> +	__le32  reserved;
> +	__le32  addr_high;
> +	__le32  addr_low;
> +};
> +
> +I40E_CHECK_CMD_LENGTH(i40e_aqc_get_applied_profiles);
> +
>  /* DCB 0x03xx*/
>
>  /* PFC Ignore (direct 0x0301)
> diff --git a/drivers/net/ethernet/intel/i40evf/i40e_common.c b/drivers/net/ethernet/intel/i40evf/i40e_common.c
> index 626fbf1..606f5f1 100644
> --- a/drivers/net/ethernet/intel/i40evf/i40e_common.c
> +++ b/drivers/net/ethernet/intel/i40evf/i40e_common.c
> @@ -1131,3 +1131,215 @@ i40e_status i40e_vf_reset(struct i40e_hw *hw)
>  	return i40e_aq_send_msg_to_pf(hw, I40E_VIRTCHNL_OP_RESET_VF,
>  				      0, NULL, 0, NULL);
>  }
> +
> +/**
> + * i40evf_aq_write_ppp - Write pipeline personalization profile (ppp)
> + * @hw: pointer to the hw struct
> + * @buff: command buffer (size in bytes = buff_size)
> + * @buff_size: buffer size in bytes
> + * @track_id: package tracking id
> + * @error_offset: returns error offset
> + * @error_info: returns error information
> + * @cmd_details: pointer to command details structure or NULL
> + **/
> +enum
> +i40e_status_code i40evf_aq_write_ppp(struct i40e_hw *hw, void *buff,
> +				     u16 buff_size, u32 track_id,
> +				   u32 *error_offset, u32 *error_info,
> +				   struct i40e_asq_cmd_details *cmd_details)
> +{
> +	struct i40e_aq_desc desc;
> +	struct i40e_aqc_write_personalization_profile *cmd =
> +		(struct i40e_aqc_write_personalization_profile *)
> +		&desc.params.raw;
> +	struct i40e_aqc_write_ppp_resp *resp;
> +	i40e_status status;
> +
> +	i40evf_fill_default_direct_cmd_desc(&desc,
> +					    i40e_aqc_opc_write_personalization_profile);
> +
> +	desc.flags |= cpu_to_le16(I40E_AQ_FLAG_BUF | I40E_AQ_FLAG_RD);
> +	if (buff_size > I40E_AQ_LARGE_BUF)
> +		desc.flags |= cpu_to_le16((u16)I40E_AQ_FLAG_LB);
> +
> +	desc.datalen = cpu_to_le16(buff_size);
> +
> +	cmd->profile_track_id = cpu_to_le32(track_id);
> +
> +	status = i40evf_asq_send_command(hw, &desc, buff, buff_size, cmd_details);
> +	if (!status) {
> +		resp = (struct i40e_aqc_write_ppp_resp *)&desc.params.raw;
> +		if (error_offset)
> +			*error_offset = le32_to_cpu(resp->error_offset);
> +		if (error_info)
> +			*error_info = le32_to_cpu(resp->error_info);
> +	}
> +
> +	return status;
> +}
> +
> +/**
> + * i40evf_aq_get_ppp_list - Read pipeline personalization profile (ppp)
> + * @hw: pointer to the hw struct
> + * @buff: command buffer (size in bytes = buff_size)
> + * @buff_size: buffer size in bytes
> + * @cmd_details: pointer to command details structure or NULL
> + **/
> +enum
> +i40e_status_code i40evf_aq_get_ppp_list(struct i40e_hw *hw, void *buff,
> +					u16 buff_size, u8 flags,
> +				      struct i40e_asq_cmd_details *cmd_details)
> +{
> +	struct i40e_aq_desc desc;
> +	struct i40e_aqc_get_applied_profiles *cmd =
> +		(struct i40e_aqc_get_applied_profiles *)&desc.params.raw;
> +	i40e_status status;
> +
> +	i40evf_fill_default_direct_cmd_desc(&desc,
> +					    i40e_aqc_opc_get_personalization_profile_list);
> +
> +	desc.flags |= cpu_to_le16((u16)I40E_AQ_FLAG_BUF);
> +	if (buff_size > I40E_AQ_LARGE_BUF)
> +		desc.flags |= cpu_to_le16((u16)I40E_AQ_FLAG_LB);
> +	desc.datalen = cpu_to_le16(buff_size);
> +
> +	cmd->flags = flags;
> +
> +	status = i40evf_asq_send_command(hw, &desc, buff, buff_size, cmd_details);
> +
> +	return status;
> +}
> +
> +/**
> + * i40evf_find_segment_in_package
> + * @segment_type: the segment type to search for (i.e., SEGMENT_TYPE_I40E)
> + * @pkg_hdr: pointer to the package header to be searched
> + *
> + * This function searches a package file for a particular segment type. On
> + * success it returns a pointer to the segment header, otherwise it will
> + * return NULL.
> + **/
> +struct i40e_generic_seg_header *
> +i40evf_find_segment_in_package(u32 segment_type,
> +			       struct i40e_package_header *pkg_hdr)
> +{
> +	struct i40e_generic_seg_header *segment;
> +	u32 i;
> +
> +	/* Search all package segments for the requested segment type */
> +	for (i = 0; i < pkg_hdr->segment_count; i++) {
> +		segment =
> +			(struct i40e_generic_seg_header *)((u8 *)pkg_hdr +
> +			 pkg_hdr->segment_offset[i]);
> +
> +		if (segment->type == segment_type)
> +			return segment;
> +	}
> +
> +	return NULL;
> +}
> +
> +/**
> + * i40evf_write_profile
> + * @hw: pointer to the hardware structure
> + * @profile: pointer to the profile segment of the package to be downloaded
> + * @track_id: package tracking id
> + *
> + * Handles the download of a complete package.
> + */
> +enum i40e_status_code
> +i40evf_write_profile(struct i40e_hw *hw, struct i40e_profile_segment *profile,
> +		     u32 track_id)
> +{
> +	i40e_status status = 0;
> +	struct i40e_section_table *sec_tbl;
> +	struct i40e_profile_section_header *sec = NULL;
> +	u32 dev_cnt;
> +	u32 vendor_dev_id;
> +	u32 *nvm;
> +	u32 section_size = 0;
> +	u32 offset = 0, info = 0;
> +	u32 i;
> +
> +	if (!track_id) {
> +		i40e_debug(hw, I40E_DEBUG_PACKAGE, "Track_id can't be 0.");
> +		return I40E_NOT_SUPPORTED;
> +	}
> +
> +	dev_cnt = profile->device_table_count;
> +
> +	for (i = 0; i < dev_cnt; i++) {
> +		vendor_dev_id = profile->device_table[i].vendor_dev_id;
> +		if ((vendor_dev_id >> 16) == PCI_VENDOR_ID_INTEL)
> +			if (hw->device_id == (vendor_dev_id & 0xFFFF))
> +				break;
> +	}
> +	if (i == dev_cnt) {
> +		i40e_debug(hw, I40E_DEBUG_PACKAGE, "Device doesn't support PPP");
> +		return I40E_ERR_DEVICE_NOT_SUPPORTED;
> +	}
> +
> +	nvm = (u32 *)&profile->device_table[dev_cnt];
> +	sec_tbl = (struct i40e_section_table *)&nvm[nvm[0] + 1];
> +
> +	for (i = 0; i < sec_tbl->section_count; i++) {
> +		sec = (struct i40e_profile_section_header *)((u8 *)profile +
> +					     sec_tbl->section_offset[i]);
> +
> +		/* Skip 'AQ', 'note' and 'name' sections */
> +		if (sec->section.type != SECTION_TYPE_MMIO)
> +			continue;
> +
> +		section_size = sec->section.size +
> +			sizeof(struct i40e_profile_section_header);
> +
> +		/* Write profile */
> +		status = i40evf_aq_write_ppp(hw, (void *)sec, (u16)section_size,
> +					     track_id, &offset, &info, NULL);
> +		if (status) {
> +			i40e_debug(hw, I40E_DEBUG_PACKAGE,
> +				   "Failed to write profile: offset %d, info %d",
> +				   offset, info);
> +			break;
> +		}
> +	}
> +	return status;
> +}
> +
> +/**
> + * i40evf_add_pinfo_to_list
> + * @hw: pointer to the hardware structure
> + * @profile: pointer to the profile segment of the package
> + * @profile_info_sec: buffer for information section
> + * @track_id: package tracking id
> + *
> + * Register a profile to the list of loaded profiles.
> + */
> +enum i40e_status_code
> +i40evf_add_pinfo_to_list(struct i40e_hw *hw,
> +			 struct i40e_profile_segment *profile,
> +			  u8 *profile_info_sec, u32 track_id)
> +{
> +	i40e_status status = 0;
> +	struct i40e_profile_section_header *sec = NULL;
> +	struct i40e_profile_info *pinfo;
> +	u32 offset = 0, info = 0;
> +
> +	sec = (struct i40e_profile_section_header *)profile_info_sec;
> +	sec->tbl_size = 1;
> +	sec->data_end = sizeof(struct i40e_profile_section_header) +
> +			sizeof(struct i40e_profile_info);
> +	sec->section.type = SECTION_TYPE_INFO;
> +	sec->section.offset = sizeof(struct i40e_profile_section_header);
> +	sec->section.size = sizeof(struct i40e_profile_info);
> +	pinfo = (struct i40e_profile_info *)(profile_info_sec +
> +					     sec->section.offset);
> +	pinfo->track_id = track_id;
> +	pinfo->version = profile->version;
> +	pinfo->op = I40E_PPP_ADD_TRACKID;
> +	memcpy(pinfo->name, profile->name, I40E_PPP_NAME_SIZE);
> +
> +	status = i40evf_aq_write_ppp(hw, (void *)sec, sec->data_end,
> +				     track_id, &offset, &info, NULL);
> +	return status;
> +}
> diff --git a/drivers/net/ethernet/intel/i40evf/i40e_prototype.h b/drivers/net/ethernet/intel/i40evf/i40e_prototype.h
> index ba6c6bd..329ce2b 100644
> --- a/drivers/net/ethernet/intel/i40evf/i40e_prototype.h
> +++ b/drivers/net/ethernet/intel/i40evf/i40e_prototype.h
> @@ -122,4 +122,21 @@ i40e_status i40e_write_phy_register(struct i40e_hw *hw, u8 page, u16 reg,
>  u8 i40e_get_phy_address(struct i40e_hw *hw, u8 dev_num);
>  i40e_status i40e_blink_phy_link_led(struct i40e_hw *hw,
>  				    u32 time, u32 interval);
> +i40e_status i40evf_aq_write_ppp(struct i40e_hw *hw, void *buff,
> +				u16 buff_size, u32 track_id,
> +				u32 *error_offset, u32 *error_info,
> +				struct i40e_asq_cmd_details *cmd_details);
> +i40e_status i40evf_aq_get_ppp_list(struct i40e_hw *hw, void *buff,
> +				   u16 buff_size, u8 flags,
> +				   struct i40e_asq_cmd_details *cmd_details);
> +struct i40e_generic_seg_header *
> +i40evf_find_segment_in_package(u32 segment_type,
> +			       struct i40e_package_header *pkg_header);
> +enum i40e_status_code
> +i40evf_write_profile(struct i40e_hw *hw, struct i40e_profile_segment *i40e_seg,
> +		     u32 track_id);
> +enum i40e_status_code
> +i40evf_add_pinfo_to_list(struct i40e_hw *hw,
> +			 struct i40e_profile_segment *profile,
> +			  u8 *profile_info_sec, u32 track_id);
>  #endif /* _I40E_PROTOTYPE_H_ */
> diff --git a/drivers/net/ethernet/intel/i40evf/i40e_type.h b/drivers/net/ethernet/intel/i40evf/i40e_type.h
> index 16bb880..44bb9f0 100644
> --- a/drivers/net/ethernet/intel/i40evf/i40e_type.h
> +++ b/drivers/net/ethernet/intel/i40evf/i40e_type.h
> @@ -78,6 +78,7 @@ enum i40e_debug_mask {
>  	I40E_DEBUG_DCB			= 0x00000400,
>  	I40E_DEBUG_DIAG			= 0x00000800,
>  	I40E_DEBUG_FD			= 0x00001000,
> +	I40E_DEBUG_PACKAGE		= 0x00002000,
>
>  	I40E_DEBUG_AQ_MESSAGE		= 0x01000000,
>  	I40E_DEBUG_AQ_DESCRIPTOR	= 0x02000000,
> @@ -1396,4 +1397,83 @@ enum i40e_reset_type {
>  #define I40E_FD_INSET_FLEX_WORD57_SHIFT		10
>  #define I40E_FD_INSET_FLEX_WORD57_MASK		(0x1ULL << \
>  					I40E_FD_INSET_FLEX_WORD57_SHIFT)
> +
> +/* Version format for PPP */
> +struct i40e_ppp_version {
> +	u8 major;
> +	u8 minor;
> +	u8 update;
> +	u8 draft;
> +};
> +
> +#define I40E_PPP_NAME_SIZE	32
> +
> +/* Package header */
> +struct i40e_package_header {
> +	struct i40e_ppp_version version;
> +	u32 segment_count;
> +	u32 segment_offset[1];
> +};
> +
> +/* Generic segment header */
> +struct i40e_generic_seg_header {
> +#define SEGMENT_TYPE_METADATA	0x00000001
> +#define SEGMENT_TYPE_NOTES	0x00000002
> +#define SEGMENT_TYPE_I40E	0x00000011
> +#define SEGMENT_TYPE_X722	0x00000012
> +	u32 type;
> +	struct i40e_ppp_version version;
> +	u32 size;
> +	char name[I40E_PPP_NAME_SIZE];
> +};
> +
> +struct i40e_metadata_segment {
> +	struct i40e_generic_seg_header header;
> +	struct i40e_ppp_version version;
> +	u32 track_id;
> +	char     name[I40E_PPP_NAME_SIZE];
> +};
> +
> +struct i40e_device_id_entry {
> +	u32 vendor_dev_id;
> +	u32 sub_vendor_dev_id;
> +};
> +
> +struct i40e_profile_segment {
> +	struct i40e_generic_seg_header header;
> +	struct i40e_ppp_version version;
> +	char name[I40E_PPP_NAME_SIZE];
> +	u32 device_table_count;
> +	struct i40e_device_id_entry device_table[1];
> +};
> +
> +struct i40e_section_table {
> +	u32 section_count;
> +	u32 section_offset[1];
> +};
> +
> +struct i40e_profile_section_header {
> +	u16 tbl_size;
> +	u16 data_end;
> +	struct {
> +#define SECTION_TYPE_INFO	0x00000010
> +#define SECTION_TYPE_MMIO	0x00000800
> +#define SECTION_TYPE_AQ		0x00000801
> +#define SECTION_TYPE_NOTE	0x80000000
> +#define SECTION_TYPE_NAME	0x80000001
> +		u32 type;
> +		u32 offset;
> +		u32 size;
> +	} section;
> +};
> +
> +struct i40e_profile_info {
> +	u32 track_id;
> +	struct i40e_ppp_version version;
> +	u8 op;
> +#define I40E_PPP_ADD_TRACKID		0x01
> +#define I40E_PPP_REMOVE_TRACKID	0x02
> +	u8 reserved[7];
> +	u8 name[I40E_PPP_NAME_SIZE];
> +};
>  #endif /* _I40E_TYPE_H_ */
>

[Intel-wired-lan] [next PATCH S70 10/12] i40e: use i40e_stop_rings_no_wait to implement PORT_SUSPENDED state

[Shannon Nelson]

On 4/13/2017 1:45 AM, Alice Michael wrote:
> From: Jacob Keller <jacob.e.keller@intel.com>
>
> This state bit was added as a way for DCB to avoid having to wait for
> the queues to disable when handling LLDP events. The logic for this was
> burried deep within stop Tx and stop Rx queue code. First, let's rename

s/burried/buried/

> it so that it does not appear to only affect Tx when infact it modifies

s/infact/in fact/

> both Tx and Rx flow. Second we can move it up into the i40e_stop_rings()
> function, and we can simply re-use the i40e_stop_rings_no_wait() so that
> we don't have to bury the implementation as deep into the call stack.
>
> An alternative might be to remove the state bit and instead attempt to
> shut down everything directly in DCP flow. This, however, is not ideal
> because it creates yet another separate shutdown routine that we'd have
> to maintain. In the current implementation any changes will be made to
> both flows.
>
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Change-ID: I68e1ccb901af320862bca395e9c9746f08e8b17c
> ---
>  drivers/net/ethernet/intel/i40e/i40e.h      |  2 +-
>  drivers/net/ethernet/intel/i40e/i40e_main.c | 16 ++++++----------
>  2 files changed, 7 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h
> index d35fec3..76bc404 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e.h
> +++ b/drivers/net/ethernet/intel/i40e/i40e.h
> @@ -149,7 +149,7 @@ enum i40e_state_t {
>  	__I40E_DOWN_REQUESTED,
>  	__I40E_FD_FLUSH_REQUESTED,
>  	__I40E_RESET_FAILED,
> -	__I40E_PORT_TX_SUSPENDED,
> +	__I40E_PORT_SUSPENDED,
>  	__I40E_VF_DISABLE,
>  };
>
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
> index 7ceb256..b9d7798 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_main.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
> @@ -4008,10 +4008,6 @@ static int i40e_vsi_control_tx(struct i40e_vsi *vsi, bool enable)
>  	for (i = 0; i < vsi->num_queue_pairs; i++, pf_q++) {
>  		i40e_control_tx_q(pf, pf_q, enable);
>
> -		/* Don't wait to disable when port Tx is suspended */
> -		if (!enable && test_bit(__I40E_PORT_TX_SUSPENDED, &pf->state))
> -			continue;
> -
>  		/* wait for the change to finish */
>  		ret = i40e_pf_txq_wait(pf, pf_q, enable);
>  		if (ret) {
> @@ -4105,10 +4101,6 @@ static int i40e_vsi_control_rx(struct i40e_vsi *vsi, bool enable)
>  	for (i = 0; i < vsi->num_queue_pairs; i++, pf_q++) {
>  		i40e_control_rx_q(pf, pf_q, enable);
>
> -		/* Don't wait to disable when port Tx is suspended */
> -		if (!enable && test_bit(__I40E_PORT_TX_SUSPENDED, &pf->state))
> -			continue;
> -
>  		/* wait for the change to finish */
>  		ret = i40e_pf_rxq_wait(pf, pf_q, enable);
>  		if (ret) {
> @@ -4151,6 +4143,10 @@ int i40e_vsi_start_rings(struct i40e_vsi *vsi)
>   **/
>  void i40e_vsi_stop_rings(struct i40e_vsi *vsi)
>  {
> +	/* When port TX is suspended, don't wait */
> +	if (test_bit(__I40E_PORT_SUSPENDED, &vsi->back->state))
> +		return i40e_vsi_stop_rings_no_wait(vsi);
> +
>  	/* do rx first for enable and last for disable
>  	 * Ignore return value, we need to shutdown whatever we can
>  	 */
> @@ -5986,7 +5982,7 @@ static int i40e_handle_lldp_event(struct i40e_pf *pf,
>  	else
>  		pf->flags &= ~I40E_FLAG_DCB_ENABLED;
>
> -	set_bit(__I40E_PORT_TX_SUSPENDED, &pf->state);
> +	set_bit(__I40E_PORT_SUSPENDED, &pf->state);
>  	/* Reconfiguration needed quiesce all VSIs */
>  	i40e_pf_quiesce_all_vsi(pf);
>
> @@ -5995,7 +5991,7 @@ static int i40e_handle_lldp_event(struct i40e_pf *pf,
>
>  	ret = i40e_resume_port_tx(pf);
>
> -	clear_bit(__I40E_PORT_TX_SUSPENDED, &pf->state);
> +	clear_bit(__I40E_PORT_SUSPENDED, &pf->state);
>  	/* In case of error no point in resuming VSIs */
>  	if (ret)
>  		goto exit;
>

[Intel-wired-lan] [next PATCH S70 01/12] i40e/i40evf: Add tracepoints

[Bowers, AndrewX]

> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
> Behalf Of Alice Michael
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-
> lan at lists.osuosl.org
> Subject: [Intel-wired-lan] [next PATCH S70 01/12] i40e/i40evf: Add
> tracepoints
> 
> From: "Peterson, Scott D" <scott.d.peterson@intel.com>
> 
> This patch adds tracepoints to the i40e and i40evf drivers to which BPF
> programs can be attached for feature testing and verification.
> It's expected that an attached BPF program will identify and count or log
> some interesting subset of traffic. The bcc-tools package is helpful there for
> containing all the BPF arcana in a handy Python wrapper. Though you can
> make these tracepoints log trace messages, the messages themselves
> probably won't be very useful (other to verify the tracepoint is being called
> while you're debugging your BPF program).
> 
> The idea here is that tracepoints have such low performance cost when
> disabled that we can leave these in the upstream drivers. This may
> eventually enable the instrumentation of unmodified customer systems
> should the need arise to verify a NIC feature is working as expected.
> In general this enables one set of feature verification tools to be used on
> these drivers whether they're built with the kernel or separately.
> 
> Users are advised against using these tracepoints for anything other than a
> diagnostic tool. They have a performance impact when enabled, and their
> exact placement and form may change as we see how well they work in
> practice for the purposes above.
> 
> Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
> Change-ID: Id6014a7322c0e6d08068114dd20bd156f2f6435e
> ---
> Testing Hints:
> 
> See that performance is unaffected while tracepoints are present but
> disabled (default state). Enable them all and see they appear in
> /sys/kernel/debug/tracing/trace when exercised.
> 
> When disabled, a tracepoint reduces to a 5-byte no-op.
> When enabled, that code is patched to jump to the tracepoint clause, which
> is located somewhere nearby.  See include/linux/jump_label.h for info in
> static keys, and samples/trace-events/trace-events-samples.[ch]
> for info on the trace events.
> 
> To test tracepoints:
> insmod i40e and bring up a link
> cd /sys/kernel/debug/tracing
> echo > trace
> echo 1 > events/i40e/enable
> send something over i40e
> more trace
> echo 0 > events/i40e/enable
> 
> i40evf trace events are in events/i40evf, and enabled separately.
> 
> A sample BPF program (CORE/example_tp_bpf.py) is available, but won't be
> submitted upstream and so is not included in this patch.
> 
> It demonstrates how to access the structures pointed to by the tracepoint
> args from BPF. See the comments in that script for details and usage. As is it
> will attach itself to one of these tracepoints and emit mostly useless
> messages when the i40e_clean_rx_irq tracepoint is invoked.
>  drivers/net/ethernet/intel/i40e/Makefile        |   3 +
>  drivers/net/ethernet/intel/i40e/i40e_main.c     |   6 +
>  drivers/net/ethernet/intel/i40e/i40e_trace.h    | 229
> ++++++++++++++++++++++++
>  drivers/net/ethernet/intel/i40e/i40e_txrx.c     |   9 +
>  drivers/net/ethernet/intel/i40evf/Makefile      |   3 +
>  drivers/net/ethernet/intel/i40evf/i40e_trace.h  | 229
> ++++++++++++++++++++++++
>  drivers/net/ethernet/intel/i40evf/i40e_txrx.c   |   9 +
>  drivers/net/ethernet/intel/i40evf/i40evf_main.c |   7 +
>  8 files changed, 495 insertions(+)

Tested-by: Andrew Bowers <andrewx.bowers@intel.com>

[Intel-wired-lan] [next PATCH S70 03/12] i40e: don't hold RTNL lock while waiting for VF reset to finish

[Bowers, AndrewX]

> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
> Behalf Of Alice Michael
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-
> lan at lists.osuosl.org
> Subject: [Intel-wired-lan] [next PATCH S70 03/12] i40e: don't hold RTNL lock
> while waiting for VF reset to finish
> 
> From: Jacob Keller <jacob.e.keller@intel.com>
> 
> We made some effort to reduce the RTNL lock scope when resetting and
> rebuilding the PF. Unfortunately we still held the RTNL lock during the VF
> reset operation, which meant that multiple PFs could not reset in parallel due
> to the global lock. For now, further reduce the scope by not holding the RTNL
> lock while resetting VFs. This allows multiple PFs to reset in a timely manner.
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Change-ID: I2fbf823a0063f24dff67676cad09f0bbf83ee4ce
> ---
>  drivers/net/ethernet/intel/i40e/i40e_main.c | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)

Tested-by: Andrew Bowers <andrewx.bowers@intel.com>

[Intel-wired-lan] [next PATCH S70 04/12] i40e: factor out queue control from i40e_vsi_control_(tx|rx)

[Bowers, AndrewX]

> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
> Behalf Of Alice Michael
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-
> lan at lists.osuosl.org
> Subject: [Intel-wired-lan] [next PATCH S70 04/12] i40e: factor out queue
> control from i40e_vsi_control_(tx|rx)
> 
> From: Jacob Keller <jacob.e.keller@intel.com>
> 
> A future patch will need to be able to handle controlling queues without
> waiting until all VSIs are handled. Factor out the direct queue modification so
> that we can easily re-use this code. The result is also a bit easier to read since
> we don't embed multiple single-letter loop counters.
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Change-ID: Id923cbfa43127b1c24d8ed4f809b1012c736d9ac
> ---
>  drivers/net/ethernet/intel/i40e/i40e_main.c | 139 ++++++++++++++++++--
> --------
>  1 file changed, 89 insertions(+), 50 deletions(-)

Tested-by: Andrew Bowers <andrewx.bowers@intel.com>

[Intel-wired-lan] [next PATCH S70 05/12] i40e: fix CONFIG_BUSY checks in i40e_set_settings function

[Bowers, AndrewX]

> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
> Behalf Of Alice Michael
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-
> lan at lists.osuosl.org
> Subject: [Intel-wired-lan] [next PATCH S70 05/12] i40e: fix CONFIG_BUSY
> checks in i40e_set_settings function
> 
> From: Jacob Keller <jacob.e.keller@intel.com>
> 
> The check for I40E_CONFIG_BUSY state bit in the i40e_set_link_ksettings
> function is fishy. First we can notice a few things about the check here.
> 
> First a similar check was introduced by commit 'c7d05ca89f8e ("i40e: driver
> ethtool core")'
> 
> Later a commit introducing the link settings was added by commit
> 'bf9c71417f72 ("i40e: Implement set_settings for ethtool")'
> 
> However, this second check was against vsi->state instead of pf->state, and
> also failed to set the bit, it only checks. That indicates the locking was not
> quite correct. The only other place that the state bit in vsi->state gets used is
> to protect the filter list.
> 
> Since this code does not care about the mac filter list,  and seems clear the
> original code should have set the pf->state bit. Fix these issues by using pf-
> >state correctly, and by actually setting the bit so that we properly lock as
> expected.
> 
> Since these checks occur while holding the rtnl_lock(), lets also add a timeout
> so that we don't potentially softlock the system.
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> ---
>  drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 38
> ++++++++++++++++++++------
>  1 file changed, 29 insertions(+), 9 deletions(-)

Tested-by: Andrew Bowers <andrewx.bowers@intel.com>

[Intel-wired-lan] [next PATCH S70 06/12] i40e: reduce wait time for adminq command completion

[Bowers, AndrewX]

> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
> Behalf Of Alice Michael
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-
> lan at lists.osuosl.org
> Subject: [Intel-wired-lan] [next PATCH S70 06/12] i40e: reduce wait time for
> adminq command completion
> 
> From: Jacob Keller <jacob.e.keller@intel.com>
> 
> When sending an adminq command, we wait for the command to complete
> in a loop. This loop waits for an entire millisecond, when in practice the
> adminq command is processed often much faster.
> 
> Change the loop to use i40e_usec_delay instead, and wait for 50 usecs each
> time instead. This appears to be about the minimum time required, based on
> some manual observation and testing.
> 
> The primary benefit of this change is reducing latency of various operations in
> the PF driver, especially when related to having a large number of VFs
> enabled.
> 
> For example, on Linux, when instantiating 128 VFs, the time to finish the
> operation dropped from about 9 seconds down to under 6 seconds.
> Additionally, the time it takes to finish a PF reset with 128 VFs dropped from
> 5.1 seconds down to 0.7 seconds.
> 
> As the examples above show, a significant portion of the delay is wasted
> waiting for admiqn operations which have already finished.
> 
> This patch shouldn't cause impact to functionality, as we still check and keep
> waiting until the command does get processed. The only expected change is
> an increase in CPU utilization as we now check for completion far more times.
> However, in practice the commands appear to generally be complete within
> the first delay window anyways.
> 
> Previously for NDIS we used i40e_msec_stall since i40e_msec_delay would
> cause the CPU to be yielded. We can now just use i40e_usec_delay instead
> because this function will behave correctly for both NDIS and non-NDIS code.
> Thus, we can simply drop the #ifdef logic entirely.
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Change-ID: If8af8388e100da0a14eaf9e1af3afadf73a958cf
> ---
>  drivers/net/ethernet/intel/i40e/i40e_adminq.c   | 4 ++--
>  drivers/net/ethernet/intel/i40e/i40e_adminq.h   | 2 +-
>  drivers/net/ethernet/intel/i40evf/i40e_adminq.c | 4 ++--
> drivers/net/ethernet/intel/i40evf/i40e_adminq.h | 2 +-
>  4 files changed, 6 insertions(+), 6 deletions(-)

Tested-by: Andrew Bowers <andrewx.bowers@intel.com>

[Intel-wired-lan] [next PATCH S70 07/12] i40e: remove I40E_FLAG_IN_NETPOLL entirely

[Bowers, AndrewX]

> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
> Behalf Of Alice Michael
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-
> lan at lists.osuosl.org
> Subject: [Intel-wired-lan] [next PATCH S70 07/12] i40e: remove
> I40E_FLAG_IN_NETPOLL entirely
> 
> From: Jacob Keller <jacob.e.keller@intel.com>
> 
> This flag was originally intended to be used to let some driver code know
> when we were running from netpoll.
> Ultimately this was not necessary and we never used it.
> Let's remove it
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Change-ID: I43b72483d91c1638071d2a7f389ab171ec5b796a
> ---
>  drivers/net/ethernet/intel/i40evf/i40evf.h | 2 --
>  1 file changed, 2 deletions(-)

Tested-by: Andrew Bowers <andrewx.bowers@intel.com>

[Intel-wired-lan] [next PATCH S70 08/12] i40e: split some code in i40e_reset_vf into helpers

[Bowers, AndrewX]

> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
> Behalf Of Alice Michael
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-
> lan at lists.osuosl.org
> Subject: [Intel-wired-lan] [next PATCH S70 08/12] i40e: split some code in
> i40e_reset_vf into helpers
> 
> From: Jacob Keller <jacob.e.keller@intel.com>
> 
> A future patch is going to want to re-use some of the code in i40e_reset_vf,
> so lets break up the beginning and ending parts into their own helper
> functions. The first function will be used to initialize the reset on a VF, while
> the second function will be used to finalize the reset and restore
> functionality.
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Change-ID: I48df808b8bf09de3c2ed8c521f57b3f0ab9e5907
> ---
>  drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c | 109 ++++++++++++++-
> ------
>  1 file changed, 72 insertions(+), 37 deletions(-)

Tested-by: Andrew Bowers <andrewx.bowers@intel.com>

[Intel-wired-lan] [next PATCH S70 09/12] i40e: reset all VFs in parallel when rebuilding PF

[Bowers, AndrewX]

> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
> Behalf Of Alice Michael
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-
> lan at lists.osuosl.org
> Subject: [Intel-wired-lan] [next PATCH S70 09/12] i40e: reset all VFs in parallel
> when rebuilding PF
> 
> From: Jacob Keller <jacob.e.keller@intel.com>
> 
> When there are a lot of active VFs, it can take multiple seconds to finish
> resetting all of them during certain flows., which can cause some VFs to fail to
> wait long enough for the reset to occur. The user might see messages like
> "Never saw reset" or "Reset never finished" and the VF driver will stop
> functioning properly.
> 
> The naive solution would be to simply increase the wait timer. We can get
> much more clever. Notice that i40e_reset_vf is run in a serialized fashion,
> and includes lots of delays.
> 
> There are two prominent delays which take most of the time. First, when we
> begin resetting VFs, we have multiple 10ms delays which accrue because we
> reset each VF in a serial fashion. These delays accumulate to almost 4
> seconds when handling the maximum number of VFs (128).
> 
> Secondly, there is a massive 50ms delay for each time we disable queues on
> a VSI. This delay is necessary to allow HW to finish disabling queues before
> we restore functionality. However, just like with the first case, we are paying
> the cost for each VF, rather than disabling all VFs and waiting once.
> 
> Both of these can be fixed, but required some previous refactoring to handle
> the special case. First, we will need the i40e_vsi_wait_queues_disabled
> function which was previously DCB specific. Second, we will need to
> implement our own i40e_vsi_stop_rings_no_wait function which will handle
> the stopping of rings without the delays.
> 
> Finally, implement an i40e_reset_all_vfs function, which will first start the
> reset of all VFs, and pay the wait cost all at once, rather than serially waiting
> for each VF before we start processing then next one. After the VF has been
> reset, we'll disable all the VF queues, and then wait for them to disable.
> Again, we'll organize the flow such that we pay the wait cost only once.
> 
> Finally, after we've disabled queues we'll go ahead and begin restoring VF
> functionality. The result is reducing the wait time by a large factor and
> ensuring that VFs do not timeout when waiting in the VF driver.
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Change-ID: Ia6e8cf8d98131b78aec89db78afb8d905c9b12be
> ---
> Testing-hints:
>   "time echo 1 > /sys/class/net/<enpX>/device/reset" will trigger
>   a request for a reset in the driver. I used the "time" command to
>   measure how long it took for the reset to finish.
> 
>  drivers/net/ethernet/intel/i40e/i40e.h             |   2 +
>  drivers/net/ethernet/intel/i40e/i40e_main.c        |  32 +++++--
>  drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c | 100
> +++++++++++++++++++++
>  drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h |   1 +
>  4 files changed, 129 insertions(+), 6 deletions(-)

Tested-by: Andrew Bowers <andrewx.bowers@intel.com>

[Intel-wired-lan] [next PATCH S70 10/12] i40e: use i40e_stop_rings_no_wait to implement PORT_SUSPENDED state

[Bowers, AndrewX]

> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
> Behalf Of Alice Michael
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-
> lan at lists.osuosl.org
> Subject: [Intel-wired-lan] [next PATCH S70 10/12] i40e: use
> i40e_stop_rings_no_wait to implement PORT_SUSPENDED state
> 
> From: Jacob Keller <jacob.e.keller@intel.com>
> 
> This state bit was added as a way for DCB to avoid having to wait for the
> queues to disable when handling LLDP events. The logic for this was burried
> deep within stop Tx and stop Rx queue code. First, let's rename it so that it
> does not appear to only affect Tx when infact it modifies both Tx and Rx flow.
> Second we can move it up into the i40e_stop_rings() function, and we can
> simply re-use the i40e_stop_rings_no_wait() so that we don't have to bury
> the implementation as deep into the call stack.
> 
> An alternative might be to remove the state bit and instead attempt to shut
> down everything directly in DCP flow. This, however, is not ideal because it
> creates yet another separate shutdown routine that we'd have to maintain.
> In the current implementation any changes will be made to both flows.
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Change-ID: I68e1ccb901af320862bca395e9c9746f08e8b17c
> ---
>  drivers/net/ethernet/intel/i40e/i40e.h      |  2 +-
>  drivers/net/ethernet/intel/i40e/i40e_main.c | 16 ++++++----------
>  2 files changed, 7 insertions(+), 11 deletions(-)

Tested-by: Andrew Bowers <andrewx.bowers@intel.com>

[Intel-wired-lan] [next PATCH S70 11/12] i40e: properly spell I40E_VF_STATE_* flags

[Bowers, AndrewX]

> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
> Behalf Of Alice Michael
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-
> lan at lists.osuosl.org
> Subject: [Intel-wired-lan] [next PATCH S70 11/12] i40e: properly spell
> I40E_VF_STATE_* flags
> 
> From: Jacob Keller <jacob.e.keller@intel.com>
> 
> These flags represent the state of the VF at various times. Do not spell them
> as _STAT_ which can be confusing to readers who may think these refer to
> statistics.
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Change-ID: I6bc092cd472e8276896a1fd7498aced2084312df
> ---
>  drivers/net/ethernet/intel/i40e/i40e_ethtool.c     |  2 +-
>  drivers/net/ethernet/intel/i40e/i40e_main.c        |  2 +-
>  drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c | 98 +++++++++++--------
> ---  drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h | 14 ++--
>  4 files changed, 58 insertions(+), 58 deletions(-)


Tested-by: Andrew Bowers <andrewx.bowers@intel.com>

[Intel-wired-lan] [next PATCH S70 12/12] i40e: make use of i40e_reset_all_vfs when initializing new VFs

[Bowers, AndrewX]

> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
> Behalf Of Alice Michael
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-
> lan at lists.osuosl.org
> Subject: [Intel-wired-lan] [next PATCH S70 12/12] i40e: make use of
> i40e_reset_all_vfs when initializing new VFs
> 
> From: Jacob Keller <jacob.e.keller@intel.com>
> 
> When allocating a large number of VFs, the driver previously used
> i40e_reset_vf in a sequence. Just as when performing a normal reset, this
> accumulates a large amount of delay for handling all of the VFs in sequence.
> This delay is mainly due to a hardware requirement to wait after initiating a
> reset on the VF.
> 
> We recently added a new function, i40e_reset_all_vfs() which can be used to
> amortize the delay time, by first triggering all VF resets, then waiting once,
> and finally cleaning up and allocating the VFs. This is almost as good as truly
> running the resets in parallel.
> 
> In order to avoid sending a spurious reset message to a client interface, we
> have a check to see whether we've assigned
> pf->num_alloc_vfs yet. This was originally intended as a way to
> distinquish the "initialization" case from the regular reset case.
> 
> Unfortunately, this means that we can't directly use i40e_reset_all_vfs yet.
> Lets avoid this check of pf->num_alloc_vfs by replacing it with a proper VSI
> state bit which we can use instead. This makes the intention much clearer
> and allows us to re-use the i40e_reset_all_vfs function directly.
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Reviewed-by: Williams, Mitch A <mitch.a.williams@intel.com>
> Change-ID: I694279b37eb6b5a91b6670182d0c15d10244fd6e
> ---
>  drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c | 10 +++++++---
> drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h |  1 +
>  2 files changed, 8 insertions(+), 3 deletions(-)

Tested-by: Andrew Bowers <andrewx.bowers@intel.com>

[Intel-wired-lan] [next PATCH S70 02/12] i40e: DCR 287 new AQ commands

[Bowers, AndrewX]

> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
> Behalf Of Alice Michael
> Sent: Thursday, April 13, 2017 1:46 AM
> To: Michael, Alice <alice.michael@intel.com>; intel-wired-
> lan at lists.osuosl.org
> Cc: Wu, Jingjing <jingjing.wu@intel.com>
> Subject: [Intel-wired-lan] [next PATCH S70 02/12] i40e: DCR 287 new AQ
> commands
> 
> From: Jingjing Wu <jingjing.wu@intel.com>
> 
> Add admin queue functions for Pipeline Personalization Profile AQ
> commands defined in DCR 287:
>  - Write Recipe Command buffer (Opcode: 0x0270)
>  - Get Applied Profiles list (Opcode: 0x0271)
> 
> Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
> Change-ID: I558b4145364140f624013af48d4bbf79d21ebb0d
> ---
>  drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h  |  34 ++++
>  drivers/net/ethernet/intel/i40e/i40e_common.c      | 212
> +++++++++++++++++++++
>  drivers/net/ethernet/intel/i40e/i40e_prototype.h   |  17 ++
>  drivers/net/ethernet/intel/i40e/i40e_type.h        |  80 ++++++++
>  .../net/ethernet/intel/i40evf/i40e_adminq_cmd.h    |  34 ++++
>  drivers/net/ethernet/intel/i40evf/i40e_common.c    | 212
> +++++++++++++++++++++
>  drivers/net/ethernet/intel/i40evf/i40e_prototype.h |  17 ++
>  drivers/net/ethernet/intel/i40evf/i40e_type.h      |  80 ++++++++
>  8 files changed, 686 insertions(+)

Tested-by: Andrew Bowers <andrewx.bowers@intel.com>