[PATCH 00/31] Support more features in Solarflare PMD

[PATCH 00/31] Support more features in Solarflare PMD

[Andrew Rybchenko]

The patch series adds a number of features to Solarflare libefx-based
PMD. Basically one patch per feature.

The patches are grouped into one series since they touch nearby lines
in either PMD feature list, or dev_ops structure, or documentation.
So, patches cannot be applied in arbitrary order.

The patch series should be applied after
[PATCH v2 00/55] Solarflare libefx-based PMD
(Message-ID: 1480436367-20749-1-git-send-email-arybchenko@solarflare.com)


Andrew Rybchenko (16):
  net/sfc: implement MCDI logging callback
  net/sfc: support parameter to choose performance profile
  net/sfc: implement ethdev hook to get basic statistics
  net/sfc: support extended statistics
  net/sfc: support flow control settings get/set
  net/sfc: support link status change interrupt
  net/sfc: implement device operation to change MTU
  net/sfc: support link speed and duplex settings
  net/sfc: support checksum offloads on receive
  net/sfc: handle received packet type info provided by HW
  net/sfc: support callback to get receive queue information
  net/sfc: support Rx free threshold
  net/sfc: add callback to get RxQ pending descriptors count
  net/sfc: add RxQ descriptor done callback
  net/sfc: support scattered Rx DMA
  net/sfc: support deferred start of receive queues

Artem Andreev (1):
  net/sfc: support link up/down

Ivan Malov (14):
  net/sfc: support promiscuous and all-multicast control
  net/sfc: support main (the first) MAC address change
  net/sfc: support multicast addresses list controls
  net/sfc: add callback to get transmit queue information
  net/sfc: support Tx free threshold
  net/sfc: support deferred start of transmit queues
  net/sfc: support VLAN offload on transmit path
  net/sfc: add basic stubs for RSS support on driver attach
  net/sfc: support RSS hash offload
  net/sfc: add callback to query RSS key and hash types config
  net/sfc: add callback to set RSS key and hash types config
  net/sfc: add callback to query RSS redirection table
  net/sfc: add callback to update RSS redirection table
  net/sfc: support firmware-assisted TSOv2

 config/common_base                   |   1 +
 doc/guides/nics/features/sfc_efx.ini |  22 +-
 doc/guides/nics/sfc_efx.rst          |  55 ++-
 drivers/net/sfc/Makefile             |   4 +
 drivers/net/sfc/efsys.h              |   8 +-
 drivers/net/sfc/sfc.c                | 126 ++++-
 drivers/net/sfc/sfc.h                |  46 ++
 drivers/net/sfc/sfc_ethdev.c         | 893 ++++++++++++++++++++++++++++++++++-
 drivers/net/sfc/sfc_ev.c             |  64 ++-
 drivers/net/sfc/sfc_ev.h             |   2 +
 drivers/net/sfc/sfc_intr.c           | 204 ++++++++
 drivers/net/sfc/sfc_kvargs.c         |   2 +
 drivers/net/sfc/sfc_kvargs.h         |  12 +
 drivers/net/sfc/sfc_mcdi.c           |  69 +++
 drivers/net/sfc/sfc_port.c           | 107 ++++-
 drivers/net/sfc/sfc_rx.c             | 288 ++++++++++-
 drivers/net/sfc/sfc_rx.h             |  16 +
 drivers/net/sfc/sfc_tso.c            | 203 ++++++++
 drivers/net/sfc/sfc_tweak.h          |   3 +
 drivers/net/sfc/sfc_tx.c             | 165 ++++++-
 drivers/net/sfc/sfc_tx.h             |  41 +-
 21 files changed, 2259 insertions(+), 72 deletions(-)
 create mode 100644 drivers/net/sfc/sfc_tso.c

-- 
2.5.5

[PATCH 01/31] net/sfc: implement MCDI logging callback

[Andrew Rybchenko]

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/sfc_efx.rst  |  6 ++++
 drivers/net/sfc/efsys.h      |  2 +-
 drivers/net/sfc/sfc.h        |  1 +
 drivers/net/sfc/sfc_ethdev.c |  1 +
 drivers/net/sfc/sfc_kvargs.c |  1 +
 drivers/net/sfc/sfc_kvargs.h |  2 ++
 drivers/net/sfc/sfc_mcdi.c   | 69 ++++++++++++++++++++++++++++++++++++++++++++
 7 files changed, 81 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index aadd775..2cca287 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -155,3 +155,9 @@ boolean parameters value.
 - ``debug_init`` [bool] (default **n**)
 
   Enable extra logging during device intialization and startup.
+
+- ``mcdi_logging`` [bool] (default **n**)
+
+  Enable extra logging of the communication with the NIC's management CPU.
+  The logging is done using RTE_LOG() with INFO level and PMD type.
+  The format is consumed by the Solarflare netlogdecode cross-platform tool.
diff --git a/drivers/net/sfc/efsys.h b/drivers/net/sfc/efsys.h
index e4d5035..d48eb4c 100644
--- a/drivers/net/sfc/efsys.h
+++ b/drivers/net/sfc/efsys.h
@@ -175,7 +175,7 @@ prefetch_read_once(const volatile void *addr)
 
 /* MCDI is required for SFN7xxx and SFN8xx */
 #define EFSYS_OPT_MCDI 1
-#define EFSYS_OPT_MCDI_LOGGING 0
+#define EFSYS_OPT_MCDI_LOGGING 1
 #define EFSYS_OPT_MCDI_PROXY_AUTH 0
 
 #define EFSYS_OPT_MAC_STATS 0
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index 29d3a6b..0064fcb 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -107,6 +107,7 @@ struct sfc_mcdi {
 	efsys_mem_t			mem;
 	enum sfc_mcdi_state		state;
 	efx_mcdi_transport_t		transport;
+	bool				logging;
 };
 
 struct sfc_intr {
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index c28082c..12309ee 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -464,4 +464,5 @@ static struct eth_driver sfc_efx_pmd = {
 RTE_PMD_REGISTER_PCI(net_sfc_efx, sfc_efx_pmd.pci_drv);
 RTE_PMD_REGISTER_PCI_TABLE(net_sfc_efx, pci_id_sfc_efx_map);
 RTE_PMD_REGISTER_PARAM_STRING(net_sfc_efx,
+	SFC_KVARG_MCDI_LOGGING "=" SFC_KVARG_VALUES_BOOL " "
 	SFC_KVARG_DEBUG_INIT "=" SFC_KVARG_VALUES_BOOL);
diff --git a/drivers/net/sfc/sfc_kvargs.c b/drivers/net/sfc/sfc_kvargs.c
index f1bab28..bbbd026 100644
--- a/drivers/net/sfc/sfc_kvargs.c
+++ b/drivers/net/sfc/sfc_kvargs.c
@@ -42,6 +42,7 @@ sfc_kvargs_parse(struct sfc_adapter *sa)
 	struct rte_devargs *devargs = sa->eth_dev->pci_dev->device.devargs;
 	const char **params = (const char *[]){
 		SFC_KVARG_DEBUG_INIT,
+		SFC_KVARG_MCDI_LOGGING,
 		NULL,
 	};
 
diff --git a/drivers/net/sfc/sfc_kvargs.h b/drivers/net/sfc/sfc_kvargs.h
index 0b53963..ffce851 100644
--- a/drivers/net/sfc/sfc_kvargs.h
+++ b/drivers/net/sfc/sfc_kvargs.h
@@ -40,6 +40,8 @@ extern "C" {
 
 #define SFC_KVARG_DEBUG_INIT		"debug_init"
 
+#define SFC_KVARG_MCDI_LOGGING		"mcdi_logging"
+
 struct sfc_adapter;
 
 int sfc_kvargs_parse(struct sfc_adapter *sa);
diff --git a/drivers/net/sfc/sfc_mcdi.c b/drivers/net/sfc/sfc_mcdi.c
index 9ba28e1..3bed2e0 100644
--- a/drivers/net/sfc/sfc_mcdi.c
+++ b/drivers/net/sfc/sfc_mcdi.c
@@ -35,6 +35,7 @@
 
 #include "sfc.h"
 #include "sfc_log.h"
+#include "sfc_kvargs.h"
 
 #define SFC_MCDI_POLL_INTERVAL_MIN_US	10		/* 10us in 1us units */
 #define SFC_MCDI_POLL_INTERVAL_MAX_US	(US_PER_S / 10)	/* 100ms in 1us units */
@@ -125,6 +126,65 @@ sfc_mcdi_exception(void *arg, efx_mcdi_exception_t eme)
 	sfc_panic(sa, "MCDI exceptions handling is not implemented\n");
 }
 
+#define SFC_MCDI_LOG_BUF_SIZE	128
+
+static size_t
+sfc_mcdi_do_log(const struct sfc_adapter *sa,
+		char *buffer, void *data, size_t data_size,
+		size_t pfxsize, size_t position)
+{
+	uint32_t *words = data;
+	/* Space separator plus 2 characters per byte */
+	const size_t word_str_space = 1 + 2 * sizeof(*words);
+	size_t i;
+
+	for (i = 0; i < data_size; i += sizeof(*words)) {
+		if (position + word_str_space >=
+		    SFC_MCDI_LOG_BUF_SIZE) {
+			/* Flush at SFC_MCDI_LOG_BUF_SIZE with backslash
+			 * at the end which is required by netlogdecode.
+			 */
+			buffer[position] = '\0';
+			sfc_info(sa, "%s \\", buffer);
+			/* Preserve prefix for the next log message */
+			position = pfxsize;
+		}
+		position += snprintf(buffer + position,
+				     SFC_MCDI_LOG_BUF_SIZE - position,
+				     " %08x", *words);
+		words++;
+	}
+	return position;
+}
+
+static void
+sfc_mcdi_logger(void *arg, efx_log_msg_t type,
+		void *header, size_t header_size,
+		void *data, size_t data_size)
+{
+	struct sfc_adapter *sa = (struct sfc_adapter *)arg;
+	char buffer[SFC_MCDI_LOG_BUF_SIZE];
+	size_t pfxsize;
+	size_t start;
+
+	if (!sa->mcdi.logging)
+		return;
+
+	/* The format including prefix added by sfc_info() is the format
+	 * consumed by the Solarflare netlogdecode tool.
+	 */
+	pfxsize = snprintf(buffer, sizeof(buffer), "MCDI RPC %s:",
+			   type == EFX_LOG_MCDI_REQUEST ? "REQ" :
+			   type == EFX_LOG_MCDI_RESPONSE ? "RESP" : "???");
+	start = sfc_mcdi_do_log(sa, buffer, header, header_size,
+				pfxsize, pfxsize);
+	start = sfc_mcdi_do_log(sa, buffer, data, data_size, pfxsize, start);
+	if (start != pfxsize) {
+		buffer[start] = '\0';
+		sfc_info(sa, "%s", buffer);
+	}
+}
+
 int
 sfc_mcdi_init(struct sfc_adapter *sa)
 {
@@ -149,12 +209,19 @@ sfc_mcdi_init(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_dma_alloc;
 
+	/* Convert negative error to positive used in the driver */
+	rc = sfc_kvargs_process(sa, SFC_KVARG_MCDI_LOGGING,
+				sfc_kvarg_bool_handler, &mcdi->logging);
+	if (rc != 0)
+		goto fail_kvargs_process;
+
 	emtp = &mcdi->transport;
 	emtp->emt_context = sa;
 	emtp->emt_dma_mem = &mcdi->mem;
 	emtp->emt_execute = sfc_mcdi_execute;
 	emtp->emt_ev_cpl = sfc_mcdi_ev_cpl;
 	emtp->emt_exception = sfc_mcdi_exception;
+	emtp->emt_logger = sfc_mcdi_logger;
 
 	sfc_log_init(sa, "init MCDI");
 	rc = efx_mcdi_init(sa->nic, emtp);
@@ -165,6 +232,8 @@ sfc_mcdi_init(struct sfc_adapter *sa)
 
 fail_mcdi_init:
 	memset(emtp, 0, sizeof(*emtp));
+
+fail_kvargs_process:
 	sfc_dma_free(sa, &mcdi->mem);
 
 fail_dma_alloc:
-- 
2.5.5

[PATCH 02/31] net/sfc: support parameter to choose performance profile

[Andrew Rybchenko]

Supported options are auto (based on NIC firmware variant and
installed licences), throughput, low-latency.

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/sfc_efx.rst  |  7 +++++++
 drivers/net/sfc/sfc.h        |  1 +
 drivers/net/sfc/sfc_ethdev.c |  1 +
 drivers/net/sfc/sfc_ev.c     | 36 +++++++++++++++++++++++++++++++++---
 drivers/net/sfc/sfc_ev.h     |  2 ++
 drivers/net/sfc/sfc_kvargs.c |  1 +
 drivers/net/sfc/sfc_kvargs.h | 10 ++++++++++
 7 files changed, 55 insertions(+), 3 deletions(-)

diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 2cca287..36d0974 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -152,6 +152,13 @@ whitelist option like "-w 02:00.0,arg1=value1,...".
 Case-insensitive 1/y/yes/on or 0/n/no/off may be used to specify
 boolean parameters value.
 
+- ``perf_profile`` [auto|throughput|low-latency] (default **throughput**)
+
+  Choose hardware tunning to be optimized for either throughput or
+  low-latency.
+  **auto** allows NIC firmware to make a choice based on
+  installed licences and firmware variant configured using **sfboot**.
+
 - ``debug_init`` [bool] (default **n**)
 
   Enable extra logging during device intialization and startup.
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index 0064fcb..d0aafa3 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -152,6 +152,7 @@ struct sfc_adapter {
 
 	unsigned int			txq_max_entries;
 
+	uint32_t			evq_flags;
 	unsigned int			evq_count;
 	struct sfc_evq_info		*evq_info;
 
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 12309ee..1df227e 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -464,5 +464,6 @@ static struct eth_driver sfc_efx_pmd = {
 RTE_PMD_REGISTER_PCI(net_sfc_efx, sfc_efx_pmd.pci_drv);
 RTE_PMD_REGISTER_PCI_TABLE(net_sfc_efx, pci_id_sfc_efx_map);
 RTE_PMD_REGISTER_PARAM_STRING(net_sfc_efx,
+	SFC_KVARG_PERF_PROFILE "=" SFC_KVARG_VALUES_PERF_PROFILE " "
 	SFC_KVARG_MCDI_LOGGING "=" SFC_KVARG_VALUES_BOOL " "
 	SFC_KVARG_DEBUG_INIT "=" SFC_KVARG_VALUES_BOOL);
diff --git a/drivers/net/sfc/sfc_ev.c b/drivers/net/sfc/sfc_ev.c
index 96b95cc..34c1127 100644
--- a/drivers/net/sfc/sfc_ev.c
+++ b/drivers/net/sfc/sfc_ev.c
@@ -39,6 +39,7 @@
 #include "sfc_ev.h"
 #include "sfc_rx.h"
 #include "sfc_tx.h"
+#include "sfc_kvargs.h"
 
 
 /* Initial delay when waiting for event queue init complete event */
@@ -365,9 +366,7 @@ sfc_ev_qstart(struct sfc_adapter *sa, unsigned int sw_index)
 
 	/* Create the common code event queue */
 	rc = efx_ev_qcreate(sa->nic, sw_index, esmp, evq_info->entries,
-			    0 /* unused on EF10 */, 0,
-			    EFX_EVQ_FLAGS_TYPE_THROUGHPUT |
-			    EFX_EVQ_FLAGS_NOTIFY_DISABLED,
+			    0 /* unused on EF10 */, 0, evq_info->flags,
 			    &evq->common);
 	if (rc != 0)
 		goto fail_ev_qcreate;
@@ -600,6 +599,25 @@ sfc_ev_qinit_info(struct sfc_adapter *sa, unsigned int sw_index)
 	SFC_ASSERT(rte_is_power_of_2(max_entries));
 
 	evq_info->max_entries = max_entries;
+	evq_info->flags = sa->evq_flags | EFX_EVQ_FLAGS_NOTIFY_DISABLED;
+
+	return 0;
+}
+
+static int
+sfc_kvarg_perf_profile_handler(__rte_unused const char *key,
+			       const char *value_str, void *opaque)
+{
+	uint64_t *value = opaque;
+
+	if (strcasecmp(value_str, SFC_KVARG_PERF_PROFILE_THROUGHPUT) == 0)
+		*value = EFX_EVQ_FLAGS_TYPE_THROUGHPUT;
+	else if (strcasecmp(value_str, SFC_KVARG_PERF_PROFILE_LOW_LATENCY) == 0)
+		*value = EFX_EVQ_FLAGS_TYPE_LOW_LATENCY;
+	else if (strcasecmp(value_str, SFC_KVARG_PERF_PROFILE_AUTO) == 0)
+		*value = EFX_EVQ_FLAGS_TYPE_AUTO;
+	else
+		return -EINVAL;
 
 	return 0;
 }
@@ -620,6 +638,16 @@ sfc_ev_init(struct sfc_adapter *sa)
 
 	sfc_log_init(sa, "entry");
 
+	sa->evq_flags = EFX_EVQ_FLAGS_TYPE_THROUGHPUT;
+	rc = sfc_kvargs_process(sa, SFC_KVARG_PERF_PROFILE,
+				sfc_kvarg_perf_profile_handler,
+				&sa->evq_flags);
+	if (rc != 0) {
+		sfc_err(sa, "invalid %s parameter value",
+			SFC_KVARG_PERF_PROFILE);
+		goto fail_kvarg_perf_profile;
+	}
+
 	sa->evq_count = sfc_ev_qcount(sa);
 	sa->mgmt_evq_index = 0;
 	rte_spinlock_init(&sa->mgmt_evq_lock);
@@ -660,6 +688,8 @@ sfc_ev_init(struct sfc_adapter *sa)
 
 fail_evqs_alloc:
 	sa->evq_count = 0;
+
+fail_kvarg_perf_profile:
 	sfc_log_init(sa, "failed %d", rc);
 	return rc;
 }
diff --git a/drivers/net/sfc/sfc_ev.h b/drivers/net/sfc/sfc_ev.h
index 110f3b6..346e3ec 100644
--- a/drivers/net/sfc/sfc_ev.h
+++ b/drivers/net/sfc/sfc_ev.h
@@ -74,6 +74,8 @@ struct sfc_evq_info {
 	unsigned int		max_entries;
 	/* Real number of EVQ entries, less or equal to max_entries */
 	unsigned int		entries;
+	/* Event queue creation flags */
+	uint32_t		flags;
 	/* NUMA-aware EVQ data structure used on datapath */
 	struct sfc_evq		*evq;
 };
diff --git a/drivers/net/sfc/sfc_kvargs.c b/drivers/net/sfc/sfc_kvargs.c
index bbbd026..2ced47c 100644
--- a/drivers/net/sfc/sfc_kvargs.c
+++ b/drivers/net/sfc/sfc_kvargs.c
@@ -43,6 +43,7 @@ sfc_kvargs_parse(struct sfc_adapter *sa)
 	const char **params = (const char *[]){
 		SFC_KVARG_DEBUG_INIT,
 		SFC_KVARG_MCDI_LOGGING,
+		SFC_KVARG_PERF_PROFILE,
 		NULL,
 	};
 
diff --git a/drivers/net/sfc/sfc_kvargs.h b/drivers/net/sfc/sfc_kvargs.h
index ffce851..2fea9c7 100644
--- a/drivers/net/sfc/sfc_kvargs.h
+++ b/drivers/net/sfc/sfc_kvargs.h
@@ -42,6 +42,16 @@ extern "C" {
 
 #define SFC_KVARG_MCDI_LOGGING		"mcdi_logging"
 
+#define SFC_KVARG_PERF_PROFILE		"perf_profile"
+
+#define SFC_KVARG_PERF_PROFILE_AUTO		"auto"
+#define SFC_KVARG_PERF_PROFILE_THROUGHPUT	"throughput"
+#define SFC_KVARG_PERF_PROFILE_LOW_LATENCY	"low-latency"
+#define SFC_KVARG_VALUES_PERF_PROFILE \
+	"[" SFC_KVARG_PERF_PROFILE_AUTO "|" \
+	    SFC_KVARG_PERF_PROFILE_THROUGHPUT "|" \
+	    SFC_KVARG_PERF_PROFILE_LOW_LATENCY "]"
+
 struct sfc_adapter;
 
 int sfc_kvargs_parse(struct sfc_adapter *sa);
-- 
2.5.5

[PATCH 03/31] net/sfc: implement ethdev hook to get basic statistics

[Andrew Rybchenko]

Does not implement any deprecated statistics.
No per-queue statistics yet.

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 +
 drivers/net/sfc/efsys.h              |  2 +-
 drivers/net/sfc/sfc.h                |  7 ++++
 drivers/net/sfc/sfc_ethdev.c         | 69 +++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_port.c           | 75 ++++++++++++++++++++++++++++++++++++
 6 files changed, 155 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 67df1c6..f55a988 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -7,6 +7,7 @@
 Link status          = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
+Basic stats          = Y
 BSD nic_uio          = Y
 Linux UIO            = Y
 Linux VFIO           = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 36d0974..cbb51de 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -48,6 +48,8 @@ SFC EFX PMD has support for:
 
 - IPv4/IPv6 TCP/UDP transmit checksum offload
 
+- Port hardware statistics
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/efsys.h b/drivers/net/sfc/efsys.h
index d48eb4c..fe8615f 100644
--- a/drivers/net/sfc/efsys.h
+++ b/drivers/net/sfc/efsys.h
@@ -178,7 +178,7 @@ prefetch_read_once(const volatile void *addr)
 #define EFSYS_OPT_MCDI_LOGGING 1
 #define EFSYS_OPT_MCDI_PROXY_AUTH 0
 
-#define EFSYS_OPT_MAC_STATS 0
+#define EFSYS_OPT_MAC_STATS 1
 
 #define EFSYS_OPT_LOOPBACK 0
 
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index d0aafa3..1189283 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -122,6 +122,12 @@ struct sfc_port {
 	unsigned int			flow_ctrl;
 	boolean_t			flow_ctrl_autoneg;
 	size_t				pdu;
+
+	rte_spinlock_t			mac_stats_lock;
+	uint64_t			*mac_stats_buf;
+	efsys_mem_t			mac_stats_dma_mem;
+
+	uint32_t		mac_stats_mask[EFX_MAC_STATS_MASK_NPAGES];
 };
 
 /* Adapter private data */
@@ -229,6 +235,7 @@ int sfc_port_start(struct sfc_adapter *sa);
 void sfc_port_stop(struct sfc_adapter *sa);
 void sfc_port_link_mode_to_info(efx_link_mode_t link_mode,
 				struct rte_eth_link *link_info);
+int sfc_port_update_mac_stats(struct sfc_adapter *sa);
 
 
 #ifdef __cplusplus
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 1df227e..f31330c 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -324,12 +324,81 @@ sfc_tx_queue_release(void *queue)
 	sfc_adapter_unlock(sa);
 }
 
+static void
+sfc_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	struct sfc_port *port = &sa->port;
+	uint64_t *mac_stats;
+
+	rte_spinlock_lock(&port->mac_stats_lock);
+
+	if (sfc_port_update_mac_stats(sa) != 0)
+		goto unlock;
+
+	mac_stats = port->mac_stats_buf;
+
+	if (EFX_MAC_STAT_SUPPORTED(port->mac_stats_mask,
+				   EFX_MAC_VADAPTER_RX_UNICAST_PACKETS)) {
+		stats->ipackets =
+			mac_stats[EFX_MAC_VADAPTER_RX_UNICAST_PACKETS] +
+			mac_stats[EFX_MAC_VADAPTER_RX_MULTICAST_PACKETS] +
+			mac_stats[EFX_MAC_VADAPTER_RX_BROADCAST_PACKETS];
+		stats->opackets =
+			mac_stats[EFX_MAC_VADAPTER_TX_UNICAST_PACKETS] +
+			mac_stats[EFX_MAC_VADAPTER_TX_MULTICAST_PACKETS] +
+			mac_stats[EFX_MAC_VADAPTER_TX_BROADCAST_PACKETS];
+		stats->ibytes =
+			mac_stats[EFX_MAC_VADAPTER_RX_UNICAST_BYTES] +
+			mac_stats[EFX_MAC_VADAPTER_RX_MULTICAST_BYTES] +
+			mac_stats[EFX_MAC_VADAPTER_RX_BROADCAST_BYTES];
+		stats->obytes =
+			mac_stats[EFX_MAC_VADAPTER_TX_UNICAST_BYTES] +
+			mac_stats[EFX_MAC_VADAPTER_TX_MULTICAST_BYTES] +
+			mac_stats[EFX_MAC_VADAPTER_TX_BROADCAST_BYTES];
+		stats->imissed = mac_stats[EFX_MAC_VADAPTER_RX_OVERFLOW];
+		stats->ierrors = mac_stats[EFX_MAC_VADAPTER_RX_BAD_PACKETS];
+		stats->oerrors = mac_stats[EFX_MAC_VADAPTER_TX_BAD_PACKETS];
+	} else {
+		stats->ipackets = mac_stats[EFX_MAC_RX_PKTS];
+		stats->opackets = mac_stats[EFX_MAC_TX_PKTS];
+		stats->ibytes = mac_stats[EFX_MAC_RX_OCTETS];
+		stats->obytes = mac_stats[EFX_MAC_TX_OCTETS];
+		/*
+		 * Take into account stats which are whenever supported
+		 * on EF10. If some stat is not supported by current
+		 * firmware variant or HW revision, it is guaranteed
+		 * to be zero in mac_stats.
+		 */
+		stats->imissed =
+			mac_stats[EFX_MAC_RX_NODESC_DROP_CNT] +
+			mac_stats[EFX_MAC_PM_TRUNC_BB_OVERFLOW] +
+			mac_stats[EFX_MAC_PM_DISCARD_BB_OVERFLOW] +
+			mac_stats[EFX_MAC_PM_TRUNC_VFIFO_FULL] +
+			mac_stats[EFX_MAC_PM_DISCARD_VFIFO_FULL] +
+			mac_stats[EFX_MAC_PM_TRUNC_QBB] +
+			mac_stats[EFX_MAC_PM_DISCARD_QBB] +
+			mac_stats[EFX_MAC_PM_DISCARD_MAPPING] +
+			mac_stats[EFX_MAC_RXDP_Q_DISABLED_PKTS] +
+			mac_stats[EFX_MAC_RXDP_DI_DROPPED_PKTS];
+		stats->ierrors =
+			mac_stats[EFX_MAC_RX_FCS_ERRORS] +
+			mac_stats[EFX_MAC_RX_ALIGN_ERRORS] +
+			mac_stats[EFX_MAC_RX_JABBER_PKTS];
+		/* no oerrors counters supported on EF10 */
+	}
+
+unlock:
+	rte_spinlock_unlock(&port->mac_stats_lock);
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
 	.dev_stop			= sfc_dev_stop,
 	.dev_close			= sfc_dev_close,
 	.link_update			= sfc_dev_link_update,
+	.stats_get			= sfc_stats_get,
 	.dev_infos_get			= sfc_dev_infos_get,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
diff --git a/drivers/net/sfc/sfc_port.c b/drivers/net/sfc/sfc_port.c
index c124181..d8ff097 100644
--- a/drivers/net/sfc/sfc_port.c
+++ b/drivers/net/sfc/sfc_port.c
@@ -32,6 +32,34 @@
 #include "sfc.h"
 #include "sfc_log.h"
 
+/**
+ * Update MAC statistics in the buffer.
+ *
+ * @param	sa	Adapter
+ *
+ * @return Status code
+ * @retval	0	Success
+ * @retval	EAGAIN	Try again
+ * @retval	ENOMEM	Memory allocation failure
+ */
+int
+sfc_port_update_mac_stats(struct sfc_adapter *sa)
+{
+	struct sfc_port *port = &sa->port;
+	int rc;
+
+	SFC_ASSERT(rte_spinlock_is_locked(&port->mac_stats_lock));
+
+	if (sa->state != SFC_ADAPTER_STARTED)
+		return EINVAL;
+
+	rc = efx_mac_stats_update(sa->nic, &port->mac_stats_dma_mem,
+				  port->mac_stats_buf, NULL);
+	if (rc != 0)
+		return rc;
+
+	return 0;
+}
 
 int
 sfc_port_start(struct sfc_adapter *sa)
@@ -67,6 +95,19 @@ sfc_port_start(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_mac_filter_set;
 
+	efx_mac_stats_get_mask(sa->nic, port->mac_stats_mask,
+			       sizeof(port->mac_stats_mask));
+
+	/* Update MAC stats using periodic DMA.
+	 * Common code always uses 1000ms update period, so period_ms
+	 * parameter only needs to be non-zero to start updates.
+	 */
+	sfc_log_init(sa, "request MAC stats DMA'ing");
+	rc = efx_mac_stats_periodic(sa->nic, &port->mac_stats_dma_mem,
+				    1000, B_FALSE);
+	if (rc != 0)
+		goto fail_mac_stats_periodic;
+
 	sfc_log_init(sa, "disable MAC drain");
 	rc = efx_mac_drain(sa->nic, B_FALSE);
 	if (rc != 0)
@@ -76,6 +117,10 @@ sfc_port_start(struct sfc_adapter *sa)
 	return 0;
 
 fail_mac_drain:
+	(void)efx_mac_stats_periodic(sa->nic, &port->mac_stats_dma_mem,
+				     0, B_FALSE);
+
+fail_mac_stats_periodic:
 fail_mac_filter_set:
 fail_mac_addr_set:
 fail_mac_pdu_set:
@@ -95,6 +140,10 @@ sfc_port_stop(struct sfc_adapter *sa)
 	sfc_log_init(sa, "entry");
 
 	efx_mac_drain(sa->nic, B_TRUE);
+
+	(void)efx_mac_stats_periodic(sa->nic, &sa->port.mac_stats_dma_mem,
+				     0, B_FALSE);
+
 	efx_port_fini(sa->nic);
 	efx_filter_fini(sa->nic);
 
@@ -106,6 +155,7 @@ sfc_port_init(struct sfc_adapter *sa)
 {
 	const struct rte_eth_dev_data *dev_data = sa->eth_dev->data;
 	struct sfc_port *port = &sa->port;
+	int rc;
 
 	sfc_log_init(sa, "entry");
 
@@ -118,15 +168,40 @@ sfc_port_init(struct sfc_adapter *sa)
 	else
 		port->pdu = EFX_MAC_PDU(dev_data->mtu);
 
+	rte_spinlock_init(&port->mac_stats_lock);
+
+	rc = ENOMEM;
+	port->mac_stats_buf = rte_calloc_socket("mac_stats_buf", EFX_MAC_NSTATS,
+						sizeof(uint64_t), 0,
+						sa->socket_id);
+	if (port->mac_stats_buf == NULL)
+		goto fail_mac_stats_buf_alloc;
+
+	rc = sfc_dma_alloc(sa, "mac_stats", 0, EFX_MAC_STATS_SIZE,
+			   sa->socket_id, &port->mac_stats_dma_mem);
+	if (rc != 0)
+		goto fail_mac_stats_dma_alloc;
+
 	sfc_log_init(sa, "done");
 	return 0;
+
+fail_mac_stats_dma_alloc:
+	rte_free(port->mac_stats_buf);
+fail_mac_stats_buf_alloc:
+	sfc_log_init(sa, "failed %d", rc);
+	return rc;
 }
 
 void
 sfc_port_fini(struct sfc_adapter *sa)
 {
+	struct sfc_port *port = &sa->port;
+
 	sfc_log_init(sa, "entry");
 
+	sfc_dma_free(sa, &port->mac_stats_dma_mem);
+	rte_free(port->mac_stats_buf);
+
 	sfc_log_init(sa, "done");
 }
 
-- 
2.5.5

[PATCH 04/31] net/sfc: support extended statistics

[Andrew Rybchenko]

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 drivers/net/sfc/efsys.h              |  2 +-
 drivers/net/sfc/sfc_ethdev.c         | 63 ++++++++++++++++++++++++++++++++++++
 3 files changed, 65 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index f55a988..698553c 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -8,6 +8,7 @@ Link status          = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
 Basic stats          = Y
+Extended stats       = Y
 BSD nic_uio          = Y
 Linux UIO            = Y
 Linux VFIO           = Y
diff --git a/drivers/net/sfc/efsys.h b/drivers/net/sfc/efsys.h
index fe8615f..0f941e6 100644
--- a/drivers/net/sfc/efsys.h
+++ b/drivers/net/sfc/efsys.h
@@ -159,7 +159,7 @@ prefetch_read_once(const volatile void *addr)
 /* Code inclusion options */
 
 
-#define EFSYS_OPT_NAMES 0
+#define EFSYS_OPT_NAMES 1
 
 /* Disable SFN5xxx/SFN6xxx since it requires specific support in the PMD */
 #define EFSYS_OPT_SIENA 0
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index f31330c..d5ae1a0 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -392,6 +392,67 @@ sfc_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
 	rte_spinlock_unlock(&port->mac_stats_lock);
 }
 
+static int
+sfc_xstats_get(struct rte_eth_dev *dev, struct rte_eth_xstat *xstats,
+	       unsigned int xstats_count)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	struct sfc_port *port = &sa->port;
+	uint64_t *mac_stats;
+	int rc;
+	unsigned int i;
+	int nstats = 0;
+
+	rte_spinlock_lock(&port->mac_stats_lock);
+
+	rc = sfc_port_update_mac_stats(sa);
+	if (rc != 0) {
+		SFC_ASSERT(rc > 0);
+		nstats = -rc;
+		goto unlock;
+	}
+
+	mac_stats = port->mac_stats_buf;
+
+	for (i = 0; i < EFX_MAC_NSTATS; ++i) {
+		if (EFX_MAC_STAT_SUPPORTED(port->mac_stats_mask, i)) {
+			if (xstats != NULL && nstats < (int)xstats_count) {
+				xstats[nstats].id = nstats;
+				xstats[nstats].value = mac_stats[i];
+			}
+			nstats++;
+		}
+	}
+
+unlock:
+	rte_spinlock_unlock(&port->mac_stats_lock);
+
+	return nstats;
+}
+
+static int
+sfc_xstats_get_names(struct rte_eth_dev *dev,
+		     struct rte_eth_xstat_name *xstats_names,
+		     unsigned int xstats_count)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	struct sfc_port *port = &sa->port;
+	unsigned int i;
+	unsigned int nstats = 0;
+
+	for (i = 0; i < EFX_MAC_NSTATS; ++i) {
+		if (EFX_MAC_STAT_SUPPORTED(port->mac_stats_mask, i)) {
+			if (xstats_names != NULL && nstats < xstats_count)
+				strncpy(xstats_names[nstats].name,
+					efx_mac_stat_name(sa->nic, i),
+					sizeof(xstats_names[0].name));
+			nstats++;
+		}
+	}
+
+	return nstats;
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -399,6 +460,8 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_close			= sfc_dev_close,
 	.link_update			= sfc_dev_link_update,
 	.stats_get			= sfc_stats_get,
+	.xstats_get			= sfc_xstats_get,
+	.xstats_get_names		= sfc_xstats_get_names,
 	.dev_infos_get			= sfc_dev_infos_get,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
-- 
2.5.5

[PATCH 05/31] net/sfc: support flow control settings get/set

[Andrew Rybchenko]

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 +
 drivers/net/sfc/sfc_ethdev.c         | 98 ++++++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_port.c           |  8 +++
 4 files changed, 109 insertions(+)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 698553c..25472f8 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -5,6 +5,7 @@
 ;
 [Features]
 Link status          = Y
+Flow control         = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
 Basic stats          = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index cbb51de..1cfed6a 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -50,6 +50,8 @@ SFC EFX PMD has support for:
 
 - Port hardware statistics
 
+- Basic flow control
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index d5ae1a0..eff648b 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -453,6 +453,102 @@ sfc_xstats_get_names(struct rte_eth_dev *dev,
 	return nstats;
 }
 
+static int
+sfc_flow_ctrl_get(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	unsigned int wanted_fc, link_fc;
+
+	memset(fc_conf, 0, sizeof(*fc_conf));
+
+	sfc_adapter_lock(sa);
+
+	if (sa->state == SFC_ADAPTER_STARTED)
+		efx_mac_fcntl_get(sa->nic, &wanted_fc, &link_fc);
+	else
+		link_fc = sa->port.flow_ctrl;
+
+	switch (link_fc) {
+	case 0:
+		fc_conf->mode = RTE_FC_NONE;
+		break;
+	case EFX_FCNTL_RESPOND:
+		fc_conf->mode = RTE_FC_RX_PAUSE;
+		break;
+	case EFX_FCNTL_GENERATE:
+		fc_conf->mode = RTE_FC_TX_PAUSE;
+		break;
+	case (EFX_FCNTL_RESPOND | EFX_FCNTL_GENERATE):
+		fc_conf->mode = RTE_FC_FULL;
+		break;
+	default:
+		sfc_err(sa, "%s: unexpected flow control value %#x",
+			__func__, link_fc);
+	}
+
+	fc_conf->autoneg = sa->port.flow_ctrl_autoneg;
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+}
+
+static int
+sfc_flow_ctrl_set(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	struct sfc_port *port = &sa->port;
+	unsigned int fcntl;
+	int rc;
+
+	if (fc_conf->high_water != 0 || fc_conf->low_water != 0 ||
+	    fc_conf->pause_time != 0 || fc_conf->send_xon != 0 ||
+	    fc_conf->mac_ctrl_frame_fwd != 0) {
+		sfc_err(sa, "unsupported flow control settings specified");
+		rc = EINVAL;
+		goto fail_inval;
+	}
+
+	switch (fc_conf->mode) {
+	case RTE_FC_NONE:
+		fcntl = 0;
+		break;
+	case RTE_FC_RX_PAUSE:
+		fcntl = EFX_FCNTL_RESPOND;
+		break;
+	case RTE_FC_TX_PAUSE:
+		fcntl = EFX_FCNTL_GENERATE;
+		break;
+	case RTE_FC_FULL:
+		fcntl = EFX_FCNTL_RESPOND | EFX_FCNTL_GENERATE;
+		break;
+	default:
+		rc = EINVAL;
+		goto fail_inval;
+	}
+
+	sfc_adapter_lock(sa);
+
+	if (sa->state == SFC_ADAPTER_STARTED) {
+		rc = efx_mac_fcntl_set(sa->nic, fcntl, fc_conf->autoneg);
+		if (rc != 0)
+			goto fail_mac_fcntl_set;
+	}
+
+	port->flow_ctrl = fcntl;
+	port->flow_ctrl_autoneg = fc_conf->autoneg;
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+
+fail_mac_fcntl_set:
+	sfc_adapter_unlock(sa);
+fail_inval:
+	SFC_ASSERT(rc > 0);
+	return -rc;
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -467,6 +563,8 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.rx_queue_release		= sfc_rx_queue_release,
 	.tx_queue_setup			= sfc_tx_queue_setup,
 	.tx_queue_release		= sfc_tx_queue_release,
+	.flow_ctrl_get			= sfc_flow_ctrl_get,
+	.flow_ctrl_set			= sfc_flow_ctrl_set,
 };
 
 static int
diff --git a/drivers/net/sfc/sfc_port.c b/drivers/net/sfc/sfc_port.c
index d8ff097..ccc0854 100644
--- a/drivers/net/sfc/sfc_port.c
+++ b/drivers/net/sfc/sfc_port.c
@@ -79,6 +79,13 @@ sfc_port_start(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_port_init;
 
+	sfc_log_init(sa, "set flow control to %#x autoneg=%u",
+		     port->flow_ctrl, port->flow_ctrl_autoneg);
+	rc = efx_mac_fcntl_set(sa->nic, port->flow_ctrl,
+			       port->flow_ctrl_autoneg);
+	if (rc != 0)
+		goto fail_mac_fcntl_set;
+
 	sfc_log_init(sa, "set MAC PDU %u", (unsigned int)port->pdu);
 	rc = efx_mac_pdu_set(sa->nic, port->pdu);
 	if (rc != 0)
@@ -124,6 +131,7 @@ sfc_port_start(struct sfc_adapter *sa)
 fail_mac_filter_set:
 fail_mac_addr_set:
 fail_mac_pdu_set:
+fail_mac_fcntl_set:
 	efx_port_fini(sa->nic);
 
 fail_port_init:
-- 
2.5.5

[PATCH 06/31] net/sfc: support link status change interrupt

[Andrew Rybchenko]

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |   1 +
 doc/guides/nics/sfc_efx.rst          |   4 +-
 drivers/net/sfc/sfc.c                |   4 +-
 drivers/net/sfc/sfc.h                |   4 +
 drivers/net/sfc/sfc_ethdev.c         |   1 +
 drivers/net/sfc/sfc_ev.c             |  30 +++++-
 drivers/net/sfc/sfc_intr.c           | 204 +++++++++++++++++++++++++++++++++++
 7 files changed, 242 insertions(+), 6 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 25472f8..693d35e 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -5,6 +5,7 @@
 ;
 [Features]
 Link status          = Y
+Link status event    = Y
 Flow control         = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 1cfed6a..94eedd1 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -44,7 +44,7 @@ SFC EFX PMD has support for:
 
 - Multiple transmit and receive queues
 
-- Link state information
+- Link state information including link status change interrupt
 
 - IPv4/IPv6 TCP/UDP transmit checksum offload
 
@@ -58,8 +58,6 @@ Non-supported Features
 
 The features not yet supported include:
 
-- Link status change interrupt
-
 - Receive queue interupts
 
 - Priority-based flow control
diff --git a/drivers/net/sfc/sfc.c b/drivers/net/sfc/sfc.c
index ef9e0d4..36044a0 100644
--- a/drivers/net/sfc/sfc.c
+++ b/drivers/net/sfc/sfc.c
@@ -116,7 +116,9 @@ sfc_check_conf(struct sfc_adapter *sa)
 		rc = EINVAL;
 	}
 
-	if (conf->intr_conf.lsc != 0) {
+	if ((conf->intr_conf.lsc != 0) &&
+	    (sa->intr.type != EFX_INTR_LINE) &&
+	    (sa->intr.type != EFX_INTR_MESSAGE)) {
 		sfc_err(sa, "Link status change interrupt not supported");
 		rc = EINVAL;
 	}
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index 1189283..257622f 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -112,6 +112,8 @@ struct sfc_mcdi {
 
 struct sfc_intr {
 	efx_intr_type_t			type;
+	rte_intr_callback_fn		handler;
+	boolean_t			lsc_intr;
 };
 
 struct sfc_evq_info;
@@ -119,6 +121,8 @@ struct sfc_rxq_info;
 struct sfc_txq_info;
 
 struct sfc_port {
+	unsigned int			lsc_seq;
+
 	unsigned int			flow_ctrl;
 	boolean_t			flow_ctrl_autoneg;
 	size_t				pdu;
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index eff648b..8c46500 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -682,6 +682,7 @@ static struct eth_driver sfc_efx_pmd = {
 	.pci_drv = {
 		.id_table = pci_id_sfc_efx_map,
 		.drv_flags =
+			RTE_PCI_DRV_INTR_LSC |
 			RTE_PCI_DRV_NEED_MAPPING,
 		.probe = rte_eth_dev_pci_probe,
 		.remove = rte_eth_dev_pci_remove,
diff --git a/drivers/net/sfc/sfc_ev.c b/drivers/net/sfc/sfc_ev.c
index 34c1127..c8a2d23 100644
--- a/drivers/net/sfc/sfc_ev.c
+++ b/drivers/net/sfc/sfc_ev.c
@@ -286,11 +286,25 @@ sfc_ev_link_change(void *arg, efx_link_mode_t link_mode)
 	struct sfc_adapter *sa = evq->sa;
 	struct rte_eth_link *dev_link = &sa->eth_dev->data->dev_link;
 	struct rte_eth_link new_link;
+	uint64_t new_link_u64;
+	uint64_t old_link_u64;
 
 	EFX_STATIC_ASSERT(sizeof(*dev_link) == sizeof(rte_atomic64_t));
 
 	sfc_port_link_mode_to_info(link_mode, &new_link);
-	rte_atomic64_set((rte_atomic64_t *)dev_link, *(uint64_t *)&new_link);
+
+	new_link_u64 = *(uint64_t *)&new_link;
+	do {
+		old_link_u64 = rte_atomic64_read((rte_atomic64_t *)dev_link);
+		if (old_link_u64 == new_link_u64)
+			break;
+
+		if (rte_atomic64_cmpset((volatile uint64_t *)dev_link,
+					old_link_u64, new_link_u64)) {
+			evq->sa->port.lsc_seq++;
+			break;
+		}
+	} while (B_TRUE);
 
 	return B_FALSE;
 }
@@ -481,6 +495,12 @@ sfc_ev_start(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_mgmt_evq_start;
 
+	if (sa->intr.lsc_intr) {
+		rc = sfc_ev_qprime(sa->evq_info[sa->mgmt_evq_index].evq);
+		if (rc != 0)
+			goto fail_evq0_prime;
+	}
+
 	rte_spinlock_unlock(&sa->mgmt_evq_lock);
 
 	/*
@@ -498,6 +518,9 @@ sfc_ev_start(struct sfc_adapter *sa)
 
 	return 0;
 
+fail_evq0_prime:
+	sfc_ev_qstop(sa, 0);
+
 fail_mgmt_evq_start:
 	rte_spinlock_unlock(&sa->mgmt_evq_lock);
 	efx_ev_fini(sa->nic);
@@ -599,7 +622,10 @@ sfc_ev_qinit_info(struct sfc_adapter *sa, unsigned int sw_index)
 	SFC_ASSERT(rte_is_power_of_2(max_entries));
 
 	evq_info->max_entries = max_entries;
-	evq_info->flags = sa->evq_flags | EFX_EVQ_FLAGS_NOTIFY_DISABLED;
+	evq_info->flags = sa->evq_flags |
+		((sa->intr.lsc_intr && sw_index == sa->mgmt_evq_index) ?
+			EFX_EVQ_FLAGS_NOTIFY_INTERRUPT :
+			EFX_EVQ_FLAGS_NOTIFY_DISABLED);
 
 	return 0;
 }
diff --git a/drivers/net/sfc/sfc_intr.c b/drivers/net/sfc/sfc_intr.c
index 1b7dcdd..e0b1693 100644
--- a/drivers/net/sfc/sfc_intr.c
+++ b/drivers/net/sfc/sfc_intr.c
@@ -27,10 +27,130 @@
  * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+/*
+ * At the momemt of writing DPDK v16.07 has notion of two types of
+ * interrupts: LSC (link status change) and RXQ (receive indication).
+ * It allows to register interrupt callback for entire device which is
+ * not intended to be used for receive indication (i.e. link status
+ * change indication only). The handler has no information which HW
+ * interrupt has triggered it, so we don't know which event queue should
+ * be polled/reprimed (except qmask in the case of legacy line interrupt).
+ */
+
+#include <rte_common.h>
+#include <rte_interrupts.h>
+
 #include "efx.h"
 
 #include "sfc.h"
 #include "sfc_log.h"
+#include "sfc_ev.h"
+
+static void
+sfc_intr_handle_mgmt_evq(struct sfc_adapter *sa)
+{
+	struct sfc_evq *evq;
+
+	rte_spinlock_lock(&sa->mgmt_evq_lock);
+
+	evq = sa->evq_info[sa->mgmt_evq_index].evq;
+
+	if (evq->init_state != SFC_EVQ_STARTED) {
+		sfc_log_init(sa, "interrupt on stopped EVQ %u", evq->evq_index);
+	} else {
+		sfc_ev_qpoll(evq);
+
+		if (sfc_ev_qprime(evq) != 0)
+			sfc_err(sa, "cannot prime EVQ %u", evq->evq_index);
+	}
+
+	rte_spinlock_unlock(&sa->mgmt_evq_lock);
+}
+
+static void
+sfc_intr_line_handler(struct rte_intr_handle *intr_handle, void *cb_arg)
+{
+	struct sfc_adapter *sa = (struct sfc_adapter *)cb_arg;
+	efx_nic_t *enp = sa->nic;
+	boolean_t fatal;
+	uint32_t qmask;
+	unsigned int lsc_seq = sa->port.lsc_seq;
+
+	sfc_log_init(sa, "entry");
+
+	if (sa->state != SFC_ADAPTER_STARTED &&
+	    sa->state != SFC_ADAPTER_STARTING &&
+	    sa->state != SFC_ADAPTER_STOPPING) {
+		sfc_log_init(sa,
+			     "interrupt on stopped adapter, don't reenable");
+		goto exit;
+	}
+
+	efx_intr_status_line(enp, &fatal, &qmask);
+	if (fatal) {
+		(void)efx_intr_disable(enp);
+		(void)efx_intr_fatal(enp);
+		sfc_err(sa, "fatal, interrupts disabled");
+		goto exit;
+	}
+
+	if (qmask & (1 << sa->mgmt_evq_index))
+		sfc_intr_handle_mgmt_evq(sa);
+
+	if (rte_intr_enable(intr_handle) != 0)
+		sfc_err(sa, "cannot reenable interrupts");
+
+	sfc_log_init(sa, "done");
+
+exit:
+	if (lsc_seq != sa->port.lsc_seq) {
+		sfc_info(sa, "link status change event: link %s",
+			 sa->eth_dev->data->dev_link.link_status ?
+			 "UP" : "DOWN");
+		_rte_eth_dev_callback_process(sa->eth_dev,
+					      RTE_ETH_EVENT_INTR_LSC, NULL);
+	}
+}
+
+static void
+sfc_intr_message_handler(struct rte_intr_handle *intr_handle, void *cb_arg)
+{
+	struct sfc_adapter *sa = (struct sfc_adapter *)cb_arg;
+	efx_nic_t *enp = sa->nic;
+	boolean_t fatal;
+	unsigned int lsc_seq = sa->port.lsc_seq;
+
+	sfc_log_init(sa, "entry");
+
+	if (sa->state != SFC_ADAPTER_STARTED &&
+	    sa->state != SFC_ADAPTER_STARTING &&
+	    sa->state != SFC_ADAPTER_STOPPING) {
+		sfc_log_init(sa, "adapter not-started, don't reenable");
+		goto exit;
+	}
+
+	efx_intr_status_message(enp, sa->mgmt_evq_index, &fatal);
+	if (fatal) {
+		(void)efx_intr_disable(enp);
+		(void)efx_intr_fatal(enp);
+		sfc_err(sa, "fatal, interrupts disabled");
+		goto exit;
+	}
+
+	sfc_intr_handle_mgmt_evq(sa);
+
+	if (rte_intr_enable(intr_handle) != 0)
+		sfc_err(sa, "cannot reenable interrupts");
+
+	sfc_log_init(sa, "done");
+
+exit:
+	if (lsc_seq != sa->port.lsc_seq) {
+		sfc_info(sa, "link status change event");
+		_rte_eth_dev_callback_process(sa->eth_dev,
+					      RTE_ETH_EVENT_INTR_LSC, NULL);
+	}
+}
 
 int
 sfc_intr_start(struct sfc_adapter *sa)
@@ -54,11 +174,49 @@ sfc_intr_start(struct sfc_adapter *sa)
 
 	intr_handle = &sa->eth_dev->pci_dev->intr_handle;
 
+	if (intr->handler != NULL) {
+		sfc_log_init(sa, "rte_intr_callback_register");
+		rc = rte_intr_callback_register(intr_handle, intr->handler,
+						(void *)sa);
+		if (rc != 0) {
+			sfc_err(sa,
+				"cannot register interrupt handler (rc=%d)",
+				rc);
+			/*
+			 * Convert error code from negative returned by RTE API
+			 * to positive used in the driver.
+			 */
+			rc = -rc;
+			goto fail_rte_intr_cb_reg;
+		}
+
+		sfc_log_init(sa, "rte_intr_enable");
+		rc = rte_intr_enable(intr_handle);
+		if (rc != 0) {
+			sfc_err(sa, "cannot enable interrupts (rc=%d)", rc);
+			/*
+			 * Convert error code from negative returned by RTE API
+			 * to positive used in the driver.
+			 */
+			rc = -rc;
+			goto fail_rte_intr_enable;
+		}
+
+		sfc_log_init(sa, "efx_intr_enable");
+		efx_intr_enable(sa->nic);
+	}
+
 	sfc_log_init(sa, "done type=%u max_intr=%d nb_efd=%u vec=%p",
 		     intr_handle->type, intr_handle->max_intr,
 		     intr_handle->nb_efd, intr_handle->intr_vec);
 	return 0;
 
+fail_rte_intr_enable:
+	rte_intr_callback_unregister(intr_handle, intr->handler, (void *)sa);
+
+fail_rte_intr_cb_reg:
+	efx_intr_fini(sa->nic);
+
 fail_intr_init:
 	sfc_log_init(sa, "failed %d", rc);
 	return rc;
@@ -67,8 +225,29 @@ sfc_intr_start(struct sfc_adapter *sa)
 void
 sfc_intr_stop(struct sfc_adapter *sa)
 {
+	struct sfc_intr *intr = &sa->intr;
+
 	sfc_log_init(sa, "entry");
 
+	if (intr->handler != NULL) {
+		struct rte_intr_handle *intr_handle;
+		int rc;
+
+		efx_intr_disable(sa->nic);
+
+		intr_handle = &sa->eth_dev->pci_dev->intr_handle;
+		if (rte_intr_disable(intr_handle) != 0)
+			sfc_err(sa, "cannot disable interrupts");
+
+		while ((rc = rte_intr_callback_unregister(intr_handle,
+				intr->handler, (void *)sa)) == -EAGAIN)
+			;
+		if (rc != 1)
+			sfc_err(sa,
+				"cannot unregister interrupt handler %d",
+				rc);
+	}
+
 	efx_intr_fini(sa->nic);
 
 	sfc_log_init(sa, "done");
@@ -77,8 +256,33 @@ sfc_intr_stop(struct sfc_adapter *sa)
 int
 sfc_intr_init(struct sfc_adapter *sa)
 {
+	struct sfc_intr *intr = &sa->intr;
+
 	sfc_log_init(sa, "entry");
 
+	intr->handler = NULL;
+	intr->lsc_intr = (sa->eth_dev->data->dev_conf.intr_conf.lsc != 0);
+	if (!intr->lsc_intr) {
+		sfc_info(sa, "LSC tracking using interrupts is disabled");
+		goto done;
+	}
+
+	switch (intr->type) {
+	case EFX_INTR_MESSAGE:
+		intr->handler = sfc_intr_message_handler;
+		break;
+	case EFX_INTR_LINE:
+		intr->handler = sfc_intr_line_handler;
+		break;
+	case EFX_INTR_INVALID:
+		sfc_warn(sa, "interrupts are not supported");
+		break;
+	default:
+		sfc_panic(sa, "unexpected EFX interrupt type %u\n", intr->type);
+		break;
+	}
+
+done:
 	sfc_log_init(sa, "done");
 	return 0;
 }
-- 
2.5.5

[PATCH 07/31] net/sfc: implement device operation to change MTU

[Andrew Rybchenko]

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  2 ++
 doc/guides/nics/sfc_efx.rst          |  4 +++
 drivers/net/sfc/sfc_ethdev.c         | 67 ++++++++++++++++++++++++++++++++++++
 3 files changed, 73 insertions(+)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 693d35e..a845bfc 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -6,6 +6,8 @@
 [Features]
 Link status          = Y
 Link status event    = Y
+MTU update           = Y
+Jumbo frame          = Y
 Flow control         = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 94eedd1..adab9fd 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -52,6 +52,10 @@ SFC EFX PMD has support for:
 
 - Basic flow control
 
+- MTU update
+
+- Jumbo frames up to 9K
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 8c46500..6690755 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -549,6 +549,72 @@ sfc_flow_ctrl_set(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf)
 	return -rc;
 }
 
+static int
+sfc_dev_set_mtu(struct rte_eth_dev *dev, uint16_t mtu)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	size_t pdu = EFX_MAC_PDU(mtu);
+	size_t old_pdu;
+	int rc;
+
+	sfc_log_init(sa, "mtu=%u", mtu);
+
+	rc = EINVAL;
+	if (pdu < EFX_MAC_PDU_MIN) {
+		sfc_err(sa, "too small MTU %u (PDU size %u less than min %u)",
+			(unsigned int)mtu, (unsigned int)pdu,
+			EFX_MAC_PDU_MIN);
+		goto fail_inval;
+	}
+	if (pdu > EFX_MAC_PDU_MAX) {
+		sfc_err(sa, "too big MTU %u (PDU size %u greater than max %u)",
+			(unsigned int)mtu, (unsigned int)pdu,
+			EFX_MAC_PDU_MAX);
+		goto fail_inval;
+	}
+
+	sfc_adapter_lock(sa);
+
+	if (pdu != sa->port.pdu) {
+		if (sa->state == SFC_ADAPTER_STARTED) {
+			sfc_stop(sa);
+
+			old_pdu = sa->port.pdu;
+			sa->port.pdu = pdu;
+			rc = sfc_start(sa);
+			if (rc != 0)
+				goto fail_start;
+		} else {
+			sa->port.pdu = pdu;
+		}
+	}
+
+	/*
+	 * The driver does not use it, but other PMDs update jumbo_frame
+	 * flag and max_rx_pkt_len when MTU is set.
+	 */
+	dev->data->dev_conf.rxmode.jumbo_frame = (mtu > ETHER_MAX_LEN);
+	dev->data->dev_conf.rxmode.max_rx_pkt_len = sa->port.pdu;
+
+	sfc_adapter_unlock(sa);
+
+	sfc_log_init(sa, "done");
+	return 0;
+
+fail_start:
+	sa->port.pdu = old_pdu;
+	if (sfc_start(sa) != 0)
+		sfc_err(sa, "cannot start with neither new (%u) nor old (%u) "
+			"PDU max size - port is stopped",
+			(unsigned int)pdu, (unsigned int)old_pdu);
+	sfc_adapter_unlock(sa);
+
+fail_inval:
+	sfc_log_init(sa, "failed %d", rc);
+	SFC_ASSERT(rc > 0);
+	return -rc;
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -559,6 +625,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.xstats_get			= sfc_xstats_get,
 	.xstats_get_names		= sfc_xstats_get_names,
 	.dev_infos_get			= sfc_dev_infos_get,
+	.mtu_set			= sfc_dev_set_mtu,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
 	.tx_queue_setup			= sfc_tx_queue_setup,
-- 
2.5.5

[PATCH 08/31] net/sfc: support link speed and duplex settings

[Andrew Rybchenko]

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 drivers/net/sfc/sfc.c                | 38 ++++++++++++++++++++++++++++++++++--
 drivers/net/sfc/sfc.h                |  3 +++
 drivers/net/sfc/sfc_ethdev.c         |  9 +++++++++
 drivers/net/sfc/sfc_port.c           |  6 ++++++
 5 files changed, 55 insertions(+), 2 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index a845bfc..60ecca0 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -4,6 +4,7 @@
 ; Refer to default.ini for the full list of available PMD features.
 ;
 [Features]
+Speed capabilities   = Y
 Link status          = Y
 Link status event    = Y
 MTU update           = Y
diff --git a/drivers/net/sfc/sfc.c b/drivers/net/sfc/sfc.c
index 36044a0..e2e6c9e 100644
--- a/drivers/net/sfc/sfc.c
+++ b/drivers/net/sfc/sfc.c
@@ -85,6 +85,33 @@ sfc_dma_free(const struct sfc_adapter *sa, efsys_mem_t *esmp)
 	memset(esmp, 0, sizeof(*esmp));
 }
 
+static uint32_t
+sfc_phy_cap_from_link_speeds(uint32_t speeds)
+{
+	uint32_t phy_caps = 0;
+
+	if (~speeds & ETH_LINK_SPEED_FIXED) {
+		phy_caps |= (1 << EFX_PHY_CAP_AN);
+		/*
+		 * If no speeds are specified in the mask, any supported
+		 * may be negotiated
+		 */
+		if (speeds == ETH_LINK_SPEED_AUTONEG)
+			phy_caps |=
+				(1 << EFX_PHY_CAP_1000FDX) |
+				(1 << EFX_PHY_CAP_10000FDX) |
+				(1 << EFX_PHY_CAP_40000FDX);
+	}
+	if (speeds & ETH_LINK_SPEED_1G)
+		phy_caps |= (1 << EFX_PHY_CAP_1000FDX);
+	if (speeds & ETH_LINK_SPEED_10G)
+		phy_caps |= (1 << EFX_PHY_CAP_10000FDX);
+	if (speeds & ETH_LINK_SPEED_40G)
+		phy_caps |= (1 << EFX_PHY_CAP_40000FDX);
+
+	return phy_caps;
+}
+
 /*
  * Check requested device level configuration.
  * Receive and transmit configuration is checked in corresponding
@@ -96,8 +123,12 @@ sfc_check_conf(struct sfc_adapter *sa)
 	const struct rte_eth_conf *conf = &sa->eth_dev->data->dev_conf;
 	int rc = 0;
 
-	if (conf->link_speeds != ETH_LINK_SPEED_AUTONEG) {
-		sfc_err(sa, "Manual link speed/duplex choice not supported");
+	sa->port.phy_adv_cap =
+		sfc_phy_cap_from_link_speeds(conf->link_speeds) &
+		sa->port.phy_adv_cap_mask;
+	if ((sa->port.phy_adv_cap & ~(1 << EFX_PHY_CAP_AN)) == 0) {
+		sfc_err(sa, "No link speeds from mask %#x are supported",
+			conf->link_speeds);
 		rc = EINVAL;
 	}
 
@@ -516,6 +547,9 @@ sfc_attach(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_intr_attach;
 
+	efx_phy_adv_cap_get(sa->nic, EFX_PHY_CAP_PERM,
+			    &sa->port.phy_adv_cap_mask);
+
 	sfc_log_init(sa, "fini nic");
 	efx_nic_fini(enp);
 
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index 257622f..5883547 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -123,6 +123,9 @@ struct sfc_txq_info;
 struct sfc_port {
 	unsigned int			lsc_seq;
 
+	uint32_t			phy_adv_cap_mask;
+	uint32_t			phy_adv_cap;
+
 	unsigned int			flow_ctrl;
 	boolean_t			flow_ctrl_autoneg;
 	size_t				pdu;
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 6690755..42c488e 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -53,6 +53,15 @@ sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 
 	dev_info->max_rx_pktlen = EFX_MAC_PDU_MAX;
 
+	/* Autonegotiation may be disabled */
+	dev_info->speed_capa = ETH_LINK_SPEED_FIXED;
+	if (sa->port.phy_adv_cap_mask & EFX_PHY_CAP_1000FDX)
+		dev_info->speed_capa |= ETH_LINK_SPEED_1G;
+	if (sa->port.phy_adv_cap_mask & EFX_PHY_CAP_10000FDX)
+		dev_info->speed_capa |= ETH_LINK_SPEED_10G;
+	if (sa->port.phy_adv_cap_mask & EFX_PHY_CAP_40000FDX)
+		dev_info->speed_capa |= ETH_LINK_SPEED_40G;
+
 	dev_info->max_rx_queues = sa->rxq_max;
 	dev_info->max_tx_queues = sa->txq_max;
 
diff --git a/drivers/net/sfc/sfc_port.c b/drivers/net/sfc/sfc_port.c
index ccc0854..1241af7 100644
--- a/drivers/net/sfc/sfc_port.c
+++ b/drivers/net/sfc/sfc_port.c
@@ -86,6 +86,11 @@ sfc_port_start(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_mac_fcntl_set;
 
+	sfc_log_init(sa, "set phy adv caps to %#x", port->phy_adv_cap);
+	rc = efx_phy_adv_cap_set(sa->nic, port->phy_adv_cap);
+	if (rc != 0)
+		goto fail_phy_adv_cap_set;
+
 	sfc_log_init(sa, "set MAC PDU %u", (unsigned int)port->pdu);
 	rc = efx_mac_pdu_set(sa->nic, port->pdu);
 	if (rc != 0)
@@ -131,6 +136,7 @@ sfc_port_start(struct sfc_adapter *sa)
 fail_mac_filter_set:
 fail_mac_addr_set:
 fail_mac_pdu_set:
+fail_phy_adv_cap_set:
 fail_mac_fcntl_set:
 	efx_port_fini(sa->nic);
 
-- 
2.5.5

[PATCH 09/31] net/sfc: support link up/down

[Andrew Rybchenko]

From: Artem Andreev <Artem.Andreev@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Artem Andreev <Artem.Andreev@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 42c488e..1716d78 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -195,6 +195,36 @@ sfc_dev_stop(struct rte_eth_dev *dev)
 	sfc_log_init(sa, "done");
 }
 
+static int
+sfc_dev_set_link_up(struct rte_eth_dev *dev)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	int rc;
+
+	sfc_log_init(sa, "entry");
+
+	sfc_adapter_lock(sa);
+	rc = sfc_start(sa);
+	sfc_adapter_unlock(sa);
+
+	SFC_ASSERT(rc >= 0);
+	return -rc;
+}
+
+static int
+sfc_dev_set_link_down(struct rte_eth_dev *dev)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+
+	sfc_log_init(sa, "entry");
+
+	sfc_adapter_lock(sa);
+	sfc_stop(sa);
+	sfc_adapter_unlock(sa);
+
+	return 0;
+}
+
 static void
 sfc_dev_close(struct rte_eth_dev *dev)
 {
@@ -628,6 +658,8 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
 	.dev_stop			= sfc_dev_stop,
+	.dev_set_link_up		= sfc_dev_set_link_up,
+	.dev_set_link_down		= sfc_dev_set_link_down,
 	.dev_close			= sfc_dev_close,
 	.link_update			= sfc_dev_link_update,
 	.stats_get			= sfc_stats_get,
-- 
2.5.5

[PATCH 10/31] net/sfc: support promiscuous and all-multicast control

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  2 ++
 doc/guides/nics/sfc_efx.rst          |  4 +++
 drivers/net/sfc/sfc.h                | 11 +++++++
 drivers/net/sfc/sfc_ethdev.c         | 57 ++++++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_port.c           | 18 +++++++++++-
 5 files changed, 91 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 60ecca0..aaea993 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -9,6 +9,8 @@ Link status          = Y
 Link status event    = Y
 MTU update           = Y
 Jumbo frame          = Y
+Promiscuous mode     = Y
+Allmulticast mode    = Y
 Flow control         = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index adab9fd..360ed98 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -56,6 +56,10 @@ SFC EFX PMD has support for:
 
 - Jumbo frames up to 9K
 
+- Promiscuous mode
+
+- Allmulticast mode
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index 5883547..f0bbaf8 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -93,6 +93,13 @@ enum sfc_adapter_state {
 	SFC_ADAPTER_NSTATES
 };
 
+enum sfc_dev_filter_mode {
+	SFC_DEV_FILTER_MODE_PROMISC = 0,
+	SFC_DEV_FILTER_MODE_ALLMULTI,
+
+	SFC_DEV_FILTER_NMODES
+};
+
 enum sfc_mcdi_state {
 	SFC_MCDI_UNINITIALIZED = 0,
 	SFC_MCDI_INITIALIZED,
@@ -130,6 +137,9 @@ struct sfc_port {
 	boolean_t			flow_ctrl_autoneg;
 	size_t				pdu;
 
+	boolean_t			promisc;
+	boolean_t			allmulti;
+
 	rte_spinlock_t			mac_stats_lock;
 	uint64_t			*mac_stats_buf;
 	efsys_mem_t			mac_stats_dma_mem;
@@ -243,6 +253,7 @@ void sfc_port_stop(struct sfc_adapter *sa);
 void sfc_port_link_mode_to_info(efx_link_mode_t link_mode,
 				struct rte_eth_link *link_info);
 int sfc_port_update_mac_stats(struct sfc_adapter *sa);
+int sfc_set_rx_mode(struct sfc_adapter *sa);
 
 
 #ifdef __cplusplus
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 1716d78..c6095ad 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -253,6 +253,59 @@ sfc_dev_close(struct rte_eth_dev *dev)
 	sfc_log_init(sa, "done");
 }
 
+static void
+sfc_dev_filter_set(struct rte_eth_dev *dev, enum sfc_dev_filter_mode mode,
+		   boolean_t enabled)
+{
+	struct sfc_port *port;
+	boolean_t *toggle;
+	struct sfc_adapter *sa = dev->data->dev_private;
+	boolean_t allmulti = (mode == SFC_DEV_FILTER_MODE_ALLMULTI);
+	const char *desc = (allmulti) ? "all-multi" : "promiscuous";
+
+	sfc_adapter_lock(sa);
+
+	port = &sa->port;
+	toggle = (allmulti) ? (&port->allmulti) : (&port->promisc);
+
+	if (*toggle != enabled) {
+		*toggle = enabled;
+
+		if ((sa->state == SFC_ADAPTER_STARTED) &&
+		    (sfc_set_rx_mode(sa) != 0)) {
+			*toggle = !(enabled);
+			sfc_warn(sa, "Failed to %s %s mode",
+				 ((enabled) ? "enable" : "disable"), desc);
+		}
+	}
+
+	sfc_adapter_unlock(sa);
+}
+
+static void
+sfc_dev_promisc_enable(struct rte_eth_dev *dev)
+{
+	sfc_dev_filter_set(dev, SFC_DEV_FILTER_MODE_PROMISC, B_TRUE);
+}
+
+static void
+sfc_dev_promisc_disable(struct rte_eth_dev *dev)
+{
+	sfc_dev_filter_set(dev, SFC_DEV_FILTER_MODE_PROMISC, B_FALSE);
+}
+
+static void
+sfc_dev_allmulti_enable(struct rte_eth_dev *dev)
+{
+	sfc_dev_filter_set(dev, SFC_DEV_FILTER_MODE_ALLMULTI, B_TRUE);
+}
+
+static void
+sfc_dev_allmulti_disable(struct rte_eth_dev *dev)
+{
+	sfc_dev_filter_set(dev, SFC_DEV_FILTER_MODE_ALLMULTI, B_FALSE);
+}
+
 static int
 sfc_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 		   uint16_t nb_rx_desc, unsigned int socket_id,
@@ -661,6 +714,10 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_set_link_up		= sfc_dev_set_link_up,
 	.dev_set_link_down		= sfc_dev_set_link_down,
 	.dev_close			= sfc_dev_close,
+	.promiscuous_enable		= sfc_dev_promisc_enable,
+	.promiscuous_disable		= sfc_dev_promisc_disable,
+	.allmulticast_enable		= sfc_dev_allmulti_enable,
+	.allmulticast_disable		= sfc_dev_allmulti_disable,
 	.link_update			= sfc_dev_link_update,
 	.stats_get			= sfc_stats_get,
 	.xstats_get			= sfc_xstats_get,
diff --git a/drivers/net/sfc/sfc_port.c b/drivers/net/sfc/sfc_port.c
index 1241af7..dc6ecdf 100644
--- a/drivers/net/sfc/sfc_port.c
+++ b/drivers/net/sfc/sfc_port.c
@@ -103,7 +103,11 @@ sfc_port_start(struct sfc_adapter *sa)
 		goto fail_mac_addr_set;
 
 	sfc_log_init(sa, "set MAC filters");
-	rc = efx_mac_filter_set(sa->nic, B_TRUE, B_TRUE, B_TRUE, B_TRUE);
+	port->promisc = (sa->eth_dev->data->promiscuous != 0) ?
+			B_TRUE : B_FALSE;
+	port->allmulti = (sa->eth_dev->data->all_multicast != 0) ?
+			 B_TRUE : B_FALSE;
+	rc = sfc_set_rx_mode(sa);
 	if (rc != 0)
 		goto fail_mac_filter_set;
 
@@ -219,6 +223,18 @@ sfc_port_fini(struct sfc_adapter *sa)
 	sfc_log_init(sa, "done");
 }
 
+int
+sfc_set_rx_mode(struct sfc_adapter *sa)
+{
+	struct sfc_port *port = &sa->port;
+	int rc;
+
+	rc = efx_mac_filter_set(sa->nic, port->promisc, B_TRUE,
+				port->promisc || port->allmulti, B_TRUE);
+
+	return rc;
+}
+
 void
 sfc_port_link_mode_to_info(efx_link_mode_t link_mode,
 			   struct rte_eth_link *link_info)
-- 
2.5.5

[PATCH 11/31] net/sfc: support main (the first) MAC address change

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 53 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index c6095ad..79e2761 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -706,6 +706,58 @@ sfc_dev_set_mtu(struct rte_eth_dev *dev, uint16_t mtu)
 	SFC_ASSERT(rc > 0);
 	return -rc;
 }
+static void
+sfc_mac_addr_set(struct rte_eth_dev *dev, struct ether_addr *mac_addr)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	const efx_nic_cfg_t *encp = efx_nic_cfg_get(sa->nic);
+	int rc;
+
+	sfc_adapter_lock(sa);
+
+	if (sa->state != SFC_ADAPTER_STARTED) {
+		sfc_info(sa, "the port is not started");
+		sfc_info(sa, "the new MAC address will be set on port start");
+
+		goto unlock;
+	}
+
+	if (encp->enc_allow_set_mac_with_installed_filters) {
+		rc = efx_mac_addr_set(sa->nic, mac_addr->addr_bytes);
+		if (rc != 0) {
+			sfc_err(sa, "cannot set MAC address (rc = %u)", rc);
+			goto unlock;
+		}
+
+		/*
+		 * Changing the MAC address by means of MCDI request
+		 * has no effect on received traffic, therefore
+		 * we also need to update unicast filters
+		 */
+		rc = sfc_set_rx_mode(sa);
+		if (rc != 0)
+			sfc_err(sa, "cannot set filter (rc = %u)", rc);
+	} else {
+		sfc_warn(sa, "cannot set MAC address with filters installed");
+		sfc_warn(sa, "adapter will be restarted to pick the new MAC");
+		sfc_warn(sa, "(some traffic may be dropped)");
+
+		/*
+		 * Since setting MAC address with filters installed is not
+		 * allowed on the adapter, one needs to simply restart adapter
+		 * so that the new MAC address will be taken from an outer
+		 * storage and set flawlessly by means of sfc_start() call
+		 */
+		sfc_stop(sa);
+		rc = sfc_start(sa);
+		if (rc != 0)
+			sfc_err(sa, "cannot restart adapter (rc = %u)", rc);
+	}
+
+unlock:
+	sfc_adapter_unlock(sa);
+}
+
 
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
@@ -730,6 +782,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.tx_queue_release		= sfc_tx_queue_release,
 	.flow_ctrl_get			= sfc_flow_ctrl_get,
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
+	.mac_addr_set			= sfc_mac_addr_set,
 };
 
 static int
-- 
2.5.5

[PATCH 12/31] net/sfc: support multicast addresses list controls

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 ++
 drivers/net/sfc/sfc_ethdev.c         | 40 ++++++++++++++++++++++++++++++++++++
 3 files changed, 43 insertions(+)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index aaea993..35633cb 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -11,6 +11,7 @@ MTU update           = Y
 Jumbo frame          = Y
 Promiscuous mode     = Y
 Allmulticast mode    = Y
+Multicast MAC filter = Y
 Flow control         = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 360ed98..529928d 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -60,6 +60,8 @@ SFC EFX PMD has support for:
 
 - Allmulticast mode
 
+- Multicast MAC filter
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 79e2761..268b76a 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -759,6 +759,45 @@ sfc_mac_addr_set(struct rte_eth_dev *dev, struct ether_addr *mac_addr)
 }
 
 
+static int
+sfc_set_mc_addr_list(struct rte_eth_dev *dev, struct ether_addr *mc_addr_set,
+		     uint32_t nb_mc_addr)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	uint8_t *mc_addrs_p;
+	uint8_t *mc_addrs;
+	int rc;
+	unsigned int i;
+
+	if (nb_mc_addr > EFX_MAC_MULTICAST_LIST_MAX) {
+		sfc_err(sa, "too many multicast addresses: %u > %u",
+			 nb_mc_addr, EFX_MAC_MULTICAST_LIST_MAX);
+		return -EINVAL;
+	}
+
+	mc_addrs_p = rte_calloc("mc-addrs", nb_mc_addr, EFX_MAC_ADDR_LEN, 0);
+	if (mc_addrs_p == NULL)
+		return -ENOMEM;
+
+	mc_addrs = mc_addrs_p;
+
+	for (i = 0; i < nb_mc_addr; ++i) {
+		(void)rte_memcpy(mc_addrs, mc_addr_set[i].addr_bytes,
+				 EFX_MAC_ADDR_LEN);
+		mc_addrs += EFX_MAC_ADDR_LEN;
+	}
+
+	rc = efx_mac_multicast_list_set(sa->nic, mc_addrs_p, nb_mc_addr);
+
+	rte_free(mc_addrs_p);
+
+	if (rc != 0)
+		sfc_err(sa, "cannot set multicast address list (rc = %u)", rc);
+
+	SFC_ASSERT(rc > 0);
+	return -rc;
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -783,6 +822,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.flow_ctrl_get			= sfc_flow_ctrl_get,
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
 	.mac_addr_set			= sfc_mac_addr_set,
+	.set_mc_addr_list		= sfc_set_mc_addr_list,
 };
 
 static int
-- 
2.5.5

[PATCH 13/31] net/sfc: support checksum offloads on receive

[Andrew Rybchenko]

IPv4 header and TCP/UDP checksums for both IPv4 and IPv6 are supported.

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  4 ++--
 doc/guides/nics/sfc_efx.rst          |  2 ++
 drivers/net/sfc/sfc_ethdev.c         |  5 +++++
 drivers/net/sfc/sfc_rx.c             | 40 ++++++++++++++++++++++++++++++++++++
 4 files changed, 49 insertions(+), 2 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 35633cb..ec1d1db 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -13,8 +13,8 @@ Promiscuous mode     = Y
 Allmulticast mode    = Y
 Multicast MAC filter = Y
 Flow control         = Y
-L3 checksum offload  = P
-L4 checksum offload  = P
+L3 checksum offload  = Y
+L4 checksum offload  = Y
 Basic stats          = Y
 Extended stats       = Y
 BSD nic_uio          = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 529928d..452a9ff 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -62,6 +62,8 @@ SFC EFX PMD has support for:
 
 - Multicast MAC filter
 
+- IPv4/IPv6 TCP/UDP receive checksum offload
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 268b76a..235582d 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -68,6 +68,11 @@ sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 	/* By default packets are dropped if no descriptors are available */
 	dev_info->default_rxconf.rx_drop_en = 1;
 
+	dev_info->rx_offload_capa =
+		DEV_RX_OFFLOAD_IPV4_CKSUM |
+		DEV_RX_OFFLOAD_UDP_CKSUM |
+		DEV_RX_OFFLOAD_TCP_CKSUM;
+
 	dev_info->tx_offload_capa =
 		DEV_TX_OFFLOAD_IPV4_CKSUM |
 		DEV_TX_OFFLOAD_UDP_CKSUM |
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index bd41131..4df4132 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -130,6 +130,45 @@ sfc_rx_qrefill(struct sfc_rxq *rxq)
 	}
 }
 
+static uint64_t
+sfc_rx_desc_flags_to_offload_flags(const unsigned int desc_flags)
+{
+	uint64_t mbuf_flags = 0;
+
+	switch (desc_flags & (EFX_PKT_IPV4 | EFX_CKSUM_IPV4)) {
+	case (EFX_PKT_IPV4 | EFX_CKSUM_IPV4):
+		mbuf_flags |= PKT_RX_IP_CKSUM_GOOD;
+		break;
+	case EFX_PKT_IPV4:
+		mbuf_flags |= PKT_RX_IP_CKSUM_BAD;
+		break;
+	default:
+		RTE_BUILD_BUG_ON(PKT_RX_IP_CKSUM_UNKNOWN != 0);
+		SFC_ASSERT((mbuf_flags & PKT_RX_IP_CKSUM_MASK) ==
+			   PKT_RX_IP_CKSUM_UNKNOWN);
+		break;
+	}
+
+	switch ((desc_flags &
+		 (EFX_PKT_TCP | EFX_PKT_UDP | EFX_CKSUM_TCPUDP))) {
+	case (EFX_PKT_TCP | EFX_CKSUM_TCPUDP):
+	case (EFX_PKT_UDP | EFX_CKSUM_TCPUDP):
+		mbuf_flags |= PKT_RX_L4_CKSUM_GOOD;
+		break;
+	case EFX_PKT_TCP:
+	case EFX_PKT_UDP:
+		mbuf_flags |= PKT_RX_L4_CKSUM_BAD;
+		break;
+	default:
+		RTE_BUILD_BUG_ON(PKT_RX_L4_CKSUM_UNKNOWN != 0);
+		SFC_ASSERT((mbuf_flags & PKT_RX_L4_CKSUM_MASK) ==
+			   PKT_RX_L4_CKSUM_UNKNOWN);
+		break;
+	}
+
+	return mbuf_flags;
+}
+
 uint16_t
 sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 {
@@ -182,6 +221,7 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 		rte_pktmbuf_data_len(m) = seg_len;
 		rte_pktmbuf_pkt_len(m) = seg_len;
 
+		m->ol_flags = sfc_rx_desc_flags_to_offload_flags(desc_flags);
 		m->packet_type = RTE_PTYPE_L2_ETHER;
 
 		*rx_pkts++ = m;
-- 
2.5.5

[PATCH 14/31] net/sfc: handle received packet type info provided by HW

[Andrew Rybchenko]

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 ++
 drivers/net/sfc/sfc_ethdev.c         | 19 +++++++++++++++++++
 drivers/net/sfc/sfc_rx.c             | 14 +++++++++++++-
 4 files changed, 35 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index ec1d1db..b5887d5 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -15,6 +15,7 @@ Multicast MAC filter = Y
 Flow control         = Y
 L3 checksum offload  = Y
 L4 checksum offload  = Y
+Packet type parsing  = Y
 Basic stats          = Y
 Extended stats       = Y
 BSD nic_uio          = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 452a9ff..57e8318 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -64,6 +64,8 @@ SFC EFX PMD has support for:
 
 - IPv4/IPv6 TCP/UDP receive checksum offload
 
+- Received packet type information
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 235582d..03c9a01 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -97,6 +97,24 @@ sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 	dev_info->tx_desc_lim.nb_align = EFX_TXQ_MINNDESCS;
 }
 
+static const uint32_t *
+sfc_dev_supported_ptypes_get(struct rte_eth_dev *dev)
+{
+	static const uint32_t ptypes[] = {
+		RTE_PTYPE_L2_ETHER,
+		RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+		RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+		RTE_PTYPE_L4_TCP,
+		RTE_PTYPE_L4_UDP,
+		RTE_PTYPE_UNKNOWN
+	};
+
+	if (dev->rx_pkt_burst == sfc_recv_pkts)
+		return ptypes;
+
+	return NULL;
+}
+
 static int
 sfc_dev_configure(struct rte_eth_dev *dev)
 {
@@ -819,6 +837,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.xstats_get			= sfc_xstats_get,
 	.xstats_get_names		= sfc_xstats_get_names,
 	.dev_infos_get			= sfc_dev_infos_get,
+	.dev_supported_ptypes_get	= sfc_dev_supported_ptypes_get,
 	.mtu_set			= sfc_dev_set_mtu,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index 4df4132..be8fa23 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -169,6 +169,18 @@ sfc_rx_desc_flags_to_offload_flags(const unsigned int desc_flags)
 	return mbuf_flags;
 }
 
+static uint32_t
+sfc_rx_desc_flags_to_packet_type(const unsigned int desc_flags)
+{
+	return RTE_PTYPE_L2_ETHER |
+		((desc_flags & EFX_PKT_IPV4) ?
+			RTE_PTYPE_L3_IPV4_EXT_UNKNOWN : 0) |
+		((desc_flags & EFX_PKT_IPV6) ?
+			RTE_PTYPE_L3_IPV6_EXT_UNKNOWN : 0) |
+		((desc_flags & EFX_PKT_TCP) ? RTE_PTYPE_L4_TCP : 0) |
+		((desc_flags & EFX_PKT_UDP) ? RTE_PTYPE_L4_UDP : 0);
+}
+
 uint16_t
 sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 {
@@ -222,7 +234,7 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 		rte_pktmbuf_pkt_len(m) = seg_len;
 
 		m->ol_flags = sfc_rx_desc_flags_to_offload_flags(desc_flags);
-		m->packet_type = RTE_PTYPE_L2_ETHER;
+		m->packet_type = sfc_rx_desc_flags_to_packet_type(desc_flags);
 
 		*rx_pkts++ = m;
 		done_pkts++;
-- 
2.5.5

[PATCH 15/31] net/sfc: support callback to get receive queue information

[Andrew Rybchenko]

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 03c9a01..e67ca61 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -821,6 +821,29 @@ sfc_set_mc_addr_list(struct rte_eth_dev *dev, struct ether_addr *mc_addr_set,
 	return -rc;
 }
 
+static void
+sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
+		      struct rte_eth_rxq_info *qinfo)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	struct sfc_rxq_info *rxq_info;
+	struct sfc_rxq *rxq;
+
+	sfc_adapter_lock(sa);
+
+	SFC_ASSERT(rx_queue_id < sa->rxq_count);
+
+	rxq_info = &sa->rxq_info[rx_queue_id];
+	rxq = rxq_info->rxq;
+	SFC_ASSERT(rxq != NULL);
+
+	qinfo->mp = rxq->refill_mb_pool;
+	qinfo->conf.rx_drop_en = 1;
+	qinfo->nb_desc = rxq_info->entries;
+
+	sfc_adapter_unlock(sa);
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -847,6 +870,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
 	.mac_addr_set			= sfc_mac_addr_set,
 	.set_mc_addr_list		= sfc_set_mc_addr_list,
+	.rxq_info_get			= sfc_rx_queue_info_get,
 };
 
 static int
-- 
2.5.5

[PATCH 16/31] net/sfc: support Rx free threshold

[Andrew Rybchenko]

Rx free threshold defines minimum number of free Rx descriptors
when Rx ring refill should be done.

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c |  1 +
 drivers/net/sfc/sfc_rx.c     | 16 ++++++++++++----
 drivers/net/sfc/sfc_rx.h     |  1 +
 3 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index e67ca61..1d7993f 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -838,6 +838,7 @@ sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 	SFC_ASSERT(rxq != NULL);
 
 	qinfo->mp = rxq->refill_mb_pool;
+	qinfo->conf.rx_free_thresh = rxq->refill_threshold;
 	qinfo->conf.rx_drop_en = 1;
 	qinfo->nb_desc = rxq_info->entries;
 
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index be8fa23..278d583 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -87,6 +87,10 @@ sfc_rx_qrefill(struct sfc_rxq *rxq)
 
 	free_space = EFX_RXQ_LIMIT(rxq->ptr_mask + 1) -
 		(added - rxq->completed);
+
+	if (free_space < rxq->refill_threshold)
+		return;
+
 	bulks = free_space / RTE_DIM(objs);
 
 	id = added & rxq->ptr_mask;
@@ -410,9 +414,10 @@ sfc_rx_qstop(struct sfc_adapter *sa, unsigned int sw_index)
 }
 
 static int
-sfc_rx_qcheck_conf(struct sfc_adapter *sa,
+sfc_rx_qcheck_conf(struct sfc_adapter *sa, uint16_t nb_rx_desc,
 		   const struct rte_eth_rxconf *rx_conf)
 {
+	const uint16_t rx_free_thresh_max = EFX_RXQ_LIMIT(nb_rx_desc);
 	int rc = 0;
 
 	if (rx_conf->rx_thresh.pthresh != 0 ||
@@ -423,8 +428,10 @@ sfc_rx_qcheck_conf(struct sfc_adapter *sa,
 		rc = EINVAL;
 	}
 
-	if (rx_conf->rx_free_thresh != 0) {
-		sfc_err(sa, "RxQ free threshold is not supported");
+	if (rx_conf->rx_free_thresh > rx_free_thresh_max) {
+		sfc_err(sa,
+			"RxQ free threshold too large: %u vs maximum %u",
+			rx_conf->rx_free_thresh, rx_free_thresh_max);
 		rc = EINVAL;
 	}
 
@@ -555,7 +562,7 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 	struct sfc_evq *evq;
 	struct sfc_rxq *rxq;
 
-	rc = sfc_rx_qcheck_conf(sa, rx_conf);
+	rc = sfc_rx_qcheck_conf(sa, nb_rx_desc, rx_conf);
 	if (rc != 0)
 		goto fail_bad_conf;
 
@@ -615,6 +622,7 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 	evq->rxq = rxq;
 	rxq->evq = evq;
 	rxq->ptr_mask = rxq_info->entries - 1;
+	rxq->refill_threshold = rx_conf->rx_free_thresh;
 	rxq->refill_mb_pool = mb_pool;
 	rxq->buf_size = buf_size;
 	rxq->hw_index = sw_index;
diff --git a/drivers/net/sfc/sfc_rx.h b/drivers/net/sfc/sfc_rx.h
index e4385b9..69318ab 100644
--- a/drivers/net/sfc/sfc_rx.h
+++ b/drivers/net/sfc/sfc_rx.h
@@ -87,6 +87,7 @@ struct sfc_rxq {
 	/* Used on refill */
 	unsigned int		added;
 	unsigned int		pushed;
+	unsigned int		refill_threshold;
 	uint8_t			port_id;
 	uint16_t		buf_size;
 	struct rte_mempool	*refill_mb_pool;
-- 
2.5.5

[PATCH 17/31] net/sfc: add callback to get RxQ pending descriptors count

[Andrew Rybchenko]

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 11 +++++++++++
 drivers/net/sfc/sfc_rx.c     | 16 ++++++++++++++++
 drivers/net/sfc/sfc_rx.h     |  3 +++
 3 files changed, 30 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 1d7993f..163f9ed 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -845,6 +845,16 @@ sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 	sfc_adapter_unlock(sa);
 }
 
+static uint32_t
+sfc_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+
+	sfc_log_init(sa, "RxQ=%u", rx_queue_id);
+
+	return sfc_rx_qdesc_npending(sa, rx_queue_id);
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -865,6 +875,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.mtu_set			= sfc_dev_set_mtu,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
+	.rx_queue_count			= sfc_rx_queue_count,
 	.tx_queue_setup			= sfc_tx_queue_setup,
 	.tx_queue_release		= sfc_tx_queue_release,
 	.flow_ctrl_get			= sfc_flow_ctrl_get,
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index 278d583..ff9d799 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -257,6 +257,22 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 	return done_pkts;
 }
 
+unsigned int
+sfc_rx_qdesc_npending(struct sfc_adapter *sa, unsigned int sw_index)
+{
+	struct sfc_rxq *rxq;
+
+	SFC_ASSERT(sw_index < sa->rxq_count);
+	rxq = sa->rxq_info[sw_index].rxq;
+
+	if (rxq == NULL || (rxq->state & SFC_RXQ_RUNNING) == 0)
+		return 0;
+
+	sfc_ev_qpoll(rxq->evq);
+
+	return rxq->pending - rxq->completed;
+}
+
 static void
 sfc_rx_qpurge(struct sfc_rxq *rxq)
 {
diff --git a/drivers/net/sfc/sfc_rx.h b/drivers/net/sfc/sfc_rx.h
index 69318ab..8064af6 100644
--- a/drivers/net/sfc/sfc_rx.h
+++ b/drivers/net/sfc/sfc_rx.h
@@ -140,6 +140,9 @@ void sfc_rx_qflush_failed(struct sfc_rxq *rxq);
 uint16_t sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 		       uint16_t nb_pkts);
 
+unsigned int sfc_rx_qdesc_npending(struct sfc_adapter *sa,
+				   unsigned int sw_index);
+
 #ifdef __cplusplus
 }
 #endif
-- 
2.5.5

[PATCH 18/31] net/sfc: add RxQ descriptor done callback

[Andrew Rybchenko]

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c |  9 +++++++++
 drivers/net/sfc/sfc_rx.c     | 11 +++++++++++
 drivers/net/sfc/sfc_rx.h     |  1 +
 3 files changed, 21 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 163f9ed..59dcdb4 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -855,6 +855,14 @@ sfc_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	return sfc_rx_qdesc_npending(sa, rx_queue_id);
 }
 
+static int
+sfc_rx_descriptor_done(void *queue, uint16_t offset)
+{
+	struct sfc_rxq *rxq = queue;
+
+	return sfc_rx_qdesc_done(rxq, offset);
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -876,6 +884,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
 	.rx_queue_count			= sfc_rx_queue_count,
+	.rx_descriptor_done		= sfc_rx_descriptor_done,
 	.tx_queue_setup			= sfc_tx_queue_setup,
 	.tx_queue_release		= sfc_tx_queue_release,
 	.flow_ctrl_get			= sfc_flow_ctrl_get,
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index ff9d799..802beb2 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -273,6 +273,17 @@ sfc_rx_qdesc_npending(struct sfc_adapter *sa, unsigned int sw_index)
 	return rxq->pending - rxq->completed;
 }
 
+int
+sfc_rx_qdesc_done(struct sfc_rxq *rxq, unsigned int offset)
+{
+	if ((rxq->state & SFC_RXQ_RUNNING) == 0)
+		return 0;
+
+	sfc_ev_qpoll(rxq->evq);
+
+	return offset < (rxq->pending - rxq->completed);
+}
+
 static void
 sfc_rx_qpurge(struct sfc_rxq *rxq)
 {
diff --git a/drivers/net/sfc/sfc_rx.h b/drivers/net/sfc/sfc_rx.h
index 8064af6..8d8e709 100644
--- a/drivers/net/sfc/sfc_rx.h
+++ b/drivers/net/sfc/sfc_rx.h
@@ -142,6 +142,7 @@ uint16_t sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 
 unsigned int sfc_rx_qdesc_npending(struct sfc_adapter *sa,
 				   unsigned int sw_index);
+int sfc_rx_qdesc_done(struct sfc_rxq *rxq, unsigned int offset);
 
 #ifdef __cplusplus
 }
-- 
2.5.5

[PATCH 19/31] net/sfc: support scattered Rx DMA

[Andrew Rybchenko]

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  4 ++--
 drivers/net/sfc/sfc_ethdev.c         |  1 +
 drivers/net/sfc/sfc_rx.c             | 39 +++++++++++++++++++++++++++---------
 4 files changed, 34 insertions(+), 11 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index b5887d5..74cc942 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -9,6 +9,7 @@ Link status          = Y
 Link status event    = Y
 MTU update           = Y
 Jumbo frame          = Y
+Scattered Rx         = Y
 Promiscuous mode     = Y
 Allmulticast mode    = Y
 Multicast MAC filter = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 57e8318..a0f25d3 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -66,6 +66,8 @@ SFC EFX PMD has support for:
 
 - Received packet type information
 
+- Scattered Rx DMA for packet that are larger that a single Rx descriptor
+
 
 Non-supported Features
 ----------------------
@@ -86,8 +88,6 @@ The features not yet supported include:
 
 - VLAN stripping
 
-- Scattered receive
-
 - LRO
 
 
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 59dcdb4..f1b655b 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -840,6 +840,7 @@ sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 	qinfo->mp = rxq->refill_mb_pool;
 	qinfo->conf.rx_free_thresh = rxq->refill_threshold;
 	qinfo->conf.rx_drop_en = 1;
+	qinfo->scattered_rx = (rxq_info->type == EFX_RXQ_TYPE_SCATTER);
 	qinfo->nb_desc = rxq_info->entries;
 
 	sfc_adapter_unlock(sa);
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index 802beb2..2909ec0 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -193,6 +193,7 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 	unsigned int prefix_size = rxq->prefix_size;
 	unsigned int done_pkts = 0;
 	boolean_t discard_next = B_FALSE;
+	struct rte_mbuf *scatter_pkt = NULL;
 
 	if (unlikely((rxq->state & SFC_RXQ_RUNNING) == 0))
 		return 0;
@@ -218,9 +219,6 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 		if (desc_flags & (EFX_ADDR_MISMATCH | EFX_DISCARD))
 			goto discard;
 
-		if (desc_flags & EFX_PKT_CONT)
-			goto discard;
-
 		if (desc_flags & EFX_PKT_PREFIX_LEN) {
 			uint16_t tmp_size;
 			int rc __rte_unused;
@@ -237,6 +235,29 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 		rte_pktmbuf_data_len(m) = seg_len;
 		rte_pktmbuf_pkt_len(m) = seg_len;
 
+		if (scatter_pkt != NULL) {
+			if (rte_pktmbuf_chain(scatter_pkt, m) != 0) {
+				rte_mempool_put(rxq->refill_mb_pool,
+						scatter_pkt);
+				goto discard;
+			}
+			/* The packet to deliver */
+			m = scatter_pkt;
+		}
+
+		if (desc_flags & EFX_PKT_CONT) {
+			/* The packet is scattered, more fragments to come */
+			scatter_pkt = m;
+			/* Futher fragments have no prefix */
+			prefix_size = 0;
+			continue;
+		}
+
+		/* Scattered packet is done */
+		scatter_pkt = NULL;
+		/* The first fragment of the packet has prefix */
+		prefix_size = rxq->prefix_size;
+
 		m->ol_flags = sfc_rx_desc_flags_to_offload_flags(desc_flags);
 		m->packet_type = sfc_rx_desc_flags_to_packet_type(desc_flags);
 
@@ -250,6 +271,9 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 		rxd->mbuf = NULL;
 	}
 
+	/* pending is only moved when entire packet is received */
+	SFC_ASSERT(scatter_pkt == NULL);
+
 	rxq->completed = completed;
 
 	sfc_rx_qrefill(rxq);
@@ -618,7 +642,9 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 
 	SFC_ASSERT(nb_rx_desc <= rxq_info->max_entries);
 	rxq_info->entries = nb_rx_desc;
-	rxq_info->type = EFX_RXQ_TYPE_DEFAULT;
+	rxq_info->type =
+		sa->eth_dev->data->dev_conf.rxmode.enable_scatter ?
+		EFX_RXQ_TYPE_SCATTER : EFX_RXQ_TYPE_DEFAULT;
 
 	evq_index = sfc_evq_index_by_rxq_sw_index(sa, sw_index);
 
@@ -806,11 +832,6 @@ sfc_rx_check_mode(struct sfc_adapter *sa, struct rte_eth_rxmode *rxmode)
 		rxmode->hw_strip_crc = 1;
 	}
 
-	if (rxmode->enable_scatter) {
-		sfc_err(sa, "Scatter on Rx not supported");
-		rc = EINVAL;
-	}
-
 	if (rxmode->enable_lro) {
 		sfc_err(sa, "LRO not supported");
 		rc = EINVAL;
-- 
2.5.5

[PATCH 20/31] net/sfc: support deferred start of receive queues

[Andrew Rybchenko]

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 ++
 drivers/net/sfc/sfc_ethdev.c         | 51 ++++++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_rx.c             | 18 +++++++------
 drivers/net/sfc/sfc_rx.h             |  2 ++
 5 files changed, 66 insertions(+), 8 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 74cc942..4a887f0 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -7,6 +7,7 @@
 Speed capabilities   = Y
 Link status          = Y
 Link status event    = Y
+Queue start/stop     = P
 MTU update           = Y
 Jumbo frame          = Y
 Scattered Rx         = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index a0f25d3..3d705bc 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -68,6 +68,8 @@ SFC EFX PMD has support for:
 
 - Scattered Rx DMA for packet that are larger that a single Rx descriptor
 
+- Deferred receive queue start
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index f1b655b..c531fdb 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -840,6 +840,7 @@ sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 	qinfo->mp = rxq->refill_mb_pool;
 	qinfo->conf.rx_free_thresh = rxq->refill_threshold;
 	qinfo->conf.rx_drop_en = 1;
+	qinfo->conf.rx_deferred_start = rxq_info->deferred_start;
 	qinfo->scattered_rx = (rxq_info->type == EFX_RXQ_TYPE_SCATTER);
 	qinfo->nb_desc = rxq_info->entries;
 
@@ -864,6 +865,54 @@ sfc_rx_descriptor_done(void *queue, uint16_t offset)
 	return sfc_rx_qdesc_done(rxq, offset);
 }
 
+static int
+sfc_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	int rc;
+
+	sfc_log_init(sa, "RxQ=%u", rx_queue_id);
+
+	sfc_adapter_lock(sa);
+
+	rc = EINVAL;
+	if (sa->state != SFC_ADAPTER_STARTED)
+		goto fail_not_started;
+
+	rc = sfc_rx_qstart(sa, rx_queue_id);
+	if (rc != 0)
+		goto fail_rx_qstart;
+
+	sa->rxq_info[rx_queue_id].deferred_started = B_TRUE;
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+
+fail_rx_qstart:
+fail_not_started:
+	sfc_adapter_unlock(sa);
+	SFC_ASSERT(rc > 0);
+	return -rc;
+}
+
+static int
+sfc_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+
+	sfc_log_init(sa, "RxQ=%u", rx_queue_id);
+
+	sfc_adapter_lock(sa);
+	sfc_rx_qstop(sa, rx_queue_id);
+
+	sa->rxq_info[rx_queue_id].deferred_started = B_FALSE;
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -882,6 +931,8 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_infos_get			= sfc_dev_infos_get,
 	.dev_supported_ptypes_get	= sfc_dev_supported_ptypes_get,
 	.mtu_set			= sfc_dev_set_mtu,
+	.rx_queue_start			= sfc_rx_queue_start,
+	.rx_queue_stop			= sfc_rx_queue_stop,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
 	.rx_queue_count			= sfc_rx_queue_count,
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index 2909ec0..3bfce1c 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -444,6 +444,9 @@ sfc_rx_qstop(struct sfc_adapter *sa, unsigned int sw_index)
 
 	rxq_info = &sa->rxq_info[sw_index];
 	rxq = rxq_info->rxq;
+
+	if (rxq->state == SFC_RXQ_INITIALIZED)
+		return;
 	SFC_ASSERT(rxq->state & SFC_RXQ_STARTED);
 
 	/* It seems to be used by DPDK for debug purposes only ('rte_ether') */
@@ -491,11 +494,6 @@ sfc_rx_qcheck_conf(struct sfc_adapter *sa, uint16_t nb_rx_desc,
 		rc = EINVAL;
 	}
 
-	if (rx_conf->rx_deferred_start != 0) {
-		sfc_err(sa, "RxQ deferred start is not supported");
-		rc = EINVAL;
-	}
-
 	return rc;
 }
 
@@ -688,6 +686,7 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 	rxq->state = SFC_RXQ_INITIALIZED;
 
 	rxq_info->rxq = rxq;
+	rxq_info->deferred_start = (rx_conf->rx_deferred_start != 0);
 
 	return 0;
 
@@ -742,9 +741,12 @@ sfc_rx_start(struct sfc_adapter *sa)
 		goto fail_rx_init;
 
 	for (sw_index = 0; sw_index < sa->rxq_count; ++sw_index) {
-		rc = sfc_rx_qstart(sa, sw_index);
-		if (rc != 0)
-			goto fail_rx_qstart;
+		if ((!sa->rxq_info[sw_index].deferred_start ||
+		     sa->rxq_info[sw_index].deferred_started)) {
+			rc = sfc_rx_qstart(sa, sw_index);
+			if (rc != 0)
+				goto fail_rx_qstart;
+		}
 	}
 
 	return 0;
diff --git a/drivers/net/sfc/sfc_rx.h b/drivers/net/sfc/sfc_rx.h
index 8d8e709..4aa6aea 100644
--- a/drivers/net/sfc/sfc_rx.h
+++ b/drivers/net/sfc/sfc_rx.h
@@ -119,6 +119,8 @@ struct sfc_rxq_info {
 	unsigned int		entries;
 	efx_rxq_type_t		type;
 	struct sfc_rxq		*rxq;
+	boolean_t		deferred_start;
+	boolean_t		deferred_started;
 };
 
 int sfc_rx_init(struct sfc_adapter *sa);
-- 
2.5.5

[PATCH 21/31] net/sfc: add callback to get transmit queue information

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index c531fdb..73e8fe2 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -847,6 +847,28 @@ sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 	sfc_adapter_unlock(sa);
 }
 
+static void
+sfc_tx_queue_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
+		      struct rte_eth_txq_info *qinfo)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	struct sfc_txq_info *txq_info;
+
+	sfc_adapter_lock(sa);
+
+	SFC_ASSERT(tx_queue_id < sa->txq_count);
+
+	txq_info = &sa->txq_info[tx_queue_id];
+	SFC_ASSERT(txq_info->txq != NULL);
+
+	memset(qinfo, 0, sizeof(*qinfo));
+
+	qinfo->conf.txq_flags = txq_info->txq->flags;
+	qinfo->nb_desc = txq_info->entries;
+
+	sfc_adapter_unlock(sa);
+}
+
 static uint32_t
 sfc_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 {
@@ -944,6 +966,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.mac_addr_set			= sfc_mac_addr_set,
 	.set_mc_addr_list		= sfc_set_mc_addr_list,
 	.rxq_info_get			= sfc_rx_queue_info_get,
+	.txq_info_get			= sfc_tx_queue_info_get,
 };
 
 static int
-- 
2.5.5

[PATCH 22/31] net/sfc: support Tx free threshold

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c |  1 +
 drivers/net/sfc/sfc_tweak.h  |  3 +++
 drivers/net/sfc/sfc_tx.c     | 14 ++++++++------
 drivers/net/sfc/sfc_tx.h     | 10 +---------
 4 files changed, 13 insertions(+), 15 deletions(-)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 73e8fe2..5d0d774 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -864,6 +864,7 @@ sfc_tx_queue_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
 	memset(qinfo, 0, sizeof(*qinfo));
 
 	qinfo->conf.txq_flags = txq_info->txq->flags;
+	qinfo->conf.tx_free_thresh = txq_info->txq->free_thresh;
 	qinfo->nb_desc = txq_info->entries;
 
 	sfc_adapter_unlock(sa);
diff --git a/drivers/net/sfc/sfc_tweak.h b/drivers/net/sfc/sfc_tweak.h
index 8a60f35..be39a5e 100644
--- a/drivers/net/sfc/sfc_tweak.h
+++ b/drivers/net/sfc/sfc_tweak.h
@@ -48,4 +48,7 @@
  */
 #define SFC_TX_XMIT_PKTS_REAP_AT_LEAST_ONCE	0
 
+/** Default free threshold follows recommendations from DPDK documentation */
+#define SFC_TX_DEFAULT_FREE_THRESH	32
+
 #endif /* _SFC_TWEAK_H_ */
diff --git a/drivers/net/sfc/sfc_tx.c b/drivers/net/sfc/sfc_tx.c
index a240610..13b24f7 100644
--- a/drivers/net/sfc/sfc_tx.c
+++ b/drivers/net/sfc/sfc_tx.c
@@ -54,7 +54,7 @@
 #define SFC_TX_QFLUSH_POLL_ATTEMPTS	(2000)
 
 static int
-sfc_tx_qcheck_conf(struct sfc_adapter *sa,
+sfc_tx_qcheck_conf(struct sfc_adapter *sa, uint16_t nb_tx_desc,
 		   const struct rte_eth_txconf *tx_conf)
 {
 	unsigned int flags = tx_conf->txq_flags;
@@ -65,9 +65,10 @@ sfc_tx_qcheck_conf(struct sfc_adapter *sa,
 		rc = EINVAL;
 	}
 
-	if (tx_conf->tx_free_thresh != 0) {
+	if (tx_conf->tx_free_thresh > EFX_TXQ_LIMIT(nb_tx_desc)) {
 		sfc_err(sa,
-			"setting explicit TX free threshold is not supported");
+			"TxQ free threshold too large: %u vs maximum %u",
+			tx_conf->tx_free_thresh, EFX_TXQ_LIMIT(nb_tx_desc));
 		rc = EINVAL;
 	}
 
@@ -147,7 +148,7 @@ sfc_tx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 
 	sfc_log_init(sa, "TxQ = %u", sw_index);
 
-	rc = sfc_tx_qcheck_conf(sa, tx_conf);
+	rc = sfc_tx_qcheck_conf(sa, nb_tx_desc, tx_conf);
 	if (rc != 0)
 		goto fail_bad_conf;
 
@@ -188,6 +189,8 @@ sfc_tx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 
 	txq->state = SFC_TXQ_INITIALIZED;
 	txq->ptr_mask = txq_info->entries - 1;
+	txq->free_thresh = (tx_conf->tx_free_thresh) ? tx_conf->tx_free_thresh :
+						     SFC_TX_DEFAULT_FREE_THRESH;
 	txq->hw_index = sw_index;
 	txq->flags = tx_conf->txq_flags;
 	txq->evq = evq;
@@ -537,8 +540,7 @@ sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 	unsigned int pkts_sent = 0;
 	efx_desc_t *pend = &txq->pend_desc[0];
 	const unsigned int hard_max_fill = EFX_TXQ_LIMIT(txq->ptr_mask + 1);
-	const unsigned int soft_max_fill = hard_max_fill -
-					   SFC_TX_MAX_PKT_DESC;
+	const unsigned int soft_max_fill = hard_max_fill - txq->free_thresh;
 	unsigned int fill_level = added - txq->completed;
 	boolean_t reap_done;
 	int rc __rte_unused;
diff --git a/drivers/net/sfc/sfc_tx.h b/drivers/net/sfc/sfc_tx.h
index fe2736b..f9eecc0 100644
--- a/drivers/net/sfc/sfc_tx.h
+++ b/drivers/net/sfc/sfc_tx.h
@@ -40,15 +40,6 @@ extern "C" {
 #endif
 
 /**
- * Estimated maximum number of segments that transmit packet consists of;
- * it is determined with respect to the expectation of a packet to consist
- * of a header plus a couple of data segments one of those crossing 4K page;
- * it is used by transmit path to avoid redundant reaping and, thus,
- * to avoid increase of latency
- */
-#define SFC_TX_MAX_PKT_DESC	4
-
-/**
  * A segment must not cross 4K boundary
  * (this is a requirement of NIC TX descriptors)
  */
@@ -85,6 +76,7 @@ struct sfc_txq {
 	unsigned int		added;
 	unsigned int		pending;
 	unsigned int		completed;
+	unsigned int		free_thresh;
 
 	unsigned int		hw_index;
 	unsigned int		flags;
-- 
2.5.5

[PATCH 23/31] net/sfc: support deferred start of transmit queues

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  2 +-
 doc/guides/nics/sfc_efx.rst          |  2 +-
 drivers/net/sfc/sfc_ethdev.c         | 51 ++++++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_tx.c             | 18 +++++++------
 drivers/net/sfc/sfc_tx.h             |  2 ++
 5 files changed, 65 insertions(+), 10 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 4a887f0..38bf9d2 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -7,7 +7,7 @@
 Speed capabilities   = Y
 Link status          = Y
 Link status event    = Y
-Queue start/stop     = P
+Queue start/stop     = Y
 MTU update           = Y
 Jumbo frame          = Y
 Scattered Rx         = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 3d705bc..c2e7149 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -68,7 +68,7 @@ SFC EFX PMD has support for:
 
 - Scattered Rx DMA for packet that are larger that a single Rx descriptor
 
-- Deferred receive queue start
+- Deferred receive and transmit queue start
 
 
 Non-supported Features
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 5d0d774..ba3c838 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -865,6 +865,7 @@ sfc_tx_queue_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
 
 	qinfo->conf.txq_flags = txq_info->txq->flags;
 	qinfo->conf.tx_free_thresh = txq_info->txq->free_thresh;
+	qinfo->conf.tx_deferred_start = txq_info->deferred_start;
 	qinfo->nb_desc = txq_info->entries;
 
 	sfc_adapter_unlock(sa);
@@ -936,6 +937,54 @@ sfc_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	return 0;
 }
 
+static int
+sfc_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	int rc;
+
+	sfc_log_init(sa, "TxQ = %u", tx_queue_id);
+
+	sfc_adapter_lock(sa);
+
+	rc = EINVAL;
+	if (sa->state != SFC_ADAPTER_STARTED)
+		goto fail_not_started;
+
+	rc = sfc_tx_qstart(sa, tx_queue_id);
+	if (rc != 0)
+		goto fail_tx_qstart;
+
+	sa->txq_info[tx_queue_id].deferred_started = B_TRUE;
+
+	sfc_adapter_unlock(sa);
+	return 0;
+
+fail_tx_qstart:
+
+fail_not_started:
+	sfc_adapter_unlock(sa);
+	SFC_ASSERT(rc > 0);
+	return -rc;
+}
+
+static int
+sfc_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+
+	sfc_log_init(sa, "TxQ = %u", tx_queue_id);
+
+	sfc_adapter_lock(sa);
+
+	sfc_tx_qstop(sa, tx_queue_id);
+
+	sa->txq_info[tx_queue_id].deferred_started = B_FALSE;
+
+	sfc_adapter_unlock(sa);
+	return 0;
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -956,6 +1005,8 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.mtu_set			= sfc_dev_set_mtu,
 	.rx_queue_start			= sfc_rx_queue_start,
 	.rx_queue_stop			= sfc_rx_queue_stop,
+	.tx_queue_start			= sfc_tx_queue_start,
+	.tx_queue_stop			= sfc_tx_queue_stop,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
 	.rx_queue_count			= sfc_rx_queue_count,
diff --git a/drivers/net/sfc/sfc_tx.c b/drivers/net/sfc/sfc_tx.c
index 13b24f7..15a6f9f 100644
--- a/drivers/net/sfc/sfc_tx.c
+++ b/drivers/net/sfc/sfc_tx.c
@@ -72,11 +72,6 @@ sfc_tx_qcheck_conf(struct sfc_adapter *sa, uint16_t nb_tx_desc,
 		rc = EINVAL;
 	}
 
-	if (tx_conf->tx_deferred_start != 0) {
-		sfc_err(sa, "TX queue deferred start is not supported (yet)");
-		rc = EINVAL;
-	}
-
 	if (tx_conf->tx_thresh.pthresh != 0 ||
 	    tx_conf->tx_thresh.hthresh != 0 ||
 	    tx_conf->tx_thresh.wthresh != 0) {
@@ -198,6 +193,7 @@ sfc_tx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 	evq->txq = txq;
 
 	txq_info->txq = txq;
+	txq_info->deferred_start = (tx_conf->tx_deferred_start != 0);
 
 	return 0;
 
@@ -425,6 +421,9 @@ sfc_tx_qstop(struct sfc_adapter *sa, unsigned int sw_index)
 
 	txq = txq_info->txq;
 
+	if (txq->state == SFC_TXQ_INITIALIZED)
+		return;
+
 	SFC_ASSERT(txq->state & SFC_TXQ_STARTED);
 
 	txq->state &= ~SFC_TXQ_RUNNING;
@@ -497,9 +496,12 @@ sfc_tx_start(struct sfc_adapter *sa)
 		goto fail_efx_tx_init;
 
 	for (sw_index = 0; sw_index < sa->txq_count; ++sw_index) {
-		rc = sfc_tx_qstart(sa, sw_index);
-		if (rc != 0)
-			goto fail_tx_qstart;
+		if (!(sa->txq_info[sw_index].deferred_start) ||
+		    sa->txq_info[sw_index].deferred_started) {
+			rc = sfc_tx_qstart(sa, sw_index);
+			if (rc != 0)
+				goto fail_tx_qstart;
+		}
 	}
 
 	return 0;
diff --git a/drivers/net/sfc/sfc_tx.h b/drivers/net/sfc/sfc_tx.h
index f9eecc0..632e3be 100644
--- a/drivers/net/sfc/sfc_tx.h
+++ b/drivers/net/sfc/sfc_tx.h
@@ -91,6 +91,8 @@ sfc_txq_sw_index(const struct sfc_txq *txq)
 struct sfc_txq_info {
 	unsigned int		entries;
 	struct sfc_txq		*txq;
+	boolean_t		deferred_start;
+	boolean_t		deferred_started;
 };
 
 int sfc_tx_init(struct sfc_adapter *sa);
-- 
2.5.5

[PATCH 24/31] net/sfc: support VLAN offload on transmit path

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 ++
 drivers/net/sfc/sfc_ethdev.c         |  8 +++++--
 drivers/net/sfc/sfc_tx.c             | 45 +++++++++++++++++++++++++++++++++++-
 drivers/net/sfc/sfc_tx.h             |  1 +
 5 files changed, 54 insertions(+), 3 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 38bf9d2..e7a1143 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -15,6 +15,7 @@ Promiscuous mode     = Y
 Allmulticast mode    = Y
 Multicast MAC filter = Y
 Flow control         = Y
+VLAN offload         = P
 L3 checksum offload  = Y
 L4 checksum offload  = Y
 Packet type parsing  = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index c2e7149..ee2ba27 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -70,6 +70,8 @@ SFC EFX PMD has support for:
 
 - Deferred receive and transmit queue start
 
+- Transmit VLAN insertion (if running firmware variant supports it)
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index ba3c838..0de17ca 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -46,6 +46,7 @@ static void
 sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 {
 	struct sfc_adapter *sa = dev->data->dev_private;
+	const efx_nic_cfg_t *encp = efx_nic_cfg_get(sa->nic);
 
 	RTE_SET_USED(dev_info);
 
@@ -78,8 +79,11 @@ sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 		DEV_TX_OFFLOAD_UDP_CKSUM |
 		DEV_TX_OFFLOAD_TCP_CKSUM;
 
-	dev_info->default_txconf.txq_flags = ETH_TXQ_FLAGS_NOVLANOFFL |
-					     ETH_TXQ_FLAGS_NOXSUMSCTP;
+	dev_info->default_txconf.txq_flags = ETH_TXQ_FLAGS_NOXSUMSCTP;
+	if (!encp->enc_hw_tx_insert_vlan_enabled)
+		dev_info->default_txconf.txq_flags |= ETH_TXQ_FLAGS_NOVLANOFFL;
+	else
+		dev_info->tx_offload_capa |= DEV_TX_OFFLOAD_VLAN_INSERT;
 
 	dev_info->rx_desc_lim.nb_max = EFX_RXQ_MAXNDESCS;
 	dev_info->rx_desc_lim.nb_min = EFX_RXQ_MINNDESCS;
diff --git a/drivers/net/sfc/sfc_tx.c b/drivers/net/sfc/sfc_tx.c
index 15a6f9f..86bcfec 100644
--- a/drivers/net/sfc/sfc_tx.c
+++ b/drivers/net/sfc/sfc_tx.c
@@ -58,6 +58,7 @@ sfc_tx_qcheck_conf(struct sfc_adapter *sa, uint16_t nb_tx_desc,
 		   const struct rte_eth_txconf *tx_conf)
 {
 	unsigned int flags = tx_conf->txq_flags;
+	const efx_nic_cfg_t *encp = efx_nic_cfg_get(sa->nic);
 	int rc = 0;
 
 	if (tx_conf->tx_rs_thresh != 0) {
@@ -80,7 +81,8 @@ sfc_tx_qcheck_conf(struct sfc_adapter *sa, uint16_t nb_tx_desc,
 		rc = EINVAL;
 	}
 
-	if ((flags & ETH_TXQ_FLAGS_NOVLANOFFL) == 0) {
+	if (!encp->enc_hw_tx_insert_vlan_enabled &&
+	    (flags & ETH_TXQ_FLAGS_NOVLANOFFL) == 0) {
 		sfc_err(sa, "VLAN offload is not supported");
 		rc = EINVAL;
 	}
@@ -384,6 +386,7 @@ sfc_tx_qstart(struct sfc_adapter *sa, unsigned int sw_index)
 		goto fail_tx_qcreate;
 
 	txq->added = txq->pending = txq->completed = desc_index;
+	txq->hw_vlan_tci = 0;
 
 	efx_tx_qenable(txq->common);
 
@@ -533,6 +536,37 @@ sfc_tx_stop(struct sfc_adapter *sa)
 	efx_tx_fini(sa->nic);
 }
 
+/*
+ * The function is used to insert or update VLAN tag;
+ * the firmware has state of the firmware tag to insert per TxQ
+ * (controlled by option descriptors), hence, if the tag of the
+ * packet to be sent is different from one remembered by the firmware,
+ * the function will update it
+ */
+static unsigned int
+sfc_tx_maybe_insert_tag(struct sfc_txq *txq, struct rte_mbuf *m,
+			efx_desc_t **pend)
+{
+	uint16_t this_tag = ((m->ol_flags & PKT_TX_VLAN_PKT) ?
+			     m->vlan_tci : 0);
+
+	if (this_tag == txq->hw_vlan_tci)
+		return 0;
+
+	/*
+	 * The expression inside SFC_ASSERT() is not desired to be checked in
+	 * a non-debug build because it might be too expensive on the data path
+	 */
+	SFC_ASSERT(efx_nic_cfg_get(txq->evq->sa->nic)->enc_hw_tx_insert_vlan_enabled);
+
+	efx_tx_qdesc_vlantci_create(txq->common, rte_cpu_to_be_16(this_tag),
+				    *pend);
+	(*pend)++;
+	txq->hw_vlan_tci = this_tag;
+
+	return 1;
+}
+
 uint16_t
 sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 {
@@ -574,6 +608,15 @@ sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		size_t			pkt_len = m_seg->pkt_len;
 		unsigned int		pkt_descs = 0;
 
+		/*
+		 * Here VLAN TCI is expected to be zero in case if no
+		 * DEV_TX_VLAN_OFFLOAD capability is advertised;
+		 * if the calling app ignores the absence of
+		 * DEV_TX_VLAN_OFFLOAD and pushes VLAN TCI, then
+		 * TX_ERROR will occur
+		 */
+		pkt_descs += sfc_tx_maybe_insert_tag(txq, m_seg, &pend);
+
 		for (; m_seg != NULL; m_seg = m_seg->next) {
 			efsys_dma_addr_t	next_frag;
 			size_t			seg_len;
diff --git a/drivers/net/sfc/sfc_tx.h b/drivers/net/sfc/sfc_tx.h
index 632e3be..4d25c6a 100644
--- a/drivers/net/sfc/sfc_tx.h
+++ b/drivers/net/sfc/sfc_tx.h
@@ -77,6 +77,7 @@ struct sfc_txq {
 	unsigned int		pending;
 	unsigned int		completed;
 	unsigned int		free_thresh;
+	uint16_t		hw_vlan_tci;
 
 	unsigned int		hw_index;
 	unsigned int		flags;
-- 
2.5.5

[PATCH 25/31] net/sfc: add basic stubs for RSS support on driver attach

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/sfc_efx.rst  |  2 ++
 drivers/net/sfc/efsys.h      |  2 +-
 drivers/net/sfc/sfc.c        | 76 +++++++++++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc.h        | 17 ++++++++++
 drivers/net/sfc/sfc_ethdev.c |  8 +++++
 drivers/net/sfc/sfc_rx.c     | 81 +++++++++++++++++++++++++++++++++++++++++++-
 drivers/net/sfc/sfc_rx.h     |  8 +++++
 7 files changed, 192 insertions(+), 2 deletions(-)

diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index ee2ba27..4f674c0 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -66,6 +66,8 @@ SFC EFX PMD has support for:
 
 - Received packet type information
 
+- Receive side scaling (RSS)
+
 - Scattered Rx DMA for packet that are larger that a single Rx descriptor
 
 - Deferred receive and transmit queue start
diff --git a/drivers/net/sfc/efsys.h b/drivers/net/sfc/efsys.h
index 0f941e6..fb2f3b5 100644
--- a/drivers/net/sfc/efsys.h
+++ b/drivers/net/sfc/efsys.h
@@ -195,7 +195,7 @@ prefetch_read_once(const volatile void *addr)
 #define EFSYS_OPT_BOOTCFG 0
 
 #define EFSYS_OPT_DIAG 0
-#define EFSYS_OPT_RX_SCALE 0
+#define EFSYS_OPT_RX_SCALE 1
 #define EFSYS_OPT_QSTATS 0
 /* Filters support is required for SFN7xxx and SFN8xx */
 #define EFSYS_OPT_FILTER 1
diff --git a/drivers/net/sfc/sfc.c b/drivers/net/sfc/sfc.c
index e2e6c9e..e79367d 100644
--- a/drivers/net/sfc/sfc.c
+++ b/drivers/net/sfc/sfc.c
@@ -484,6 +484,73 @@ sfc_mem_bar_fini(struct sfc_adapter *sa)
 	memset(ebp, 0, sizeof(*ebp));
 }
 
+#if EFSYS_OPT_RX_SCALE
+/*
+ * A fixed RSS key which has a property of being symmetric
+ * (symmetrical flows are distributed to the same CPU)
+ * and also known to give a uniform distribution
+ * (a good distribution of traffic between different CPUs)
+ */
+static const uint8_t default_rss_key[SFC_RSS_KEY_SIZE] = {
+	0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a,
+	0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a,
+	0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a,
+	0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a,
+	0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a,
+};
+#endif
+
+static int
+sfc_set_rss_defaults(struct sfc_adapter *sa)
+{
+#if EFSYS_OPT_RX_SCALE
+	int rc;
+
+	rc = efx_intr_init(sa->nic, sa->intr.type, NULL);
+	if (rc != 0)
+		goto fail_intr_init;
+
+	rc = efx_ev_init(sa->nic);
+	if (rc != 0)
+		goto fail_ev_init;
+
+	rc = efx_rx_init(sa->nic);
+	if (rc != 0)
+		goto fail_rx_init;
+
+	rc = efx_rx_scale_support_get(sa->nic, &sa->rss_support);
+	if (rc != 0)
+		goto fail_scale_support_get;
+
+	rc = efx_rx_hash_support_get(sa->nic, &sa->hash_support);
+	if (rc != 0)
+		goto fail_hash_support_get;
+
+	efx_rx_fini(sa->nic);
+	efx_ev_fini(sa->nic);
+	efx_intr_fini(sa->nic);
+
+	sa->rss_hash_types = sfc_rte_to_efx_hash_type(SFC_RSS_OFFLOADS);
+
+	rte_memcpy(sa->rss_key, default_rss_key, sizeof(sa->rss_key));
+
+	return 0;
+
+fail_hash_support_get:
+fail_scale_support_get:
+fail_rx_init:
+	efx_ev_fini(sa->nic);
+
+fail_ev_init:
+	efx_intr_fini(sa->nic);
+
+fail_intr_init:
+	return rc;
+#else
+	return 0;
+#endif
+}
+
 int
 sfc_attach(struct sfc_adapter *sa)
 {
@@ -550,6 +617,10 @@ sfc_attach(struct sfc_adapter *sa)
 	efx_phy_adv_cap_get(sa->nic, EFX_PHY_CAP_PERM,
 			    &sa->port.phy_adv_cap_mask);
 
+	rc = sfc_set_rss_defaults(sa);
+	if (rc != 0)
+		goto fail_set_rss_defaults;
+
 	sfc_log_init(sa, "fini nic");
 	efx_nic_fini(enp);
 
@@ -558,7 +629,12 @@ sfc_attach(struct sfc_adapter *sa)
 	sfc_log_init(sa, "done");
 	return 0;
 
+fail_set_rss_defaults:
+	sfc_intr_detach(sa);
+
 fail_intr_attach:
+	efx_nic_fini(sa->nic);
+
 fail_estimate_rsrc_limits:
 fail_nic_reset:
 	sfc_log_init(sa, "unprobe nic");
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index f0bbaf8..01dbfb6 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -42,6 +42,13 @@
 extern "C" {
 #endif
 
+#if EFSYS_OPT_RX_SCALE
+/** RSS key length (bytes) */
+#define	SFC_RSS_KEY_SIZE	40
+/** RSS hash offloads mask */
+#define	SFC_RSS_OFFLOADS	(ETH_RSS_IP | ETH_RSS_TCP)
+#endif
+
 /*
  * +---------------+
  * | UNINITIALIZED |<-----------+
@@ -187,6 +194,16 @@ struct sfc_adapter {
 
 	unsigned int			txq_count;
 	struct sfc_txq_info		*txq_info;
+
+	unsigned int			rss_channels;
+
+#if EFSYS_OPT_RX_SCALE
+	efx_rx_scale_support_t		rss_support;
+	efx_rx_hash_support_t		hash_support;
+	unsigned int			rss_hash_types;
+	unsigned int			rss_tbl[EFX_RSS_TBL_SIZE];
+	uint8_t				rss_key[SFC_RSS_KEY_SIZE];
+#endif
 };
 
 /*
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 0de17ca..b17607f 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -85,6 +85,14 @@ sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 	else
 		dev_info->tx_offload_capa |= DEV_TX_OFFLOAD_VLAN_INSERT;
 
+#if EFSYS_OPT_RX_SCALE
+	if (sa->rss_support != EFX_RX_SCALE_UNAVAILABLE) {
+		dev_info->reta_size = EFX_RSS_TBL_SIZE;
+		dev_info->hash_key_size = SFC_RSS_KEY_SIZE;
+		dev_info->flow_type_rss_offloads = SFC_RSS_OFFLOADS;
+	}
+#endif
+
 	dev_info->rx_desc_lim.nb_max = EFX_RXQ_MAXNDESCS;
 	dev_info->rx_desc_lim.nb_min = EFX_RXQ_MINNDESCS;
 	/* The RXQ hardware requires that the descriptor count is a power
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index 3bfce1c..f0fe1b8 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -411,7 +411,8 @@ sfc_rx_qstart(struct sfc_adapter *sa, unsigned int sw_index)
 
 	if (sw_index == 0) {
 		rc = efx_mac_filter_default_rxq_set(sa->nic, rxq->common,
-						    B_FALSE);
+						    (sa->rss_channels > 1) ?
+						    B_TRUE : B_FALSE);
 		if (rc != 0)
 			goto fail_mac_filter_default_rxq_set;
 	}
@@ -683,6 +684,11 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 	rxq->batch_max = encp->enc_rx_batch_max;
 	rxq->prefix_size = encp->enc_rx_prefix_size;
 
+#if EFSYS_OPT_RX_SCALE
+	if (sa->hash_support == EFX_RX_HASH_AVAILABLE)
+		rxq->flags |= SFC_RXQ_RSS_HASH;
+#endif
+
 	rxq->state = SFC_RXQ_INITIALIZED;
 
 	rxq_info->rxq = rxq;
@@ -728,6 +734,56 @@ sfc_rx_qfini(struct sfc_adapter *sa, unsigned int sw_index)
 	rte_free(rxq);
 }
 
+#if EFSYS_OPT_RX_SCALE
+unsigned int
+sfc_rte_to_efx_hash_type(uint64_t rss_hf)
+{
+	unsigned int efx_hash_types = 0;
+
+	if ((rss_hf & (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 |
+		       ETH_RSS_NONFRAG_IPV4_OTHER)) != 0)
+		efx_hash_types |= (1 << EFX_RX_HASH_IPV4);
+
+	if ((rss_hf & ETH_RSS_NONFRAG_IPV4_TCP) != 0)
+		efx_hash_types |= (1 << EFX_RX_HASH_TCPIPV4);
+
+	if ((rss_hf & (ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6 |
+			ETH_RSS_NONFRAG_IPV6_OTHER | ETH_RSS_IPV6_EX)) != 0)
+		efx_hash_types |= (1 << EFX_RX_HASH_IPV6);
+
+	if ((rss_hf & (ETH_RSS_NONFRAG_IPV6_TCP | ETH_RSS_IPV6_TCP_EX)) != 0)
+		efx_hash_types |= (1 << EFX_RX_HASH_TCPIPV6);
+
+	return efx_hash_types;
+}
+#endif
+
+static int
+sfc_rx_rss_config(struct sfc_adapter *sa)
+{
+	int rc = 0;
+
+#if EFSYS_OPT_RX_SCALE
+	if (sa->rss_channels > 1) {
+		rc = efx_rx_scale_mode_set(sa->nic, EFX_RX_HASHALG_TOEPLITZ,
+					   sa->rss_hash_types, B_TRUE);
+		if (rc != 0)
+			goto finish;
+
+		rc = efx_rx_scale_key_set(sa->nic, sa->rss_key,
+					  sizeof(sa->rss_key));
+		if (rc != 0)
+			goto finish;
+
+		rc = efx_rx_scale_tbl_set(sa->nic, sa->rss_tbl,
+					  sizeof(sa->rss_tbl));
+	}
+
+finish:
+#endif
+	return rc;
+}
+
 int
 sfc_rx_start(struct sfc_adapter *sa)
 {
@@ -740,6 +796,10 @@ sfc_rx_start(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_rx_init;
 
+	rc = sfc_rx_rss_config(sa);
+	if (rc != 0)
+		goto fail_rss_config;
+
 	for (sw_index = 0; sw_index < sa->rxq_count; ++sw_index) {
 		if ((!sa->rxq_info[sw_index].deferred_start ||
 		     sa->rxq_info[sw_index].deferred_started)) {
@@ -755,6 +815,7 @@ sfc_rx_start(struct sfc_adapter *sa)
 	while (sw_index-- > 0)
 		sfc_rx_qstop(sa, sw_index);
 
+fail_rss_config:
 	efx_rx_fini(sa->nic);
 
 fail_rx_init:
@@ -801,6 +862,14 @@ sfc_rx_check_mode(struct sfc_adapter *sa, struct rte_eth_rxmode *rxmode)
 	case ETH_MQ_RX_NONE:
 		/* No special checks are required */
 		break;
+#if EFSYS_OPT_RX_SCALE
+	case ETH_MQ_RX_RSS:
+		if (sa->rss_support == EFX_RX_SCALE_UNAVAILABLE) {
+			sfc_err(sa, "RSS is not available");
+			rc = EINVAL;
+		}
+		break;
+#endif
 	default:
 		sfc_err(sa, "Rx multi-queue mode %u not supported",
 			rxmode->mq_mode);
@@ -876,6 +945,16 @@ sfc_rx_init(struct sfc_adapter *sa)
 			goto fail_rx_qinit_info;
 	}
 
+#if EFSYS_OPT_RX_SCALE
+	sa->rss_channels = (dev_conf->rxmode.mq_mode == ETH_MQ_RX_RSS) ?
+			   MIN(sa->rxq_count, EFX_MAXRSS) : 1;
+
+	if (sa->rss_channels > 1) {
+		for (sw_index = 0; sw_index < EFX_RSS_TBL_SIZE; ++sw_index)
+			sa->rss_tbl[sw_index] = sw_index % sa->rss_channels;
+	}
+#endif
+
 	return 0;
 
 fail_rx_qinit_info:
diff --git a/drivers/net/sfc/sfc_rx.h b/drivers/net/sfc/sfc_rx.h
index 4aa6aea..2b8b9eb 100644
--- a/drivers/net/sfc/sfc_rx.h
+++ b/drivers/net/sfc/sfc_rx.h
@@ -83,6 +83,10 @@ struct sfc_rxq {
 	unsigned int		completed;
 	uint16_t		batch_max;
 	uint16_t		prefix_size;
+#if EFSYS_OPT_RX_SCALE
+	unsigned int		flags;
+#define	SFC_RXQ_RSS_HASH	0x1
+#endif
 
 	/* Used on refill */
 	unsigned int		added;
@@ -146,6 +150,10 @@ unsigned int sfc_rx_qdesc_npending(struct sfc_adapter *sa,
 				   unsigned int sw_index);
 int sfc_rx_qdesc_done(struct sfc_rxq *rxq, unsigned int offset);
 
+#if EFSYS_OPT_RX_SCALE
+unsigned int sfc_rte_to_efx_hash_type(uint64_t rss_hf);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
-- 
2.5.5

[PATCH 26/31] net/sfc: support RSS hash offload

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Extract RSS hash provided by the HW in the prefix and put it to mbuf.

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 ++
 drivers/net/sfc/sfc_rx.c             | 31 ++++++++++++++++++++++++++++++-
 3 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index e7a1143..debea27 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -14,6 +14,7 @@ Scattered Rx         = Y
 Promiscuous mode     = Y
 Allmulticast mode    = Y
 Multicast MAC filter = Y
+RSS hash             = Y
 Flow control         = Y
 VLAN offload         = P
 L3 checksum offload  = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 4f674c0..64489de 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -68,6 +68,8 @@ SFC EFX PMD has support for:
 
 - Receive side scaling (RSS)
 
+- RSS hash
+
 - Scattered Rx DMA for packet that are larger that a single Rx descriptor
 
 - Deferred receive and transmit queue start
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index f0fe1b8..1e254a8 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -185,6 +185,28 @@ sfc_rx_desc_flags_to_packet_type(const unsigned int desc_flags)
 		((desc_flags & EFX_PKT_UDP) ? RTE_PTYPE_L4_UDP : 0);
 }
 
+static void
+sfc_rx_set_rss_hash(struct sfc_rxq *rxq, unsigned int flags, struct rte_mbuf *m)
+{
+#if EFSYS_OPT_RX_SCALE
+	uint8_t *mbuf_data;
+
+
+	if ((rxq->flags & SFC_RXQ_RSS_HASH) == 0)
+		return;
+
+	mbuf_data = rte_pktmbuf_mtod(m, uint8_t *);
+
+	if (flags & (EFX_PKT_IPV4 | EFX_PKT_IPV6)) {
+		m->hash.rss = efx_pseudo_hdr_hash_get(rxq->common,
+						      EFX_RX_HASHALG_TOEPLITZ,
+						      mbuf_data);
+
+		m->ol_flags |= PKT_RX_RSS_HASH;
+	}
+#endif
+}
+
 uint16_t
 sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 {
@@ -231,7 +253,6 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 			seg_len = rxd->size - prefix_size;
 		}
 
-		m->data_off += prefix_size;
 		rte_pktmbuf_data_len(m) = seg_len;
 		rte_pktmbuf_pkt_len(m) = seg_len;
 
@@ -261,6 +282,14 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 		m->ol_flags = sfc_rx_desc_flags_to_offload_flags(desc_flags);
 		m->packet_type = sfc_rx_desc_flags_to_packet_type(desc_flags);
 
+		/*
+		 * Extract RSS hash from the packet prefix and
+		 * set the corresponding field (if needed and possible)
+		 */
+		sfc_rx_set_rss_hash(rxq, desc_flags, m);
+
+		m->data_off += prefix_size;
+
 		*rx_pkts++ = m;
 		done_pkts++;
 		continue;
-- 
2.5.5

[PATCH 27/31] net/sfc: add callback to query RSS key and hash types config

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 33 +++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_rx.c     | 22 ++++++++++++++++++++++
 drivers/net/sfc/sfc_rx.h     |  1 +
 3 files changed, 56 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index b17607f..c78d798 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -997,6 +997,36 @@ sfc_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
 	return 0;
 }
 
+#if EFSYS_OPT_RX_SCALE
+static int
+sfc_dev_rss_hash_conf_get(struct rte_eth_dev *dev,
+			  struct rte_eth_rss_conf *rss_conf)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+
+	if ((sa->rss_channels == 1) ||
+	    (sa->rss_support != EFX_RX_SCALE_EXCLUSIVE))
+		return -ENOTSUP;
+
+	sfc_adapter_lock(sa);
+
+	/*
+	 * Mapping of hash configuration between RTE and EFX is not one-to-one,
+	 * hence, conversion is done here to derive a correct set of ETH_RSS
+	 * flags which corresponds to the active EFX configuration stored
+	 * locally in 'sfc_adapter' and kept up-to-date
+	 */
+	rss_conf->rss_hf = sfc_efx_to_rte_hash_type(sa->rss_hash_types);
+	rss_conf->rss_key_len = SFC_RSS_KEY_SIZE;
+	if (rss_conf->rss_key != NULL)
+		rte_memcpy(rss_conf->rss_key, sa->rss_key, SFC_RSS_KEY_SIZE);
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+}
+#endif
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -1028,6 +1058,9 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.flow_ctrl_get			= sfc_flow_ctrl_get,
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
 	.mac_addr_set			= sfc_mac_addr_set,
+#if EFSYS_OPT_RX_SCALE
+	.rss_hash_conf_get		= sfc_dev_rss_hash_conf_get,
+#endif
 	.set_mc_addr_list		= sfc_set_mc_addr_list,
 	.rxq_info_get			= sfc_rx_queue_info_get,
 	.txq_info_get			= sfc_tx_queue_info_get,
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index 1e254a8..ca26b1e 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -785,6 +785,28 @@ sfc_rte_to_efx_hash_type(uint64_t rss_hf)
 
 	return efx_hash_types;
 }
+
+uint64_t
+sfc_efx_to_rte_hash_type(unsigned int efx_hash_types)
+{
+	uint64_t rss_hf = 0;
+
+	if ((efx_hash_types & (1 << EFX_RX_HASH_IPV4)) != 0)
+		rss_hf |= (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 |
+			   ETH_RSS_NONFRAG_IPV4_OTHER);
+
+	if ((efx_hash_types & (1 << EFX_RX_HASH_TCPIPV4)) != 0)
+		rss_hf |= ETH_RSS_NONFRAG_IPV4_TCP;
+
+	if ((efx_hash_types & (1 << EFX_RX_HASH_IPV6)) != 0)
+		rss_hf |= (ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6 |
+			   ETH_RSS_NONFRAG_IPV6_OTHER | ETH_RSS_IPV6_EX);
+
+	if ((efx_hash_types & (1 << EFX_RX_HASH_TCPIPV6)) != 0)
+		rss_hf |= (ETH_RSS_NONFRAG_IPV6_TCP | ETH_RSS_IPV6_TCP_EX);
+
+	return rss_hf;
+}
 #endif
 
 static int
diff --git a/drivers/net/sfc/sfc_rx.h b/drivers/net/sfc/sfc_rx.h
index 2b8b9eb..aa7264c 100644
--- a/drivers/net/sfc/sfc_rx.h
+++ b/drivers/net/sfc/sfc_rx.h
@@ -152,6 +152,7 @@ int sfc_rx_qdesc_done(struct sfc_rxq *rxq, unsigned int offset);
 
 #if EFSYS_OPT_RX_SCALE
 unsigned int sfc_rte_to_efx_hash_type(uint64_t rss_hf);
+uint64_t sfc_efx_to_rte_hash_type(unsigned int efx_hash_types);
 #endif
 
 #ifdef __cplusplus
-- 
2.5.5

[PATCH 28/31] net/sfc: add callback to set RSS key and hash types config

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 drivers/net/sfc/sfc_ethdev.c         | 63 ++++++++++++++++++++++++++++++++++++
 2 files changed, 64 insertions(+)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index debea27..4f6f117 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -15,6 +15,7 @@ Promiscuous mode     = Y
 Allmulticast mode    = Y
 Multicast MAC filter = Y
 RSS hash             = Y
+RSS key update       = Y
 Flow control         = Y
 VLAN offload         = P
 L3 checksum offload  = Y
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index c78d798..f9a766c 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -1025,6 +1025,68 @@ sfc_dev_rss_hash_conf_get(struct rte_eth_dev *dev,
 
 	return 0;
 }
+
+static int
+sfc_dev_rss_hash_update(struct rte_eth_dev *dev,
+			struct rte_eth_rss_conf *rss_conf)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	unsigned int efx_hash_types;
+	int rc = 0;
+
+	if ((sa->rss_channels == 1) ||
+	    (sa->rss_support != EFX_RX_SCALE_EXCLUSIVE)) {
+		sfc_err(sa, "RSS is not available");
+		return -ENOTSUP;
+	}
+
+	if ((rss_conf->rss_key != NULL) &&
+	    (rss_conf->rss_key_len != sizeof(sa->rss_key))) {
+		sfc_err(sa, "RSS key size is wrong (should be %lu)",
+			sizeof(sa->rss_key));
+		return -EINVAL;
+	}
+
+	if ((rss_conf->rss_hf & ~SFC_RSS_OFFLOADS) != 0) {
+		sfc_err(sa, "unsupported hash functions requested");
+		return -EINVAL;
+	}
+
+	sfc_adapter_lock(sa);
+
+	efx_hash_types = sfc_rte_to_efx_hash_type(rss_conf->rss_hf);
+
+	rc = efx_rx_scale_mode_set(sa->nic, EFX_RX_HASHALG_TOEPLITZ,
+				   efx_hash_types, B_TRUE);
+	if (rc != 0)
+		goto fail_scale_mode_set;
+
+	if (rss_conf->rss_key != NULL) {
+		if (sa->state == SFC_ADAPTER_STARTED) {
+			rc = efx_rx_scale_key_set(sa->nic, rss_conf->rss_key,
+						  sizeof(sa->rss_key));
+			if (rc != 0)
+				goto fail_scale_key_set;
+		}
+
+		rte_memcpy(sa->rss_key, rss_conf->rss_key, sizeof(sa->rss_key));
+	}
+
+	sa->rss_hash_types = efx_hash_types;
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+
+fail_scale_key_set:
+	if (efx_rx_scale_mode_set(sa->nic, EFX_RX_HASHALG_TOEPLITZ,
+				  sa->rss_hash_types, B_TRUE) != 0)
+		sfc_err(sa, "failed to restore RSS mode");
+
+fail_scale_mode_set:
+	sfc_adapter_unlock(sa);
+	return -rc;
+}
 #endif
 
 static const struct eth_dev_ops sfc_eth_dev_ops = {
@@ -1059,6 +1121,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
 	.mac_addr_set			= sfc_mac_addr_set,
 #if EFSYS_OPT_RX_SCALE
+	.rss_hash_update		= sfc_dev_rss_hash_update,
 	.rss_hash_conf_get		= sfc_dev_rss_hash_conf_get,
 #endif
 	.set_mc_addr_list		= sfc_set_mc_addr_list,
-- 
2.5.5

[PATCH 29/31] net/sfc: add callback to query RSS redirection table

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index f9a766c..0cd96ac 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -1087,6 +1087,36 @@ sfc_dev_rss_hash_update(struct rte_eth_dev *dev,
 	sfc_adapter_unlock(sa);
 	return -rc;
 }
+
+static int
+sfc_dev_rss_reta_query(struct rte_eth_dev *dev,
+		       struct rte_eth_rss_reta_entry64 *reta_conf,
+		       uint16_t reta_size)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	int entry;
+
+	if ((sa->rss_channels == 1) ||
+	    (sa->rss_support != EFX_RX_SCALE_EXCLUSIVE))
+		return -ENOTSUP;
+
+	if (reta_size != EFX_RSS_TBL_SIZE)
+		return -EINVAL;
+
+	sfc_adapter_lock(sa);
+
+	for (entry = 0; entry < reta_size; entry++) {
+		int grp = entry / RTE_RETA_GROUP_SIZE;
+		int grp_idx = entry % RTE_RETA_GROUP_SIZE;
+
+		if ((reta_conf[grp].mask >> grp_idx) & 1)
+			reta_conf[grp].reta[grp_idx] = sa->rss_tbl[entry];
+	}
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+}
 #endif
 
 static const struct eth_dev_ops sfc_eth_dev_ops = {
@@ -1121,6 +1151,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
 	.mac_addr_set			= sfc_mac_addr_set,
 #if EFSYS_OPT_RX_SCALE
+	.reta_query			= sfc_dev_rss_reta_query,
 	.rss_hash_update		= sfc_dev_rss_hash_update,
 	.rss_hash_conf_get		= sfc_dev_rss_hash_conf_get,
 #endif
-- 
2.5.5

[PATCH 30/31] net/sfc: add callback to update RSS redirection table

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 drivers/net/sfc/sfc_ethdev.c         | 60 ++++++++++++++++++++++++++++++++++++
 2 files changed, 61 insertions(+)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 4f6f117..07c58d5 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -16,6 +16,7 @@ Allmulticast mode    = Y
 Multicast MAC filter = Y
 RSS hash             = Y
 RSS key update       = Y
+RSS reta update      = Y
 Flow control         = Y
 VLAN offload         = P
 L3 checksum offload  = Y
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 0cd96ac..09cc46b 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -1117,6 +1117,65 @@ sfc_dev_rss_reta_query(struct rte_eth_dev *dev,
 
 	return 0;
 }
+
+static int
+sfc_dev_rss_reta_update(struct rte_eth_dev *dev,
+			struct rte_eth_rss_reta_entry64 *reta_conf,
+			uint16_t reta_size)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	unsigned int *rss_tbl_new;
+	uint16_t entry;
+	int rc;
+
+
+	if ((sa->rss_channels == 1) ||
+	    (sa->rss_support != EFX_RX_SCALE_EXCLUSIVE)) {
+		sfc_err(sa, "RSS is not available");
+		return -ENOTSUP;
+	}
+
+	if (reta_size != EFX_RSS_TBL_SIZE) {
+		sfc_err(sa, "RETA size is wrong (should be %hu)",
+			EFX_RSS_TBL_SIZE);
+		return -EINVAL;
+	}
+
+	rss_tbl_new = rte_zmalloc("rss_tbl_new", sizeof(sa->rss_tbl), 0);
+	if (rss_tbl_new == NULL)
+		return -ENOMEM;
+
+	sfc_adapter_lock(sa);
+
+	rte_memcpy(rss_tbl_new, sa->rss_tbl, sizeof(sa->rss_tbl));
+
+	for (entry = 0; entry < reta_size; entry++) {
+		int grp_idx = entry % RTE_RETA_GROUP_SIZE;
+		struct rte_eth_rss_reta_entry64 *grp;
+
+		grp = &reta_conf[entry / RTE_RETA_GROUP_SIZE];
+
+		if (grp->mask & (1ull << grp_idx)) {
+			if (grp->reta[grp_idx] >= sa->rss_channels) {
+				rc = EINVAL;
+				goto bad_reta_entry;
+			}
+			rss_tbl_new[entry] = grp->reta[grp_idx];
+		}
+	}
+
+	rc = efx_rx_scale_tbl_set(sa->nic, rss_tbl_new, EFX_RSS_TBL_SIZE);
+	if (rc == 0)
+		rte_memcpy(sa->rss_tbl, rss_tbl_new, sizeof(sa->rss_tbl));
+
+bad_reta_entry:
+	sfc_adapter_unlock(sa);
+
+	rte_free(rss_tbl_new);
+
+	SFC_ASSERT(rc >= 0);
+	return -rc;
+}
 #endif
 
 static const struct eth_dev_ops sfc_eth_dev_ops = {
@@ -1151,6 +1210,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
 	.mac_addr_set			= sfc_mac_addr_set,
 #if EFSYS_OPT_RX_SCALE
+	.reta_update			= sfc_dev_rss_reta_update,
 	.reta_query			= sfc_dev_rss_reta_query,
 	.rss_hash_update		= sfc_dev_rss_hash_update,
 	.rss_hash_conf_get		= sfc_dev_rss_hash_conf_get,
-- 
2.5.5

[PATCH 31/31] net/sfc: support firmware-assisted TSOv2

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Mark Spender <mspender@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 config/common_base                   |   1 +
 doc/guides/nics/features/sfc_efx.ini |   1 +
 doc/guides/nics/sfc_efx.rst          |   8 ++
 drivers/net/sfc/Makefile             |   4 +
 drivers/net/sfc/sfc.c                |   8 ++
 drivers/net/sfc/sfc.h                |   2 +
 drivers/net/sfc/sfc_ethdev.c         |   3 +
 drivers/net/sfc/sfc_tso.c            | 203 +++++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_tx.c             |  88 ++++++++++++++-
 drivers/net/sfc/sfc_tx.h             |  28 +++++
 10 files changed, 343 insertions(+), 3 deletions(-)
 create mode 100644 drivers/net/sfc/sfc_tso.c

diff --git a/config/common_base b/config/common_base
index 1eb8eea..8bb2a00 100644
--- a/config/common_base
+++ b/config/common_base
@@ -342,6 +342,7 @@ CONFIG_RTE_LIBRTE_PMD_XENVIRT=n
 #
 CONFIG_RTE_LIBRTE_SFC_EFX_PMD=y
 CONFIG_RTE_LIBRTE_SFC_EFX_DEBUG=n
+CONFIG_RTE_LIBRTE_SFC_EFX_TSO=n
 
 #
 # Compile null PMD
diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 07c58d5..3a15baa 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -11,6 +11,7 @@ Queue start/stop     = Y
 MTU update           = Y
 Jumbo frame          = Y
 Scattered Rx         = Y
+TSO                  = Y
 Promiscuous mode     = Y
 Allmulticast mode    = Y
 Multicast MAC filter = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 64489de..f826726 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -60,6 +60,8 @@ SFC EFX PMD has support for:
 
 - Allmulticast mode
 
+- TCP segmentation offload (TSO)
+
 - Multicast MAC filter
 
 - IPv4/IPv6 TCP/UDP receive checksum offload
@@ -166,6 +168,12 @@ Please note that enabling debugging options may affect system performance.
 
   Enable compilation of the extra run-time consistency checks.
 
+- ``CONFIG_RTE_LIBRTE_SFC_EFX_TSO`` (default **n**)
+
+  Toggle TCP segmentation offload support.
+  Enabling the feature limits the number of available transmit queues
+  significantly due to the limited number of adapter TSO contexts.
+
 
 Per-Device Parameters
 ~~~~~~~~~~~~~~~~~~~~~
diff --git a/drivers/net/sfc/Makefile b/drivers/net/sfc/Makefile
index dd099b2..14d6536 100644
--- a/drivers/net/sfc/Makefile
+++ b/drivers/net/sfc/Makefile
@@ -90,6 +90,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += sfc_port.c
 SRCS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += sfc_rx.c
 SRCS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += sfc_tx.c
 
+SRCS-$(CONFIG_RTE_LIBRTE_SFC_EFX_TSO) += sfc_tso.c
+
 VPATH += $(SRCDIR)/base
 
 SRCS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += efx_bootcfg.c
@@ -139,4 +141,6 @@ DEPDIRS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += lib/librte_mempool
 DEPDIRS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += lib/librte_mbuf
 
+DEPDIRS-$(CONFIG_RTE_LIBRTE_SFC_EFX_TSO) += lib/librte_net
+
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/sfc/sfc.c b/drivers/net/sfc/sfc.c
index e79367d..02a56f7 100644
--- a/drivers/net/sfc/sfc.c
+++ b/drivers/net/sfc/sfc.c
@@ -621,6 +621,14 @@ sfc_attach(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_set_rss_defaults;
 
+#ifdef RTE_LIBRTE_SFC_EFX_TSO
+	sa->tso = efx_nic_cfg_get(sa->nic)->enc_fw_assisted_tso_v2_enabled;
+	if (!sa->tso)
+		sfc_warn(sa, "TSO support isn't available on this adapter");
+#else /* !RTE_LIBRTE_SFC_EFX_TSO */
+	sa->tso = B_FALSE;
+#endif /* RTE_LIBRTE_SFC_EFX_TSO */
+
 	sfc_log_init(sa, "fini nic");
 	efx_nic_fini(enp);
 
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index 01dbfb6..df9c20b 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -195,6 +195,8 @@ struct sfc_adapter {
 	unsigned int			txq_count;
 	struct sfc_txq_info		*txq_info;
 
+	boolean_t			tso;
+
 	unsigned int			rss_channels;
 
 #if EFSYS_OPT_RX_SCALE
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 09cc46b..1fb763a 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -93,6 +93,9 @@ sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 	}
 #endif
 
+	if (sa->tso)
+		dev_info->tx_offload_capa |= DEV_TX_OFFLOAD_TCP_TSO;
+
 	dev_info->rx_desc_lim.nb_max = EFX_RXQ_MAXNDESCS;
 	dev_info->rx_desc_lim.nb_min = EFX_RXQ_MINNDESCS;
 	/* The RXQ hardware requires that the descriptor count is a power
diff --git a/drivers/net/sfc/sfc_tso.c b/drivers/net/sfc/sfc_tso.c
new file mode 100644
index 0000000..fa8c04d
--- /dev/null
+++ b/drivers/net/sfc/sfc_tso.c
@@ -0,0 +1,203 @@
+/*-
+ * Copyright (c) 2016 Solarflare Communications Inc.
+ * All rights reserved.
+ *
+ * This software was jointly developed between OKTET Labs (under contract
+ * for Solarflare) and Solarflare Communications, Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright notice,
+ *    this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright notice,
+ *    this list of conditions and the following disclaimer in the documentation
+ *    and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+ * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+ * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
+ * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ip.h>
+#include <rte_tcp.h>
+
+#include "sfc.h"
+#include "sfc_debug.h"
+#include "sfc_tx.h"
+#include "sfc_ev.h"
+
+/** Standard TSO header length */
+#define SFC_TSOH_STD_LEN        256
+
+/** The number of TSO option descriptors that precede the packet descriptors */
+#define SFC_TSO_OPDESCS_IDX_SHIFT	2
+
+int
+sfc_tso_alloc_tsoh_objs(struct sfc_adapter *sa, unsigned int sw_index,
+			struct sfc_txq *txq, unsigned int socket_id)
+{
+	unsigned int i;
+	struct sfc_txq_info *txq_info = &sa->txq_info[sw_index];
+
+	for (i = 0; i < txq_info->entries; ++i) {
+		txq->sw_ring[i].tsoh = rte_malloc_socket("sfc-txq-tsoh-obj",
+							 SFC_TSOH_STD_LEN,
+							 SFC_TX_SEG_BOUNDARY,
+							 socket_id);
+		if (txq->sw_ring[i].tsoh == NULL)
+			goto fail_alloc_tsoh_objs;
+	}
+
+	return 0;
+
+fail_alloc_tsoh_objs:
+	while (i > 0)
+		rte_free(txq->sw_ring[--i].tsoh);
+
+	return ENOMEM;
+}
+
+void
+sfc_tso_free_tsoh_objs(struct sfc_adapter *sa, unsigned int sw_index)
+{
+	unsigned int txds;
+	struct sfc_txq_info *txq_info = &sa->txq_info[sw_index];
+	struct sfc_txq *txq = txq_info->txq;
+
+	for (txds = 0; txds < txq_info->entries; ++txds) {
+		rte_free(txq->sw_ring[txds].tsoh);
+		txq->sw_ring[txds].tsoh = NULL;
+	}
+}
+
+static void
+sfc_tso_prepare_header(struct sfc_txq *txq, struct rte_mbuf **in_seg,
+		       size_t *in_off, unsigned int idx, size_t bytes_left)
+{
+	struct rte_mbuf *m = *in_seg;
+	size_t bytes_to_copy = 0;
+	uint8_t *tsoh = txq->sw_ring[idx & txq->ptr_mask].tsoh;
+
+	do {
+		bytes_to_copy = MIN(bytes_left, m->data_len);
+
+		rte_memcpy(tsoh, rte_pktmbuf_mtod(m, uint8_t *),
+			   bytes_to_copy);
+
+		bytes_left -= bytes_to_copy;
+		tsoh += bytes_to_copy;
+
+		if (bytes_left > 0) {
+			m = m->next;
+			SFC_ASSERT(m != NULL);
+		}
+	} while (bytes_left > 0);
+
+	if (bytes_to_copy == m->data_len) {
+		*in_seg = m->next;
+		*in_off = 0;
+	} else {
+		*in_seg = m;
+		*in_off = bytes_to_copy;
+	}
+}
+
+int
+sfc_tso_do(struct sfc_txq *txq, unsigned int idx, struct rte_mbuf **in_seg,
+	   size_t *in_off, efx_desc_t **pend, unsigned int *pkt_descs,
+	   size_t *pkt_len)
+{
+	uint8_t *tsoh;
+	const struct tcp_hdr *th;
+	efsys_dma_addr_t header_paddr;
+	efsys_dma_addr_t paddr_next_frag;
+	uint16_t packet_id;
+	uint32_t sent_seq;
+	struct rte_mbuf *m = *in_seg;
+	size_t nh_off = m->l2_len; /* IP header offset */
+	size_t tcph_off = m->l2_len + m->l3_len; /* TCP header offset */
+	size_t header_len = m->l2_len + m->l3_len + m->l4_len;
+	const efx_nic_cfg_t *encp = efx_nic_cfg_get(txq->evq->sa->nic);
+
+	idx += SFC_TSO_OPDESCS_IDX_SHIFT;
+
+	/* Packets which have too big headers should be discarded */
+	if (unlikely(header_len > SFC_TSOH_STD_LEN))
+		return EMSGSIZE;
+
+	/*
+	 * The TCP header must start at most 208 bytes into the frame.
+	 * If it starts later than this then the NIC won't realise
+	 * it's a TCP packet and TSO edits won't be applied
+	 */
+	if (unlikely(tcph_off > encp->enc_tx_tso_tcp_header_offset_limit))
+		return EMSGSIZE;
+
+	header_paddr = rte_pktmbuf_mtophys(m);
+	paddr_next_frag = P2ROUNDUP(header_paddr + 1, SFC_TX_SEG_BOUNDARY);
+
+	/*
+	 * Sometimes headers may be split across multiple mbufs. In such cases
+	 * we need to glue those pieces and store them in some temporary place.
+	 * Also, packet headers must be contiguous in memory, so that
+	 * they can be referred to with a single DMA descriptor. Hence, handle
+	 * the case where the original header crosses a 4K memory boundary
+	 */
+	if ((m->data_len < header_len) ||
+	    ((paddr_next_frag - header_paddr) < header_len)) {
+		sfc_tso_prepare_header(txq, in_seg, in_off, idx, header_len);
+		tsoh = txq->sw_ring[idx & txq->ptr_mask].tsoh;
+
+		header_paddr = rte_malloc_virt2phy((void *)tsoh);
+	} else {
+		if (m->data_len == header_len) {
+			*in_off = 0;
+			*in_seg = m->next;
+		} else {
+			*in_off = header_len;
+		}
+
+		tsoh = rte_pktmbuf_mtod(m, uint8_t *);
+	}
+
+	/* Handle IP header */
+	if (m->ol_flags & PKT_TX_IPV4) {
+		const struct ipv4_hdr *iphe4;
+
+		iphe4 = (const struct ipv4_hdr *)(tsoh + nh_off);
+		rte_memcpy(&packet_id, &iphe4->packet_id, sizeof(uint16_t));
+		packet_id = rte_be_to_cpu_16(packet_id);
+	} else if (m->ol_flags & PKT_TX_IPV6) {
+		packet_id = 0;
+	} else {
+		return EINVAL;
+	}
+
+	/* Handle TCP header */
+	th = (const struct tcp_hdr *)(tsoh + tcph_off);
+
+	rte_memcpy(&sent_seq, &th->sent_seq, sizeof(uint32_t));
+	sent_seq = rte_be_to_cpu_32(sent_seq);
+
+	efx_tx_qdesc_tso2_create(txq->common, packet_id, sent_seq, m->tso_segsz,
+				 *pend, EFX_TX_FATSOV2_OPT_NDESCS);
+
+	*pend += EFX_TX_FATSOV2_OPT_NDESCS;
+	*pkt_descs += EFX_TX_FATSOV2_OPT_NDESCS;
+
+	efx_tx_qdesc_dma_create(txq->common, header_paddr, header_len,
+				B_FALSE, (*pend)++);
+	(*pkt_descs)++;
+	*pkt_len -= header_len;
+
+	return 0;
+}
diff --git a/drivers/net/sfc/sfc_tx.c b/drivers/net/sfc/sfc_tx.c
index 86bcfec..cb21f78 100644
--- a/drivers/net/sfc/sfc_tx.c
+++ b/drivers/net/sfc/sfc_tx.c
@@ -184,6 +184,12 @@ sfc_tx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 	if (txq->sw_ring == NULL)
 		goto fail_desc_alloc;
 
+	if (sa->tso) {
+		rc = sfc_tso_alloc_tsoh_objs(sa, sw_index, txq, socket_id);
+		if (rc != 0)
+			goto fail_alloc_tsoh_objs;
+	}
+
 	txq->state = SFC_TXQ_INITIALIZED;
 	txq->ptr_mask = txq_info->entries - 1;
 	txq->free_thresh = (tx_conf->tx_free_thresh) ? tx_conf->tx_free_thresh :
@@ -199,6 +205,9 @@ sfc_tx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 
 	return 0;
 
+fail_alloc_tsoh_objs:
+	rte_free(txq->sw_ring);
+
 fail_desc_alloc:
 	rte_free(txq->pend_desc);
 
@@ -234,6 +243,8 @@ sfc_tx_qfini(struct sfc_adapter *sa, unsigned int sw_index)
 	SFC_ASSERT(txq != NULL);
 	SFC_ASSERT(txq->state == SFC_TXQ_INITIALIZED);
 
+	sfc_tso_free_tsoh_objs(sa, sw_index);
+
 	txq_info->txq = NULL;
 	txq_info->entries = 0;
 
@@ -300,6 +311,11 @@ sfc_tx_init(struct sfc_adapter *sa)
 
 	sa->txq_count = sa->eth_dev->data->nb_tx_queues;
 
+	if (sa->tso)
+		sa->txq_count = MIN(sa->txq_count,
+		   efx_nic_cfg_get(sa->nic)->enc_fw_assisted_tso_v2_n_contexts /
+		   efx_nic_cfg_get(sa->nic)->enc_hw_pf_count);
+
 	sa->txq_info = rte_calloc_socket("sfc-txqs", sa->txq_count,
 					 sizeof(sa->txq_info[0]), 0,
 					 sa->socket_id);
@@ -373,17 +389,25 @@ sfc_tx_qstart(struct sfc_adapter *sa, unsigned int sw_index)
 	 * hence, we always enable it here
 	 */
 	if ((txq->flags & ETH_TXQ_FLAGS_NOXSUMTCP) ||
-	    (txq->flags & ETH_TXQ_FLAGS_NOXSUMUDP))
+	    (txq->flags & ETH_TXQ_FLAGS_NOXSUMUDP)) {
 		flags = EFX_TXQ_CKSUM_IPV4;
-	else
+	} else {
 		flags = EFX_TXQ_CKSUM_IPV4 | EFX_TXQ_CKSUM_TCPUDP;
 
+		if (sa->tso)
+			flags |= EFX_TXQ_FATSOV2;
+	}
+
 	rc = efx_tx_qcreate(sa->nic, sw_index, 0, &txq->mem,
 			    txq_info->entries, 0 /* not used on EF10 */,
 			    flags, evq->common,
 			    &txq->common, &desc_index);
-	if (rc != 0)
+	if (rc != 0) {
+		if (sa->tso && (rc == ENOSPC))
+			sfc_err(sa, "ran out of TSO contexts");
+
 		goto fail_tx_qcreate;
+	}
 
 	txq->added = txq->pending = txq->completed = desc_index;
 	txq->hw_vlan_tci = 0;
@@ -494,6 +518,13 @@ sfc_tx_start(struct sfc_adapter *sa)
 
 	sfc_log_init(sa, "txq_count = %u", sa->txq_count);
 
+	if (sa->tso) {
+		if (!efx_nic_cfg_get(sa->nic)->enc_fw_assisted_tso_v2_enabled) {
+			sfc_warn(sa, "TSO support was unable to be restored");
+			sa->tso = B_FALSE;
+		}
+	}
+
 	rc = efx_tx_init(sa->nic);
 	if (rc != 0)
 		goto fail_efx_tx_init;
@@ -607,6 +638,7 @@ sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		struct rte_mbuf		*m_seg = *pktp;
 		size_t			pkt_len = m_seg->pkt_len;
 		unsigned int		pkt_descs = 0;
+		size_t			in_off = 0;
 
 		/*
 		 * Here VLAN TCI is expected to be zero in case if no
@@ -617,6 +649,46 @@ sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		 */
 		pkt_descs += sfc_tx_maybe_insert_tag(txq, m_seg, &pend);
 
+#ifdef RTE_LIBRTE_SFC_EFX_TSO
+		if (m_seg->ol_flags & PKT_TX_TCP_SEG) {
+			/*
+			 * We expect correct 'pkt->l[2, 3, 4]_len' values
+			 * to be set correctly by the caller
+			 */
+			if (sfc_tso_do(txq, added, &m_seg, &in_off, &pend,
+				       &pkt_descs, &pkt_len) != 0) {
+				/* We may have reached this place for
+				 * one of the following reasons:
+				 *
+				 * 1) Packet header length is greater
+				 *    than SFC_TSOH_STD_LEN
+				 * 2) TCP header starts at more then
+				 *    208 bytes into the frame
+				 *
+				 * We will deceive RTE saying that we have sent
+				 * the packet, but we will actually drop it.
+				 * Hence, we should revert 'pend' to the
+				 * previous state (in case we have added
+				 * VLAN descriptor) and start processing
+				 * another one packet. But the original
+				 * mbuf shouldn't be orphaned
+				 */
+				pend -= pkt_descs;
+
+				rte_pktmbuf_free(*pktp);
+
+				continue;
+			}
+
+			/*
+			 * We've only added 2 FATSOv2 option descriptors
+			 * and 1 descriptor for the linearized packet header.
+			 * The outstanding work will be done in the same manner
+			 * as for the usual non-TSO path
+			 */
+		}
+#endif /* RTE_LIBRTE_SFC_EFX_TSO */
+
 		for (; m_seg != NULL; m_seg = m_seg->next) {
 			efsys_dma_addr_t	next_frag;
 			size_t			seg_len;
@@ -624,6 +696,16 @@ sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 			seg_len = m_seg->data_len;
 			next_frag = rte_mbuf_data_dma_addr(m_seg);
 
+			/*
+			 * If we've started TSO transaction few steps earlier,
+			 * we'll skip packet header using an offset in the
+			 * current segment (which has been set to the
+			 * first one containing payload)
+			 */
+			seg_len -= in_off;
+			next_frag += in_off;
+			in_off = 0;
+
 			do {
 				efsys_dma_addr_t	frag_addr = next_frag;
 				size_t			frag_len;
diff --git a/drivers/net/sfc/sfc_tx.h b/drivers/net/sfc/sfc_tx.h
index 4d25c6a..d4298e4 100644
--- a/drivers/net/sfc/sfc_tx.h
+++ b/drivers/net/sfc/sfc_tx.h
@@ -50,6 +50,9 @@ struct sfc_evq;
 
 struct sfc_tx_sw_desc {
 	struct rte_mbuf		*mbuf;
+#ifdef RTE_LIBRTE_SFC_EFX_TSO
+	uint8_t			*tsoh;	/* Buffer to store TSO header */
+#endif /* RTE_LIBRTE_SFC_EFX_TSO */
 };
 
 enum sfc_txq_state_bit {
@@ -113,6 +116,31 @@ void sfc_tx_stop(struct sfc_adapter *sa);
 uint16_t sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 		       uint16_t nb_pkts);
 
+#ifdef RTE_LIBRTE_SFC_EFX_TSO
+/* From 'sfc_tso.c' */
+int sfc_tso_alloc_tsoh_objs(struct sfc_adapter *sa, unsigned int sw_index,
+			    struct sfc_txq *txq, unsigned int socket_id);
+void sfc_tso_free_tsoh_objs(struct sfc_adapter *sa, unsigned int sw_index);
+int sfc_tso_do(struct sfc_txq *txq, unsigned int idx, struct rte_mbuf **in_seg,
+	       size_t *in_off, efx_desc_t **pend, unsigned int *pkt_descs,
+	       size_t *pkt_len);
+#else /* !RTE_LIBRTE_SFC_EFX_TSO */
+static inline int
+sfc_tso_alloc_tsoh_objs(__rte_unused struct sfc_adapter *sa,
+			__rte_unused unsigned int sw_index,
+			__rte_unused struct sfc_txq *txq,
+			__rte_unused unsigned int socket_id)
+{
+	return 0;
+}
+
+static inline void
+sfc_tso_free_tsoh_objs(__rte_unused struct sfc_adapter *sa,
+		       __rte_unused unsigned int sw_index)
+{
+}
+#endif /* RTE_LIBRTE_SFC_EFX_TSO */
+
 #ifdef __cplusplus
 }
 #endif
-- 
2.5.5

Re: [PATCH 00/31] Support more features in Solarflare PMD

[Ferruh Yigit]

On 12/2/2016 7:44 AM, Andrew Rybchenko wrote:
> The patch series adds a number of features to Solarflare libefx-based
> PMD. Basically one patch per feature.
> 
> The patches are grouped into one series since they touch nearby lines
> in either PMD feature list, or dev_ops structure, or documentation.
> So, patches cannot be applied in arbitrary order.
> 
> The patch series should be applied after
> [PATCH v2 00/55] Solarflare libefx-based PMD
> (Message-ID: 1480436367-20749-1-git-send-email-arybchenko@solarflare.com)
> 
> 
> Andrew Rybchenko (16):
>   net/sfc: implement MCDI logging callback
>   net/sfc: support parameter to choose performance profile
>   net/sfc: implement ethdev hook to get basic statistics
>   net/sfc: support extended statistics
>   net/sfc: support flow control settings get/set
>   net/sfc: support link status change interrupt
>   net/sfc: implement device operation to change MTU
>   net/sfc: support link speed and duplex settings
>   net/sfc: support checksum offloads on receive
>   net/sfc: handle received packet type info provided by HW
>   net/sfc: support callback to get receive queue information
>   net/sfc: support Rx free threshold
>   net/sfc: add callback to get RxQ pending descriptors count
>   net/sfc: add RxQ descriptor done callback
>   net/sfc: support scattered Rx DMA
>   net/sfc: support deferred start of receive queues
> 
> Artem Andreev (1):
>   net/sfc: support link up/down
> 
> Ivan Malov (14):
>   net/sfc: support promiscuous and all-multicast control
>   net/sfc: support main (the first) MAC address change
>   net/sfc: support multicast addresses list controls
>   net/sfc: add callback to get transmit queue information
>   net/sfc: support Tx free threshold
>   net/sfc: support deferred start of transmit queues
>   net/sfc: support VLAN offload on transmit path
>   net/sfc: add basic stubs for RSS support on driver attach
>   net/sfc: support RSS hash offload
>   net/sfc: add callback to query RSS key and hash types config
>   net/sfc: add callback to set RSS key and hash types config
>   net/sfc: add callback to query RSS redirection table
>   net/sfc: add callback to update RSS redirection table
>   net/sfc: support firmware-assisted TSOv2

Hi Andrew,

I am getting following build errors for clang [1] and ICC [2]. I have
not investigated the root cause, just copy-pasting here.

For ICC, since you explicitly noted it is not supported, and reported
warning is known, I believe it is safe the ignore this warning via
"-wd188" CFLAGS option in the Makefile.

Thanks,
ferruh



[1] clang
.../drivers/net/sfc/sfc_ethdev.c:1143:4: error: format specifies type
'unsigned short' but the argument has type 'int' [-Werror,-Wformat]
                        EFX_RSS_TBL_SIZE);
                        ^~~~~~~~~~~~~~~~
.../drivers/net/sfc/base/efx.h:1866:26: note: expanded from macro
'EFX_RSS_TBL_SIZE'
#define EFX_RSS_TBL_SIZE        128     /* Rows in RX indirection table */
                                ^~~
.../drivers/net/sfc/sfc_log.h:51:19: note: expanded from macro 'sfc_err'
        SFC_LOG(sa, ERR, __VA_ARGS__)
                         ^~~~~~~~~~~
.../drivers/net/sfc/sfc_log.h:47:18: note: expanded from macro 'SFC_LOG'
                                RTE_FMT_TAIL(__VA_ARGS__,)));           \
                                             ^~~~~~~~~~~
.../x86_64-native-linuxapp-clang/include/rte_common.h:345:32: note:
expanded from macro 'RTE_FMT_TAIL'
#define RTE_FMT_TAIL(fmt, ...) __VA_ARGS__
                               ^~~~~~~~~~~
.../x86_64-native-linuxapp-clang/include/rte_common.h:343:39: note:
expanded from macro 'RTE_FMT'
#define RTE_FMT(fmt, ...) fmt "%.0s", __VA_ARGS__ ""
                                      ^~~~~~~~~~~
.../x86_64-native-linuxapp-clang/include/rte_log.h:258:32: note:
expanded from macro 'RTE_LOG'
                 RTE_LOGTYPE_ ## t, # t ": " __VA_ARGS__)
                                             ^~~~~~~~~~~



[2] ICC
.../drivers/net/sfc/sfc_ethdev.c(1063): error #188: enumerated type
mixed with another type
                                   efx_hash_types, B_TRUE);
                                   ^

.../drivers/net/sfc/sfc_ethdev.c(1086): error #188: enumerated type
mixed with another type
                                  sa->rss_hash_types, B_TRUE) != 0)
                                  ^

compilation aborted for .../drivers/net/sfc/sfc_ethdev.c (code 2)
make[7]: *** [sfc_ethdev.o] Error 2
make[7]: *** Waiting for unfinished jobs....
.../drivers/net/sfc/sfc_rx.c(820): error #188: enumerated type mixed
with another type
                                           sa->rss_hash_types, B_TRUE);
                                           ^

compilation aborted for .../drivers/net/sfc/sfc_rx.c (code 2)

Re: [PATCH 00/31] Support more features in Solarflare PMD

[Andrew Rybchenko]

On 12/09/2016 08:34 PM, Ferruh Yigit wrote:
> On 12/2/2016 7:44 AM, Andrew Rybchenko wrote:
>> The patch series adds a number of features to Solarflare libefx-based
>> PMD. Basically one patch per feature.
>>
>> The patches are grouped into one series since they touch nearby lines
>> in either PMD feature list, or dev_ops structure, or documentation.
>> So, patches cannot be applied in arbitrary order.
>>
>> The patch series should be applied after
>> [PATCH v2 00/55] Solarflare libefx-based PMD
>> (Message-ID: 1480436367-20749-1-git-send-email-arybchenko@solarflare.com)
>>
>>
>> Andrew Rybchenko (16):
>>    net/sfc: implement MCDI logging callback
>>    net/sfc: support parameter to choose performance profile
>>    net/sfc: implement ethdev hook to get basic statistics
>>    net/sfc: support extended statistics
>>    net/sfc: support flow control settings get/set
>>    net/sfc: support link status change interrupt
>>    net/sfc: implement device operation to change MTU
>>    net/sfc: support link speed and duplex settings
>>    net/sfc: support checksum offloads on receive
>>    net/sfc: handle received packet type info provided by HW
>>    net/sfc: support callback to get receive queue information
>>    net/sfc: support Rx free threshold
>>    net/sfc: add callback to get RxQ pending descriptors count
>>    net/sfc: add RxQ descriptor done callback
>>    net/sfc: support scattered Rx DMA
>>    net/sfc: support deferred start of receive queues
>>
>> Artem Andreev (1):
>>    net/sfc: support link up/down
>>
>> Ivan Malov (14):
>>    net/sfc: support promiscuous and all-multicast control
>>    net/sfc: support main (the first) MAC address change
>>    net/sfc: support multicast addresses list controls
>>    net/sfc: add callback to get transmit queue information
>>    net/sfc: support Tx free threshold
>>    net/sfc: support deferred start of transmit queues
>>    net/sfc: support VLAN offload on transmit path
>>    net/sfc: add basic stubs for RSS support on driver attach
>>    net/sfc: support RSS hash offload
>>    net/sfc: add callback to query RSS key and hash types config
>>    net/sfc: add callback to set RSS key and hash types config
>>    net/sfc: add callback to query RSS redirection table
>>    net/sfc: add callback to update RSS redirection table
>>    net/sfc: support firmware-assisted TSOv2
> Hi Andrew,
>
> I am getting following build errors for clang [1] and ICC [2]. I have
> not investigated the root cause, just copy-pasting here.
>
> For ICC, since you explicitly noted it is not supported, and reported
> warning is known, I believe it is safe the ignore this warning via
> "-wd188" CFLAGS option in the Makefile.

Hi Ferruh,

I think I prefer to fix these warnings in v2. May be other compilers 
will become more pedantic in the future.

Thanks,
Andrew.

>
> Thanks,
> ferruh
>
>
>
> [1] clang
> .../drivers/net/sfc/sfc_ethdev.c:1143:4: error: format specifies type
> 'unsigned short' but the argument has type 'int' [-Werror,-Wformat]
>                          EFX_RSS_TBL_SIZE);
>                          ^~~~~~~~~~~~~~~~
> .../drivers/net/sfc/base/efx.h:1866:26: note: expanded from macro
> 'EFX_RSS_TBL_SIZE'
> #define EFX_RSS_TBL_SIZE        128     /* Rows in RX indirection table */
>                                  ^~~
> .../drivers/net/sfc/sfc_log.h:51:19: note: expanded from macro 'sfc_err'
>          SFC_LOG(sa, ERR, __VA_ARGS__)
>                           ^~~~~~~~~~~
> .../drivers/net/sfc/sfc_log.h:47:18: note: expanded from macro 'SFC_LOG'
>                                  RTE_FMT_TAIL(__VA_ARGS__,)));           \
>                                               ^~~~~~~~~~~
> .../x86_64-native-linuxapp-clang/include/rte_common.h:345:32: note:
> expanded from macro 'RTE_FMT_TAIL'
> #define RTE_FMT_TAIL(fmt, ...) __VA_ARGS__
>                                 ^~~~~~~~~~~
> .../x86_64-native-linuxapp-clang/include/rte_common.h:343:39: note:
> expanded from macro 'RTE_FMT'
> #define RTE_FMT(fmt, ...) fmt "%.0s", __VA_ARGS__ ""
>                                        ^~~~~~~~~~~
> .../x86_64-native-linuxapp-clang/include/rte_log.h:258:32: note:
> expanded from macro 'RTE_LOG'
>                   RTE_LOGTYPE_ ## t, # t ": " __VA_ARGS__)
>                                               ^~~~~~~~~~~
>
>
>
> [2] ICC
> .../drivers/net/sfc/sfc_ethdev.c(1063): error #188: enumerated type
> mixed with another type
>                                     efx_hash_types, B_TRUE);
>                                     ^
>
> .../drivers/net/sfc/sfc_ethdev.c(1086): error #188: enumerated type
> mixed with another type
>                                    sa->rss_hash_types, B_TRUE) != 0)
>                                    ^
>
> compilation aborted for .../drivers/net/sfc/sfc_ethdev.c (code 2)
> make[7]: *** [sfc_ethdev.o] Error 2
> make[7]: *** Waiting for unfinished jobs....
> .../drivers/net/sfc/sfc_rx.c(820): error #188: enumerated type mixed
> with another type
>                                             sa->rss_hash_types, B_TRUE);
>                                             ^
>
> compilation aborted for .../drivers/net/sfc/sfc_rx.c (code 2)
>

[PATCH v2 00/32] Support more features in Solarflare PMD

[Andrew Rybchenko]

The patch series adds a number of features to Solarflare libefx-based
PMD. Basically one patch per feature.

The patches are grouped into one series since they touch nearby lines
in either PMD feature list, or dev_ops structure, or documentation.
So, patches cannot be applied in arbitrary order.

---

v2:
* Fix ICC and clang warnings
* Slightly change sfc_tso_{alloc,free}_tsoh_objs() prototypes


Andrew Rybchenko (17):
  net/sfc: implement MCDI logging callback
  net/sfc: support parameter to choose performance profile
  net/sfc: implement ethdev hook to get basic statistics
  net/sfc: support extended statistics
  net/sfc: support flow control settings get/set
  net/sfc: support link status change interrupt
  net/sfc: implement device operation to change MTU
  net/sfc: support link speed and duplex settings
  net/sfc: support checksum offloads on receive
  net/sfc: handle received packet type info provided by HW
  net/sfc: support callback to get receive queue information
  net/sfc: support Rx free threshold
  net/sfc: add callback to get RxQ pending descriptors count
  net/sfc: add RxQ descriptor done callback
  net/sfc: support scattered Rx DMA
  net/sfc: support deferred start of receive queues
  net/sfc/base: do not use enum type when values are bitmask

Artem Andreev (1):
  net/sfc: support link up/down

Ivan Malov (14):
  net/sfc: support promiscuous and all-multicast control
  net/sfc: support main (the first) MAC address change
  net/sfc: support multicast addresses list controls
  net/sfc: add callback to get transmit queue information
  net/sfc: support Tx free threshold
  net/sfc: support deferred start of transmit queues
  net/sfc: support VLAN offload on transmit path
  net/sfc: add basic stubs for RSS support on driver attach
  net/sfc: support RSS hash offload
  net/sfc: add callback to query RSS key and hash types config
  net/sfc: add callback to set RSS key and hash types config
  net/sfc: add callback to query RSS redirection table
  net/sfc: add callback to update RSS redirection table
  net/sfc: support firmware-assisted TSOv2

 config/common_base                   |   1 +
 doc/guides/nics/features/sfc_efx.ini |  22 +-
 doc/guides/nics/sfc_efx.rst          |  58 ++-
 drivers/net/sfc/Makefile             |   4 +
 drivers/net/sfc/base/ef10_rx.c       |   8 +-
 drivers/net/sfc/base/efx.h           |  12 +-
 drivers/net/sfc/base/efx_rx.c        |   8 +-
 drivers/net/sfc/efsys.h              |   8 +-
 drivers/net/sfc/sfc.c                | 126 ++++-
 drivers/net/sfc/sfc.h                |  46 ++
 drivers/net/sfc/sfc_ethdev.c         | 893 ++++++++++++++++++++++++++++++++++-
 drivers/net/sfc/sfc_ev.c             |  64 ++-
 drivers/net/sfc/sfc_ev.h             |   2 +
 drivers/net/sfc/sfc_intr.c           | 204 ++++++++
 drivers/net/sfc/sfc_kvargs.c         |   2 +
 drivers/net/sfc/sfc_kvargs.h         |  12 +
 drivers/net/sfc/sfc_mcdi.c           |  69 +++
 drivers/net/sfc/sfc_port.c           | 107 ++++-
 drivers/net/sfc/sfc_rx.c             | 288 ++++++++++-
 drivers/net/sfc/sfc_rx.h             |  16 +
 drivers/net/sfc/sfc_tso.c            | 200 ++++++++
 drivers/net/sfc/sfc_tweak.h          |   3 +
 drivers/net/sfc/sfc_tx.c             | 166 ++++++-
 drivers/net/sfc/sfc_tx.h             |  41 +-
 24 files changed, 2274 insertions(+), 86 deletions(-)
 create mode 100644 drivers/net/sfc/sfc_tso.c

-- 
2.5.5

[PATCH v2 01/32] net/sfc: implement MCDI logging callback

[Andrew Rybchenko]

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/sfc_efx.rst  |  6 ++++
 drivers/net/sfc/efsys.h      |  2 +-
 drivers/net/sfc/sfc.h        |  1 +
 drivers/net/sfc/sfc_ethdev.c |  1 +
 drivers/net/sfc/sfc_kvargs.c |  1 +
 drivers/net/sfc/sfc_kvargs.h |  2 ++
 drivers/net/sfc/sfc_mcdi.c   | 69 ++++++++++++++++++++++++++++++++++++++++++++
 7 files changed, 81 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index aadd775..2cca287 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -155,3 +155,9 @@ boolean parameters value.
 - ``debug_init`` [bool] (default **n**)
 
   Enable extra logging during device intialization and startup.
+
+- ``mcdi_logging`` [bool] (default **n**)
+
+  Enable extra logging of the communication with the NIC's management CPU.
+  The logging is done using RTE_LOG() with INFO level and PMD type.
+  The format is consumed by the Solarflare netlogdecode cross-platform tool.
diff --git a/drivers/net/sfc/efsys.h b/drivers/net/sfc/efsys.h
index e4d5035..d48eb4c 100644
--- a/drivers/net/sfc/efsys.h
+++ b/drivers/net/sfc/efsys.h
@@ -175,7 +175,7 @@ prefetch_read_once(const volatile void *addr)
 
 /* MCDI is required for SFN7xxx and SFN8xx */
 #define EFSYS_OPT_MCDI 1
-#define EFSYS_OPT_MCDI_LOGGING 0
+#define EFSYS_OPT_MCDI_LOGGING 1
 #define EFSYS_OPT_MCDI_PROXY_AUTH 0
 
 #define EFSYS_OPT_MAC_STATS 0
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index 995dfe6..46c6fb6 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -107,6 +107,7 @@ struct sfc_mcdi {
 	efsys_mem_t			mem;
 	enum sfc_mcdi_state		state;
 	efx_mcdi_transport_t		transport;
+	bool				logging;
 };
 
 struct sfc_intr {
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index c28082c..12309ee 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -464,4 +464,5 @@ static struct eth_driver sfc_efx_pmd = {
 RTE_PMD_REGISTER_PCI(net_sfc_efx, sfc_efx_pmd.pci_drv);
 RTE_PMD_REGISTER_PCI_TABLE(net_sfc_efx, pci_id_sfc_efx_map);
 RTE_PMD_REGISTER_PARAM_STRING(net_sfc_efx,
+	SFC_KVARG_MCDI_LOGGING "=" SFC_KVARG_VALUES_BOOL " "
 	SFC_KVARG_DEBUG_INIT "=" SFC_KVARG_VALUES_BOOL);
diff --git a/drivers/net/sfc/sfc_kvargs.c b/drivers/net/sfc/sfc_kvargs.c
index f1bab28..bbbd026 100644
--- a/drivers/net/sfc/sfc_kvargs.c
+++ b/drivers/net/sfc/sfc_kvargs.c
@@ -42,6 +42,7 @@ sfc_kvargs_parse(struct sfc_adapter *sa)
 	struct rte_devargs *devargs = sa->eth_dev->pci_dev->device.devargs;
 	const char **params = (const char *[]){
 		SFC_KVARG_DEBUG_INIT,
+		SFC_KVARG_MCDI_LOGGING,
 		NULL,
 	};
 
diff --git a/drivers/net/sfc/sfc_kvargs.h b/drivers/net/sfc/sfc_kvargs.h
index 0b53963..ffce851 100644
--- a/drivers/net/sfc/sfc_kvargs.h
+++ b/drivers/net/sfc/sfc_kvargs.h
@@ -40,6 +40,8 @@ extern "C" {
 
 #define SFC_KVARG_DEBUG_INIT		"debug_init"
 
+#define SFC_KVARG_MCDI_LOGGING		"mcdi_logging"
+
 struct sfc_adapter;
 
 int sfc_kvargs_parse(struct sfc_adapter *sa);
diff --git a/drivers/net/sfc/sfc_mcdi.c b/drivers/net/sfc/sfc_mcdi.c
index 9ba28e1..3bed2e0 100644
--- a/drivers/net/sfc/sfc_mcdi.c
+++ b/drivers/net/sfc/sfc_mcdi.c
@@ -35,6 +35,7 @@
 
 #include "sfc.h"
 #include "sfc_log.h"
+#include "sfc_kvargs.h"
 
 #define SFC_MCDI_POLL_INTERVAL_MIN_US	10		/* 10us in 1us units */
 #define SFC_MCDI_POLL_INTERVAL_MAX_US	(US_PER_S / 10)	/* 100ms in 1us units */
@@ -125,6 +126,65 @@ sfc_mcdi_exception(void *arg, efx_mcdi_exception_t eme)
 	sfc_panic(sa, "MCDI exceptions handling is not implemented\n");
 }
 
+#define SFC_MCDI_LOG_BUF_SIZE	128
+
+static size_t
+sfc_mcdi_do_log(const struct sfc_adapter *sa,
+		char *buffer, void *data, size_t data_size,
+		size_t pfxsize, size_t position)
+{
+	uint32_t *words = data;
+	/* Space separator plus 2 characters per byte */
+	const size_t word_str_space = 1 + 2 * sizeof(*words);
+	size_t i;
+
+	for (i = 0; i < data_size; i += sizeof(*words)) {
+		if (position + word_str_space >=
+		    SFC_MCDI_LOG_BUF_SIZE) {
+			/* Flush at SFC_MCDI_LOG_BUF_SIZE with backslash
+			 * at the end which is required by netlogdecode.
+			 */
+			buffer[position] = '\0';
+			sfc_info(sa, "%s \\", buffer);
+			/* Preserve prefix for the next log message */
+			position = pfxsize;
+		}
+		position += snprintf(buffer + position,
+				     SFC_MCDI_LOG_BUF_SIZE - position,
+				     " %08x", *words);
+		words++;
+	}
+	return position;
+}
+
+static void
+sfc_mcdi_logger(void *arg, efx_log_msg_t type,
+		void *header, size_t header_size,
+		void *data, size_t data_size)
+{
+	struct sfc_adapter *sa = (struct sfc_adapter *)arg;
+	char buffer[SFC_MCDI_LOG_BUF_SIZE];
+	size_t pfxsize;
+	size_t start;
+
+	if (!sa->mcdi.logging)
+		return;
+
+	/* The format including prefix added by sfc_info() is the format
+	 * consumed by the Solarflare netlogdecode tool.
+	 */
+	pfxsize = snprintf(buffer, sizeof(buffer), "MCDI RPC %s:",
+			   type == EFX_LOG_MCDI_REQUEST ? "REQ" :
+			   type == EFX_LOG_MCDI_RESPONSE ? "RESP" : "???");
+	start = sfc_mcdi_do_log(sa, buffer, header, header_size,
+				pfxsize, pfxsize);
+	start = sfc_mcdi_do_log(sa, buffer, data, data_size, pfxsize, start);
+	if (start != pfxsize) {
+		buffer[start] = '\0';
+		sfc_info(sa, "%s", buffer);
+	}
+}
+
 int
 sfc_mcdi_init(struct sfc_adapter *sa)
 {
@@ -149,12 +209,19 @@ sfc_mcdi_init(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_dma_alloc;
 
+	/* Convert negative error to positive used in the driver */
+	rc = sfc_kvargs_process(sa, SFC_KVARG_MCDI_LOGGING,
+				sfc_kvarg_bool_handler, &mcdi->logging);
+	if (rc != 0)
+		goto fail_kvargs_process;
+
 	emtp = &mcdi->transport;
 	emtp->emt_context = sa;
 	emtp->emt_dma_mem = &mcdi->mem;
 	emtp->emt_execute = sfc_mcdi_execute;
 	emtp->emt_ev_cpl = sfc_mcdi_ev_cpl;
 	emtp->emt_exception = sfc_mcdi_exception;
+	emtp->emt_logger = sfc_mcdi_logger;
 
 	sfc_log_init(sa, "init MCDI");
 	rc = efx_mcdi_init(sa->nic, emtp);
@@ -165,6 +232,8 @@ sfc_mcdi_init(struct sfc_adapter *sa)
 
 fail_mcdi_init:
 	memset(emtp, 0, sizeof(*emtp));
+
+fail_kvargs_process:
 	sfc_dma_free(sa, &mcdi->mem);
 
 fail_dma_alloc:
-- 
2.5.5

[PATCH v2 02/32] net/sfc: support parameter to choose performance profile

[Andrew Rybchenko]

Supported options are auto (based on NIC firmware variant and
installed licences), throughput, low-latency.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/sfc_efx.rst  |  7 +++++++
 drivers/net/sfc/sfc.h        |  1 +
 drivers/net/sfc/sfc_ethdev.c |  1 +
 drivers/net/sfc/sfc_ev.c     | 36 +++++++++++++++++++++++++++++++++---
 drivers/net/sfc/sfc_ev.h     |  2 ++
 drivers/net/sfc/sfc_kvargs.c |  1 +
 drivers/net/sfc/sfc_kvargs.h | 10 ++++++++++
 7 files changed, 55 insertions(+), 3 deletions(-)

diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 2cca287..36d0974 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -152,6 +152,13 @@ whitelist option like "-w 02:00.0,arg1=value1,...".
 Case-insensitive 1/y/yes/on or 0/n/no/off may be used to specify
 boolean parameters value.
 
+- ``perf_profile`` [auto|throughput|low-latency] (default **throughput**)
+
+  Choose hardware tunning to be optimized for either throughput or
+  low-latency.
+  **auto** allows NIC firmware to make a choice based on
+  installed licences and firmware variant configured using **sfboot**.
+
 - ``debug_init`` [bool] (default **n**)
 
   Enable extra logging during device intialization and startup.
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index 46c6fb6..00b186f 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -152,6 +152,7 @@ struct sfc_adapter {
 
 	unsigned int			txq_max_entries;
 
+	uint32_t			evq_flags;
 	unsigned int			evq_count;
 	struct sfc_evq_info		*evq_info;
 
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 12309ee..1df227e 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -464,5 +464,6 @@ static struct eth_driver sfc_efx_pmd = {
 RTE_PMD_REGISTER_PCI(net_sfc_efx, sfc_efx_pmd.pci_drv);
 RTE_PMD_REGISTER_PCI_TABLE(net_sfc_efx, pci_id_sfc_efx_map);
 RTE_PMD_REGISTER_PARAM_STRING(net_sfc_efx,
+	SFC_KVARG_PERF_PROFILE "=" SFC_KVARG_VALUES_PERF_PROFILE " "
 	SFC_KVARG_MCDI_LOGGING "=" SFC_KVARG_VALUES_BOOL " "
 	SFC_KVARG_DEBUG_INIT "=" SFC_KVARG_VALUES_BOOL);
diff --git a/drivers/net/sfc/sfc_ev.c b/drivers/net/sfc/sfc_ev.c
index af3c7b2..de42845 100644
--- a/drivers/net/sfc/sfc_ev.c
+++ b/drivers/net/sfc/sfc_ev.c
@@ -40,6 +40,7 @@
 #include "sfc_ev.h"
 #include "sfc_rx.h"
 #include "sfc_tx.h"
+#include "sfc_kvargs.h"
 
 
 /* Initial delay when waiting for event queue init complete event */
@@ -405,9 +406,7 @@ sfc_ev_qstart(struct sfc_adapter *sa, unsigned int sw_index)
 
 	/* Create the common code event queue */
 	rc = efx_ev_qcreate(sa->nic, sw_index, esmp, evq_info->entries,
-			    0 /* unused on EF10 */, 0,
-			    EFX_EVQ_FLAGS_TYPE_THROUGHPUT |
-			    EFX_EVQ_FLAGS_NOTIFY_DISABLED,
+			    0 /* unused on EF10 */, 0, evq_info->flags,
 			    &evq->common);
 	if (rc != 0)
 		goto fail_ev_qcreate;
@@ -640,6 +639,25 @@ sfc_ev_qinit_info(struct sfc_adapter *sa, unsigned int sw_index)
 	SFC_ASSERT(rte_is_power_of_2(max_entries));
 
 	evq_info->max_entries = max_entries;
+	evq_info->flags = sa->evq_flags | EFX_EVQ_FLAGS_NOTIFY_DISABLED;
+
+	return 0;
+}
+
+static int
+sfc_kvarg_perf_profile_handler(__rte_unused const char *key,
+			       const char *value_str, void *opaque)
+{
+	uint64_t *value = opaque;
+
+	if (strcasecmp(value_str, SFC_KVARG_PERF_PROFILE_THROUGHPUT) == 0)
+		*value = EFX_EVQ_FLAGS_TYPE_THROUGHPUT;
+	else if (strcasecmp(value_str, SFC_KVARG_PERF_PROFILE_LOW_LATENCY) == 0)
+		*value = EFX_EVQ_FLAGS_TYPE_LOW_LATENCY;
+	else if (strcasecmp(value_str, SFC_KVARG_PERF_PROFILE_AUTO) == 0)
+		*value = EFX_EVQ_FLAGS_TYPE_AUTO;
+	else
+		return -EINVAL;
 
 	return 0;
 }
@@ -660,6 +678,16 @@ sfc_ev_init(struct sfc_adapter *sa)
 
 	sfc_log_init(sa, "entry");
 
+	sa->evq_flags = EFX_EVQ_FLAGS_TYPE_THROUGHPUT;
+	rc = sfc_kvargs_process(sa, SFC_KVARG_PERF_PROFILE,
+				sfc_kvarg_perf_profile_handler,
+				&sa->evq_flags);
+	if (rc != 0) {
+		sfc_err(sa, "invalid %s parameter value",
+			SFC_KVARG_PERF_PROFILE);
+		goto fail_kvarg_perf_profile;
+	}
+
 	sa->evq_count = sfc_ev_qcount(sa);
 	sa->mgmt_evq_index = 0;
 	rte_spinlock_init(&sa->mgmt_evq_lock);
@@ -700,6 +728,8 @@ sfc_ev_init(struct sfc_adapter *sa)
 
 fail_evqs_alloc:
 	sa->evq_count = 0;
+
+fail_kvarg_perf_profile:
 	sfc_log_init(sa, "failed %d", rc);
 	return rc;
 }
diff --git a/drivers/net/sfc/sfc_ev.h b/drivers/net/sfc/sfc_ev.h
index 110f3b6..346e3ec 100644
--- a/drivers/net/sfc/sfc_ev.h
+++ b/drivers/net/sfc/sfc_ev.h
@@ -74,6 +74,8 @@ struct sfc_evq_info {
 	unsigned int		max_entries;
 	/* Real number of EVQ entries, less or equal to max_entries */
 	unsigned int		entries;
+	/* Event queue creation flags */
+	uint32_t		flags;
 	/* NUMA-aware EVQ data structure used on datapath */
 	struct sfc_evq		*evq;
 };
diff --git a/drivers/net/sfc/sfc_kvargs.c b/drivers/net/sfc/sfc_kvargs.c
index bbbd026..2ced47c 100644
--- a/drivers/net/sfc/sfc_kvargs.c
+++ b/drivers/net/sfc/sfc_kvargs.c
@@ -43,6 +43,7 @@ sfc_kvargs_parse(struct sfc_adapter *sa)
 	const char **params = (const char *[]){
 		SFC_KVARG_DEBUG_INIT,
 		SFC_KVARG_MCDI_LOGGING,
+		SFC_KVARG_PERF_PROFILE,
 		NULL,
 	};
 
diff --git a/drivers/net/sfc/sfc_kvargs.h b/drivers/net/sfc/sfc_kvargs.h
index ffce851..2fea9c7 100644
--- a/drivers/net/sfc/sfc_kvargs.h
+++ b/drivers/net/sfc/sfc_kvargs.h
@@ -42,6 +42,16 @@ extern "C" {
 
 #define SFC_KVARG_MCDI_LOGGING		"mcdi_logging"
 
+#define SFC_KVARG_PERF_PROFILE		"perf_profile"
+
+#define SFC_KVARG_PERF_PROFILE_AUTO		"auto"
+#define SFC_KVARG_PERF_PROFILE_THROUGHPUT	"throughput"
+#define SFC_KVARG_PERF_PROFILE_LOW_LATENCY	"low-latency"
+#define SFC_KVARG_VALUES_PERF_PROFILE \
+	"[" SFC_KVARG_PERF_PROFILE_AUTO "|" \
+	    SFC_KVARG_PERF_PROFILE_THROUGHPUT "|" \
+	    SFC_KVARG_PERF_PROFILE_LOW_LATENCY "]"
+
 struct sfc_adapter;
 
 int sfc_kvargs_parse(struct sfc_adapter *sa);
-- 
2.5.5

[PATCH v2 03/32] net/sfc: implement ethdev hook to get basic statistics

[Andrew Rybchenko]

Does not implement any deprecated statistics.
No per-queue statistics yet.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 +
 drivers/net/sfc/efsys.h              |  2 +-
 drivers/net/sfc/sfc.h                |  7 ++++
 drivers/net/sfc/sfc_ethdev.c         | 69 +++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_port.c           | 75 ++++++++++++++++++++++++++++++++++++
 6 files changed, 155 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 67df1c6..f55a988 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -7,6 +7,7 @@
 Link status          = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
+Basic stats          = Y
 BSD nic_uio          = Y
 Linux UIO            = Y
 Linux VFIO           = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 36d0974..cbb51de 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -48,6 +48,8 @@ SFC EFX PMD has support for:
 
 - IPv4/IPv6 TCP/UDP transmit checksum offload
 
+- Port hardware statistics
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/efsys.h b/drivers/net/sfc/efsys.h
index d48eb4c..fe8615f 100644
--- a/drivers/net/sfc/efsys.h
+++ b/drivers/net/sfc/efsys.h
@@ -178,7 +178,7 @@ prefetch_read_once(const volatile void *addr)
 #define EFSYS_OPT_MCDI_LOGGING 1
 #define EFSYS_OPT_MCDI_PROXY_AUTH 0
 
-#define EFSYS_OPT_MAC_STATS 0
+#define EFSYS_OPT_MAC_STATS 1
 
 #define EFSYS_OPT_LOOPBACK 0
 
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index 00b186f..e9f5183 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -122,6 +122,12 @@ struct sfc_port {
 	unsigned int			flow_ctrl;
 	boolean_t			flow_ctrl_autoneg;
 	size_t				pdu;
+
+	rte_spinlock_t			mac_stats_lock;
+	uint64_t			*mac_stats_buf;
+	efsys_mem_t			mac_stats_dma_mem;
+
+	uint32_t		mac_stats_mask[EFX_MAC_STATS_MASK_NPAGES];
 };
 
 /* Adapter private data */
@@ -235,6 +241,7 @@ int sfc_port_start(struct sfc_adapter *sa);
 void sfc_port_stop(struct sfc_adapter *sa);
 void sfc_port_link_mode_to_info(efx_link_mode_t link_mode,
 				struct rte_eth_link *link_info);
+int sfc_port_update_mac_stats(struct sfc_adapter *sa);
 
 
 #ifdef __cplusplus
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 1df227e..f31330c 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -324,12 +324,81 @@ sfc_tx_queue_release(void *queue)
 	sfc_adapter_unlock(sa);
 }
 
+static void
+sfc_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	struct sfc_port *port = &sa->port;
+	uint64_t *mac_stats;
+
+	rte_spinlock_lock(&port->mac_stats_lock);
+
+	if (sfc_port_update_mac_stats(sa) != 0)
+		goto unlock;
+
+	mac_stats = port->mac_stats_buf;
+
+	if (EFX_MAC_STAT_SUPPORTED(port->mac_stats_mask,
+				   EFX_MAC_VADAPTER_RX_UNICAST_PACKETS)) {
+		stats->ipackets =
+			mac_stats[EFX_MAC_VADAPTER_RX_UNICAST_PACKETS] +
+			mac_stats[EFX_MAC_VADAPTER_RX_MULTICAST_PACKETS] +
+			mac_stats[EFX_MAC_VADAPTER_RX_BROADCAST_PACKETS];
+		stats->opackets =
+			mac_stats[EFX_MAC_VADAPTER_TX_UNICAST_PACKETS] +
+			mac_stats[EFX_MAC_VADAPTER_TX_MULTICAST_PACKETS] +
+			mac_stats[EFX_MAC_VADAPTER_TX_BROADCAST_PACKETS];
+		stats->ibytes =
+			mac_stats[EFX_MAC_VADAPTER_RX_UNICAST_BYTES] +
+			mac_stats[EFX_MAC_VADAPTER_RX_MULTICAST_BYTES] +
+			mac_stats[EFX_MAC_VADAPTER_RX_BROADCAST_BYTES];
+		stats->obytes =
+			mac_stats[EFX_MAC_VADAPTER_TX_UNICAST_BYTES] +
+			mac_stats[EFX_MAC_VADAPTER_TX_MULTICAST_BYTES] +
+			mac_stats[EFX_MAC_VADAPTER_TX_BROADCAST_BYTES];
+		stats->imissed = mac_stats[EFX_MAC_VADAPTER_RX_OVERFLOW];
+		stats->ierrors = mac_stats[EFX_MAC_VADAPTER_RX_BAD_PACKETS];
+		stats->oerrors = mac_stats[EFX_MAC_VADAPTER_TX_BAD_PACKETS];
+	} else {
+		stats->ipackets = mac_stats[EFX_MAC_RX_PKTS];
+		stats->opackets = mac_stats[EFX_MAC_TX_PKTS];
+		stats->ibytes = mac_stats[EFX_MAC_RX_OCTETS];
+		stats->obytes = mac_stats[EFX_MAC_TX_OCTETS];
+		/*
+		 * Take into account stats which are whenever supported
+		 * on EF10. If some stat is not supported by current
+		 * firmware variant or HW revision, it is guaranteed
+		 * to be zero in mac_stats.
+		 */
+		stats->imissed =
+			mac_stats[EFX_MAC_RX_NODESC_DROP_CNT] +
+			mac_stats[EFX_MAC_PM_TRUNC_BB_OVERFLOW] +
+			mac_stats[EFX_MAC_PM_DISCARD_BB_OVERFLOW] +
+			mac_stats[EFX_MAC_PM_TRUNC_VFIFO_FULL] +
+			mac_stats[EFX_MAC_PM_DISCARD_VFIFO_FULL] +
+			mac_stats[EFX_MAC_PM_TRUNC_QBB] +
+			mac_stats[EFX_MAC_PM_DISCARD_QBB] +
+			mac_stats[EFX_MAC_PM_DISCARD_MAPPING] +
+			mac_stats[EFX_MAC_RXDP_Q_DISABLED_PKTS] +
+			mac_stats[EFX_MAC_RXDP_DI_DROPPED_PKTS];
+		stats->ierrors =
+			mac_stats[EFX_MAC_RX_FCS_ERRORS] +
+			mac_stats[EFX_MAC_RX_ALIGN_ERRORS] +
+			mac_stats[EFX_MAC_RX_JABBER_PKTS];
+		/* no oerrors counters supported on EF10 */
+	}
+
+unlock:
+	rte_spinlock_unlock(&port->mac_stats_lock);
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
 	.dev_stop			= sfc_dev_stop,
 	.dev_close			= sfc_dev_close,
 	.link_update			= sfc_dev_link_update,
+	.stats_get			= sfc_stats_get,
 	.dev_infos_get			= sfc_dev_infos_get,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
diff --git a/drivers/net/sfc/sfc_port.c b/drivers/net/sfc/sfc_port.c
index c124181..d8ff097 100644
--- a/drivers/net/sfc/sfc_port.c
+++ b/drivers/net/sfc/sfc_port.c
@@ -32,6 +32,34 @@
 #include "sfc.h"
 #include "sfc_log.h"
 
+/**
+ * Update MAC statistics in the buffer.
+ *
+ * @param	sa	Adapter
+ *
+ * @return Status code
+ * @retval	0	Success
+ * @retval	EAGAIN	Try again
+ * @retval	ENOMEM	Memory allocation failure
+ */
+int
+sfc_port_update_mac_stats(struct sfc_adapter *sa)
+{
+	struct sfc_port *port = &sa->port;
+	int rc;
+
+	SFC_ASSERT(rte_spinlock_is_locked(&port->mac_stats_lock));
+
+	if (sa->state != SFC_ADAPTER_STARTED)
+		return EINVAL;
+
+	rc = efx_mac_stats_update(sa->nic, &port->mac_stats_dma_mem,
+				  port->mac_stats_buf, NULL);
+	if (rc != 0)
+		return rc;
+
+	return 0;
+}
 
 int
 sfc_port_start(struct sfc_adapter *sa)
@@ -67,6 +95,19 @@ sfc_port_start(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_mac_filter_set;
 
+	efx_mac_stats_get_mask(sa->nic, port->mac_stats_mask,
+			       sizeof(port->mac_stats_mask));
+
+	/* Update MAC stats using periodic DMA.
+	 * Common code always uses 1000ms update period, so period_ms
+	 * parameter only needs to be non-zero to start updates.
+	 */
+	sfc_log_init(sa, "request MAC stats DMA'ing");
+	rc = efx_mac_stats_periodic(sa->nic, &port->mac_stats_dma_mem,
+				    1000, B_FALSE);
+	if (rc != 0)
+		goto fail_mac_stats_periodic;
+
 	sfc_log_init(sa, "disable MAC drain");
 	rc = efx_mac_drain(sa->nic, B_FALSE);
 	if (rc != 0)
@@ -76,6 +117,10 @@ sfc_port_start(struct sfc_adapter *sa)
 	return 0;
 
 fail_mac_drain:
+	(void)efx_mac_stats_periodic(sa->nic, &port->mac_stats_dma_mem,
+				     0, B_FALSE);
+
+fail_mac_stats_periodic:
 fail_mac_filter_set:
 fail_mac_addr_set:
 fail_mac_pdu_set:
@@ -95,6 +140,10 @@ sfc_port_stop(struct sfc_adapter *sa)
 	sfc_log_init(sa, "entry");
 
 	efx_mac_drain(sa->nic, B_TRUE);
+
+	(void)efx_mac_stats_periodic(sa->nic, &sa->port.mac_stats_dma_mem,
+				     0, B_FALSE);
+
 	efx_port_fini(sa->nic);
 	efx_filter_fini(sa->nic);
 
@@ -106,6 +155,7 @@ sfc_port_init(struct sfc_adapter *sa)
 {
 	const struct rte_eth_dev_data *dev_data = sa->eth_dev->data;
 	struct sfc_port *port = &sa->port;
+	int rc;
 
 	sfc_log_init(sa, "entry");
 
@@ -118,15 +168,40 @@ sfc_port_init(struct sfc_adapter *sa)
 	else
 		port->pdu = EFX_MAC_PDU(dev_data->mtu);
 
+	rte_spinlock_init(&port->mac_stats_lock);
+
+	rc = ENOMEM;
+	port->mac_stats_buf = rte_calloc_socket("mac_stats_buf", EFX_MAC_NSTATS,
+						sizeof(uint64_t), 0,
+						sa->socket_id);
+	if (port->mac_stats_buf == NULL)
+		goto fail_mac_stats_buf_alloc;
+
+	rc = sfc_dma_alloc(sa, "mac_stats", 0, EFX_MAC_STATS_SIZE,
+			   sa->socket_id, &port->mac_stats_dma_mem);
+	if (rc != 0)
+		goto fail_mac_stats_dma_alloc;
+
 	sfc_log_init(sa, "done");
 	return 0;
+
+fail_mac_stats_dma_alloc:
+	rte_free(port->mac_stats_buf);
+fail_mac_stats_buf_alloc:
+	sfc_log_init(sa, "failed %d", rc);
+	return rc;
 }
 
 void
 sfc_port_fini(struct sfc_adapter *sa)
 {
+	struct sfc_port *port = &sa->port;
+
 	sfc_log_init(sa, "entry");
 
+	sfc_dma_free(sa, &port->mac_stats_dma_mem);
+	rte_free(port->mac_stats_buf);
+
 	sfc_log_init(sa, "done");
 }
 
-- 
2.5.5

[PATCH v2 04/32] net/sfc: support extended statistics

[Andrew Rybchenko]

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  3 ++
 drivers/net/sfc/efsys.h              |  2 +-
 drivers/net/sfc/sfc_ethdev.c         | 63 ++++++++++++++++++++++++++++++++++++
 4 files changed, 68 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index f55a988..698553c 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -8,6 +8,7 @@ Link status          = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
 Basic stats          = Y
+Extended stats       = Y
 BSD nic_uio          = Y
 Linux UIO            = Y
 Linux VFIO           = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index cbb51de..eb9d26d 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -50,6 +50,9 @@ SFC EFX PMD has support for:
 
 - Port hardware statistics
 
+- Extended statistics (see Solarflare Server Adapter User's Guide for
+  the statistics description)
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/efsys.h b/drivers/net/sfc/efsys.h
index fe8615f..0f941e6 100644
--- a/drivers/net/sfc/efsys.h
+++ b/drivers/net/sfc/efsys.h
@@ -159,7 +159,7 @@ prefetch_read_once(const volatile void *addr)
 /* Code inclusion options */
 
 
-#define EFSYS_OPT_NAMES 0
+#define EFSYS_OPT_NAMES 1
 
 /* Disable SFN5xxx/SFN6xxx since it requires specific support in the PMD */
 #define EFSYS_OPT_SIENA 0
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index f31330c..d5ae1a0 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -392,6 +392,67 @@ sfc_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
 	rte_spinlock_unlock(&port->mac_stats_lock);
 }
 
+static int
+sfc_xstats_get(struct rte_eth_dev *dev, struct rte_eth_xstat *xstats,
+	       unsigned int xstats_count)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	struct sfc_port *port = &sa->port;
+	uint64_t *mac_stats;
+	int rc;
+	unsigned int i;
+	int nstats = 0;
+
+	rte_spinlock_lock(&port->mac_stats_lock);
+
+	rc = sfc_port_update_mac_stats(sa);
+	if (rc != 0) {
+		SFC_ASSERT(rc > 0);
+		nstats = -rc;
+		goto unlock;
+	}
+
+	mac_stats = port->mac_stats_buf;
+
+	for (i = 0; i < EFX_MAC_NSTATS; ++i) {
+		if (EFX_MAC_STAT_SUPPORTED(port->mac_stats_mask, i)) {
+			if (xstats != NULL && nstats < (int)xstats_count) {
+				xstats[nstats].id = nstats;
+				xstats[nstats].value = mac_stats[i];
+			}
+			nstats++;
+		}
+	}
+
+unlock:
+	rte_spinlock_unlock(&port->mac_stats_lock);
+
+	return nstats;
+}
+
+static int
+sfc_xstats_get_names(struct rte_eth_dev *dev,
+		     struct rte_eth_xstat_name *xstats_names,
+		     unsigned int xstats_count)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	struct sfc_port *port = &sa->port;
+	unsigned int i;
+	unsigned int nstats = 0;
+
+	for (i = 0; i < EFX_MAC_NSTATS; ++i) {
+		if (EFX_MAC_STAT_SUPPORTED(port->mac_stats_mask, i)) {
+			if (xstats_names != NULL && nstats < xstats_count)
+				strncpy(xstats_names[nstats].name,
+					efx_mac_stat_name(sa->nic, i),
+					sizeof(xstats_names[0].name));
+			nstats++;
+		}
+	}
+
+	return nstats;
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -399,6 +460,8 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_close			= sfc_dev_close,
 	.link_update			= sfc_dev_link_update,
 	.stats_get			= sfc_stats_get,
+	.xstats_get			= sfc_xstats_get,
+	.xstats_get_names		= sfc_xstats_get_names,
 	.dev_infos_get			= sfc_dev_infos_get,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
-- 
2.5.5

[PATCH v2 05/32] net/sfc: support flow control settings get/set

[Andrew Rybchenko]

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 +
 drivers/net/sfc/sfc_ethdev.c         | 98 ++++++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_port.c           |  8 +++
 4 files changed, 109 insertions(+)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 698553c..25472f8 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -5,6 +5,7 @@
 ;
 [Features]
 Link status          = Y
+Flow control         = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
 Basic stats          = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index eb9d26d..06b0c14 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -53,6 +53,8 @@ SFC EFX PMD has support for:
 - Extended statistics (see Solarflare Server Adapter User's Guide for
   the statistics description)
 
+- Basic flow control
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index d5ae1a0..eff648b 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -453,6 +453,102 @@ sfc_xstats_get_names(struct rte_eth_dev *dev,
 	return nstats;
 }
 
+static int
+sfc_flow_ctrl_get(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	unsigned int wanted_fc, link_fc;
+
+	memset(fc_conf, 0, sizeof(*fc_conf));
+
+	sfc_adapter_lock(sa);
+
+	if (sa->state == SFC_ADAPTER_STARTED)
+		efx_mac_fcntl_get(sa->nic, &wanted_fc, &link_fc);
+	else
+		link_fc = sa->port.flow_ctrl;
+
+	switch (link_fc) {
+	case 0:
+		fc_conf->mode = RTE_FC_NONE;
+		break;
+	case EFX_FCNTL_RESPOND:
+		fc_conf->mode = RTE_FC_RX_PAUSE;
+		break;
+	case EFX_FCNTL_GENERATE:
+		fc_conf->mode = RTE_FC_TX_PAUSE;
+		break;
+	case (EFX_FCNTL_RESPOND | EFX_FCNTL_GENERATE):
+		fc_conf->mode = RTE_FC_FULL;
+		break;
+	default:
+		sfc_err(sa, "%s: unexpected flow control value %#x",
+			__func__, link_fc);
+	}
+
+	fc_conf->autoneg = sa->port.flow_ctrl_autoneg;
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+}
+
+static int
+sfc_flow_ctrl_set(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	struct sfc_port *port = &sa->port;
+	unsigned int fcntl;
+	int rc;
+
+	if (fc_conf->high_water != 0 || fc_conf->low_water != 0 ||
+	    fc_conf->pause_time != 0 || fc_conf->send_xon != 0 ||
+	    fc_conf->mac_ctrl_frame_fwd != 0) {
+		sfc_err(sa, "unsupported flow control settings specified");
+		rc = EINVAL;
+		goto fail_inval;
+	}
+
+	switch (fc_conf->mode) {
+	case RTE_FC_NONE:
+		fcntl = 0;
+		break;
+	case RTE_FC_RX_PAUSE:
+		fcntl = EFX_FCNTL_RESPOND;
+		break;
+	case RTE_FC_TX_PAUSE:
+		fcntl = EFX_FCNTL_GENERATE;
+		break;
+	case RTE_FC_FULL:
+		fcntl = EFX_FCNTL_RESPOND | EFX_FCNTL_GENERATE;
+		break;
+	default:
+		rc = EINVAL;
+		goto fail_inval;
+	}
+
+	sfc_adapter_lock(sa);
+
+	if (sa->state == SFC_ADAPTER_STARTED) {
+		rc = efx_mac_fcntl_set(sa->nic, fcntl, fc_conf->autoneg);
+		if (rc != 0)
+			goto fail_mac_fcntl_set;
+	}
+
+	port->flow_ctrl = fcntl;
+	port->flow_ctrl_autoneg = fc_conf->autoneg;
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+
+fail_mac_fcntl_set:
+	sfc_adapter_unlock(sa);
+fail_inval:
+	SFC_ASSERT(rc > 0);
+	return -rc;
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -467,6 +563,8 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.rx_queue_release		= sfc_rx_queue_release,
 	.tx_queue_setup			= sfc_tx_queue_setup,
 	.tx_queue_release		= sfc_tx_queue_release,
+	.flow_ctrl_get			= sfc_flow_ctrl_get,
+	.flow_ctrl_set			= sfc_flow_ctrl_set,
 };
 
 static int
diff --git a/drivers/net/sfc/sfc_port.c b/drivers/net/sfc/sfc_port.c
index d8ff097..ccc0854 100644
--- a/drivers/net/sfc/sfc_port.c
+++ b/drivers/net/sfc/sfc_port.c
@@ -79,6 +79,13 @@ sfc_port_start(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_port_init;
 
+	sfc_log_init(sa, "set flow control to %#x autoneg=%u",
+		     port->flow_ctrl, port->flow_ctrl_autoneg);
+	rc = efx_mac_fcntl_set(sa->nic, port->flow_ctrl,
+			       port->flow_ctrl_autoneg);
+	if (rc != 0)
+		goto fail_mac_fcntl_set;
+
 	sfc_log_init(sa, "set MAC PDU %u", (unsigned int)port->pdu);
 	rc = efx_mac_pdu_set(sa->nic, port->pdu);
 	if (rc != 0)
@@ -124,6 +131,7 @@ sfc_port_start(struct sfc_adapter *sa)
 fail_mac_filter_set:
 fail_mac_addr_set:
 fail_mac_pdu_set:
+fail_mac_fcntl_set:
 	efx_port_fini(sa->nic);
 
 fail_port_init:
-- 
2.5.5

[PATCH v2 06/32] net/sfc: support link status change interrupt

[Andrew Rybchenko]

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |   1 +
 doc/guides/nics/sfc_efx.rst          |   4 +-
 drivers/net/sfc/sfc.c                |   4 +-
 drivers/net/sfc/sfc.h                |   4 +
 drivers/net/sfc/sfc_ethdev.c         |   1 +
 drivers/net/sfc/sfc_ev.c             |  30 +++++-
 drivers/net/sfc/sfc_intr.c           | 204 +++++++++++++++++++++++++++++++++++
 7 files changed, 242 insertions(+), 6 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 25472f8..693d35e 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -5,6 +5,7 @@
 ;
 [Features]
 Link status          = Y
+Link status event    = Y
 Flow control         = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 06b0c14..58f8242 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -44,7 +44,7 @@ SFC EFX PMD has support for:
 
 - Multiple transmit and receive queues
 
-- Link state information
+- Link state information including link status change interrupt
 
 - IPv4/IPv6 TCP/UDP transmit checksum offload
 
@@ -61,8 +61,6 @@ Non-supported Features
 
 The features not yet supported include:
 
-- Link status change interrupt
-
 - Receive queue interupts
 
 - Priority-based flow control
diff --git a/drivers/net/sfc/sfc.c b/drivers/net/sfc/sfc.c
index ef9e0d4..36044a0 100644
--- a/drivers/net/sfc/sfc.c
+++ b/drivers/net/sfc/sfc.c
@@ -116,7 +116,9 @@ sfc_check_conf(struct sfc_adapter *sa)
 		rc = EINVAL;
 	}
 
-	if (conf->intr_conf.lsc != 0) {
+	if ((conf->intr_conf.lsc != 0) &&
+	    (sa->intr.type != EFX_INTR_LINE) &&
+	    (sa->intr.type != EFX_INTR_MESSAGE)) {
 		sfc_err(sa, "Link status change interrupt not supported");
 		rc = EINVAL;
 	}
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index e9f5183..be33464 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -112,6 +112,8 @@ struct sfc_mcdi {
 
 struct sfc_intr {
 	efx_intr_type_t			type;
+	rte_intr_callback_fn		handler;
+	boolean_t			lsc_intr;
 };
 
 struct sfc_evq_info;
@@ -119,6 +121,8 @@ struct sfc_rxq_info;
 struct sfc_txq_info;
 
 struct sfc_port {
+	unsigned int			lsc_seq;
+
 	unsigned int			flow_ctrl;
 	boolean_t			flow_ctrl_autoneg;
 	size_t				pdu;
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index eff648b..8c46500 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -682,6 +682,7 @@ static struct eth_driver sfc_efx_pmd = {
 	.pci_drv = {
 		.id_table = pci_id_sfc_efx_map,
 		.drv_flags =
+			RTE_PCI_DRV_INTR_LSC |
 			RTE_PCI_DRV_NEED_MAPPING,
 		.probe = rte_eth_dev_pci_probe,
 		.remove = rte_eth_dev_pci_remove,
diff --git a/drivers/net/sfc/sfc_ev.c b/drivers/net/sfc/sfc_ev.c
index de42845..c788986 100644
--- a/drivers/net/sfc/sfc_ev.c
+++ b/drivers/net/sfc/sfc_ev.c
@@ -287,11 +287,25 @@ sfc_ev_link_change(void *arg, efx_link_mode_t link_mode)
 	struct sfc_adapter *sa = evq->sa;
 	struct rte_eth_link *dev_link = &sa->eth_dev->data->dev_link;
 	struct rte_eth_link new_link;
+	uint64_t new_link_u64;
+	uint64_t old_link_u64;
 
 	EFX_STATIC_ASSERT(sizeof(*dev_link) == sizeof(rte_atomic64_t));
 
 	sfc_port_link_mode_to_info(link_mode, &new_link);
-	rte_atomic64_set((rte_atomic64_t *)dev_link, *(uint64_t *)&new_link);
+
+	new_link_u64 = *(uint64_t *)&new_link;
+	do {
+		old_link_u64 = rte_atomic64_read((rte_atomic64_t *)dev_link);
+		if (old_link_u64 == new_link_u64)
+			break;
+
+		if (rte_atomic64_cmpset((volatile uint64_t *)dev_link,
+					old_link_u64, new_link_u64)) {
+			evq->sa->port.lsc_seq++;
+			break;
+		}
+	} while (B_TRUE);
 
 	return B_FALSE;
 }
@@ -521,6 +535,12 @@ sfc_ev_start(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_mgmt_evq_start;
 
+	if (sa->intr.lsc_intr) {
+		rc = sfc_ev_qprime(sa->evq_info[sa->mgmt_evq_index].evq);
+		if (rc != 0)
+			goto fail_evq0_prime;
+	}
+
 	rte_spinlock_unlock(&sa->mgmt_evq_lock);
 
 	/*
@@ -538,6 +558,9 @@ sfc_ev_start(struct sfc_adapter *sa)
 
 	return 0;
 
+fail_evq0_prime:
+	sfc_ev_qstop(sa, 0);
+
 fail_mgmt_evq_start:
 	rte_spinlock_unlock(&sa->mgmt_evq_lock);
 	efx_ev_fini(sa->nic);
@@ -639,7 +662,10 @@ sfc_ev_qinit_info(struct sfc_adapter *sa, unsigned int sw_index)
 	SFC_ASSERT(rte_is_power_of_2(max_entries));
 
 	evq_info->max_entries = max_entries;
-	evq_info->flags = sa->evq_flags | EFX_EVQ_FLAGS_NOTIFY_DISABLED;
+	evq_info->flags = sa->evq_flags |
+		((sa->intr.lsc_intr && sw_index == sa->mgmt_evq_index) ?
+			EFX_EVQ_FLAGS_NOTIFY_INTERRUPT :
+			EFX_EVQ_FLAGS_NOTIFY_DISABLED);
 
 	return 0;
 }
diff --git a/drivers/net/sfc/sfc_intr.c b/drivers/net/sfc/sfc_intr.c
index 1b7dcdd..e0b1693 100644
--- a/drivers/net/sfc/sfc_intr.c
+++ b/drivers/net/sfc/sfc_intr.c
@@ -27,10 +27,130 @@
  * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+/*
+ * At the momemt of writing DPDK v16.07 has notion of two types of
+ * interrupts: LSC (link status change) and RXQ (receive indication).
+ * It allows to register interrupt callback for entire device which is
+ * not intended to be used for receive indication (i.e. link status
+ * change indication only). The handler has no information which HW
+ * interrupt has triggered it, so we don't know which event queue should
+ * be polled/reprimed (except qmask in the case of legacy line interrupt).
+ */
+
+#include <rte_common.h>
+#include <rte_interrupts.h>
+
 #include "efx.h"
 
 #include "sfc.h"
 #include "sfc_log.h"
+#include "sfc_ev.h"
+
+static void
+sfc_intr_handle_mgmt_evq(struct sfc_adapter *sa)
+{
+	struct sfc_evq *evq;
+
+	rte_spinlock_lock(&sa->mgmt_evq_lock);
+
+	evq = sa->evq_info[sa->mgmt_evq_index].evq;
+
+	if (evq->init_state != SFC_EVQ_STARTED) {
+		sfc_log_init(sa, "interrupt on stopped EVQ %u", evq->evq_index);
+	} else {
+		sfc_ev_qpoll(evq);
+
+		if (sfc_ev_qprime(evq) != 0)
+			sfc_err(sa, "cannot prime EVQ %u", evq->evq_index);
+	}
+
+	rte_spinlock_unlock(&sa->mgmt_evq_lock);
+}
+
+static void
+sfc_intr_line_handler(struct rte_intr_handle *intr_handle, void *cb_arg)
+{
+	struct sfc_adapter *sa = (struct sfc_adapter *)cb_arg;
+	efx_nic_t *enp = sa->nic;
+	boolean_t fatal;
+	uint32_t qmask;
+	unsigned int lsc_seq = sa->port.lsc_seq;
+
+	sfc_log_init(sa, "entry");
+
+	if (sa->state != SFC_ADAPTER_STARTED &&
+	    sa->state != SFC_ADAPTER_STARTING &&
+	    sa->state != SFC_ADAPTER_STOPPING) {
+		sfc_log_init(sa,
+			     "interrupt on stopped adapter, don't reenable");
+		goto exit;
+	}
+
+	efx_intr_status_line(enp, &fatal, &qmask);
+	if (fatal) {
+		(void)efx_intr_disable(enp);
+		(void)efx_intr_fatal(enp);
+		sfc_err(sa, "fatal, interrupts disabled");
+		goto exit;
+	}
+
+	if (qmask & (1 << sa->mgmt_evq_index))
+		sfc_intr_handle_mgmt_evq(sa);
+
+	if (rte_intr_enable(intr_handle) != 0)
+		sfc_err(sa, "cannot reenable interrupts");
+
+	sfc_log_init(sa, "done");
+
+exit:
+	if (lsc_seq != sa->port.lsc_seq) {
+		sfc_info(sa, "link status change event: link %s",
+			 sa->eth_dev->data->dev_link.link_status ?
+			 "UP" : "DOWN");
+		_rte_eth_dev_callback_process(sa->eth_dev,
+					      RTE_ETH_EVENT_INTR_LSC, NULL);
+	}
+}
+
+static void
+sfc_intr_message_handler(struct rte_intr_handle *intr_handle, void *cb_arg)
+{
+	struct sfc_adapter *sa = (struct sfc_adapter *)cb_arg;
+	efx_nic_t *enp = sa->nic;
+	boolean_t fatal;
+	unsigned int lsc_seq = sa->port.lsc_seq;
+
+	sfc_log_init(sa, "entry");
+
+	if (sa->state != SFC_ADAPTER_STARTED &&
+	    sa->state != SFC_ADAPTER_STARTING &&
+	    sa->state != SFC_ADAPTER_STOPPING) {
+		sfc_log_init(sa, "adapter not-started, don't reenable");
+		goto exit;
+	}
+
+	efx_intr_status_message(enp, sa->mgmt_evq_index, &fatal);
+	if (fatal) {
+		(void)efx_intr_disable(enp);
+		(void)efx_intr_fatal(enp);
+		sfc_err(sa, "fatal, interrupts disabled");
+		goto exit;
+	}
+
+	sfc_intr_handle_mgmt_evq(sa);
+
+	if (rte_intr_enable(intr_handle) != 0)
+		sfc_err(sa, "cannot reenable interrupts");
+
+	sfc_log_init(sa, "done");
+
+exit:
+	if (lsc_seq != sa->port.lsc_seq) {
+		sfc_info(sa, "link status change event");
+		_rte_eth_dev_callback_process(sa->eth_dev,
+					      RTE_ETH_EVENT_INTR_LSC, NULL);
+	}
+}
 
 int
 sfc_intr_start(struct sfc_adapter *sa)
@@ -54,11 +174,49 @@ sfc_intr_start(struct sfc_adapter *sa)
 
 	intr_handle = &sa->eth_dev->pci_dev->intr_handle;
 
+	if (intr->handler != NULL) {
+		sfc_log_init(sa, "rte_intr_callback_register");
+		rc = rte_intr_callback_register(intr_handle, intr->handler,
+						(void *)sa);
+		if (rc != 0) {
+			sfc_err(sa,
+				"cannot register interrupt handler (rc=%d)",
+				rc);
+			/*
+			 * Convert error code from negative returned by RTE API
+			 * to positive used in the driver.
+			 */
+			rc = -rc;
+			goto fail_rte_intr_cb_reg;
+		}
+
+		sfc_log_init(sa, "rte_intr_enable");
+		rc = rte_intr_enable(intr_handle);
+		if (rc != 0) {
+			sfc_err(sa, "cannot enable interrupts (rc=%d)", rc);
+			/*
+			 * Convert error code from negative returned by RTE API
+			 * to positive used in the driver.
+			 */
+			rc = -rc;
+			goto fail_rte_intr_enable;
+		}
+
+		sfc_log_init(sa, "efx_intr_enable");
+		efx_intr_enable(sa->nic);
+	}
+
 	sfc_log_init(sa, "done type=%u max_intr=%d nb_efd=%u vec=%p",
 		     intr_handle->type, intr_handle->max_intr,
 		     intr_handle->nb_efd, intr_handle->intr_vec);
 	return 0;
 
+fail_rte_intr_enable:
+	rte_intr_callback_unregister(intr_handle, intr->handler, (void *)sa);
+
+fail_rte_intr_cb_reg:
+	efx_intr_fini(sa->nic);
+
 fail_intr_init:
 	sfc_log_init(sa, "failed %d", rc);
 	return rc;
@@ -67,8 +225,29 @@ sfc_intr_start(struct sfc_adapter *sa)
 void
 sfc_intr_stop(struct sfc_adapter *sa)
 {
+	struct sfc_intr *intr = &sa->intr;
+
 	sfc_log_init(sa, "entry");
 
+	if (intr->handler != NULL) {
+		struct rte_intr_handle *intr_handle;
+		int rc;
+
+		efx_intr_disable(sa->nic);
+
+		intr_handle = &sa->eth_dev->pci_dev->intr_handle;
+		if (rte_intr_disable(intr_handle) != 0)
+			sfc_err(sa, "cannot disable interrupts");
+
+		while ((rc = rte_intr_callback_unregister(intr_handle,
+				intr->handler, (void *)sa)) == -EAGAIN)
+			;
+		if (rc != 1)
+			sfc_err(sa,
+				"cannot unregister interrupt handler %d",
+				rc);
+	}
+
 	efx_intr_fini(sa->nic);
 
 	sfc_log_init(sa, "done");
@@ -77,8 +256,33 @@ sfc_intr_stop(struct sfc_adapter *sa)
 int
 sfc_intr_init(struct sfc_adapter *sa)
 {
+	struct sfc_intr *intr = &sa->intr;
+
 	sfc_log_init(sa, "entry");
 
+	intr->handler = NULL;
+	intr->lsc_intr = (sa->eth_dev->data->dev_conf.intr_conf.lsc != 0);
+	if (!intr->lsc_intr) {
+		sfc_info(sa, "LSC tracking using interrupts is disabled");
+		goto done;
+	}
+
+	switch (intr->type) {
+	case EFX_INTR_MESSAGE:
+		intr->handler = sfc_intr_message_handler;
+		break;
+	case EFX_INTR_LINE:
+		intr->handler = sfc_intr_line_handler;
+		break;
+	case EFX_INTR_INVALID:
+		sfc_warn(sa, "interrupts are not supported");
+		break;
+	default:
+		sfc_panic(sa, "unexpected EFX interrupt type %u\n", intr->type);
+		break;
+	}
+
+done:
 	sfc_log_init(sa, "done");
 	return 0;
 }
-- 
2.5.5

[PATCH v2 07/32] net/sfc: implement device operation to change MTU

[Andrew Rybchenko]

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  2 ++
 doc/guides/nics/sfc_efx.rst          |  4 +++
 drivers/net/sfc/sfc_ethdev.c         | 67 ++++++++++++++++++++++++++++++++++++
 3 files changed, 73 insertions(+)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 693d35e..a845bfc 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -6,6 +6,8 @@
 [Features]
 Link status          = Y
 Link status event    = Y
+MTU update           = Y
+Jumbo frame          = Y
 Flow control         = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 58f8242..c482d77 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -55,6 +55,10 @@ SFC EFX PMD has support for:
 
 - Basic flow control
 
+- MTU update
+
+- Jumbo frames up to 9K
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 8c46500..6690755 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -549,6 +549,72 @@ sfc_flow_ctrl_set(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf)
 	return -rc;
 }
 
+static int
+sfc_dev_set_mtu(struct rte_eth_dev *dev, uint16_t mtu)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	size_t pdu = EFX_MAC_PDU(mtu);
+	size_t old_pdu;
+	int rc;
+
+	sfc_log_init(sa, "mtu=%u", mtu);
+
+	rc = EINVAL;
+	if (pdu < EFX_MAC_PDU_MIN) {
+		sfc_err(sa, "too small MTU %u (PDU size %u less than min %u)",
+			(unsigned int)mtu, (unsigned int)pdu,
+			EFX_MAC_PDU_MIN);
+		goto fail_inval;
+	}
+	if (pdu > EFX_MAC_PDU_MAX) {
+		sfc_err(sa, "too big MTU %u (PDU size %u greater than max %u)",
+			(unsigned int)mtu, (unsigned int)pdu,
+			EFX_MAC_PDU_MAX);
+		goto fail_inval;
+	}
+
+	sfc_adapter_lock(sa);
+
+	if (pdu != sa->port.pdu) {
+		if (sa->state == SFC_ADAPTER_STARTED) {
+			sfc_stop(sa);
+
+			old_pdu = sa->port.pdu;
+			sa->port.pdu = pdu;
+			rc = sfc_start(sa);
+			if (rc != 0)
+				goto fail_start;
+		} else {
+			sa->port.pdu = pdu;
+		}
+	}
+
+	/*
+	 * The driver does not use it, but other PMDs update jumbo_frame
+	 * flag and max_rx_pkt_len when MTU is set.
+	 */
+	dev->data->dev_conf.rxmode.jumbo_frame = (mtu > ETHER_MAX_LEN);
+	dev->data->dev_conf.rxmode.max_rx_pkt_len = sa->port.pdu;
+
+	sfc_adapter_unlock(sa);
+
+	sfc_log_init(sa, "done");
+	return 0;
+
+fail_start:
+	sa->port.pdu = old_pdu;
+	if (sfc_start(sa) != 0)
+		sfc_err(sa, "cannot start with neither new (%u) nor old (%u) "
+			"PDU max size - port is stopped",
+			(unsigned int)pdu, (unsigned int)old_pdu);
+	sfc_adapter_unlock(sa);
+
+fail_inval:
+	sfc_log_init(sa, "failed %d", rc);
+	SFC_ASSERT(rc > 0);
+	return -rc;
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -559,6 +625,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.xstats_get			= sfc_xstats_get,
 	.xstats_get_names		= sfc_xstats_get_names,
 	.dev_infos_get			= sfc_dev_infos_get,
+	.mtu_set			= sfc_dev_set_mtu,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
 	.tx_queue_setup			= sfc_tx_queue_setup,
-- 
2.5.5

[PATCH v2 08/32] net/sfc: support link speed and duplex settings

[Andrew Rybchenko]

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 drivers/net/sfc/sfc.c                | 38 ++++++++++++++++++++++++++++++++++--
 drivers/net/sfc/sfc.h                |  3 +++
 drivers/net/sfc/sfc_ethdev.c         |  9 +++++++++
 drivers/net/sfc/sfc_port.c           |  6 ++++++
 5 files changed, 55 insertions(+), 2 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index a845bfc..60ecca0 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -4,6 +4,7 @@
 ; Refer to default.ini for the full list of available PMD features.
 ;
 [Features]
+Speed capabilities   = Y
 Link status          = Y
 Link status event    = Y
 MTU update           = Y
diff --git a/drivers/net/sfc/sfc.c b/drivers/net/sfc/sfc.c
index 36044a0..e2e6c9e 100644
--- a/drivers/net/sfc/sfc.c
+++ b/drivers/net/sfc/sfc.c
@@ -85,6 +85,33 @@ sfc_dma_free(const struct sfc_adapter *sa, efsys_mem_t *esmp)
 	memset(esmp, 0, sizeof(*esmp));
 }
 
+static uint32_t
+sfc_phy_cap_from_link_speeds(uint32_t speeds)
+{
+	uint32_t phy_caps = 0;
+
+	if (~speeds & ETH_LINK_SPEED_FIXED) {
+		phy_caps |= (1 << EFX_PHY_CAP_AN);
+		/*
+		 * If no speeds are specified in the mask, any supported
+		 * may be negotiated
+		 */
+		if (speeds == ETH_LINK_SPEED_AUTONEG)
+			phy_caps |=
+				(1 << EFX_PHY_CAP_1000FDX) |
+				(1 << EFX_PHY_CAP_10000FDX) |
+				(1 << EFX_PHY_CAP_40000FDX);
+	}
+	if (speeds & ETH_LINK_SPEED_1G)
+		phy_caps |= (1 << EFX_PHY_CAP_1000FDX);
+	if (speeds & ETH_LINK_SPEED_10G)
+		phy_caps |= (1 << EFX_PHY_CAP_10000FDX);
+	if (speeds & ETH_LINK_SPEED_40G)
+		phy_caps |= (1 << EFX_PHY_CAP_40000FDX);
+
+	return phy_caps;
+}
+
 /*
  * Check requested device level configuration.
  * Receive and transmit configuration is checked in corresponding
@@ -96,8 +123,12 @@ sfc_check_conf(struct sfc_adapter *sa)
 	const struct rte_eth_conf *conf = &sa->eth_dev->data->dev_conf;
 	int rc = 0;
 
-	if (conf->link_speeds != ETH_LINK_SPEED_AUTONEG) {
-		sfc_err(sa, "Manual link speed/duplex choice not supported");
+	sa->port.phy_adv_cap =
+		sfc_phy_cap_from_link_speeds(conf->link_speeds) &
+		sa->port.phy_adv_cap_mask;
+	if ((sa->port.phy_adv_cap & ~(1 << EFX_PHY_CAP_AN)) == 0) {
+		sfc_err(sa, "No link speeds from mask %#x are supported",
+			conf->link_speeds);
 		rc = EINVAL;
 	}
 
@@ -516,6 +547,9 @@ sfc_attach(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_intr_attach;
 
+	efx_phy_adv_cap_get(sa->nic, EFX_PHY_CAP_PERM,
+			    &sa->port.phy_adv_cap_mask);
+
 	sfc_log_init(sa, "fini nic");
 	efx_nic_fini(enp);
 
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index be33464..ff0bc3c 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -123,6 +123,9 @@ struct sfc_txq_info;
 struct sfc_port {
 	unsigned int			lsc_seq;
 
+	uint32_t			phy_adv_cap_mask;
+	uint32_t			phy_adv_cap;
+
 	unsigned int			flow_ctrl;
 	boolean_t			flow_ctrl_autoneg;
 	size_t				pdu;
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 6690755..42c488e 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -53,6 +53,15 @@ sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 
 	dev_info->max_rx_pktlen = EFX_MAC_PDU_MAX;
 
+	/* Autonegotiation may be disabled */
+	dev_info->speed_capa = ETH_LINK_SPEED_FIXED;
+	if (sa->port.phy_adv_cap_mask & EFX_PHY_CAP_1000FDX)
+		dev_info->speed_capa |= ETH_LINK_SPEED_1G;
+	if (sa->port.phy_adv_cap_mask & EFX_PHY_CAP_10000FDX)
+		dev_info->speed_capa |= ETH_LINK_SPEED_10G;
+	if (sa->port.phy_adv_cap_mask & EFX_PHY_CAP_40000FDX)
+		dev_info->speed_capa |= ETH_LINK_SPEED_40G;
+
 	dev_info->max_rx_queues = sa->rxq_max;
 	dev_info->max_tx_queues = sa->txq_max;
 
diff --git a/drivers/net/sfc/sfc_port.c b/drivers/net/sfc/sfc_port.c
index ccc0854..1241af7 100644
--- a/drivers/net/sfc/sfc_port.c
+++ b/drivers/net/sfc/sfc_port.c
@@ -86,6 +86,11 @@ sfc_port_start(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_mac_fcntl_set;
 
+	sfc_log_init(sa, "set phy adv caps to %#x", port->phy_adv_cap);
+	rc = efx_phy_adv_cap_set(sa->nic, port->phy_adv_cap);
+	if (rc != 0)
+		goto fail_phy_adv_cap_set;
+
 	sfc_log_init(sa, "set MAC PDU %u", (unsigned int)port->pdu);
 	rc = efx_mac_pdu_set(sa->nic, port->pdu);
 	if (rc != 0)
@@ -131,6 +136,7 @@ sfc_port_start(struct sfc_adapter *sa)
 fail_mac_filter_set:
 fail_mac_addr_set:
 fail_mac_pdu_set:
+fail_phy_adv_cap_set:
 fail_mac_fcntl_set:
 	efx_port_fini(sa->nic);
 
-- 
2.5.5

[PATCH v2 09/32] net/sfc: support link up/down

[Andrew Rybchenko]

From: Artem Andreev <Artem.Andreev@oktetlabs.ru>

Signed-off-by: Artem Andreev <Artem.Andreev@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 42c488e..1716d78 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -195,6 +195,36 @@ sfc_dev_stop(struct rte_eth_dev *dev)
 	sfc_log_init(sa, "done");
 }
 
+static int
+sfc_dev_set_link_up(struct rte_eth_dev *dev)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	int rc;
+
+	sfc_log_init(sa, "entry");
+
+	sfc_adapter_lock(sa);
+	rc = sfc_start(sa);
+	sfc_adapter_unlock(sa);
+
+	SFC_ASSERT(rc >= 0);
+	return -rc;
+}
+
+static int
+sfc_dev_set_link_down(struct rte_eth_dev *dev)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+
+	sfc_log_init(sa, "entry");
+
+	sfc_adapter_lock(sa);
+	sfc_stop(sa);
+	sfc_adapter_unlock(sa);
+
+	return 0;
+}
+
 static void
 sfc_dev_close(struct rte_eth_dev *dev)
 {
@@ -628,6 +658,8 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
 	.dev_stop			= sfc_dev_stop,
+	.dev_set_link_up		= sfc_dev_set_link_up,
+	.dev_set_link_down		= sfc_dev_set_link_down,
 	.dev_close			= sfc_dev_close,
 	.link_update			= sfc_dev_link_update,
 	.stats_get			= sfc_stats_get,
-- 
2.5.5

[PATCH v2 10/32] net/sfc: support promiscuous and all-multicast control

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  2 ++
 doc/guides/nics/sfc_efx.rst          |  4 +++
 drivers/net/sfc/sfc.h                | 11 +++++++
 drivers/net/sfc/sfc_ethdev.c         | 57 ++++++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_port.c           | 18 +++++++++++-
 5 files changed, 91 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 60ecca0..aaea993 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -9,6 +9,8 @@ Link status          = Y
 Link status event    = Y
 MTU update           = Y
 Jumbo frame          = Y
+Promiscuous mode     = Y
+Allmulticast mode    = Y
 Flow control         = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index c482d77..984da9c 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -59,6 +59,10 @@ SFC EFX PMD has support for:
 
 - Jumbo frames up to 9K
 
+- Promiscuous mode
+
+- Allmulticast mode
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index ff0bc3c..7b135e1 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -93,6 +93,13 @@ enum sfc_adapter_state {
 	SFC_ADAPTER_NSTATES
 };
 
+enum sfc_dev_filter_mode {
+	SFC_DEV_FILTER_MODE_PROMISC = 0,
+	SFC_DEV_FILTER_MODE_ALLMULTI,
+
+	SFC_DEV_FILTER_NMODES
+};
+
 enum sfc_mcdi_state {
 	SFC_MCDI_UNINITIALIZED = 0,
 	SFC_MCDI_INITIALIZED,
@@ -130,6 +137,9 @@ struct sfc_port {
 	boolean_t			flow_ctrl_autoneg;
 	size_t				pdu;
 
+	boolean_t			promisc;
+	boolean_t			allmulti;
+
 	rte_spinlock_t			mac_stats_lock;
 	uint64_t			*mac_stats_buf;
 	efsys_mem_t			mac_stats_dma_mem;
@@ -249,6 +259,7 @@ void sfc_port_stop(struct sfc_adapter *sa);
 void sfc_port_link_mode_to_info(efx_link_mode_t link_mode,
 				struct rte_eth_link *link_info);
 int sfc_port_update_mac_stats(struct sfc_adapter *sa);
+int sfc_set_rx_mode(struct sfc_adapter *sa);
 
 
 #ifdef __cplusplus
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 1716d78..c6095ad 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -253,6 +253,59 @@ sfc_dev_close(struct rte_eth_dev *dev)
 	sfc_log_init(sa, "done");
 }
 
+static void
+sfc_dev_filter_set(struct rte_eth_dev *dev, enum sfc_dev_filter_mode mode,
+		   boolean_t enabled)
+{
+	struct sfc_port *port;
+	boolean_t *toggle;
+	struct sfc_adapter *sa = dev->data->dev_private;
+	boolean_t allmulti = (mode == SFC_DEV_FILTER_MODE_ALLMULTI);
+	const char *desc = (allmulti) ? "all-multi" : "promiscuous";
+
+	sfc_adapter_lock(sa);
+
+	port = &sa->port;
+	toggle = (allmulti) ? (&port->allmulti) : (&port->promisc);
+
+	if (*toggle != enabled) {
+		*toggle = enabled;
+
+		if ((sa->state == SFC_ADAPTER_STARTED) &&
+		    (sfc_set_rx_mode(sa) != 0)) {
+			*toggle = !(enabled);
+			sfc_warn(sa, "Failed to %s %s mode",
+				 ((enabled) ? "enable" : "disable"), desc);
+		}
+	}
+
+	sfc_adapter_unlock(sa);
+}
+
+static void
+sfc_dev_promisc_enable(struct rte_eth_dev *dev)
+{
+	sfc_dev_filter_set(dev, SFC_DEV_FILTER_MODE_PROMISC, B_TRUE);
+}
+
+static void
+sfc_dev_promisc_disable(struct rte_eth_dev *dev)
+{
+	sfc_dev_filter_set(dev, SFC_DEV_FILTER_MODE_PROMISC, B_FALSE);
+}
+
+static void
+sfc_dev_allmulti_enable(struct rte_eth_dev *dev)
+{
+	sfc_dev_filter_set(dev, SFC_DEV_FILTER_MODE_ALLMULTI, B_TRUE);
+}
+
+static void
+sfc_dev_allmulti_disable(struct rte_eth_dev *dev)
+{
+	sfc_dev_filter_set(dev, SFC_DEV_FILTER_MODE_ALLMULTI, B_FALSE);
+}
+
 static int
 sfc_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 		   uint16_t nb_rx_desc, unsigned int socket_id,
@@ -661,6 +714,10 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_set_link_up		= sfc_dev_set_link_up,
 	.dev_set_link_down		= sfc_dev_set_link_down,
 	.dev_close			= sfc_dev_close,
+	.promiscuous_enable		= sfc_dev_promisc_enable,
+	.promiscuous_disable		= sfc_dev_promisc_disable,
+	.allmulticast_enable		= sfc_dev_allmulti_enable,
+	.allmulticast_disable		= sfc_dev_allmulti_disable,
 	.link_update			= sfc_dev_link_update,
 	.stats_get			= sfc_stats_get,
 	.xstats_get			= sfc_xstats_get,
diff --git a/drivers/net/sfc/sfc_port.c b/drivers/net/sfc/sfc_port.c
index 1241af7..dc6ecdf 100644
--- a/drivers/net/sfc/sfc_port.c
+++ b/drivers/net/sfc/sfc_port.c
@@ -103,7 +103,11 @@ sfc_port_start(struct sfc_adapter *sa)
 		goto fail_mac_addr_set;
 
 	sfc_log_init(sa, "set MAC filters");
-	rc = efx_mac_filter_set(sa->nic, B_TRUE, B_TRUE, B_TRUE, B_TRUE);
+	port->promisc = (sa->eth_dev->data->promiscuous != 0) ?
+			B_TRUE : B_FALSE;
+	port->allmulti = (sa->eth_dev->data->all_multicast != 0) ?
+			 B_TRUE : B_FALSE;
+	rc = sfc_set_rx_mode(sa);
 	if (rc != 0)
 		goto fail_mac_filter_set;
 
@@ -219,6 +223,18 @@ sfc_port_fini(struct sfc_adapter *sa)
 	sfc_log_init(sa, "done");
 }
 
+int
+sfc_set_rx_mode(struct sfc_adapter *sa)
+{
+	struct sfc_port *port = &sa->port;
+	int rc;
+
+	rc = efx_mac_filter_set(sa->nic, port->promisc, B_TRUE,
+				port->promisc || port->allmulti, B_TRUE);
+
+	return rc;
+}
+
 void
 sfc_port_link_mode_to_info(efx_link_mode_t link_mode,
 			   struct rte_eth_link *link_info)
-- 
2.5.5

[PATCH v2 11/32] net/sfc: support main (the first) MAC address change

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 53 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index c6095ad..79e2761 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -706,6 +706,58 @@ sfc_dev_set_mtu(struct rte_eth_dev *dev, uint16_t mtu)
 	SFC_ASSERT(rc > 0);
 	return -rc;
 }
+static void
+sfc_mac_addr_set(struct rte_eth_dev *dev, struct ether_addr *mac_addr)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	const efx_nic_cfg_t *encp = efx_nic_cfg_get(sa->nic);
+	int rc;
+
+	sfc_adapter_lock(sa);
+
+	if (sa->state != SFC_ADAPTER_STARTED) {
+		sfc_info(sa, "the port is not started");
+		sfc_info(sa, "the new MAC address will be set on port start");
+
+		goto unlock;
+	}
+
+	if (encp->enc_allow_set_mac_with_installed_filters) {
+		rc = efx_mac_addr_set(sa->nic, mac_addr->addr_bytes);
+		if (rc != 0) {
+			sfc_err(sa, "cannot set MAC address (rc = %u)", rc);
+			goto unlock;
+		}
+
+		/*
+		 * Changing the MAC address by means of MCDI request
+		 * has no effect on received traffic, therefore
+		 * we also need to update unicast filters
+		 */
+		rc = sfc_set_rx_mode(sa);
+		if (rc != 0)
+			sfc_err(sa, "cannot set filter (rc = %u)", rc);
+	} else {
+		sfc_warn(sa, "cannot set MAC address with filters installed");
+		sfc_warn(sa, "adapter will be restarted to pick the new MAC");
+		sfc_warn(sa, "(some traffic may be dropped)");
+
+		/*
+		 * Since setting MAC address with filters installed is not
+		 * allowed on the adapter, one needs to simply restart adapter
+		 * so that the new MAC address will be taken from an outer
+		 * storage and set flawlessly by means of sfc_start() call
+		 */
+		sfc_stop(sa);
+		rc = sfc_start(sa);
+		if (rc != 0)
+			sfc_err(sa, "cannot restart adapter (rc = %u)", rc);
+	}
+
+unlock:
+	sfc_adapter_unlock(sa);
+}
+
 
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
@@ -730,6 +782,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.tx_queue_release		= sfc_tx_queue_release,
 	.flow_ctrl_get			= sfc_flow_ctrl_get,
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
+	.mac_addr_set			= sfc_mac_addr_set,
 };
 
 static int
-- 
2.5.5

[PATCH v2 12/32] net/sfc: support multicast addresses list controls

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 ++
 drivers/net/sfc/sfc_ethdev.c         | 40 ++++++++++++++++++++++++++++++++++++
 3 files changed, 43 insertions(+)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index aaea993..35633cb 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -11,6 +11,7 @@ MTU update           = Y
 Jumbo frame          = Y
 Promiscuous mode     = Y
 Allmulticast mode    = Y
+Multicast MAC filter = Y
 Flow control         = Y
 L3 checksum offload  = P
 L4 checksum offload  = P
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 984da9c..43dd4cf 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -63,6 +63,8 @@ SFC EFX PMD has support for:
 
 - Allmulticast mode
 
+- Multicast MAC filter
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 79e2761..268b76a 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -759,6 +759,45 @@ sfc_mac_addr_set(struct rte_eth_dev *dev, struct ether_addr *mac_addr)
 }
 
 
+static int
+sfc_set_mc_addr_list(struct rte_eth_dev *dev, struct ether_addr *mc_addr_set,
+		     uint32_t nb_mc_addr)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	uint8_t *mc_addrs_p;
+	uint8_t *mc_addrs;
+	int rc;
+	unsigned int i;
+
+	if (nb_mc_addr > EFX_MAC_MULTICAST_LIST_MAX) {
+		sfc_err(sa, "too many multicast addresses: %u > %u",
+			 nb_mc_addr, EFX_MAC_MULTICAST_LIST_MAX);
+		return -EINVAL;
+	}
+
+	mc_addrs_p = rte_calloc("mc-addrs", nb_mc_addr, EFX_MAC_ADDR_LEN, 0);
+	if (mc_addrs_p == NULL)
+		return -ENOMEM;
+
+	mc_addrs = mc_addrs_p;
+
+	for (i = 0; i < nb_mc_addr; ++i) {
+		(void)rte_memcpy(mc_addrs, mc_addr_set[i].addr_bytes,
+				 EFX_MAC_ADDR_LEN);
+		mc_addrs += EFX_MAC_ADDR_LEN;
+	}
+
+	rc = efx_mac_multicast_list_set(sa->nic, mc_addrs_p, nb_mc_addr);
+
+	rte_free(mc_addrs_p);
+
+	if (rc != 0)
+		sfc_err(sa, "cannot set multicast address list (rc = %u)", rc);
+
+	SFC_ASSERT(rc > 0);
+	return -rc;
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -783,6 +822,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.flow_ctrl_get			= sfc_flow_ctrl_get,
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
 	.mac_addr_set			= sfc_mac_addr_set,
+	.set_mc_addr_list		= sfc_set_mc_addr_list,
 };
 
 static int
-- 
2.5.5

[PATCH v2 13/32] net/sfc: support checksum offloads on receive

[Andrew Rybchenko]

IPv4 header and TCP/UDP checksums for both IPv4 and IPv6 are supported.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  4 ++--
 doc/guides/nics/sfc_efx.rst          |  2 ++
 drivers/net/sfc/sfc_ethdev.c         |  5 +++++
 drivers/net/sfc/sfc_rx.c             | 40 ++++++++++++++++++++++++++++++++++++
 4 files changed, 49 insertions(+), 2 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 35633cb..ec1d1db 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -13,8 +13,8 @@ Promiscuous mode     = Y
 Allmulticast mode    = Y
 Multicast MAC filter = Y
 Flow control         = Y
-L3 checksum offload  = P
-L4 checksum offload  = P
+L3 checksum offload  = Y
+L4 checksum offload  = Y
 Basic stats          = Y
 Extended stats       = Y
 BSD nic_uio          = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 43dd4cf..8724eae 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -65,6 +65,8 @@ SFC EFX PMD has support for:
 
 - Multicast MAC filter
 
+- IPv4/IPv6 TCP/UDP receive checksum offload
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 268b76a..235582d 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -68,6 +68,11 @@ sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 	/* By default packets are dropped if no descriptors are available */
 	dev_info->default_rxconf.rx_drop_en = 1;
 
+	dev_info->rx_offload_capa =
+		DEV_RX_OFFLOAD_IPV4_CKSUM |
+		DEV_RX_OFFLOAD_UDP_CKSUM |
+		DEV_RX_OFFLOAD_TCP_CKSUM;
+
 	dev_info->tx_offload_capa =
 		DEV_TX_OFFLOAD_IPV4_CKSUM |
 		DEV_TX_OFFLOAD_UDP_CKSUM |
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index bd41131..4df4132 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -130,6 +130,45 @@ sfc_rx_qrefill(struct sfc_rxq *rxq)
 	}
 }
 
+static uint64_t
+sfc_rx_desc_flags_to_offload_flags(const unsigned int desc_flags)
+{
+	uint64_t mbuf_flags = 0;
+
+	switch (desc_flags & (EFX_PKT_IPV4 | EFX_CKSUM_IPV4)) {
+	case (EFX_PKT_IPV4 | EFX_CKSUM_IPV4):
+		mbuf_flags |= PKT_RX_IP_CKSUM_GOOD;
+		break;
+	case EFX_PKT_IPV4:
+		mbuf_flags |= PKT_RX_IP_CKSUM_BAD;
+		break;
+	default:
+		RTE_BUILD_BUG_ON(PKT_RX_IP_CKSUM_UNKNOWN != 0);
+		SFC_ASSERT((mbuf_flags & PKT_RX_IP_CKSUM_MASK) ==
+			   PKT_RX_IP_CKSUM_UNKNOWN);
+		break;
+	}
+
+	switch ((desc_flags &
+		 (EFX_PKT_TCP | EFX_PKT_UDP | EFX_CKSUM_TCPUDP))) {
+	case (EFX_PKT_TCP | EFX_CKSUM_TCPUDP):
+	case (EFX_PKT_UDP | EFX_CKSUM_TCPUDP):
+		mbuf_flags |= PKT_RX_L4_CKSUM_GOOD;
+		break;
+	case EFX_PKT_TCP:
+	case EFX_PKT_UDP:
+		mbuf_flags |= PKT_RX_L4_CKSUM_BAD;
+		break;
+	default:
+		RTE_BUILD_BUG_ON(PKT_RX_L4_CKSUM_UNKNOWN != 0);
+		SFC_ASSERT((mbuf_flags & PKT_RX_L4_CKSUM_MASK) ==
+			   PKT_RX_L4_CKSUM_UNKNOWN);
+		break;
+	}
+
+	return mbuf_flags;
+}
+
 uint16_t
 sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 {
@@ -182,6 +221,7 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 		rte_pktmbuf_data_len(m) = seg_len;
 		rte_pktmbuf_pkt_len(m) = seg_len;
 
+		m->ol_flags = sfc_rx_desc_flags_to_offload_flags(desc_flags);
 		m->packet_type = RTE_PTYPE_L2_ETHER;
 
 		*rx_pkts++ = m;
-- 
2.5.5

[PATCH v2 14/32] net/sfc: handle received packet type info provided by HW

[Andrew Rybchenko]

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 ++
 drivers/net/sfc/sfc_ethdev.c         | 19 +++++++++++++++++++
 drivers/net/sfc/sfc_rx.c             | 14 +++++++++++++-
 4 files changed, 35 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index ec1d1db..b5887d5 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -15,6 +15,7 @@ Multicast MAC filter = Y
 Flow control         = Y
 L3 checksum offload  = Y
 L4 checksum offload  = Y
+Packet type parsing  = Y
 Basic stats          = Y
 Extended stats       = Y
 BSD nic_uio          = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 8724eae..1333f8b 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -67,6 +67,8 @@ SFC EFX PMD has support for:
 
 - IPv4/IPv6 TCP/UDP receive checksum offload
 
+- Received packet type information
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 235582d..03c9a01 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -97,6 +97,24 @@ sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 	dev_info->tx_desc_lim.nb_align = EFX_TXQ_MINNDESCS;
 }
 
+static const uint32_t *
+sfc_dev_supported_ptypes_get(struct rte_eth_dev *dev)
+{
+	static const uint32_t ptypes[] = {
+		RTE_PTYPE_L2_ETHER,
+		RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+		RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+		RTE_PTYPE_L4_TCP,
+		RTE_PTYPE_L4_UDP,
+		RTE_PTYPE_UNKNOWN
+	};
+
+	if (dev->rx_pkt_burst == sfc_recv_pkts)
+		return ptypes;
+
+	return NULL;
+}
+
 static int
 sfc_dev_configure(struct rte_eth_dev *dev)
 {
@@ -819,6 +837,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.xstats_get			= sfc_xstats_get,
 	.xstats_get_names		= sfc_xstats_get_names,
 	.dev_infos_get			= sfc_dev_infos_get,
+	.dev_supported_ptypes_get	= sfc_dev_supported_ptypes_get,
 	.mtu_set			= sfc_dev_set_mtu,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index 4df4132..be8fa23 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -169,6 +169,18 @@ sfc_rx_desc_flags_to_offload_flags(const unsigned int desc_flags)
 	return mbuf_flags;
 }
 
+static uint32_t
+sfc_rx_desc_flags_to_packet_type(const unsigned int desc_flags)
+{
+	return RTE_PTYPE_L2_ETHER |
+		((desc_flags & EFX_PKT_IPV4) ?
+			RTE_PTYPE_L3_IPV4_EXT_UNKNOWN : 0) |
+		((desc_flags & EFX_PKT_IPV6) ?
+			RTE_PTYPE_L3_IPV6_EXT_UNKNOWN : 0) |
+		((desc_flags & EFX_PKT_TCP) ? RTE_PTYPE_L4_TCP : 0) |
+		((desc_flags & EFX_PKT_UDP) ? RTE_PTYPE_L4_UDP : 0);
+}
+
 uint16_t
 sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 {
@@ -222,7 +234,7 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 		rte_pktmbuf_pkt_len(m) = seg_len;
 
 		m->ol_flags = sfc_rx_desc_flags_to_offload_flags(desc_flags);
-		m->packet_type = RTE_PTYPE_L2_ETHER;
+		m->packet_type = sfc_rx_desc_flags_to_packet_type(desc_flags);
 
 		*rx_pkts++ = m;
 		done_pkts++;
-- 
2.5.5

[PATCH v2 15/32] net/sfc: support callback to get receive queue information

[Andrew Rybchenko]

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 03c9a01..e67ca61 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -821,6 +821,29 @@ sfc_set_mc_addr_list(struct rte_eth_dev *dev, struct ether_addr *mc_addr_set,
 	return -rc;
 }
 
+static void
+sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
+		      struct rte_eth_rxq_info *qinfo)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	struct sfc_rxq_info *rxq_info;
+	struct sfc_rxq *rxq;
+
+	sfc_adapter_lock(sa);
+
+	SFC_ASSERT(rx_queue_id < sa->rxq_count);
+
+	rxq_info = &sa->rxq_info[rx_queue_id];
+	rxq = rxq_info->rxq;
+	SFC_ASSERT(rxq != NULL);
+
+	qinfo->mp = rxq->refill_mb_pool;
+	qinfo->conf.rx_drop_en = 1;
+	qinfo->nb_desc = rxq_info->entries;
+
+	sfc_adapter_unlock(sa);
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -847,6 +870,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
 	.mac_addr_set			= sfc_mac_addr_set,
 	.set_mc_addr_list		= sfc_set_mc_addr_list,
+	.rxq_info_get			= sfc_rx_queue_info_get,
 };
 
 static int
-- 
2.5.5

[PATCH v2 16/32] net/sfc: support Rx free threshold

[Andrew Rybchenko]

Rx free threshold defines minimum number of free Rx descriptors
when Rx ring refill should be done.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c |  1 +
 drivers/net/sfc/sfc_rx.c     | 16 ++++++++++++----
 drivers/net/sfc/sfc_rx.h     |  1 +
 3 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index e67ca61..1d7993f 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -838,6 +838,7 @@ sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 	SFC_ASSERT(rxq != NULL);
 
 	qinfo->mp = rxq->refill_mb_pool;
+	qinfo->conf.rx_free_thresh = rxq->refill_threshold;
 	qinfo->conf.rx_drop_en = 1;
 	qinfo->nb_desc = rxq_info->entries;
 
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index be8fa23..278d583 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -87,6 +87,10 @@ sfc_rx_qrefill(struct sfc_rxq *rxq)
 
 	free_space = EFX_RXQ_LIMIT(rxq->ptr_mask + 1) -
 		(added - rxq->completed);
+
+	if (free_space < rxq->refill_threshold)
+		return;
+
 	bulks = free_space / RTE_DIM(objs);
 
 	id = added & rxq->ptr_mask;
@@ -410,9 +414,10 @@ sfc_rx_qstop(struct sfc_adapter *sa, unsigned int sw_index)
 }
 
 static int
-sfc_rx_qcheck_conf(struct sfc_adapter *sa,
+sfc_rx_qcheck_conf(struct sfc_adapter *sa, uint16_t nb_rx_desc,
 		   const struct rte_eth_rxconf *rx_conf)
 {
+	const uint16_t rx_free_thresh_max = EFX_RXQ_LIMIT(nb_rx_desc);
 	int rc = 0;
 
 	if (rx_conf->rx_thresh.pthresh != 0 ||
@@ -423,8 +428,10 @@ sfc_rx_qcheck_conf(struct sfc_adapter *sa,
 		rc = EINVAL;
 	}
 
-	if (rx_conf->rx_free_thresh != 0) {
-		sfc_err(sa, "RxQ free threshold is not supported");
+	if (rx_conf->rx_free_thresh > rx_free_thresh_max) {
+		sfc_err(sa,
+			"RxQ free threshold too large: %u vs maximum %u",
+			rx_conf->rx_free_thresh, rx_free_thresh_max);
 		rc = EINVAL;
 	}
 
@@ -555,7 +562,7 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 	struct sfc_evq *evq;
 	struct sfc_rxq *rxq;
 
-	rc = sfc_rx_qcheck_conf(sa, rx_conf);
+	rc = sfc_rx_qcheck_conf(sa, nb_rx_desc, rx_conf);
 	if (rc != 0)
 		goto fail_bad_conf;
 
@@ -615,6 +622,7 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 	evq->rxq = rxq;
 	rxq->evq = evq;
 	rxq->ptr_mask = rxq_info->entries - 1;
+	rxq->refill_threshold = rx_conf->rx_free_thresh;
 	rxq->refill_mb_pool = mb_pool;
 	rxq->buf_size = buf_size;
 	rxq->hw_index = sw_index;
diff --git a/drivers/net/sfc/sfc_rx.h b/drivers/net/sfc/sfc_rx.h
index e4385b9..69318ab 100644
--- a/drivers/net/sfc/sfc_rx.h
+++ b/drivers/net/sfc/sfc_rx.h
@@ -87,6 +87,7 @@ struct sfc_rxq {
 	/* Used on refill */
 	unsigned int		added;
 	unsigned int		pushed;
+	unsigned int		refill_threshold;
 	uint8_t			port_id;
 	uint16_t		buf_size;
 	struct rte_mempool	*refill_mb_pool;
-- 
2.5.5

[PATCH v2 17/32] net/sfc: add callback to get RxQ pending descriptors count

[Andrew Rybchenko]

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 11 +++++++++++
 drivers/net/sfc/sfc_rx.c     | 16 ++++++++++++++++
 drivers/net/sfc/sfc_rx.h     |  3 +++
 3 files changed, 30 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 1d7993f..163f9ed 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -845,6 +845,16 @@ sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 	sfc_adapter_unlock(sa);
 }
 
+static uint32_t
+sfc_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+
+	sfc_log_init(sa, "RxQ=%u", rx_queue_id);
+
+	return sfc_rx_qdesc_npending(sa, rx_queue_id);
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -865,6 +875,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.mtu_set			= sfc_dev_set_mtu,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
+	.rx_queue_count			= sfc_rx_queue_count,
 	.tx_queue_setup			= sfc_tx_queue_setup,
 	.tx_queue_release		= sfc_tx_queue_release,
 	.flow_ctrl_get			= sfc_flow_ctrl_get,
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index 278d583..ff9d799 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -257,6 +257,22 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 	return done_pkts;
 }
 
+unsigned int
+sfc_rx_qdesc_npending(struct sfc_adapter *sa, unsigned int sw_index)
+{
+	struct sfc_rxq *rxq;
+
+	SFC_ASSERT(sw_index < sa->rxq_count);
+	rxq = sa->rxq_info[sw_index].rxq;
+
+	if (rxq == NULL || (rxq->state & SFC_RXQ_RUNNING) == 0)
+		return 0;
+
+	sfc_ev_qpoll(rxq->evq);
+
+	return rxq->pending - rxq->completed;
+}
+
 static void
 sfc_rx_qpurge(struct sfc_rxq *rxq)
 {
diff --git a/drivers/net/sfc/sfc_rx.h b/drivers/net/sfc/sfc_rx.h
index 69318ab..8064af6 100644
--- a/drivers/net/sfc/sfc_rx.h
+++ b/drivers/net/sfc/sfc_rx.h
@@ -140,6 +140,9 @@ void sfc_rx_qflush_failed(struct sfc_rxq *rxq);
 uint16_t sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 		       uint16_t nb_pkts);
 
+unsigned int sfc_rx_qdesc_npending(struct sfc_adapter *sa,
+				   unsigned int sw_index);
+
 #ifdef __cplusplus
 }
 #endif
-- 
2.5.5

[PATCH v2 18/32] net/sfc: add RxQ descriptor done callback

[Andrew Rybchenko]

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c |  9 +++++++++
 drivers/net/sfc/sfc_rx.c     | 11 +++++++++++
 drivers/net/sfc/sfc_rx.h     |  1 +
 3 files changed, 21 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 163f9ed..59dcdb4 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -855,6 +855,14 @@ sfc_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	return sfc_rx_qdesc_npending(sa, rx_queue_id);
 }
 
+static int
+sfc_rx_descriptor_done(void *queue, uint16_t offset)
+{
+	struct sfc_rxq *rxq = queue;
+
+	return sfc_rx_qdesc_done(rxq, offset);
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -876,6 +884,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
 	.rx_queue_count			= sfc_rx_queue_count,
+	.rx_descriptor_done		= sfc_rx_descriptor_done,
 	.tx_queue_setup			= sfc_tx_queue_setup,
 	.tx_queue_release		= sfc_tx_queue_release,
 	.flow_ctrl_get			= sfc_flow_ctrl_get,
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index ff9d799..802beb2 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -273,6 +273,17 @@ sfc_rx_qdesc_npending(struct sfc_adapter *sa, unsigned int sw_index)
 	return rxq->pending - rxq->completed;
 }
 
+int
+sfc_rx_qdesc_done(struct sfc_rxq *rxq, unsigned int offset)
+{
+	if ((rxq->state & SFC_RXQ_RUNNING) == 0)
+		return 0;
+
+	sfc_ev_qpoll(rxq->evq);
+
+	return offset < (rxq->pending - rxq->completed);
+}
+
 static void
 sfc_rx_qpurge(struct sfc_rxq *rxq)
 {
diff --git a/drivers/net/sfc/sfc_rx.h b/drivers/net/sfc/sfc_rx.h
index 8064af6..8d8e709 100644
--- a/drivers/net/sfc/sfc_rx.h
+++ b/drivers/net/sfc/sfc_rx.h
@@ -142,6 +142,7 @@ uint16_t sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 
 unsigned int sfc_rx_qdesc_npending(struct sfc_adapter *sa,
 				   unsigned int sw_index);
+int sfc_rx_qdesc_done(struct sfc_rxq *rxq, unsigned int offset);
 
 #ifdef __cplusplus
 }
-- 
2.5.5

[PATCH v2 19/32] net/sfc: support scattered Rx DMA

[Andrew Rybchenko]

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  4 ++--
 drivers/net/sfc/sfc_ethdev.c         |  1 +
 drivers/net/sfc/sfc_rx.c             | 39 +++++++++++++++++++++++++++---------
 4 files changed, 34 insertions(+), 11 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index b5887d5..74cc942 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -9,6 +9,7 @@ Link status          = Y
 Link status event    = Y
 MTU update           = Y
 Jumbo frame          = Y
+Scattered Rx         = Y
 Promiscuous mode     = Y
 Allmulticast mode    = Y
 Multicast MAC filter = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 1333f8b..2e30dfe 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -69,6 +69,8 @@ SFC EFX PMD has support for:
 
 - Received packet type information
 
+- Scattered Rx DMA for packet that are larger that a single Rx descriptor
+
 
 Non-supported Features
 ----------------------
@@ -89,8 +91,6 @@ The features not yet supported include:
 
 - VLAN stripping
 
-- Scattered receive
-
 - LRO
 
 
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 59dcdb4..f1b655b 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -840,6 +840,7 @@ sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 	qinfo->mp = rxq->refill_mb_pool;
 	qinfo->conf.rx_free_thresh = rxq->refill_threshold;
 	qinfo->conf.rx_drop_en = 1;
+	qinfo->scattered_rx = (rxq_info->type == EFX_RXQ_TYPE_SCATTER);
 	qinfo->nb_desc = rxq_info->entries;
 
 	sfc_adapter_unlock(sa);
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index 802beb2..2909ec0 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -193,6 +193,7 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 	unsigned int prefix_size = rxq->prefix_size;
 	unsigned int done_pkts = 0;
 	boolean_t discard_next = B_FALSE;
+	struct rte_mbuf *scatter_pkt = NULL;
 
 	if (unlikely((rxq->state & SFC_RXQ_RUNNING) == 0))
 		return 0;
@@ -218,9 +219,6 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 		if (desc_flags & (EFX_ADDR_MISMATCH | EFX_DISCARD))
 			goto discard;
 
-		if (desc_flags & EFX_PKT_CONT)
-			goto discard;
-
 		if (desc_flags & EFX_PKT_PREFIX_LEN) {
 			uint16_t tmp_size;
 			int rc __rte_unused;
@@ -237,6 +235,29 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 		rte_pktmbuf_data_len(m) = seg_len;
 		rte_pktmbuf_pkt_len(m) = seg_len;
 
+		if (scatter_pkt != NULL) {
+			if (rte_pktmbuf_chain(scatter_pkt, m) != 0) {
+				rte_mempool_put(rxq->refill_mb_pool,
+						scatter_pkt);
+				goto discard;
+			}
+			/* The packet to deliver */
+			m = scatter_pkt;
+		}
+
+		if (desc_flags & EFX_PKT_CONT) {
+			/* The packet is scattered, more fragments to come */
+			scatter_pkt = m;
+			/* Futher fragments have no prefix */
+			prefix_size = 0;
+			continue;
+		}
+
+		/* Scattered packet is done */
+		scatter_pkt = NULL;
+		/* The first fragment of the packet has prefix */
+		prefix_size = rxq->prefix_size;
+
 		m->ol_flags = sfc_rx_desc_flags_to_offload_flags(desc_flags);
 		m->packet_type = sfc_rx_desc_flags_to_packet_type(desc_flags);
 
@@ -250,6 +271,9 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 		rxd->mbuf = NULL;
 	}
 
+	/* pending is only moved when entire packet is received */
+	SFC_ASSERT(scatter_pkt == NULL);
+
 	rxq->completed = completed;
 
 	sfc_rx_qrefill(rxq);
@@ -618,7 +642,9 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 
 	SFC_ASSERT(nb_rx_desc <= rxq_info->max_entries);
 	rxq_info->entries = nb_rx_desc;
-	rxq_info->type = EFX_RXQ_TYPE_DEFAULT;
+	rxq_info->type =
+		sa->eth_dev->data->dev_conf.rxmode.enable_scatter ?
+		EFX_RXQ_TYPE_SCATTER : EFX_RXQ_TYPE_DEFAULT;
 
 	evq_index = sfc_evq_index_by_rxq_sw_index(sa, sw_index);
 
@@ -806,11 +832,6 @@ sfc_rx_check_mode(struct sfc_adapter *sa, struct rte_eth_rxmode *rxmode)
 		rxmode->hw_strip_crc = 1;
 	}
 
-	if (rxmode->enable_scatter) {
-		sfc_err(sa, "Scatter on Rx not supported");
-		rc = EINVAL;
-	}
-
 	if (rxmode->enable_lro) {
 		sfc_err(sa, "LRO not supported");
 		rc = EINVAL;
-- 
2.5.5

[PATCH v2 20/32] net/sfc: support deferred start of receive queues

[Andrew Rybchenko]

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 ++
 drivers/net/sfc/sfc_ethdev.c         | 51 ++++++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_rx.c             | 18 +++++++------
 drivers/net/sfc/sfc_rx.h             |  2 ++
 5 files changed, 66 insertions(+), 8 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 74cc942..4a887f0 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -7,6 +7,7 @@
 Speed capabilities   = Y
 Link status          = Y
 Link status event    = Y
+Queue start/stop     = P
 MTU update           = Y
 Jumbo frame          = Y
 Scattered Rx         = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 2e30dfe..b0beaf1 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -71,6 +71,8 @@ SFC EFX PMD has support for:
 
 - Scattered Rx DMA for packet that are larger that a single Rx descriptor
 
+- Deferred receive queue start
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index f1b655b..c531fdb 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -840,6 +840,7 @@ sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 	qinfo->mp = rxq->refill_mb_pool;
 	qinfo->conf.rx_free_thresh = rxq->refill_threshold;
 	qinfo->conf.rx_drop_en = 1;
+	qinfo->conf.rx_deferred_start = rxq_info->deferred_start;
 	qinfo->scattered_rx = (rxq_info->type == EFX_RXQ_TYPE_SCATTER);
 	qinfo->nb_desc = rxq_info->entries;
 
@@ -864,6 +865,54 @@ sfc_rx_descriptor_done(void *queue, uint16_t offset)
 	return sfc_rx_qdesc_done(rxq, offset);
 }
 
+static int
+sfc_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	int rc;
+
+	sfc_log_init(sa, "RxQ=%u", rx_queue_id);
+
+	sfc_adapter_lock(sa);
+
+	rc = EINVAL;
+	if (sa->state != SFC_ADAPTER_STARTED)
+		goto fail_not_started;
+
+	rc = sfc_rx_qstart(sa, rx_queue_id);
+	if (rc != 0)
+		goto fail_rx_qstart;
+
+	sa->rxq_info[rx_queue_id].deferred_started = B_TRUE;
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+
+fail_rx_qstart:
+fail_not_started:
+	sfc_adapter_unlock(sa);
+	SFC_ASSERT(rc > 0);
+	return -rc;
+}
+
+static int
+sfc_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+
+	sfc_log_init(sa, "RxQ=%u", rx_queue_id);
+
+	sfc_adapter_lock(sa);
+	sfc_rx_qstop(sa, rx_queue_id);
+
+	sa->rxq_info[rx_queue_id].deferred_started = B_FALSE;
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -882,6 +931,8 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_infos_get			= sfc_dev_infos_get,
 	.dev_supported_ptypes_get	= sfc_dev_supported_ptypes_get,
 	.mtu_set			= sfc_dev_set_mtu,
+	.rx_queue_start			= sfc_rx_queue_start,
+	.rx_queue_stop			= sfc_rx_queue_stop,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
 	.rx_queue_count			= sfc_rx_queue_count,
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index 2909ec0..3bfce1c 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -444,6 +444,9 @@ sfc_rx_qstop(struct sfc_adapter *sa, unsigned int sw_index)
 
 	rxq_info = &sa->rxq_info[sw_index];
 	rxq = rxq_info->rxq;
+
+	if (rxq->state == SFC_RXQ_INITIALIZED)
+		return;
 	SFC_ASSERT(rxq->state & SFC_RXQ_STARTED);
 
 	/* It seems to be used by DPDK for debug purposes only ('rte_ether') */
@@ -491,11 +494,6 @@ sfc_rx_qcheck_conf(struct sfc_adapter *sa, uint16_t nb_rx_desc,
 		rc = EINVAL;
 	}
 
-	if (rx_conf->rx_deferred_start != 0) {
-		sfc_err(sa, "RxQ deferred start is not supported");
-		rc = EINVAL;
-	}
-
 	return rc;
 }
 
@@ -688,6 +686,7 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 	rxq->state = SFC_RXQ_INITIALIZED;
 
 	rxq_info->rxq = rxq;
+	rxq_info->deferred_start = (rx_conf->rx_deferred_start != 0);
 
 	return 0;
 
@@ -742,9 +741,12 @@ sfc_rx_start(struct sfc_adapter *sa)
 		goto fail_rx_init;
 
 	for (sw_index = 0; sw_index < sa->rxq_count; ++sw_index) {
-		rc = sfc_rx_qstart(sa, sw_index);
-		if (rc != 0)
-			goto fail_rx_qstart;
+		if ((!sa->rxq_info[sw_index].deferred_start ||
+		     sa->rxq_info[sw_index].deferred_started)) {
+			rc = sfc_rx_qstart(sa, sw_index);
+			if (rc != 0)
+				goto fail_rx_qstart;
+		}
 	}
 
 	return 0;
diff --git a/drivers/net/sfc/sfc_rx.h b/drivers/net/sfc/sfc_rx.h
index 8d8e709..4aa6aea 100644
--- a/drivers/net/sfc/sfc_rx.h
+++ b/drivers/net/sfc/sfc_rx.h
@@ -119,6 +119,8 @@ struct sfc_rxq_info {
 	unsigned int		entries;
 	efx_rxq_type_t		type;
 	struct sfc_rxq		*rxq;
+	boolean_t		deferred_start;
+	boolean_t		deferred_started;
 };
 
 int sfc_rx_init(struct sfc_adapter *sa);
-- 
2.5.5

[PATCH v2 21/32] net/sfc: add callback to get transmit queue information

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index c531fdb..73e8fe2 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -847,6 +847,28 @@ sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 	sfc_adapter_unlock(sa);
 }
 
+static void
+sfc_tx_queue_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
+		      struct rte_eth_txq_info *qinfo)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	struct sfc_txq_info *txq_info;
+
+	sfc_adapter_lock(sa);
+
+	SFC_ASSERT(tx_queue_id < sa->txq_count);
+
+	txq_info = &sa->txq_info[tx_queue_id];
+	SFC_ASSERT(txq_info->txq != NULL);
+
+	memset(qinfo, 0, sizeof(*qinfo));
+
+	qinfo->conf.txq_flags = txq_info->txq->flags;
+	qinfo->nb_desc = txq_info->entries;
+
+	sfc_adapter_unlock(sa);
+}
+
 static uint32_t
 sfc_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 {
@@ -944,6 +966,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.mac_addr_set			= sfc_mac_addr_set,
 	.set_mc_addr_list		= sfc_set_mc_addr_list,
 	.rxq_info_get			= sfc_rx_queue_info_get,
+	.txq_info_get			= sfc_tx_queue_info_get,
 };
 
 static int
-- 
2.5.5

[PATCH v2 22/32] net/sfc: support Tx free threshold

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c |  1 +
 drivers/net/sfc/sfc_tweak.h  |  3 +++
 drivers/net/sfc/sfc_tx.c     | 14 ++++++++------
 drivers/net/sfc/sfc_tx.h     | 10 +---------
 4 files changed, 13 insertions(+), 15 deletions(-)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 73e8fe2..5d0d774 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -864,6 +864,7 @@ sfc_tx_queue_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
 	memset(qinfo, 0, sizeof(*qinfo));
 
 	qinfo->conf.txq_flags = txq_info->txq->flags;
+	qinfo->conf.tx_free_thresh = txq_info->txq->free_thresh;
 	qinfo->nb_desc = txq_info->entries;
 
 	sfc_adapter_unlock(sa);
diff --git a/drivers/net/sfc/sfc_tweak.h b/drivers/net/sfc/sfc_tweak.h
index 8a60f35..be39a5e 100644
--- a/drivers/net/sfc/sfc_tweak.h
+++ b/drivers/net/sfc/sfc_tweak.h
@@ -48,4 +48,7 @@
  */
 #define SFC_TX_XMIT_PKTS_REAP_AT_LEAST_ONCE	0
 
+/** Default free threshold follows recommendations from DPDK documentation */
+#define SFC_TX_DEFAULT_FREE_THRESH	32
+
 #endif /* _SFC_TWEAK_H_ */
diff --git a/drivers/net/sfc/sfc_tx.c b/drivers/net/sfc/sfc_tx.c
index a240610..13b24f7 100644
--- a/drivers/net/sfc/sfc_tx.c
+++ b/drivers/net/sfc/sfc_tx.c
@@ -54,7 +54,7 @@
 #define SFC_TX_QFLUSH_POLL_ATTEMPTS	(2000)
 
 static int
-sfc_tx_qcheck_conf(struct sfc_adapter *sa,
+sfc_tx_qcheck_conf(struct sfc_adapter *sa, uint16_t nb_tx_desc,
 		   const struct rte_eth_txconf *tx_conf)
 {
 	unsigned int flags = tx_conf->txq_flags;
@@ -65,9 +65,10 @@ sfc_tx_qcheck_conf(struct sfc_adapter *sa,
 		rc = EINVAL;
 	}
 
-	if (tx_conf->tx_free_thresh != 0) {
+	if (tx_conf->tx_free_thresh > EFX_TXQ_LIMIT(nb_tx_desc)) {
 		sfc_err(sa,
-			"setting explicit TX free threshold is not supported");
+			"TxQ free threshold too large: %u vs maximum %u",
+			tx_conf->tx_free_thresh, EFX_TXQ_LIMIT(nb_tx_desc));
 		rc = EINVAL;
 	}
 
@@ -147,7 +148,7 @@ sfc_tx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 
 	sfc_log_init(sa, "TxQ = %u", sw_index);
 
-	rc = sfc_tx_qcheck_conf(sa, tx_conf);
+	rc = sfc_tx_qcheck_conf(sa, nb_tx_desc, tx_conf);
 	if (rc != 0)
 		goto fail_bad_conf;
 
@@ -188,6 +189,8 @@ sfc_tx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 
 	txq->state = SFC_TXQ_INITIALIZED;
 	txq->ptr_mask = txq_info->entries - 1;
+	txq->free_thresh = (tx_conf->tx_free_thresh) ? tx_conf->tx_free_thresh :
+						     SFC_TX_DEFAULT_FREE_THRESH;
 	txq->hw_index = sw_index;
 	txq->flags = tx_conf->txq_flags;
 	txq->evq = evq;
@@ -537,8 +540,7 @@ sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 	unsigned int pkts_sent = 0;
 	efx_desc_t *pend = &txq->pend_desc[0];
 	const unsigned int hard_max_fill = EFX_TXQ_LIMIT(txq->ptr_mask + 1);
-	const unsigned int soft_max_fill = hard_max_fill -
-					   SFC_TX_MAX_PKT_DESC;
+	const unsigned int soft_max_fill = hard_max_fill - txq->free_thresh;
 	unsigned int fill_level = added - txq->completed;
 	boolean_t reap_done;
 	int rc __rte_unused;
diff --git a/drivers/net/sfc/sfc_tx.h b/drivers/net/sfc/sfc_tx.h
index fe2736b..f9eecc0 100644
--- a/drivers/net/sfc/sfc_tx.h
+++ b/drivers/net/sfc/sfc_tx.h
@@ -40,15 +40,6 @@ extern "C" {
 #endif
 
 /**
- * Estimated maximum number of segments that transmit packet consists of;
- * it is determined with respect to the expectation of a packet to consist
- * of a header plus a couple of data segments one of those crossing 4K page;
- * it is used by transmit path to avoid redundant reaping and, thus,
- * to avoid increase of latency
- */
-#define SFC_TX_MAX_PKT_DESC	4
-
-/**
  * A segment must not cross 4K boundary
  * (this is a requirement of NIC TX descriptors)
  */
@@ -85,6 +76,7 @@ struct sfc_txq {
 	unsigned int		added;
 	unsigned int		pending;
 	unsigned int		completed;
+	unsigned int		free_thresh;
 
 	unsigned int		hw_index;
 	unsigned int		flags;
-- 
2.5.5

[PATCH v2 23/32] net/sfc: support deferred start of transmit queues

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  2 +-
 doc/guides/nics/sfc_efx.rst          |  2 +-
 drivers/net/sfc/sfc_ethdev.c         | 51 ++++++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_tx.c             | 18 +++++++------
 drivers/net/sfc/sfc_tx.h             |  2 ++
 5 files changed, 65 insertions(+), 10 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 4a887f0..38bf9d2 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -7,7 +7,7 @@
 Speed capabilities   = Y
 Link status          = Y
 Link status event    = Y
-Queue start/stop     = P
+Queue start/stop     = Y
 MTU update           = Y
 Jumbo frame          = Y
 Scattered Rx         = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index b0beaf1..304dc95 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -71,7 +71,7 @@ SFC EFX PMD has support for:
 
 - Scattered Rx DMA for packet that are larger that a single Rx descriptor
 
-- Deferred receive queue start
+- Deferred receive and transmit queue start
 
 
 Non-supported Features
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 5d0d774..ba3c838 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -865,6 +865,7 @@ sfc_tx_queue_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
 
 	qinfo->conf.txq_flags = txq_info->txq->flags;
 	qinfo->conf.tx_free_thresh = txq_info->txq->free_thresh;
+	qinfo->conf.tx_deferred_start = txq_info->deferred_start;
 	qinfo->nb_desc = txq_info->entries;
 
 	sfc_adapter_unlock(sa);
@@ -936,6 +937,54 @@ sfc_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	return 0;
 }
 
+static int
+sfc_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	int rc;
+
+	sfc_log_init(sa, "TxQ = %u", tx_queue_id);
+
+	sfc_adapter_lock(sa);
+
+	rc = EINVAL;
+	if (sa->state != SFC_ADAPTER_STARTED)
+		goto fail_not_started;
+
+	rc = sfc_tx_qstart(sa, tx_queue_id);
+	if (rc != 0)
+		goto fail_tx_qstart;
+
+	sa->txq_info[tx_queue_id].deferred_started = B_TRUE;
+
+	sfc_adapter_unlock(sa);
+	return 0;
+
+fail_tx_qstart:
+
+fail_not_started:
+	sfc_adapter_unlock(sa);
+	SFC_ASSERT(rc > 0);
+	return -rc;
+}
+
+static int
+sfc_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+
+	sfc_log_init(sa, "TxQ = %u", tx_queue_id);
+
+	sfc_adapter_lock(sa);
+
+	sfc_tx_qstop(sa, tx_queue_id);
+
+	sa->txq_info[tx_queue_id].deferred_started = B_FALSE;
+
+	sfc_adapter_unlock(sa);
+	return 0;
+}
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -956,6 +1005,8 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.mtu_set			= sfc_dev_set_mtu,
 	.rx_queue_start			= sfc_rx_queue_start,
 	.rx_queue_stop			= sfc_rx_queue_stop,
+	.tx_queue_start			= sfc_tx_queue_start,
+	.tx_queue_stop			= sfc_tx_queue_stop,
 	.rx_queue_setup			= sfc_rx_queue_setup,
 	.rx_queue_release		= sfc_rx_queue_release,
 	.rx_queue_count			= sfc_rx_queue_count,
diff --git a/drivers/net/sfc/sfc_tx.c b/drivers/net/sfc/sfc_tx.c
index 13b24f7..15a6f9f 100644
--- a/drivers/net/sfc/sfc_tx.c
+++ b/drivers/net/sfc/sfc_tx.c
@@ -72,11 +72,6 @@ sfc_tx_qcheck_conf(struct sfc_adapter *sa, uint16_t nb_tx_desc,
 		rc = EINVAL;
 	}
 
-	if (tx_conf->tx_deferred_start != 0) {
-		sfc_err(sa, "TX queue deferred start is not supported (yet)");
-		rc = EINVAL;
-	}
-
 	if (tx_conf->tx_thresh.pthresh != 0 ||
 	    tx_conf->tx_thresh.hthresh != 0 ||
 	    tx_conf->tx_thresh.wthresh != 0) {
@@ -198,6 +193,7 @@ sfc_tx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 	evq->txq = txq;
 
 	txq_info->txq = txq;
+	txq_info->deferred_start = (tx_conf->tx_deferred_start != 0);
 
 	return 0;
 
@@ -425,6 +421,9 @@ sfc_tx_qstop(struct sfc_adapter *sa, unsigned int sw_index)
 
 	txq = txq_info->txq;
 
+	if (txq->state == SFC_TXQ_INITIALIZED)
+		return;
+
 	SFC_ASSERT(txq->state & SFC_TXQ_STARTED);
 
 	txq->state &= ~SFC_TXQ_RUNNING;
@@ -497,9 +496,12 @@ sfc_tx_start(struct sfc_adapter *sa)
 		goto fail_efx_tx_init;
 
 	for (sw_index = 0; sw_index < sa->txq_count; ++sw_index) {
-		rc = sfc_tx_qstart(sa, sw_index);
-		if (rc != 0)
-			goto fail_tx_qstart;
+		if (!(sa->txq_info[sw_index].deferred_start) ||
+		    sa->txq_info[sw_index].deferred_started) {
+			rc = sfc_tx_qstart(sa, sw_index);
+			if (rc != 0)
+				goto fail_tx_qstart;
+		}
 	}
 
 	return 0;
diff --git a/drivers/net/sfc/sfc_tx.h b/drivers/net/sfc/sfc_tx.h
index f9eecc0..632e3be 100644
--- a/drivers/net/sfc/sfc_tx.h
+++ b/drivers/net/sfc/sfc_tx.h
@@ -91,6 +91,8 @@ sfc_txq_sw_index(const struct sfc_txq *txq)
 struct sfc_txq_info {
 	unsigned int		entries;
 	struct sfc_txq		*txq;
+	boolean_t		deferred_start;
+	boolean_t		deferred_started;
 };
 
 int sfc_tx_init(struct sfc_adapter *sa);
-- 
2.5.5

[PATCH v2 24/32] net/sfc: support VLAN offload on transmit path

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 ++
 drivers/net/sfc/sfc_ethdev.c         |  8 +++++--
 drivers/net/sfc/sfc_tx.c             | 45 +++++++++++++++++++++++++++++++++++-
 drivers/net/sfc/sfc_tx.h             |  1 +
 5 files changed, 54 insertions(+), 3 deletions(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 38bf9d2..e7a1143 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -15,6 +15,7 @@ Promiscuous mode     = Y
 Allmulticast mode    = Y
 Multicast MAC filter = Y
 Flow control         = Y
+VLAN offload         = P
 L3 checksum offload  = Y
 L4 checksum offload  = Y
 Packet type parsing  = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 304dc95..2244e7a 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -73,6 +73,8 @@ SFC EFX PMD has support for:
 
 - Deferred receive and transmit queue start
 
+- Transmit VLAN insertion (if running firmware variant supports it)
+
 
 Non-supported Features
 ----------------------
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index ba3c838..0de17ca 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -46,6 +46,7 @@ static void
 sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 {
 	struct sfc_adapter *sa = dev->data->dev_private;
+	const efx_nic_cfg_t *encp = efx_nic_cfg_get(sa->nic);
 
 	RTE_SET_USED(dev_info);
 
@@ -78,8 +79,11 @@ sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 		DEV_TX_OFFLOAD_UDP_CKSUM |
 		DEV_TX_OFFLOAD_TCP_CKSUM;
 
-	dev_info->default_txconf.txq_flags = ETH_TXQ_FLAGS_NOVLANOFFL |
-					     ETH_TXQ_FLAGS_NOXSUMSCTP;
+	dev_info->default_txconf.txq_flags = ETH_TXQ_FLAGS_NOXSUMSCTP;
+	if (!encp->enc_hw_tx_insert_vlan_enabled)
+		dev_info->default_txconf.txq_flags |= ETH_TXQ_FLAGS_NOVLANOFFL;
+	else
+		dev_info->tx_offload_capa |= DEV_TX_OFFLOAD_VLAN_INSERT;
 
 	dev_info->rx_desc_lim.nb_max = EFX_RXQ_MAXNDESCS;
 	dev_info->rx_desc_lim.nb_min = EFX_RXQ_MINNDESCS;
diff --git a/drivers/net/sfc/sfc_tx.c b/drivers/net/sfc/sfc_tx.c
index 15a6f9f..86bcfec 100644
--- a/drivers/net/sfc/sfc_tx.c
+++ b/drivers/net/sfc/sfc_tx.c
@@ -58,6 +58,7 @@ sfc_tx_qcheck_conf(struct sfc_adapter *sa, uint16_t nb_tx_desc,
 		   const struct rte_eth_txconf *tx_conf)
 {
 	unsigned int flags = tx_conf->txq_flags;
+	const efx_nic_cfg_t *encp = efx_nic_cfg_get(sa->nic);
 	int rc = 0;
 
 	if (tx_conf->tx_rs_thresh != 0) {
@@ -80,7 +81,8 @@ sfc_tx_qcheck_conf(struct sfc_adapter *sa, uint16_t nb_tx_desc,
 		rc = EINVAL;
 	}
 
-	if ((flags & ETH_TXQ_FLAGS_NOVLANOFFL) == 0) {
+	if (!encp->enc_hw_tx_insert_vlan_enabled &&
+	    (flags & ETH_TXQ_FLAGS_NOVLANOFFL) == 0) {
 		sfc_err(sa, "VLAN offload is not supported");
 		rc = EINVAL;
 	}
@@ -384,6 +386,7 @@ sfc_tx_qstart(struct sfc_adapter *sa, unsigned int sw_index)
 		goto fail_tx_qcreate;
 
 	txq->added = txq->pending = txq->completed = desc_index;
+	txq->hw_vlan_tci = 0;
 
 	efx_tx_qenable(txq->common);
 
@@ -533,6 +536,37 @@ sfc_tx_stop(struct sfc_adapter *sa)
 	efx_tx_fini(sa->nic);
 }
 
+/*
+ * The function is used to insert or update VLAN tag;
+ * the firmware has state of the firmware tag to insert per TxQ
+ * (controlled by option descriptors), hence, if the tag of the
+ * packet to be sent is different from one remembered by the firmware,
+ * the function will update it
+ */
+static unsigned int
+sfc_tx_maybe_insert_tag(struct sfc_txq *txq, struct rte_mbuf *m,
+			efx_desc_t **pend)
+{
+	uint16_t this_tag = ((m->ol_flags & PKT_TX_VLAN_PKT) ?
+			     m->vlan_tci : 0);
+
+	if (this_tag == txq->hw_vlan_tci)
+		return 0;
+
+	/*
+	 * The expression inside SFC_ASSERT() is not desired to be checked in
+	 * a non-debug build because it might be too expensive on the data path
+	 */
+	SFC_ASSERT(efx_nic_cfg_get(txq->evq->sa->nic)->enc_hw_tx_insert_vlan_enabled);
+
+	efx_tx_qdesc_vlantci_create(txq->common, rte_cpu_to_be_16(this_tag),
+				    *pend);
+	(*pend)++;
+	txq->hw_vlan_tci = this_tag;
+
+	return 1;
+}
+
 uint16_t
 sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 {
@@ -574,6 +608,15 @@ sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		size_t			pkt_len = m_seg->pkt_len;
 		unsigned int		pkt_descs = 0;
 
+		/*
+		 * Here VLAN TCI is expected to be zero in case if no
+		 * DEV_TX_VLAN_OFFLOAD capability is advertised;
+		 * if the calling app ignores the absence of
+		 * DEV_TX_VLAN_OFFLOAD and pushes VLAN TCI, then
+		 * TX_ERROR will occur
+		 */
+		pkt_descs += sfc_tx_maybe_insert_tag(txq, m_seg, &pend);
+
 		for (; m_seg != NULL; m_seg = m_seg->next) {
 			efsys_dma_addr_t	next_frag;
 			size_t			seg_len;
diff --git a/drivers/net/sfc/sfc_tx.h b/drivers/net/sfc/sfc_tx.h
index 632e3be..4d25c6a 100644
--- a/drivers/net/sfc/sfc_tx.h
+++ b/drivers/net/sfc/sfc_tx.h
@@ -77,6 +77,7 @@ struct sfc_txq {
 	unsigned int		pending;
 	unsigned int		completed;
 	unsigned int		free_thresh;
+	uint16_t		hw_vlan_tci;
 
 	unsigned int		hw_index;
 	unsigned int		flags;
-- 
2.5.5

[PATCH v2 25/32] net/sfc/base: do not use enum type when values are bitmask

[Andrew Rybchenko]

From: Andrew Rybchenko <Andrew.Rybchenko@oktetlabs.ru>

ICC complains that enumerated type mixed with another type.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 drivers/net/sfc/base/ef10_rx.c |  8 ++++----
 drivers/net/sfc/base/efx.h     | 12 ++++++------
 drivers/net/sfc/base/efx_rx.c  |  8 ++++----
 3 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/drivers/net/sfc/base/ef10_rx.c b/drivers/net/sfc/base/ef10_rx.c
index 2bcd823..b65faed 100644
--- a/drivers/net/sfc/base/ef10_rx.c
+++ b/drivers/net/sfc/base/ef10_rx.c
@@ -304,13 +304,13 @@ efx_mcdi_rss_context_set_flags(
 
 	MCDI_IN_POPULATE_DWORD_4(req, RSS_CONTEXT_SET_FLAGS_IN_FLAGS,
 	    RSS_CONTEXT_SET_FLAGS_IN_TOEPLITZ_IPV4_EN,
-	    (type & (1U << EFX_RX_HASH_IPV4)) ? 1 : 0,
+	    (type & EFX_RX_HASH_IPV4) ? 1 : 0,
 	    RSS_CONTEXT_SET_FLAGS_IN_TOEPLITZ_TCPV4_EN,
-	    (type & (1U << EFX_RX_HASH_TCPIPV4)) ? 1 : 0,
+	    (type & EFX_RX_HASH_TCPIPV4) ? 1 : 0,
 	    RSS_CONTEXT_SET_FLAGS_IN_TOEPLITZ_IPV6_EN,
-	    (type & (1U << EFX_RX_HASH_IPV6)) ? 1 : 0,
+	    (type & EFX_RX_HASH_IPV6) ? 1 : 0,
 	    RSS_CONTEXT_SET_FLAGS_IN_TOEPLITZ_TCPV6_EN,
-	    (type & (1U << EFX_RX_HASH_TCPIPV6)) ? 1 : 0);
+	    (type & EFX_RX_HASH_TCPIPV6) ? 1 : 0);
 
 	efx_mcdi_execute(enp, &req);
 
diff --git a/drivers/net/sfc/base/efx.h b/drivers/net/sfc/base/efx.h
index 025721f..0815d7a 100644
--- a/drivers/net/sfc/base/efx.h
+++ b/drivers/net/sfc/base/efx.h
@@ -1851,12 +1851,12 @@ typedef enum efx_rx_hash_alg_e {
 	EFX_RX_HASHALG_TOEPLITZ
 } efx_rx_hash_alg_t;
 
-typedef enum efx_rx_hash_type_e {
-	EFX_RX_HASH_IPV4 = 0,
-	EFX_RX_HASH_TCPIPV4,
-	EFX_RX_HASH_IPV6,
-	EFX_RX_HASH_TCPIPV6,
-} efx_rx_hash_type_t;
+#define	EFX_RX_HASH_IPV4	(1U << 0)
+#define	EFX_RX_HASH_TCPIPV4	(1U << 1)
+#define	EFX_RX_HASH_IPV6	(1U << 2)
+#define	EFX_RX_HASH_TCPIPV6	(1U << 3)
+
+typedef unsigned int efx_rx_hash_type_t;
 
 typedef enum efx_rx_hash_support_e {
 	EFX_RX_HASH_UNAVAILABLE = 0,	/* Hardware hash not inserted */
diff --git a/drivers/net/sfc/base/efx_rx.c b/drivers/net/sfc/base/efx_rx.c
index 330d2aa..c815634 100644
--- a/drivers/net/sfc/base/efx_rx.c
+++ b/drivers/net/sfc/base/efx_rx.c
@@ -786,12 +786,12 @@ siena_rx_scale_mode_set(
 
 	case EFX_RX_HASHALG_TOEPLITZ:
 		EFX_RX_TOEPLITZ_IPV4_HASH(enp, insert,
-		    type & (1 << EFX_RX_HASH_IPV4),
-		    type & (1 << EFX_RX_HASH_TCPIPV4));
+		    type & EFX_RX_HASH_IPV4,
+		    type & EFX_RX_HASH_TCPIPV4);
 
 		EFX_RX_TOEPLITZ_IPV6_HASH(enp,
-		    type & (1 << EFX_RX_HASH_IPV6),
-		    type & (1 << EFX_RX_HASH_TCPIPV6),
+		    type & EFX_RX_HASH_IPV6,
+		    type & EFX_RX_HASH_TCPIPV6,
 		    rc);
 		if (rc != 0)
 			goto fail1;
-- 
2.5.5

[PATCH v2 26/32] net/sfc: add basic stubs for RSS support on driver attach

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/sfc_efx.rst  |  2 ++
 drivers/net/sfc/efsys.h      |  2 +-
 drivers/net/sfc/sfc.c        | 76 +++++++++++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc.h        | 17 ++++++++++
 drivers/net/sfc/sfc_ethdev.c |  8 +++++
 drivers/net/sfc/sfc_rx.c     | 81 +++++++++++++++++++++++++++++++++++++++++++-
 drivers/net/sfc/sfc_rx.h     |  8 +++++
 7 files changed, 192 insertions(+), 2 deletions(-)

diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 2244e7a..17e81dd 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -69,6 +69,8 @@ SFC EFX PMD has support for:
 
 - Received packet type information
 
+- Receive side scaling (RSS)
+
 - Scattered Rx DMA for packet that are larger that a single Rx descriptor
 
 - Deferred receive and transmit queue start
diff --git a/drivers/net/sfc/efsys.h b/drivers/net/sfc/efsys.h
index 0f941e6..fb2f3b5 100644
--- a/drivers/net/sfc/efsys.h
+++ b/drivers/net/sfc/efsys.h
@@ -195,7 +195,7 @@ prefetch_read_once(const volatile void *addr)
 #define EFSYS_OPT_BOOTCFG 0
 
 #define EFSYS_OPT_DIAG 0
-#define EFSYS_OPT_RX_SCALE 0
+#define EFSYS_OPT_RX_SCALE 1
 #define EFSYS_OPT_QSTATS 0
 /* Filters support is required for SFN7xxx and SFN8xx */
 #define EFSYS_OPT_FILTER 1
diff --git a/drivers/net/sfc/sfc.c b/drivers/net/sfc/sfc.c
index e2e6c9e..e79367d 100644
--- a/drivers/net/sfc/sfc.c
+++ b/drivers/net/sfc/sfc.c
@@ -484,6 +484,73 @@ sfc_mem_bar_fini(struct sfc_adapter *sa)
 	memset(ebp, 0, sizeof(*ebp));
 }
 
+#if EFSYS_OPT_RX_SCALE
+/*
+ * A fixed RSS key which has a property of being symmetric
+ * (symmetrical flows are distributed to the same CPU)
+ * and also known to give a uniform distribution
+ * (a good distribution of traffic between different CPUs)
+ */
+static const uint8_t default_rss_key[SFC_RSS_KEY_SIZE] = {
+	0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a,
+	0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a,
+	0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a,
+	0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a,
+	0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a, 0x6d, 0x5a,
+};
+#endif
+
+static int
+sfc_set_rss_defaults(struct sfc_adapter *sa)
+{
+#if EFSYS_OPT_RX_SCALE
+	int rc;
+
+	rc = efx_intr_init(sa->nic, sa->intr.type, NULL);
+	if (rc != 0)
+		goto fail_intr_init;
+
+	rc = efx_ev_init(sa->nic);
+	if (rc != 0)
+		goto fail_ev_init;
+
+	rc = efx_rx_init(sa->nic);
+	if (rc != 0)
+		goto fail_rx_init;
+
+	rc = efx_rx_scale_support_get(sa->nic, &sa->rss_support);
+	if (rc != 0)
+		goto fail_scale_support_get;
+
+	rc = efx_rx_hash_support_get(sa->nic, &sa->hash_support);
+	if (rc != 0)
+		goto fail_hash_support_get;
+
+	efx_rx_fini(sa->nic);
+	efx_ev_fini(sa->nic);
+	efx_intr_fini(sa->nic);
+
+	sa->rss_hash_types = sfc_rte_to_efx_hash_type(SFC_RSS_OFFLOADS);
+
+	rte_memcpy(sa->rss_key, default_rss_key, sizeof(sa->rss_key));
+
+	return 0;
+
+fail_hash_support_get:
+fail_scale_support_get:
+fail_rx_init:
+	efx_ev_fini(sa->nic);
+
+fail_ev_init:
+	efx_intr_fini(sa->nic);
+
+fail_intr_init:
+	return rc;
+#else
+	return 0;
+#endif
+}
+
 int
 sfc_attach(struct sfc_adapter *sa)
 {
@@ -550,6 +617,10 @@ sfc_attach(struct sfc_adapter *sa)
 	efx_phy_adv_cap_get(sa->nic, EFX_PHY_CAP_PERM,
 			    &sa->port.phy_adv_cap_mask);
 
+	rc = sfc_set_rss_defaults(sa);
+	if (rc != 0)
+		goto fail_set_rss_defaults;
+
 	sfc_log_init(sa, "fini nic");
 	efx_nic_fini(enp);
 
@@ -558,7 +629,12 @@ sfc_attach(struct sfc_adapter *sa)
 	sfc_log_init(sa, "done");
 	return 0;
 
+fail_set_rss_defaults:
+	sfc_intr_detach(sa);
+
 fail_intr_attach:
+	efx_nic_fini(sa->nic);
+
 fail_estimate_rsrc_limits:
 fail_nic_reset:
 	sfc_log_init(sa, "unprobe nic");
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index 7b135e1..d02d1c0 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -42,6 +42,13 @@
 extern "C" {
 #endif
 
+#if EFSYS_OPT_RX_SCALE
+/** RSS key length (bytes) */
+#define SFC_RSS_KEY_SIZE	40
+/** RSS hash offloads mask */
+#define SFC_RSS_OFFLOADS	(ETH_RSS_IP | ETH_RSS_TCP)
+#endif
+
 /*
  * +---------------+
  * | UNINITIALIZED |<-----------+
@@ -187,6 +194,16 @@ struct sfc_adapter {
 
 	unsigned int			txq_count;
 	struct sfc_txq_info		*txq_info;
+
+	unsigned int			rss_channels;
+
+#if EFSYS_OPT_RX_SCALE
+	efx_rx_scale_support_t		rss_support;
+	efx_rx_hash_support_t		hash_support;
+	efx_rx_hash_type_t		rss_hash_types;
+	unsigned int			rss_tbl[EFX_RSS_TBL_SIZE];
+	uint8_t				rss_key[SFC_RSS_KEY_SIZE];
+#endif
 };
 
 /*
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 0de17ca..b17607f 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -85,6 +85,14 @@ sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 	else
 		dev_info->tx_offload_capa |= DEV_TX_OFFLOAD_VLAN_INSERT;
 
+#if EFSYS_OPT_RX_SCALE
+	if (sa->rss_support != EFX_RX_SCALE_UNAVAILABLE) {
+		dev_info->reta_size = EFX_RSS_TBL_SIZE;
+		dev_info->hash_key_size = SFC_RSS_KEY_SIZE;
+		dev_info->flow_type_rss_offloads = SFC_RSS_OFFLOADS;
+	}
+#endif
+
 	dev_info->rx_desc_lim.nb_max = EFX_RXQ_MAXNDESCS;
 	dev_info->rx_desc_lim.nb_min = EFX_RXQ_MINNDESCS;
 	/* The RXQ hardware requires that the descriptor count is a power
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index 3bfce1c..36a7d71 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -411,7 +411,8 @@ sfc_rx_qstart(struct sfc_adapter *sa, unsigned int sw_index)
 
 	if (sw_index == 0) {
 		rc = efx_mac_filter_default_rxq_set(sa->nic, rxq->common,
-						    B_FALSE);
+						    (sa->rss_channels > 1) ?
+						    B_TRUE : B_FALSE);
 		if (rc != 0)
 			goto fail_mac_filter_default_rxq_set;
 	}
@@ -683,6 +684,11 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 	rxq->batch_max = encp->enc_rx_batch_max;
 	rxq->prefix_size = encp->enc_rx_prefix_size;
 
+#if EFSYS_OPT_RX_SCALE
+	if (sa->hash_support == EFX_RX_HASH_AVAILABLE)
+		rxq->flags |= SFC_RXQ_RSS_HASH;
+#endif
+
 	rxq->state = SFC_RXQ_INITIALIZED;
 
 	rxq_info->rxq = rxq;
@@ -728,6 +734,56 @@ sfc_rx_qfini(struct sfc_adapter *sa, unsigned int sw_index)
 	rte_free(rxq);
 }
 
+#if EFSYS_OPT_RX_SCALE
+efx_rx_hash_type_t
+sfc_rte_to_efx_hash_type(uint64_t rss_hf)
+{
+	efx_rx_hash_type_t efx_hash_types = 0;
+
+	if ((rss_hf & (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 |
+		       ETH_RSS_NONFRAG_IPV4_OTHER)) != 0)
+		efx_hash_types |= EFX_RX_HASH_IPV4;
+
+	if ((rss_hf & ETH_RSS_NONFRAG_IPV4_TCP) != 0)
+		efx_hash_types |= EFX_RX_HASH_TCPIPV4;
+
+	if ((rss_hf & (ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6 |
+			ETH_RSS_NONFRAG_IPV6_OTHER | ETH_RSS_IPV6_EX)) != 0)
+		efx_hash_types |= EFX_RX_HASH_IPV6;
+
+	if ((rss_hf & (ETH_RSS_NONFRAG_IPV6_TCP | ETH_RSS_IPV6_TCP_EX)) != 0)
+		efx_hash_types |= EFX_RX_HASH_TCPIPV6;
+
+	return efx_hash_types;
+}
+#endif
+
+static int
+sfc_rx_rss_config(struct sfc_adapter *sa)
+{
+	int rc = 0;
+
+#if EFSYS_OPT_RX_SCALE
+	if (sa->rss_channels > 1) {
+		rc = efx_rx_scale_mode_set(sa->nic, EFX_RX_HASHALG_TOEPLITZ,
+					   sa->rss_hash_types, B_TRUE);
+		if (rc != 0)
+			goto finish;
+
+		rc = efx_rx_scale_key_set(sa->nic, sa->rss_key,
+					  sizeof(sa->rss_key));
+		if (rc != 0)
+			goto finish;
+
+		rc = efx_rx_scale_tbl_set(sa->nic, sa->rss_tbl,
+					  sizeof(sa->rss_tbl));
+	}
+
+finish:
+#endif
+	return rc;
+}
+
 int
 sfc_rx_start(struct sfc_adapter *sa)
 {
@@ -740,6 +796,10 @@ sfc_rx_start(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_rx_init;
 
+	rc = sfc_rx_rss_config(sa);
+	if (rc != 0)
+		goto fail_rss_config;
+
 	for (sw_index = 0; sw_index < sa->rxq_count; ++sw_index) {
 		if ((!sa->rxq_info[sw_index].deferred_start ||
 		     sa->rxq_info[sw_index].deferred_started)) {
@@ -755,6 +815,7 @@ sfc_rx_start(struct sfc_adapter *sa)
 	while (sw_index-- > 0)
 		sfc_rx_qstop(sa, sw_index);
 
+fail_rss_config:
 	efx_rx_fini(sa->nic);
 
 fail_rx_init:
@@ -801,6 +862,14 @@ sfc_rx_check_mode(struct sfc_adapter *sa, struct rte_eth_rxmode *rxmode)
 	case ETH_MQ_RX_NONE:
 		/* No special checks are required */
 		break;
+#if EFSYS_OPT_RX_SCALE
+	case ETH_MQ_RX_RSS:
+		if (sa->rss_support == EFX_RX_SCALE_UNAVAILABLE) {
+			sfc_err(sa, "RSS is not available");
+			rc = EINVAL;
+		}
+		break;
+#endif
 	default:
 		sfc_err(sa, "Rx multi-queue mode %u not supported",
 			rxmode->mq_mode);
@@ -876,6 +945,16 @@ sfc_rx_init(struct sfc_adapter *sa)
 			goto fail_rx_qinit_info;
 	}
 
+#if EFSYS_OPT_RX_SCALE
+	sa->rss_channels = (dev_conf->rxmode.mq_mode == ETH_MQ_RX_RSS) ?
+			   MIN(sa->rxq_count, EFX_MAXRSS) : 1;
+
+	if (sa->rss_channels > 1) {
+		for (sw_index = 0; sw_index < EFX_RSS_TBL_SIZE; ++sw_index)
+			sa->rss_tbl[sw_index] = sw_index % sa->rss_channels;
+	}
+#endif
+
 	return 0;
 
 fail_rx_qinit_info:
diff --git a/drivers/net/sfc/sfc_rx.h b/drivers/net/sfc/sfc_rx.h
index 4aa6aea..c0cb17a 100644
--- a/drivers/net/sfc/sfc_rx.h
+++ b/drivers/net/sfc/sfc_rx.h
@@ -83,6 +83,10 @@ struct sfc_rxq {
 	unsigned int		completed;
 	uint16_t		batch_max;
 	uint16_t		prefix_size;
+#if EFSYS_OPT_RX_SCALE
+	unsigned int		flags;
+#define SFC_RXQ_RSS_HASH	0x1
+#endif
 
 	/* Used on refill */
 	unsigned int		added;
@@ -146,6 +150,10 @@ unsigned int sfc_rx_qdesc_npending(struct sfc_adapter *sa,
 				   unsigned int sw_index);
 int sfc_rx_qdesc_done(struct sfc_rxq *rxq, unsigned int offset);
 
+#if EFSYS_OPT_RX_SCALE
+efx_rx_hash_type_t sfc_rte_to_efx_hash_type(uint64_t rss_hf);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
-- 
2.5.5

[PATCH v2 27/32] net/sfc: support RSS hash offload

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Extract RSS hash provided by the HW in the prefix and put it to mbuf.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 doc/guides/nics/sfc_efx.rst          |  2 ++
 drivers/net/sfc/sfc_rx.c             | 31 ++++++++++++++++++++++++++++++-
 3 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index e7a1143..debea27 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -14,6 +14,7 @@ Scattered Rx         = Y
 Promiscuous mode     = Y
 Allmulticast mode    = Y
 Multicast MAC filter = Y
+RSS hash             = Y
 Flow control         = Y
 VLAN offload         = P
 L3 checksum offload  = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index 17e81dd..bc45b17 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -71,6 +71,8 @@ SFC EFX PMD has support for:
 
 - Receive side scaling (RSS)
 
+- RSS hash
+
 - Scattered Rx DMA for packet that are larger that a single Rx descriptor
 
 - Deferred receive and transmit queue start
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index 36a7d71..9b507c3 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -185,6 +185,28 @@ sfc_rx_desc_flags_to_packet_type(const unsigned int desc_flags)
 		((desc_flags & EFX_PKT_UDP) ? RTE_PTYPE_L4_UDP : 0);
 }
 
+static void
+sfc_rx_set_rss_hash(struct sfc_rxq *rxq, unsigned int flags, struct rte_mbuf *m)
+{
+#if EFSYS_OPT_RX_SCALE
+	uint8_t *mbuf_data;
+
+
+	if ((rxq->flags & SFC_RXQ_RSS_HASH) == 0)
+		return;
+
+	mbuf_data = rte_pktmbuf_mtod(m, uint8_t *);
+
+	if (flags & (EFX_PKT_IPV4 | EFX_PKT_IPV6)) {
+		m->hash.rss = efx_pseudo_hdr_hash_get(rxq->common,
+						      EFX_RX_HASHALG_TOEPLITZ,
+						      mbuf_data);
+
+		m->ol_flags |= PKT_RX_RSS_HASH;
+	}
+#endif
+}
+
 uint16_t
 sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 {
@@ -231,7 +253,6 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 			seg_len = rxd->size - prefix_size;
 		}
 
-		m->data_off += prefix_size;
 		rte_pktmbuf_data_len(m) = seg_len;
 		rte_pktmbuf_pkt_len(m) = seg_len;
 
@@ -261,6 +282,14 @@ sfc_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 		m->ol_flags = sfc_rx_desc_flags_to_offload_flags(desc_flags);
 		m->packet_type = sfc_rx_desc_flags_to_packet_type(desc_flags);
 
+		/*
+		 * Extract RSS hash from the packet prefix and
+		 * set the corresponding field (if needed and possible)
+		 */
+		sfc_rx_set_rss_hash(rxq, desc_flags, m);
+
+		m->data_off += prefix_size;
+
 		*rx_pkts++ = m;
 		done_pkts++;
 		continue;
-- 
2.5.5

[PATCH v2 28/32] net/sfc: add callback to query RSS key and hash types config

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 33 +++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_rx.c     | 22 ++++++++++++++++++++++
 drivers/net/sfc/sfc_rx.h     |  1 +
 3 files changed, 56 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index b17607f..c78d798 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -997,6 +997,36 @@ sfc_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
 	return 0;
 }
 
+#if EFSYS_OPT_RX_SCALE
+static int
+sfc_dev_rss_hash_conf_get(struct rte_eth_dev *dev,
+			  struct rte_eth_rss_conf *rss_conf)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+
+	if ((sa->rss_channels == 1) ||
+	    (sa->rss_support != EFX_RX_SCALE_EXCLUSIVE))
+		return -ENOTSUP;
+
+	sfc_adapter_lock(sa);
+
+	/*
+	 * Mapping of hash configuration between RTE and EFX is not one-to-one,
+	 * hence, conversion is done here to derive a correct set of ETH_RSS
+	 * flags which corresponds to the active EFX configuration stored
+	 * locally in 'sfc_adapter' and kept up-to-date
+	 */
+	rss_conf->rss_hf = sfc_efx_to_rte_hash_type(sa->rss_hash_types);
+	rss_conf->rss_key_len = SFC_RSS_KEY_SIZE;
+	if (rss_conf->rss_key != NULL)
+		rte_memcpy(rss_conf->rss_key, sa->rss_key, SFC_RSS_KEY_SIZE);
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+}
+#endif
+
 static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.dev_configure			= sfc_dev_configure,
 	.dev_start			= sfc_dev_start,
@@ -1028,6 +1058,9 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.flow_ctrl_get			= sfc_flow_ctrl_get,
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
 	.mac_addr_set			= sfc_mac_addr_set,
+#if EFSYS_OPT_RX_SCALE
+	.rss_hash_conf_get		= sfc_dev_rss_hash_conf_get,
+#endif
 	.set_mc_addr_list		= sfc_set_mc_addr_list,
 	.rxq_info_get			= sfc_rx_queue_info_get,
 	.txq_info_get			= sfc_tx_queue_info_get,
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index 9b507c3..906536e 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -785,6 +785,28 @@ sfc_rte_to_efx_hash_type(uint64_t rss_hf)
 
 	return efx_hash_types;
 }
+
+uint64_t
+sfc_efx_to_rte_hash_type(efx_rx_hash_type_t efx_hash_types)
+{
+	uint64_t rss_hf = 0;
+
+	if ((efx_hash_types & EFX_RX_HASH_IPV4) != 0)
+		rss_hf |= (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 |
+			   ETH_RSS_NONFRAG_IPV4_OTHER);
+
+	if ((efx_hash_types & EFX_RX_HASH_TCPIPV4) != 0)
+		rss_hf |= ETH_RSS_NONFRAG_IPV4_TCP;
+
+	if ((efx_hash_types & EFX_RX_HASH_IPV6) != 0)
+		rss_hf |= (ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6 |
+			   ETH_RSS_NONFRAG_IPV6_OTHER | ETH_RSS_IPV6_EX);
+
+	if ((efx_hash_types & EFX_RX_HASH_TCPIPV6) != 0)
+		rss_hf |= (ETH_RSS_NONFRAG_IPV6_TCP | ETH_RSS_IPV6_TCP_EX);
+
+	return rss_hf;
+}
 #endif
 
 static int
diff --git a/drivers/net/sfc/sfc_rx.h b/drivers/net/sfc/sfc_rx.h
index c0cb17a..45b1d77 100644
--- a/drivers/net/sfc/sfc_rx.h
+++ b/drivers/net/sfc/sfc_rx.h
@@ -152,6 +152,7 @@ int sfc_rx_qdesc_done(struct sfc_rxq *rxq, unsigned int offset);
 
 #if EFSYS_OPT_RX_SCALE
 efx_rx_hash_type_t sfc_rte_to_efx_hash_type(uint64_t rss_hf);
+uint64_t sfc_efx_to_rte_hash_type(efx_rx_hash_type_t efx_hash_types);
 #endif
 
 #ifdef __cplusplus
-- 
2.5.5

[PATCH v2 29/32] net/sfc: add callback to set RSS key and hash types config

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 drivers/net/sfc/sfc_ethdev.c         | 63 ++++++++++++++++++++++++++++++++++++
 2 files changed, 64 insertions(+)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index debea27..4f6f117 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -15,6 +15,7 @@ Promiscuous mode     = Y
 Allmulticast mode    = Y
 Multicast MAC filter = Y
 RSS hash             = Y
+RSS key update       = Y
 Flow control         = Y
 VLAN offload         = P
 L3 checksum offload  = Y
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index c78d798..f9a766c 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -1025,6 +1025,68 @@ sfc_dev_rss_hash_conf_get(struct rte_eth_dev *dev,
 
 	return 0;
 }
+
+static int
+sfc_dev_rss_hash_update(struct rte_eth_dev *dev,
+			struct rte_eth_rss_conf *rss_conf)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	unsigned int efx_hash_types;
+	int rc = 0;
+
+	if ((sa->rss_channels == 1) ||
+	    (sa->rss_support != EFX_RX_SCALE_EXCLUSIVE)) {
+		sfc_err(sa, "RSS is not available");
+		return -ENOTSUP;
+	}
+
+	if ((rss_conf->rss_key != NULL) &&
+	    (rss_conf->rss_key_len != sizeof(sa->rss_key))) {
+		sfc_err(sa, "RSS key size is wrong (should be %lu)",
+			sizeof(sa->rss_key));
+		return -EINVAL;
+	}
+
+	if ((rss_conf->rss_hf & ~SFC_RSS_OFFLOADS) != 0) {
+		sfc_err(sa, "unsupported hash functions requested");
+		return -EINVAL;
+	}
+
+	sfc_adapter_lock(sa);
+
+	efx_hash_types = sfc_rte_to_efx_hash_type(rss_conf->rss_hf);
+
+	rc = efx_rx_scale_mode_set(sa->nic, EFX_RX_HASHALG_TOEPLITZ,
+				   efx_hash_types, B_TRUE);
+	if (rc != 0)
+		goto fail_scale_mode_set;
+
+	if (rss_conf->rss_key != NULL) {
+		if (sa->state == SFC_ADAPTER_STARTED) {
+			rc = efx_rx_scale_key_set(sa->nic, rss_conf->rss_key,
+						  sizeof(sa->rss_key));
+			if (rc != 0)
+				goto fail_scale_key_set;
+		}
+
+		rte_memcpy(sa->rss_key, rss_conf->rss_key, sizeof(sa->rss_key));
+	}
+
+	sa->rss_hash_types = efx_hash_types;
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+
+fail_scale_key_set:
+	if (efx_rx_scale_mode_set(sa->nic, EFX_RX_HASHALG_TOEPLITZ,
+				  sa->rss_hash_types, B_TRUE) != 0)
+		sfc_err(sa, "failed to restore RSS mode");
+
+fail_scale_mode_set:
+	sfc_adapter_unlock(sa);
+	return -rc;
+}
 #endif
 
 static const struct eth_dev_ops sfc_eth_dev_ops = {
@@ -1059,6 +1121,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
 	.mac_addr_set			= sfc_mac_addr_set,
 #if EFSYS_OPT_RX_SCALE
+	.rss_hash_update		= sfc_dev_rss_hash_update,
 	.rss_hash_conf_get		= sfc_dev_rss_hash_conf_get,
 #endif
 	.set_mc_addr_list		= sfc_set_mc_addr_list,
-- 
2.5.5

[PATCH v2 30/32] net/sfc: add callback to query RSS redirection table

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 drivers/net/sfc/sfc_ethdev.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index f9a766c..0cd96ac 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -1087,6 +1087,36 @@ sfc_dev_rss_hash_update(struct rte_eth_dev *dev,
 	sfc_adapter_unlock(sa);
 	return -rc;
 }
+
+static int
+sfc_dev_rss_reta_query(struct rte_eth_dev *dev,
+		       struct rte_eth_rss_reta_entry64 *reta_conf,
+		       uint16_t reta_size)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	int entry;
+
+	if ((sa->rss_channels == 1) ||
+	    (sa->rss_support != EFX_RX_SCALE_EXCLUSIVE))
+		return -ENOTSUP;
+
+	if (reta_size != EFX_RSS_TBL_SIZE)
+		return -EINVAL;
+
+	sfc_adapter_lock(sa);
+
+	for (entry = 0; entry < reta_size; entry++) {
+		int grp = entry / RTE_RETA_GROUP_SIZE;
+		int grp_idx = entry % RTE_RETA_GROUP_SIZE;
+
+		if ((reta_conf[grp].mask >> grp_idx) & 1)
+			reta_conf[grp].reta[grp_idx] = sa->rss_tbl[entry];
+	}
+
+	sfc_adapter_unlock(sa);
+
+	return 0;
+}
 #endif
 
 static const struct eth_dev_ops sfc_eth_dev_ops = {
@@ -1121,6 +1151,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
 	.mac_addr_set			= sfc_mac_addr_set,
 #if EFSYS_OPT_RX_SCALE
+	.reta_query			= sfc_dev_rss_reta_query,
 	.rss_hash_update		= sfc_dev_rss_hash_update,
 	.rss_hash_conf_get		= sfc_dev_rss_hash_conf_get,
 #endif
-- 
2.5.5

[PATCH v2 31/32] net/sfc: add callback to update RSS redirection table

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 doc/guides/nics/features/sfc_efx.ini |  1 +
 drivers/net/sfc/sfc_ethdev.c         | 60 ++++++++++++++++++++++++++++++++++++
 2 files changed, 61 insertions(+)

diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 4f6f117..07c58d5 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -16,6 +16,7 @@ Allmulticast mode    = Y
 Multicast MAC filter = Y
 RSS hash             = Y
 RSS key update       = Y
+RSS reta update      = Y
 Flow control         = Y
 VLAN offload         = P
 L3 checksum offload  = Y
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 0cd96ac..f45072c 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -1117,6 +1117,65 @@ sfc_dev_rss_reta_query(struct rte_eth_dev *dev,
 
 	return 0;
 }
+
+static int
+sfc_dev_rss_reta_update(struct rte_eth_dev *dev,
+			struct rte_eth_rss_reta_entry64 *reta_conf,
+			uint16_t reta_size)
+{
+	struct sfc_adapter *sa = dev->data->dev_private;
+	unsigned int *rss_tbl_new;
+	uint16_t entry;
+	int rc;
+
+
+	if ((sa->rss_channels == 1) ||
+	    (sa->rss_support != EFX_RX_SCALE_EXCLUSIVE)) {
+		sfc_err(sa, "RSS is not available");
+		return -ENOTSUP;
+	}
+
+	if (reta_size != EFX_RSS_TBL_SIZE) {
+		sfc_err(sa, "RETA size is wrong (should be %u)",
+			EFX_RSS_TBL_SIZE);
+		return -EINVAL;
+	}
+
+	rss_tbl_new = rte_zmalloc("rss_tbl_new", sizeof(sa->rss_tbl), 0);
+	if (rss_tbl_new == NULL)
+		return -ENOMEM;
+
+	sfc_adapter_lock(sa);
+
+	rte_memcpy(rss_tbl_new, sa->rss_tbl, sizeof(sa->rss_tbl));
+
+	for (entry = 0; entry < reta_size; entry++) {
+		int grp_idx = entry % RTE_RETA_GROUP_SIZE;
+		struct rte_eth_rss_reta_entry64 *grp;
+
+		grp = &reta_conf[entry / RTE_RETA_GROUP_SIZE];
+
+		if (grp->mask & (1ull << grp_idx)) {
+			if (grp->reta[grp_idx] >= sa->rss_channels) {
+				rc = EINVAL;
+				goto bad_reta_entry;
+			}
+			rss_tbl_new[entry] = grp->reta[grp_idx];
+		}
+	}
+
+	rc = efx_rx_scale_tbl_set(sa->nic, rss_tbl_new, EFX_RSS_TBL_SIZE);
+	if (rc == 0)
+		rte_memcpy(sa->rss_tbl, rss_tbl_new, sizeof(sa->rss_tbl));
+
+bad_reta_entry:
+	sfc_adapter_unlock(sa);
+
+	rte_free(rss_tbl_new);
+
+	SFC_ASSERT(rc >= 0);
+	return -rc;
+}
 #endif
 
 static const struct eth_dev_ops sfc_eth_dev_ops = {
@@ -1151,6 +1210,7 @@ static const struct eth_dev_ops sfc_eth_dev_ops = {
 	.flow_ctrl_set			= sfc_flow_ctrl_set,
 	.mac_addr_set			= sfc_mac_addr_set,
 #if EFSYS_OPT_RX_SCALE
+	.reta_update			= sfc_dev_rss_reta_update,
 	.reta_query			= sfc_dev_rss_reta_query,
 	.rss_hash_update		= sfc_dev_rss_hash_update,
 	.rss_hash_conf_get		= sfc_dev_rss_hash_conf_get,
-- 
2.5.5

[PATCH v2 32/32] net/sfc: support firmware-assisted TSOv2

[Andrew Rybchenko]

From: Ivan Malov <ivan.malov@oktetlabs.ru>

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andrew Lee <alee@solarflare.com>
Reviewed-by: Mark Spender <mspender@solarflare.com>
Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
---
 config/common_base                   |   1 +
 doc/guides/nics/features/sfc_efx.ini |   1 +
 doc/guides/nics/sfc_efx.rst          |   8 ++
 drivers/net/sfc/Makefile             |   4 +
 drivers/net/sfc/sfc.c                |   8 ++
 drivers/net/sfc/sfc.h                |   2 +
 drivers/net/sfc/sfc_ethdev.c         |   3 +
 drivers/net/sfc/sfc_tso.c            | 200 +++++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_tx.c             |  89 +++++++++++++++-
 drivers/net/sfc/sfc_tx.h             |  28 +++++
 10 files changed, 341 insertions(+), 3 deletions(-)
 create mode 100644 drivers/net/sfc/sfc_tso.c

diff --git a/config/common_base b/config/common_base
index 59cb830..faee944 100644
--- a/config/common_base
+++ b/config/common_base
@@ -343,6 +343,7 @@ CONFIG_RTE_LIBRTE_PMD_XENVIRT=n
 #
 CONFIG_RTE_LIBRTE_SFC_EFX_PMD=y
 CONFIG_RTE_LIBRTE_SFC_EFX_DEBUG=n
+CONFIG_RTE_LIBRTE_SFC_EFX_TSO=n
 
 #
 # Compile null PMD
diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini
index 07c58d5..3a15baa 100644
--- a/doc/guides/nics/features/sfc_efx.ini
+++ b/doc/guides/nics/features/sfc_efx.ini
@@ -11,6 +11,7 @@ Queue start/stop     = Y
 MTU update           = Y
 Jumbo frame          = Y
 Scattered Rx         = Y
+TSO                  = Y
 Promiscuous mode     = Y
 Allmulticast mode    = Y
 Multicast MAC filter = Y
diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst
index bc45b17..6be4fba 100644
--- a/doc/guides/nics/sfc_efx.rst
+++ b/doc/guides/nics/sfc_efx.rst
@@ -63,6 +63,8 @@ SFC EFX PMD has support for:
 
 - Allmulticast mode
 
+- TCP segmentation offload (TSO)
+
 - Multicast MAC filter
 
 - IPv4/IPv6 TCP/UDP receive checksum offload
@@ -169,6 +171,12 @@ Please note that enabling debugging options may affect system performance.
 
   Enable compilation of the extra run-time consistency checks.
 
+- ``CONFIG_RTE_LIBRTE_SFC_EFX_TSO`` (default **n**)
+
+  Toggle TCP segmentation offload support.
+  Enabling the feature limits the number of available transmit queues
+  significantly due to the limited number of adapter TSO contexts.
+
 
 Per-Device Parameters
 ~~~~~~~~~~~~~~~~~~~~~
diff --git a/drivers/net/sfc/Makefile b/drivers/net/sfc/Makefile
index dd099b2..14d6536 100644
--- a/drivers/net/sfc/Makefile
+++ b/drivers/net/sfc/Makefile
@@ -90,6 +90,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += sfc_port.c
 SRCS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += sfc_rx.c
 SRCS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += sfc_tx.c
 
+SRCS-$(CONFIG_RTE_LIBRTE_SFC_EFX_TSO) += sfc_tso.c
+
 VPATH += $(SRCDIR)/base
 
 SRCS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += efx_bootcfg.c
@@ -139,4 +141,6 @@ DEPDIRS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += lib/librte_mempool
 DEPDIRS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += lib/librte_mbuf
 
+DEPDIRS-$(CONFIG_RTE_LIBRTE_SFC_EFX_TSO) += lib/librte_net
+
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/sfc/sfc.c b/drivers/net/sfc/sfc.c
index e79367d..02a56f7 100644
--- a/drivers/net/sfc/sfc.c
+++ b/drivers/net/sfc/sfc.c
@@ -621,6 +621,14 @@ sfc_attach(struct sfc_adapter *sa)
 	if (rc != 0)
 		goto fail_set_rss_defaults;
 
+#ifdef RTE_LIBRTE_SFC_EFX_TSO
+	sa->tso = efx_nic_cfg_get(sa->nic)->enc_fw_assisted_tso_v2_enabled;
+	if (!sa->tso)
+		sfc_warn(sa, "TSO support isn't available on this adapter");
+#else /* !RTE_LIBRTE_SFC_EFX_TSO */
+	sa->tso = B_FALSE;
+#endif /* RTE_LIBRTE_SFC_EFX_TSO */
+
 	sfc_log_init(sa, "fini nic");
 	efx_nic_fini(enp);
 
diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h
index d02d1c0..6716acd 100644
--- a/drivers/net/sfc/sfc.h
+++ b/drivers/net/sfc/sfc.h
@@ -195,6 +195,8 @@ struct sfc_adapter {
 	unsigned int			txq_count;
 	struct sfc_txq_info		*txq_info;
 
+	boolean_t			tso;
+
 	unsigned int			rss_channels;
 
 #if EFSYS_OPT_RX_SCALE
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index f45072c..dd5ca5c 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -93,6 +93,9 @@ sfc_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 	}
 #endif
 
+	if (sa->tso)
+		dev_info->tx_offload_capa |= DEV_TX_OFFLOAD_TCP_TSO;
+
 	dev_info->rx_desc_lim.nb_max = EFX_RXQ_MAXNDESCS;
 	dev_info->rx_desc_lim.nb_min = EFX_RXQ_MINNDESCS;
 	/* The RXQ hardware requires that the descriptor count is a power
diff --git a/drivers/net/sfc/sfc_tso.c b/drivers/net/sfc/sfc_tso.c
new file mode 100644
index 0000000..68d84c9
--- /dev/null
+++ b/drivers/net/sfc/sfc_tso.c
@@ -0,0 +1,200 @@
+/*-
+ * Copyright (c) 2016 Solarflare Communications Inc.
+ * All rights reserved.
+ *
+ * This software was jointly developed between OKTET Labs (under contract
+ * for Solarflare) and Solarflare Communications, Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright notice,
+ *    this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright notice,
+ *    this list of conditions and the following disclaimer in the documentation
+ *    and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+ * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+ * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
+ * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ip.h>
+#include <rte_tcp.h>
+
+#include "sfc.h"
+#include "sfc_debug.h"
+#include "sfc_tx.h"
+#include "sfc_ev.h"
+
+/** Standard TSO header length */
+#define SFC_TSOH_STD_LEN        256
+
+/** The number of TSO option descriptors that precede the packet descriptors */
+#define SFC_TSO_OPDESCS_IDX_SHIFT	2
+
+int
+sfc_tso_alloc_tsoh_objs(struct sfc_tx_sw_desc *sw_ring,
+			unsigned int txq_entries, unsigned int socket_id)
+{
+	unsigned int i;
+
+	for (i = 0; i < txq_entries; ++i) {
+		sw_ring[i].tsoh = rte_malloc_socket("sfc-txq-tsoh-obj",
+						    SFC_TSOH_STD_LEN,
+						    SFC_TX_SEG_BOUNDARY,
+						    socket_id);
+		if (sw_ring[i].tsoh == NULL)
+			goto fail_alloc_tsoh_objs;
+	}
+
+	return 0;
+
+fail_alloc_tsoh_objs:
+	while (i > 0)
+		rte_free(sw_ring[--i].tsoh);
+
+	return ENOMEM;
+}
+
+void
+sfc_tso_free_tsoh_objs(struct sfc_tx_sw_desc *sw_ring, unsigned int txq_entries)
+{
+	unsigned int i;
+
+	for (i = 0; i < txq_entries; ++i) {
+		rte_free(sw_ring[i].tsoh);
+		sw_ring[i].tsoh = NULL;
+	}
+}
+
+static void
+sfc_tso_prepare_header(struct sfc_txq *txq, struct rte_mbuf **in_seg,
+		       size_t *in_off, unsigned int idx, size_t bytes_left)
+{
+	struct rte_mbuf *m = *in_seg;
+	size_t bytes_to_copy = 0;
+	uint8_t *tsoh = txq->sw_ring[idx & txq->ptr_mask].tsoh;
+
+	do {
+		bytes_to_copy = MIN(bytes_left, m->data_len);
+
+		rte_memcpy(tsoh, rte_pktmbuf_mtod(m, uint8_t *),
+			   bytes_to_copy);
+
+		bytes_left -= bytes_to_copy;
+		tsoh += bytes_to_copy;
+
+		if (bytes_left > 0) {
+			m = m->next;
+			SFC_ASSERT(m != NULL);
+		}
+	} while (bytes_left > 0);
+
+	if (bytes_to_copy == m->data_len) {
+		*in_seg = m->next;
+		*in_off = 0;
+	} else {
+		*in_seg = m;
+		*in_off = bytes_to_copy;
+	}
+}
+
+int
+sfc_tso_do(struct sfc_txq *txq, unsigned int idx, struct rte_mbuf **in_seg,
+	   size_t *in_off, efx_desc_t **pend, unsigned int *pkt_descs,
+	   size_t *pkt_len)
+{
+	uint8_t *tsoh;
+	const struct tcp_hdr *th;
+	efsys_dma_addr_t header_paddr;
+	efsys_dma_addr_t paddr_next_frag;
+	uint16_t packet_id;
+	uint32_t sent_seq;
+	struct rte_mbuf *m = *in_seg;
+	size_t nh_off = m->l2_len; /* IP header offset */
+	size_t tcph_off = m->l2_len + m->l3_len; /* TCP header offset */
+	size_t header_len = m->l2_len + m->l3_len + m->l4_len;
+	const efx_nic_cfg_t *encp = efx_nic_cfg_get(txq->evq->sa->nic);
+
+	idx += SFC_TSO_OPDESCS_IDX_SHIFT;
+
+	/* Packets which have too big headers should be discarded */
+	if (unlikely(header_len > SFC_TSOH_STD_LEN))
+		return EMSGSIZE;
+
+	/*
+	 * The TCP header must start at most 208 bytes into the frame.
+	 * If it starts later than this then the NIC won't realise
+	 * it's a TCP packet and TSO edits won't be applied
+	 */
+	if (unlikely(tcph_off > encp->enc_tx_tso_tcp_header_offset_limit))
+		return EMSGSIZE;
+
+	header_paddr = rte_pktmbuf_mtophys(m);
+	paddr_next_frag = P2ROUNDUP(header_paddr + 1, SFC_TX_SEG_BOUNDARY);
+
+	/*
+	 * Sometimes headers may be split across multiple mbufs. In such cases
+	 * we need to glue those pieces and store them in some temporary place.
+	 * Also, packet headers must be contiguous in memory, so that
+	 * they can be referred to with a single DMA descriptor. Hence, handle
+	 * the case where the original header crosses a 4K memory boundary
+	 */
+	if ((m->data_len < header_len) ||
+	    ((paddr_next_frag - header_paddr) < header_len)) {
+		sfc_tso_prepare_header(txq, in_seg, in_off, idx, header_len);
+		tsoh = txq->sw_ring[idx & txq->ptr_mask].tsoh;
+
+		header_paddr = rte_malloc_virt2phy((void *)tsoh);
+	} else {
+		if (m->data_len == header_len) {
+			*in_off = 0;
+			*in_seg = m->next;
+		} else {
+			*in_off = header_len;
+		}
+
+		tsoh = rte_pktmbuf_mtod(m, uint8_t *);
+	}
+
+	/* Handle IP header */
+	if (m->ol_flags & PKT_TX_IPV4) {
+		const struct ipv4_hdr *iphe4;
+
+		iphe4 = (const struct ipv4_hdr *)(tsoh + nh_off);
+		rte_memcpy(&packet_id, &iphe4->packet_id, sizeof(uint16_t));
+		packet_id = rte_be_to_cpu_16(packet_id);
+	} else if (m->ol_flags & PKT_TX_IPV6) {
+		packet_id = 0;
+	} else {
+		return EINVAL;
+	}
+
+	/* Handle TCP header */
+	th = (const struct tcp_hdr *)(tsoh + tcph_off);
+
+	rte_memcpy(&sent_seq, &th->sent_seq, sizeof(uint32_t));
+	sent_seq = rte_be_to_cpu_32(sent_seq);
+
+	efx_tx_qdesc_tso2_create(txq->common, packet_id, sent_seq, m->tso_segsz,
+				 *pend, EFX_TX_FATSOV2_OPT_NDESCS);
+
+	*pend += EFX_TX_FATSOV2_OPT_NDESCS;
+	*pkt_descs += EFX_TX_FATSOV2_OPT_NDESCS;
+
+	efx_tx_qdesc_dma_create(txq->common, header_paddr, header_len,
+				B_FALSE, (*pend)++);
+	(*pkt_descs)++;
+	*pkt_len -= header_len;
+
+	return 0;
+}
diff --git a/drivers/net/sfc/sfc_tx.c b/drivers/net/sfc/sfc_tx.c
index 86bcfec..3e64c0f 100644
--- a/drivers/net/sfc/sfc_tx.c
+++ b/drivers/net/sfc/sfc_tx.c
@@ -184,6 +184,13 @@ sfc_tx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 	if (txq->sw_ring == NULL)
 		goto fail_desc_alloc;
 
+	if (sa->tso) {
+		rc = sfc_tso_alloc_tsoh_objs(txq->sw_ring, txq_info->entries,
+					     socket_id);
+		if (rc != 0)
+			goto fail_alloc_tsoh_objs;
+	}
+
 	txq->state = SFC_TXQ_INITIALIZED;
 	txq->ptr_mask = txq_info->entries - 1;
 	txq->free_thresh = (tx_conf->tx_free_thresh) ? tx_conf->tx_free_thresh :
@@ -199,6 +206,9 @@ sfc_tx_qinit(struct sfc_adapter *sa, unsigned int sw_index,
 
 	return 0;
 
+fail_alloc_tsoh_objs:
+	rte_free(txq->sw_ring);
+
 fail_desc_alloc:
 	rte_free(txq->pend_desc);
 
@@ -234,6 +244,8 @@ sfc_tx_qfini(struct sfc_adapter *sa, unsigned int sw_index)
 	SFC_ASSERT(txq != NULL);
 	SFC_ASSERT(txq->state == SFC_TXQ_INITIALIZED);
 
+	sfc_tso_free_tsoh_objs(txq->sw_ring, txq_info->entries);
+
 	txq_info->txq = NULL;
 	txq_info->entries = 0;
 
@@ -300,6 +312,11 @@ sfc_tx_init(struct sfc_adapter *sa)
 
 	sa->txq_count = sa->eth_dev->data->nb_tx_queues;
 
+	if (sa->tso)
+		sa->txq_count = MIN(sa->txq_count,
+		   efx_nic_cfg_get(sa->nic)->enc_fw_assisted_tso_v2_n_contexts /
+		   efx_nic_cfg_get(sa->nic)->enc_hw_pf_count);
+
 	sa->txq_info = rte_calloc_socket("sfc-txqs", sa->txq_count,
 					 sizeof(sa->txq_info[0]), 0,
 					 sa->socket_id);
@@ -373,17 +390,25 @@ sfc_tx_qstart(struct sfc_adapter *sa, unsigned int sw_index)
 	 * hence, we always enable it here
 	 */
 	if ((txq->flags & ETH_TXQ_FLAGS_NOXSUMTCP) ||
-	    (txq->flags & ETH_TXQ_FLAGS_NOXSUMUDP))
+	    (txq->flags & ETH_TXQ_FLAGS_NOXSUMUDP)) {
 		flags = EFX_TXQ_CKSUM_IPV4;
-	else
+	} else {
 		flags = EFX_TXQ_CKSUM_IPV4 | EFX_TXQ_CKSUM_TCPUDP;
 
+		if (sa->tso)
+			flags |= EFX_TXQ_FATSOV2;
+	}
+
 	rc = efx_tx_qcreate(sa->nic, sw_index, 0, &txq->mem,
 			    txq_info->entries, 0 /* not used on EF10 */,
 			    flags, evq->common,
 			    &txq->common, &desc_index);
-	if (rc != 0)
+	if (rc != 0) {
+		if (sa->tso && (rc == ENOSPC))
+			sfc_err(sa, "ran out of TSO contexts");
+
 		goto fail_tx_qcreate;
+	}
 
 	txq->added = txq->pending = txq->completed = desc_index;
 	txq->hw_vlan_tci = 0;
@@ -494,6 +519,13 @@ sfc_tx_start(struct sfc_adapter *sa)
 
 	sfc_log_init(sa, "txq_count = %u", sa->txq_count);
 
+	if (sa->tso) {
+		if (!efx_nic_cfg_get(sa->nic)->enc_fw_assisted_tso_v2_enabled) {
+			sfc_warn(sa, "TSO support was unable to be restored");
+			sa->tso = B_FALSE;
+		}
+	}
+
 	rc = efx_tx_init(sa->nic);
 	if (rc != 0)
 		goto fail_efx_tx_init;
@@ -607,6 +639,7 @@ sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		struct rte_mbuf		*m_seg = *pktp;
 		size_t			pkt_len = m_seg->pkt_len;
 		unsigned int		pkt_descs = 0;
+		size_t			in_off = 0;
 
 		/*
 		 * Here VLAN TCI is expected to be zero in case if no
@@ -617,6 +650,46 @@ sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		 */
 		pkt_descs += sfc_tx_maybe_insert_tag(txq, m_seg, &pend);
 
+#ifdef RTE_LIBRTE_SFC_EFX_TSO
+		if (m_seg->ol_flags & PKT_TX_TCP_SEG) {
+			/*
+			 * We expect correct 'pkt->l[2, 3, 4]_len' values
+			 * to be set correctly by the caller
+			 */
+			if (sfc_tso_do(txq, added, &m_seg, &in_off, &pend,
+				       &pkt_descs, &pkt_len) != 0) {
+				/* We may have reached this place for
+				 * one of the following reasons:
+				 *
+				 * 1) Packet header length is greater
+				 *    than SFC_TSOH_STD_LEN
+				 * 2) TCP header starts at more then
+				 *    208 bytes into the frame
+				 *
+				 * We will deceive RTE saying that we have sent
+				 * the packet, but we will actually drop it.
+				 * Hence, we should revert 'pend' to the
+				 * previous state (in case we have added
+				 * VLAN descriptor) and start processing
+				 * another one packet. But the original
+				 * mbuf shouldn't be orphaned
+				 */
+				pend -= pkt_descs;
+
+				rte_pktmbuf_free(*pktp);
+
+				continue;
+			}
+
+			/*
+			 * We've only added 2 FATSOv2 option descriptors
+			 * and 1 descriptor for the linearized packet header.
+			 * The outstanding work will be done in the same manner
+			 * as for the usual non-TSO path
+			 */
+		}
+#endif /* RTE_LIBRTE_SFC_EFX_TSO */
+
 		for (; m_seg != NULL; m_seg = m_seg->next) {
 			efsys_dma_addr_t	next_frag;
 			size_t			seg_len;
@@ -624,6 +697,16 @@ sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 			seg_len = m_seg->data_len;
 			next_frag = rte_mbuf_data_dma_addr(m_seg);
 
+			/*
+			 * If we've started TSO transaction few steps earlier,
+			 * we'll skip packet header using an offset in the
+			 * current segment (which has been set to the
+			 * first one containing payload)
+			 */
+			seg_len -= in_off;
+			next_frag += in_off;
+			in_off = 0;
+
 			do {
 				efsys_dma_addr_t	frag_addr = next_frag;
 				size_t			frag_len;
diff --git a/drivers/net/sfc/sfc_tx.h b/drivers/net/sfc/sfc_tx.h
index 4d25c6a..581e2aa 100644
--- a/drivers/net/sfc/sfc_tx.h
+++ b/drivers/net/sfc/sfc_tx.h
@@ -50,6 +50,9 @@ struct sfc_evq;
 
 struct sfc_tx_sw_desc {
 	struct rte_mbuf		*mbuf;
+#ifdef RTE_LIBRTE_SFC_EFX_TSO
+	uint8_t			*tsoh;	/* Buffer to store TSO header */
+#endif /* RTE_LIBRTE_SFC_EFX_TSO */
 };
 
 enum sfc_txq_state_bit {
@@ -113,6 +116,31 @@ void sfc_tx_stop(struct sfc_adapter *sa);
 uint16_t sfc_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 		       uint16_t nb_pkts);
 
+#ifdef RTE_LIBRTE_SFC_EFX_TSO
+/* From 'sfc_tso.c' */
+int sfc_tso_alloc_tsoh_objs(struct sfc_tx_sw_desc *sw_ring,
+			    unsigned int txq_entries, unsigned int socket_id);
+void sfc_tso_free_tsoh_objs(struct sfc_tx_sw_desc *sw_ring,
+			    unsigned int txq_entries);
+int sfc_tso_do(struct sfc_txq *txq, unsigned int idx, struct rte_mbuf **in_seg,
+	       size_t *in_off, efx_desc_t **pend, unsigned int *pkt_descs,
+	       size_t *pkt_len);
+#else /* !RTE_LIBRTE_SFC_EFX_TSO */
+static inline int
+sfc_tso_alloc_tsoh_objs(__rte_unused struct sfc_tx_sw_desc *sw_ring,
+			__rte_unused unsigned int txq_entries,
+			__rte_unused unsigned int socket_id)
+{
+	return 0;
+}
+
+static inline void
+sfc_tso_free_tsoh_objs(__rte_unused struct sfc_tx_sw_desc *sw_ring,
+		       __rte_unused unsigned int txq_entries)
+{
+}
+#endif /* RTE_LIBRTE_SFC_EFX_TSO */
+
 #ifdef __cplusplus
 }
 #endif
-- 
2.5.5

Re: [PATCH v2 00/32] Support more features in Solarflare PMD

[Ferruh Yigit]

On 12/15/2016 12:50 PM, Andrew Rybchenko wrote:
> The patch series adds a number of features to Solarflare libefx-based
> PMD. Basically one patch per feature.
> 
> The patches are grouped into one series since they touch nearby lines
> in either PMD feature list, or dev_ops structure, or documentation.
> So, patches cannot be applied in arbitrary order.
> 
> ---
> 
> v2:
> * Fix ICC and clang warnings
> * Slightly change sfc_tso_{alloc,free}_tsoh_objs() prototypes
> 
> 
> Andrew Rybchenko (17):
>   net/sfc: implement MCDI logging callback
>   net/sfc: support parameter to choose performance profile
>   net/sfc: implement ethdev hook to get basic statistics
>   net/sfc: support extended statistics
>   net/sfc: support flow control settings get/set
>   net/sfc: support link status change interrupt
>   net/sfc: implement device operation to change MTU
>   net/sfc: support link speed and duplex settings
>   net/sfc: support checksum offloads on receive
>   net/sfc: handle received packet type info provided by HW
>   net/sfc: support callback to get receive queue information
>   net/sfc: support Rx free threshold
>   net/sfc: add callback to get RxQ pending descriptors count
>   net/sfc: add RxQ descriptor done callback
>   net/sfc: support scattered Rx DMA
>   net/sfc: support deferred start of receive queues
>   net/sfc/base: do not use enum type when values are bitmask
> 
> Artem Andreev (1):
>   net/sfc: support link up/down
> 
> Ivan Malov (14):
>   net/sfc: support promiscuous and all-multicast control
>   net/sfc: support main (the first) MAC address change
>   net/sfc: support multicast addresses list controls
>   net/sfc: add callback to get transmit queue information
>   net/sfc: support Tx free threshold
>   net/sfc: support deferred start of transmit queues
>   net/sfc: support VLAN offload on transmit path
>   net/sfc: add basic stubs for RSS support on driver attach
>   net/sfc: support RSS hash offload
>   net/sfc: add callback to query RSS key and hash types config
>   net/sfc: add callback to set RSS key and hash types config
>   net/sfc: add callback to query RSS redirection table
>   net/sfc: add callback to update RSS redirection table
>   net/sfc: support firmware-assisted TSOv2
> 

<...>

Series applied to dpdk-next-net/master, thanks.

Re: [PATCH v2 25/32] net/sfc/base: do not use enum type when values are bitmask

[Ferruh Yigit]

On 12/15/2016 12:51 PM, Andrew Rybchenko wrote:
> From: Andrew Rybchenko <Andrew.Rybchenko@oktetlabs.ru>

Hi Andrew,

For this patch "Signed-off" and "From" mail addresses are different.
Since both are same person and you, is there any problem using "Andrew
Rybchenko <arybchenko@solarflare.com>" as author of the patch?

Thanks,
ferruh

> 
> ICC complains that enumerated type mixed with another type.
> 
> Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
> Reviewed-by: Andrew Lee <alee@solarflare.com>
> Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
> ---
<...>

Re: [PATCH v2 25/32] net/sfc/base: do not use enum type when values are bitmask

[Andrew Rybchenko]

On 01/18/2017 02:30 PM, Ferruh Yigit wrote:
> On 12/15/2016 12:51 PM, Andrew Rybchenko wrote:
>> From: Andrew Rybchenko <Andrew.Rybchenko@oktetlabs.ru>
> Hi Andrew,
>
> For this patch "Signed-off" and "From" mail addresses are different.
> Since both are same person and you, is there any problem using "Andrew
> Rybchenko <arybchenko@solarflare.com>" as author of the patch?

Yes, please, use "Andrew Rybchenko <arybchenko@solarflare.com>"
It was my mistake in configs.

Thanks,
Andrew.
> Thanks,
> ferruh
>
>> ICC complains that enumerated type mixed with another type.
>>
>> Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
>> Reviewed-by: Andrew Lee <alee@solarflare.com>
>> Reviewed-by: Robert Stonehouse <rstonehouse@solarflare.com>
>> ---
> <...>