[PATCH 00/50] LiquidIO PMD

[PATCH 00/50] LiquidIO PMD

[Shijith Thotton]

The patch series provides initial version of virtual function poll mode
driver for Cavium LiquidIO II server adapters. This version adds support
for LiquidIO II CN2350 210SV adapter.

Patch series includes driver documentation doc/guides/nics/liquidio.rst
and list of supported features doc/guides/nics/features/liquidio.ini.
Updated release notes to notify the addition of new PMD.

Shijith Thotton (50):
  net/liquidio/base: hardware register definitions
  config: liquidio PMD configuration
  net/liquidio: added PMD version map file
  net/liquidio: definitions for log
  maintainers: claim responsibility for LiquidIO PMD
  net/liquidio: liquidio VF PMD Driver registration
  net/liquidio: added Makefile
  net/liquidio/base: macros to read and write register
  net/liquidio: liquidio device init
  net/liquidio: add API to disable io queues
  net/liquidio: add API to setup io queue registers
  net/liquidio: add mbox APIs for PF/VF communication
  net/liquidio: add API to setup mbox registers
  net/liquidio: add API for VF/PF handshake
  net/liquidio: add API for VF FLR
  net/liquidio: add APIs to allocate and free IQ
  net/liquidio: add API to setup instruction queue
  net/liquidio: add API to allocate and free command pool
  net/liquidio: add API to allocate and free soft command
  net/liquidio: add APIs for response list
  net/liquidio: add APIs to send packet to device
  net/liquidio: add API to configure device
  net/liquidio: add API to setup Rx queue
  net/liquidio: initialize Rx queue
  net/liquidio: add Rx data path
  net/liquidio: add API to release Rx queue
  net/liquidio: add API to setup Tx queue
  net/liquidio: add APIs for sg list
  net/liquidio: add API to enable and disable IO queues
  net/liquidio: add Tx data path for single segment
  net/liquidio: add Tx data path for multiple segments
  net/liquidio: add APIs to flush IQ and free buffers
  net/liquidio: add API to release Tx queue
  net/liquidio: add API to start device and check link
  net/liquidio: add API for link update
  net/liquidio: add API to alloc and send command
  net/liquidio: add API to control Rx
  net/liquidio: add RSS support
  net/liquidio: add API to get device info
  net/liquidio: add API to set MTU
  net/liquidio: add API to enable and disable multicast
  net/liquidio: add API to set link up and down
  net/liquidio: add API to configure udp tunnel port
  net/liquidio: add support for Rx stats
  net/liquidio: add support for Tx stats
  net/liquidio: add APIs for hardware stats
  net/liquidio: add API for dev stop
  net/liquidio: add API for dev close
  net/liquidio: add API to add and remove VLAN port
  doc: added documents

 MAINTAINERS                                  |    7 +
 config/common_base                           |   11 +
 doc/guides/nics/features/liquidio.ini        |   29 +
 doc/guides/nics/index.rst                    |    1 +
 doc/guides/nics/liquidio.rst                 |  269 ++++
 doc/guides/rel_notes/release_17_05.rst       |    3 +
 drivers/net/Makefile                         |    1 +
 drivers/net/liquidio/Makefile                |   62 +
 drivers/net/liquidio/base/lio_23xx_reg.h     |  194 +++
 drivers/net/liquidio/base/lio_23xx_vf.c      |  586 ++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h      |   97 ++
 drivers/net/liquidio/base/lio_hw_defs.h      |  249 ++++
 drivers/net/liquidio/base/lio_mbox.c         |  275 ++++
 drivers/net/liquidio/base/lio_mbox.h         |  131 ++
 drivers/net/liquidio/lio_ethdev.c            | 2040 ++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h            |  204 +++
 drivers/net/liquidio/lio_logs.h              |   91 ++
 drivers/net/liquidio/lio_rxtx.c              | 1885 ++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h              |  769 ++++++++++
 drivers/net/liquidio/lio_struct.h            |  689 +++++++++
 drivers/net/liquidio/rte_pmd_lio_version.map |    4 +
 mk/rte.app.mk                                |    1 +
 22 files changed, 7598 insertions(+)
 create mode 100644 doc/guides/nics/features/liquidio.ini
 create mode 100644 doc/guides/nics/liquidio.rst
 create mode 100644 drivers/net/liquidio/Makefile
 create mode 100644 drivers/net/liquidio/base/lio_23xx_reg.h
 create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.c
 create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.h
 create mode 100644 drivers/net/liquidio/base/lio_hw_defs.h
 create mode 100644 drivers/net/liquidio/base/lio_mbox.c
 create mode 100644 drivers/net/liquidio/base/lio_mbox.h
 create mode 100644 drivers/net/liquidio/lio_ethdev.c
 create mode 100644 drivers/net/liquidio/lio_ethdev.h
 create mode 100644 drivers/net/liquidio/lio_logs.h
 create mode 100644 drivers/net/liquidio/lio_rxtx.c
 create mode 100644 drivers/net/liquidio/lio_rxtx.h
 create mode 100644 drivers/net/liquidio/lio_struct.h
 create mode 100644 drivers/net/liquidio/rte_pmd_lio_version.map

-- 
1.8.3.1

[PATCH 01/50] net/liquidio/base: hardware register definitions

[Shijith Thotton]

Add HW register definitions for LiquidIO II CN23XX adapter.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_reg.h | 194 +++++++++++++++++++++++++++++++
 1 file changed, 194 insertions(+)
 create mode 100644 drivers/net/liquidio/base/lio_23xx_reg.h

diff --git a/drivers/net/liquidio/base/lio_23xx_reg.h b/drivers/net/liquidio/base/lio_23xx_reg.h
new file mode 100644
index 0000000..794bc2c
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_23xx_reg.h
@@ -0,0 +1,194 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_23XX_REG_H_
+#define _LIO_23XX_REG_H_
+
+/* ###################### REQUEST QUEUE ######################### */
+
+/* 64 registers for Input Queues Start Addr - SLI_PKT(0..63)_INSTR_BADDR */
+#define CN23XX_SLI_PKT_INSTR_BADDR_START64	0x10010
+
+/* 64 registers for Input Doorbell - SLI_PKT(0..63)_INSTR_BAOFF_DBELL */
+#define CN23XX_SLI_PKT_INSTR_BADDR_DBELL_START	0x10020
+
+/* 64 registers for Input Queue size - SLI_PKT(0..63)_INSTR_FIFO_RSIZE */
+#define CN23XX_SLI_PKT_INSTR_FIFO_RSIZE_START	0x10030
+
+/* 64 registers for Input Queue Instr Count - SLI_PKT_IN_DONE(0..63)_CNTS */
+#define CN23XX_SLI_PKT_IN_DONE_CNTS_START64	0x10040
+
+/* 64 registers (64-bit) - ES, RO, NS, Arbitration for Input Queue Data &
+ * gather list fetches. SLI_PKT(0..63)_INPUT_CONTROL.
+ */
+#define CN23XX_SLI_PKT_INPUT_CONTROL_START64	0x10000
+
+/* ------- Request Queue Macros --------- */
+
+/* Each Input Queue register is at a 16-byte Offset in BAR0 */
+#define CN23XX_IQ_OFFSET			0x20000
+
+#define CN23XX_SLI_IQ_PKT_CONTROL64(iq)					\
+	(CN23XX_SLI_PKT_INPUT_CONTROL_START64 + ((iq) * CN23XX_IQ_OFFSET))
+
+#define CN23XX_SLI_IQ_BASE_ADDR64(iq)					\
+	(CN23XX_SLI_PKT_INSTR_BADDR_START64 + ((iq) * CN23XX_IQ_OFFSET))
+
+#define CN23XX_SLI_IQ_SIZE(iq)						\
+	(CN23XX_SLI_PKT_INSTR_FIFO_RSIZE_START + ((iq) * CN23XX_IQ_OFFSET))
+
+#define CN23XX_SLI_IQ_DOORBELL(iq)					\
+	(CN23XX_SLI_PKT_INSTR_BADDR_DBELL_START + ((iq) * CN23XX_IQ_OFFSET))
+
+#define CN23XX_SLI_IQ_INSTR_COUNT64(iq)					\
+	(CN23XX_SLI_PKT_IN_DONE_CNTS_START64 + ((iq) * CN23XX_IQ_OFFSET))
+
+/* Number of instructions to be read in one MAC read request.
+ * setting to Max value(4)
+ */
+#define CN23XX_PKT_INPUT_CTL_RDSIZE			(3 << 25)
+#define CN23XX_PKT_INPUT_CTL_IS_64B			(1 << 24)
+#define CN23XX_PKT_INPUT_CTL_RST			(1 << 23)
+#define CN23XX_PKT_INPUT_CTL_QUIET			(1 << 28)
+#define CN23XX_PKT_INPUT_CTL_RING_ENB			(1 << 22)
+#define CN23XX_PKT_INPUT_CTL_DATA_ES_64B_SWAP		(1 << 6)
+#define CN23XX_PKT_INPUT_CTL_USE_CSR			(1 << 4)
+#define CN23XX_PKT_INPUT_CTL_GATHER_ES_64B_SWAP		(2)
+
+/* These bits[47:44] select the Physical function number within the MAC */
+#define CN23XX_PKT_INPUT_CTL_PF_NUM_POS		45
+/* These bits[43:32] select the function number within the PF */
+#define CN23XX_PKT_INPUT_CTL_VF_NUM_POS		32
+
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+#define CN23XX_PKT_INPUT_CTL_MASK			\
+	(CN23XX_PKT_INPUT_CTL_RDSIZE |			\
+	 CN23XX_PKT_INPUT_CTL_DATA_ES_64B_SWAP |	\
+	 CN23XX_PKT_INPUT_CTL_USE_CSR)
+#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+#define CN23XX_PKT_INPUT_CTL_MASK			\
+	(CN23XX_PKT_INPUT_CTL_RDSIZE |			\
+	 CN23XX_PKT_INPUT_CTL_DATA_ES_64B_SWAP |	\
+	 CN23XX_PKT_INPUT_CTL_USE_CSR |			\
+	 CN23XX_PKT_INPUT_CTL_GATHER_ES_64B_SWAP)
+#endif
+
+/* ############################ OUTPUT QUEUE ######################### */
+
+/* 64 registers for Output queue control - SLI_PKT(0..63)_OUTPUT_CONTROL */
+#define CN23XX_SLI_PKT_OUTPUT_CONTROL_START	0x10050
+
+/* 64 registers for Output queue buffer and info size
+ * SLI_PKT(0..63)_OUT_SIZE
+ */
+#define CN23XX_SLI_PKT_OUT_SIZE			0x10060
+
+/* 64 registers for Output Queue Start Addr - SLI_PKT(0..63)_SLIST_BADDR */
+#define CN23XX_SLI_SLIST_BADDR_START64		0x10070
+
+/* 64 registers for Output Queue Packet Credits
+ * SLI_PKT(0..63)_SLIST_BAOFF_DBELL
+ */
+#define CN23XX_SLI_PKT_SLIST_BAOFF_DBELL_START	0x10080
+
+/* 64 registers for Output Queue size - SLI_PKT(0..63)_SLIST_FIFO_RSIZE */
+#define CN23XX_SLI_PKT_SLIST_FIFO_RSIZE_START	0x10090
+
+/* 64 registers for Output Queue Packet Count - SLI_PKT(0..63)_CNTS */
+#define CN23XX_SLI_PKT_CNTS_START		0x100B0
+
+/* Each Output Queue register is at a 16-byte Offset in BAR0 */
+#define CN23XX_OQ_OFFSET			0x20000
+
+/* ------- Output Queue Macros --------- */
+
+#define CN23XX_SLI_OQ_PKT_CONTROL(oq)					\
+	(CN23XX_SLI_PKT_OUTPUT_CONTROL_START + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_BASE_ADDR64(oq)					\
+	(CN23XX_SLI_SLIST_BADDR_START64 + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_SIZE(oq)						\
+	(CN23XX_SLI_PKT_SLIST_FIFO_RSIZE_START + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_BUFF_INFO_SIZE(oq)				\
+	(CN23XX_SLI_PKT_OUT_SIZE + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_PKTS_SENT(oq)					\
+	(CN23XX_SLI_PKT_CNTS_START + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_PKTS_CREDIT(oq)					\
+	(CN23XX_SLI_PKT_SLIST_BAOFF_DBELL_START + ((oq) * CN23XX_OQ_OFFSET))
+
+/* ------------------ Masks ---------------- */
+#define CN23XX_PKT_OUTPUT_CTL_IPTR		(1 << 11)
+#define CN23XX_PKT_OUTPUT_CTL_ES		(1 << 9)
+#define CN23XX_PKT_OUTPUT_CTL_NSR		(1 << 8)
+#define CN23XX_PKT_OUTPUT_CTL_ROR		(1 << 7)
+#define CN23XX_PKT_OUTPUT_CTL_DPTR		(1 << 6)
+#define CN23XX_PKT_OUTPUT_CTL_BMODE		(1 << 5)
+#define CN23XX_PKT_OUTPUT_CTL_ES_P		(1 << 3)
+#define CN23XX_PKT_OUTPUT_CTL_NSR_P		(1 << 2)
+#define CN23XX_PKT_OUTPUT_CTL_ROR_P		(1 << 1)
+#define CN23XX_PKT_OUTPUT_CTL_RING_ENB		(1 << 0)
+
+/* Rings per Virtual Function [RO] */
+#define CN23XX_PKT_INPUT_CTL_RPVF_MASK		0x3F
+#define CN23XX_PKT_INPUT_CTL_RPVF_POS		48
+
+/* These bits[47:44][RO] give the Physical function
+ * number info within the MAC
+ */
+#define CN23XX_PKT_INPUT_CTL_PF_NUM_MASK	0x7
+
+/* These bits[43:32][RO] give the virtual function
+ * number info within the PF
+ */
+#define CN23XX_PKT_INPUT_CTL_VF_NUM_MASK	0x1FFF
+
+/* ######################### Mailbox Reg Macros ######################## */
+#define CN23XX_SLI_PKT_PF_VF_MBOX_SIG_START	0x10200
+#define CN23XX_VF_SLI_PKT_MBOX_INT_START	0x10210
+
+#define CN23XX_SLI_MBOX_OFFSET			0x20000
+#define CN23XX_SLI_MBOX_SIG_IDX_OFFSET		0x8
+
+#define CN23XX_SLI_PKT_PF_VF_MBOX_SIG(q, idx)				\
+	(CN23XX_SLI_PKT_PF_VF_MBOX_SIG_START +				\
+	 ((q) * CN23XX_SLI_MBOX_OFFSET +				\
+	  (idx) * CN23XX_SLI_MBOX_SIG_IDX_OFFSET))
+
+#define CN23XX_VF_SLI_PKT_MBOX_INT(q)					\
+	(CN23XX_VF_SLI_PKT_MBOX_INT_START + ((q) * CN23XX_SLI_MBOX_OFFSET))
+
+#endif /* _LIO_23XX_REG_H_ */
-- 
1.8.3.1

[PATCH 02/50] config: liquidio PMD configuration

[Shijith Thotton]

Add config file options.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 config/common_base | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/config/common_base b/config/common_base
index 63756c4..71de4fc 100644
--- a/config/common_base
+++ b/config/common_base
@@ -287,6 +287,17 @@ CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_DRIVER=n
 CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_MBOX=n
 
 #
+# Compile burst-oriented Cavium LiquidIO PMD driver
+#
+CONFIG_RTE_LIBRTE_LIO_PMD=y
+CONFIG_RTE_LIBRTE_LIO_DEBUG_DRIVER=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_INIT=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_RX=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_TX=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_MBOX=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_REGS=n
+
+#
 # Compile burst-oriented VIRTIO PMD driver
 #
 CONFIG_RTE_LIBRTE_VIRTIO_PMD=y
-- 
1.8.3.1

[PATCH 03/50] net/liquidio: added PMD version map file

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/rte_pmd_lio_version.map | 4 ++++
 1 file changed, 4 insertions(+)
 create mode 100644 drivers/net/liquidio/rte_pmd_lio_version.map

diff --git a/drivers/net/liquidio/rte_pmd_lio_version.map b/drivers/net/liquidio/rte_pmd_lio_version.map
new file mode 100644
index 0000000..8591cc0
--- /dev/null
+++ b/drivers/net/liquidio/rte_pmd_lio_version.map
@@ -0,0 +1,4 @@
+DPDK_17.05 {
+
+	local: *;
+};
-- 
1.8.3.1

[PATCH 04/50] net/liquidio: definitions for log

[Shijith Thotton]

Define macros used for log and make use of config file options to
enable them.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_logs.h | 91 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 91 insertions(+)
 create mode 100644 drivers/net/liquidio/lio_logs.h

diff --git a/drivers/net/liquidio/lio_logs.h b/drivers/net/liquidio/lio_logs.h
new file mode 100644
index 0000000..a4c9ca4
--- /dev/null
+++ b/drivers/net/liquidio/lio_logs.h
@@ -0,0 +1,91 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_LOGS_H_
+#define _LIO_LOGS_H_
+
+#define lio_dev_printf(lio_dev, level, fmt, args...)			\
+	RTE_LOG(level, PMD, "%s" fmt, (lio_dev)->dev_string, ##args)
+
+#define lio_dev_info(lio_dev, fmt, args...)				\
+	lio_dev_printf(lio_dev, INFO, "INFO: " fmt, ##args)
+
+#define lio_dev_err(lio_dev, fmt, args...)				\
+	lio_dev_printf(lio_dev, ERR, "ERROR: %s() " fmt, __func__, ##args)
+
+#define PMD_INIT_LOG(level, fmt, args...) RTE_LOG(level, PMD, fmt, ## args)
+
+/* Enable these through config options */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_INIT
+#define PMD_INIT_FUNC_TRACE() PMD_INIT_LOG(DEBUG, "%s() >>\n", __func__)
+#else /* !RTE_LIBRTE_LIO_DEBUG_INIT */
+#define PMD_INIT_FUNC_TRACE() do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_INIT */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_DRIVER
+#define lio_dev_dbg(lio_dev, fmt, args...)				\
+	lio_dev_printf(lio_dev, DEBUG, "DEBUG: %s() " fmt, __func__, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_DRIVER */
+#define lio_dev_dbg(lio_dev, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_DRIVER */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_RX
+#define PMD_RX_LOG(lio_dev, level, fmt, args...)			\
+	lio_dev_printf(lio_dev, level, "RX: %s() " fmt, __func__, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_RX */
+#define PMD_RX_LOG(lio_dev, level, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_RX */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_TX
+#define PMD_TX_LOG(lio_dev, level, fmt, args...)			\
+	lio_dev_printf(lio_dev, level, "TX: %s() " fmt, __func__, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_TX */
+#define PMD_TX_LOG(lio_dev, level, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_TX */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_MBOX
+#define PMD_MBOX_LOG(lio_dev, level, fmt, args...)			\
+	lio_dev_printf(lio_dev, level, "MBOX: %s() " fmt, __func__, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_MBOX */
+#define PMD_MBOX_LOG(level, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_MBOX */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_REGS
+#define PMD_REGS_LOG(lio_dev, fmt, args...)				\
+	lio_dev_printf(lio_dev, DEBUG, "REGS: " fmt, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_REGS */
+#define PMD_REGS_LOG(level, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_REGS */
+
+#endif  /* _LIO_LOGS_H_ */
-- 
1.8.3.1

[PATCH 05/50] maintainers: claim responsibility for LiquidIO PMD

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
---
 MAINTAINERS | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index b4617fc..a63b7f7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -299,6 +299,13 @@ M: Maciej Czekaj <maciej.czekaj@caviumnetworks.com>
 F: drivers/net/thunderx/
 F: doc/guides/nics/thunderx.rst
 
+Cavium LiquidIO
+M: Shijith Thotton <shijith.thotton@cavium.com>
+M: Srisivasubramanian Srinivasan <ssrinivasan@cavium.com>
+F: drivers/net/liquidio/
+F: doc/guides/nics/liquidio.rst
+F: doc/guides/nics/features/liquidio.ini
+
 Chelsio cxgbe
 M: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
 F: drivers/net/cxgbe/
-- 
1.8.3.1

Re: [PATCH 05/50] maintainers: claim responsibility for LiquidIO PMD

[Ferruh Yigit]

On 2/21/2017 9:26 AM, Shijith Thotton wrote:
> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> ---
>  MAINTAINERS | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index b4617fc..a63b7f7 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -299,6 +299,13 @@ M: Maciej Czekaj <maciej.czekaj@caviumnetworks.com>
>  F: drivers/net/thunderx/
>  F: doc/guides/nics/thunderx.rst
>  
> +Cavium LiquidIO
> +M: Shijith Thotton <shijith.thotton@cavium.com>
> +M: Srisivasubramanian Srinivasan <ssrinivasan@cavium.com>
> +F: drivers/net/liquidio/
> +F: doc/guides/nics/liquidio.rst
> +F: doc/guides/nics/features/liquidio.ini

Since "Srisivasubramanian Srinivasan" too added as a maintainer, it can
be good to get his explicit Ack to this patch.

> +
>  Chelsio cxgbe
>  M: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
>  F: drivers/net/cxgbe/
> 

Re: [PATCH 05/50] maintainers: claim responsibility for LiquidIO PMD

[Shijith Thotton]

On Thursday 23 February 2017 07:58 PM, Ferruh Yigit wrote:
> On 2/21/2017 9:26 AM, Shijith Thotton wrote:
>> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
>> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
>> ---
>>  MAINTAINERS | 7 +++++++
>>  1 file changed, 7 insertions(+)
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index b4617fc..a63b7f7 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -299,6 +299,13 @@ M: Maciej Czekaj <maciej.czekaj@caviumnetworks.com>
>>  F: drivers/net/thunderx/
>>  F: doc/guides/nics/thunderx.rst
>>  
>> +Cavium LiquidIO
>> +M: Shijith Thotton <shijith.thotton@cavium.com>
>> +M: Srisivasubramanian Srinivasan <ssrinivasan@cavium.com>
>> +F: drivers/net/liquidio/
>> +F: doc/guides/nics/liquidio.rst
>> +F: doc/guides/nics/features/liquidio.ini
> 
> Since "Srisivasubramanian Srinivasan" too added as a maintainer, it can
> be good to get his explicit Ack to this patch.

Sure. Will add Ack from Srisivasubramanian.

> 
>> +
>>  Chelsio cxgbe
>>  M: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
>>  F: drivers/net/cxgbe/
>>
> 

Thanks,
Shijith

[PATCH 06/50] net/liquidio: liquidio VF PMD Driver registration

[Shijith Thotton]

Register LiquidIO PMD (net_liovf) and define APIs to init and
uninit.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  44 ++++++++++++
 drivers/net/liquidio/lio_ethdev.c       | 120 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  40 +++++++++++
 drivers/net/liquidio/lio_struct.h       |  65 +++++++++++++++++
 4 files changed, 269 insertions(+)
 create mode 100644 drivers/net/liquidio/base/lio_hw_defs.h
 create mode 100644 drivers/net/liquidio/lio_ethdev.c
 create mode 100644 drivers/net/liquidio/lio_ethdev.h
 create mode 100644 drivers/net/liquidio/lio_struct.h

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
new file mode 100644
index 0000000..db42f3e
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -0,0 +1,44 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_HW_DEFS_H_
+#define _LIO_HW_DEFS_H_
+
+#ifndef PCI_VENDOR_ID_CAVIUM
+#define PCI_VENDOR_ID_CAVIUM	0x177D
+#endif
+
+#define LIO_CN23XX_VF_VID	0x9712
+
+#define LIO_DEVICE_NAME_LEN		32
+#endif /* _LIO_HW_DEFS_H_ */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
new file mode 100644
index 0000000..b808a55
--- /dev/null
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -0,0 +1,120 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+#include <rte_alarm.h>
+
+#include "lio_logs.h"
+#include "lio_struct.h"
+#include "lio_ethdev.h"
+
+static int
+lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
+{
+	PMD_INIT_FUNC_TRACE();
+
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return -EPERM;
+
+	rte_free(eth_dev->data->mac_addrs);
+	eth_dev->data->mac_addrs = NULL;
+
+	return 0;
+}
+
+static int
+lio_eth_dev_init(struct rte_eth_dev *eth_dev)
+{
+	struct rte_pci_device *pdev = RTE_DEV_TO_PCI(eth_dev->device);
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	int mac_addr_size = 0;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* Primary does the initialization. */
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	rte_eth_copy_pci_info(eth_dev, pdev);
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_DETACHABLE;
+
+	if (pdev->mem_resource[0].addr) {
+		lio_dev->hw_addr = pdev->mem_resource[0].addr;
+	} else {
+		PMD_INIT_LOG(ERR, "ERROR: Failed to map BAR0\n");
+		return -ENODEV;
+	}
+
+	lio_dev->eth_dev = eth_dev;
+	/* set lio device print string */
+	snprintf(lio_dev->dev_string, sizeof(lio_dev->dev_string),
+		 "%s[%02x:%02x.%x]", pdev->driver->driver.name,
+		 pdev->addr.bus, pdev->addr.devid, pdev->addr.function);
+
+	lio_dev->port_id = eth_dev->data->port_id;
+
+	mac_addr_size = ETHER_ADDR_LEN;
+
+	eth_dev->data->mac_addrs = rte_zmalloc("lio", mac_addr_size, 0);
+	if (eth_dev->data->mac_addrs == NULL) {
+		lio_dev_err(lio_dev,
+			    "MAC addresses memory allocation failed\n");
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+/* Set of PCI devices this driver supports */
+static const struct rte_pci_id pci_id_liovf_map[] = {
+	{ RTE_PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, LIO_CN23XX_VF_VID) },
+	{ .vendor_id = 0, /* sentinel */ }
+};
+
+static struct eth_driver rte_liovf_pmd = {
+	.pci_drv = {
+		.id_table	= pci_id_liovf_map,
+		.drv_flags      = RTE_PCI_DRV_NEED_MAPPING,
+		.probe		= rte_eth_dev_pci_probe,
+		.remove		= rte_eth_dev_pci_remove,
+	},
+	.eth_dev_init		= lio_eth_dev_init,
+	.eth_dev_uninit		= lio_eth_dev_uninit,
+	.dev_private_size	= sizeof(struct lio_device),
+};
+
+RTE_PMD_REGISTER_PCI(net_liovf, rte_liovf_pmd.pci_drv);
+RTE_PMD_REGISTER_PCI_TABLE(net_liovf, pci_id_liovf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_liovf, "* igb_uio | vfio");
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
new file mode 100644
index 0000000..76c9072
--- /dev/null
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -0,0 +1,40 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_ETHDEV_H_
+#define _LIO_ETHDEV_H_
+
+#include <stdint.h>
+
+#define LIO_DEV(_eth_dev)		((_eth_dev)->data->dev_private)
+#endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
new file mode 100644
index 0000000..dcf99ce
--- /dev/null
+++ b/drivers/net/liquidio/lio_struct.h
@@ -0,0 +1,65 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_STRUCT_H_
+#define _LIO_STRUCT_H_
+
+#include <stdio.h>
+#include <stdint.h>
+#include <sys/queue.h>
+
+#include <rte_spinlock.h>
+#include <rte_atomic.h>
+
+#include "lio_hw_defs.h"
+
+/* -----------------------  THE LIO DEVICE  --------------------------- */
+/** The lio device.
+ *  Each lio device has this structure to represent all its
+ *  components.
+ */
+struct lio_device {
+	uint8_t *hw_addr;
+
+	char dev_string[LIO_DEVICE_NAME_LEN]; /* Device print string */
+
+	struct rte_eth_dev      *eth_dev;
+
+	uint8_t max_rx_queues;
+	uint8_t max_tx_queues;
+	uint8_t nb_rx_queues;
+	uint8_t nb_tx_queues;
+	uint8_t port_configured;
+	uint8_t port_id;
+};
+#endif /* _LIO_STRUCT_H_ */
-- 
1.8.3.1

Re: [PATCH 06/50] net/liquidio: liquidio VF PMD Driver registration

[Ferruh Yigit]

On 2/21/2017 9:26 AM, Shijith Thotton wrote:
> Register LiquidIO PMD (net_liovf) and define APIs to init and
> uninit.
> 
> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
<...>

> +
> +	mac_addr_size = ETHER_ADDR_LEN;
> +
> +	eth_dev->data->mac_addrs = rte_zmalloc("lio", mac_addr_size, 0);

"mac_addr_size" only used here, your call but it is possible to remove
variable and use ETHER_ADDR_LEN here directly.

Re: [PATCH 06/50] net/liquidio: liquidio VF PMD Driver registration

[Shijith Thotton]

On Thursday 23 February 2017 07:59 PM, Ferruh Yigit wrote:
> On 2/21/2017 9:26 AM, Shijith Thotton wrote:
>> Register LiquidIO PMD (net_liovf) and define APIs to init and
>> uninit.
>>
>> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
>> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
>> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
>> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
>> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
> <...>
> 
>> +
>> +	mac_addr_size = ETHER_ADDR_LEN;
>> +
>> +	eth_dev->data->mac_addrs = rte_zmalloc("lio", mac_addr_size, 0);
> 
> "mac_addr_size" only used here, your call but it is possible to remove
> variable and use ETHER_ADDR_LEN here directly.
> 

I agree. Will remove variable "mac_addr_size".

[PATCH 07/50] net/liquidio: added Makefile

[Shijith Thotton]

Added Makefile and made build changes.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/Makefile          |  1 +
 drivers/net/liquidio/Makefile | 59 +++++++++++++++++++++++++++++++++++++++++++
 mk/rte.app.mk                 |  1 +
 3 files changed, 61 insertions(+)
 create mode 100644 drivers/net/liquidio/Makefile

diff --git a/drivers/net/Makefile b/drivers/net/Makefile
index ab60cb8..2093e09 100644
--- a/drivers/net/Makefile
+++ b/drivers/net/Makefile
@@ -41,6 +41,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_ENIC_PMD) += enic
 DIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k
 DIRS-$(CONFIG_RTE_LIBRTE_I40E_PMD) += i40e
 DIRS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) += ixgbe
+DIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += liquidio
 DIRS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += mlx4
 DIRS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5
 DIRS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD) += mpipe
diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
new file mode 100644
index 0000000..25685a7
--- /dev/null
+++ b/drivers/net/liquidio/Makefile
@@ -0,0 +1,59 @@
+#
+#   BSD LICENSE
+#
+#   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Cavium, Inc. nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+LIB = librte_pmd_lio.a
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)/base -I$(SRCDIR)
+
+EXPORT_MAP := rte_pmd_lio_version.map
+
+LIBABIVER := 1
+
+VPATH += $(RTE_SDK)/drivers/net/liquidio/base
+
+#
+# all source are stored in SRCS-y
+#
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_ethdev.c
+
+# this lib depends upon:
+DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_eal lib/librte_ether
+DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_mempool lib/librte_mbuf
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 236da9c..837cca5 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -116,6 +116,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD)      += -lrte_pmd_ixgbe
 ifeq ($(CONFIG_RTE_LIBRTE_KNI),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KNI)        += -lrte_pmd_kni
 endif
+_LDLIBS-$(CONFIG_RTE_LIBRTE_LIO_PMD)        += -lrte_pmd_lio
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -lrte_pmd_mlx4 -libverbs
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -lrte_pmd_mlx5 -libverbs
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lrte_pmd_mpipe -lgxio
-- 
1.8.3.1

Re: [PATCH 07/50] net/liquidio: added Makefile

[Ferruh Yigit]

On 2/21/2017 9:26 AM, Shijith Thotton wrote:
> Added Makefile and made build changes.
> 
> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
> ---

Thanks for splitting into functional patches, that helps a lot. But set
starts with adding some set of files not used at this point. To build
step by step incremental logic, can I suggest:


1) Make this first patch of the set with some updates:
  a) make SRC-y empty, not add lio_ethdev.c yet, it will come with 6/50
  b) squash patch 2/50 with some updates [1]
  c) squash patch 3/50 to this patch
  d) squash patch 5/50 with some updates [2]

2) second patch can be 1/50

3) third patch can be 4/50, with adding DEBUG config option to config file

4) fourth patch can be 6/50, with adding lio_ethdev.c to SRC-y in Makefile

5) Rest can be same, only last patch that add document should update
maintainers file to add documents to the list.


[1]
Only keep:
--->
 #
+# Compile burst-oriented Cavium LiquidIO PMD driver
+#
+CONFIG_RTE_LIBRTE_LIO_PMD=y
+
--->

Leave rest of the DEBUG config option for the patch where lio_logs.h
added, patch 4/50.


[2]
Only keep:
--->
+Cavium LiquidIO
+M: Shijith Thotton <shijith.thotton@cavium.com>
+M: Srisivasubramanian Srinivasan <ssrinivasan@cavium.com>
+F: drivers/net/liquidio/
+
--->

Leave rest of document files to the patch where document added, patch 50/50.


Thanks,
ferruh

>  drivers/net/Makefile          |  1 +
>  drivers/net/liquidio/Makefile | 59 +++++++++++++++++++++++++++++++++++++++++++
>  mk/rte.app.mk                 |  1 +
>  3 files changed, 61 insertions(+)
>  create mode 100644 drivers/net/liquidio/Makefile
> 
> diff --git a/drivers/net/Makefile b/drivers/net/Makefile
> index ab60cb8..2093e09 100644
> --- a/drivers/net/Makefile
> +++ b/drivers/net/Makefile
> @@ -41,6 +41,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_ENIC_PMD) += enic
>  DIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k
>  DIRS-$(CONFIG_RTE_LIBRTE_I40E_PMD) += i40e
>  DIRS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) += ixgbe
> +DIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += liquidio
>  DIRS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += mlx4
>  DIRS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5
>  DIRS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD) += mpipe
> diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
> new file mode 100644
> index 0000000..25685a7
> --- /dev/null
> +++ b/drivers/net/liquidio/Makefile
> @@ -0,0 +1,59 @@
> +#
> +#   BSD LICENSE
> +#
> +#   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
> +#   All rights reserved.
> +#
> +#   Redistribution and use in source and binary forms, with or without
> +#   modification, are permitted provided that the following conditions
> +#   are met:
> +#
> +#     * Redistributions of source code must retain the above copyright
> +#       notice, this list of conditions and the following disclaimer.
> +#     * Redistributions in binary form must reproduce the above copyright
> +#       notice, this list of conditions and the following disclaimer in
> +#       the documentation and/or other materials provided with the
> +#       distribution.
> +#     * Neither the name of Cavium, Inc. nor the names of its
> +#       contributors may be used to endorse or promote products derived
> +#       from this software without specific prior written permission.
> +#
> +#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +#   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +#
> +
> +include $(RTE_SDK)/mk/rte.vars.mk
> +
> +#
> +# library name
> +#
> +LIB = librte_pmd_lio.a
> +
> +CFLAGS += -O3
> +CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)/base -I$(SRCDIR)
> +
> +EXPORT_MAP := rte_pmd_lio_version.map
> +
> +LIBABIVER := 1
> +
> +VPATH += $(RTE_SDK)/drivers/net/liquidio/base
> +
> +#
> +# all source are stored in SRCS-y
> +#
> +SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_ethdev.c
> +
> +# this lib depends upon:
> +DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_eal lib/librte_ether
> +DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_mempool lib/librte_mbuf
> +
> +include $(RTE_SDK)/mk/rte.lib.mk
> diff --git a/mk/rte.app.mk b/mk/rte.app.mk
> index 236da9c..837cca5 100644
> --- a/mk/rte.app.mk
> +++ b/mk/rte.app.mk
> @@ -116,6 +116,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD)      += -lrte_pmd_ixgbe
>  ifeq ($(CONFIG_RTE_LIBRTE_KNI),y)
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KNI)        += -lrte_pmd_kni
>  endif
> +_LDLIBS-$(CONFIG_RTE_LIBRTE_LIO_PMD)        += -lrte_pmd_lio
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -lrte_pmd_mlx4 -libverbs
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -lrte_pmd_mlx5 -libverbs
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lrte_pmd_mpipe -lgxio
> 

[PATCH 08/50] net/liquidio/base: macros to read and write register

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h | 67 +++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index db42f3e..673c2d6 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -34,6 +34,8 @@
 #ifndef _LIO_HW_DEFS_H_
 #define _LIO_HW_DEFS_H_
 
+#include <rte_io.h>
+
 #ifndef PCI_VENDOR_ID_CAVIUM
 #define PCI_VENDOR_ID_CAVIUM	0x177D
 #endif
@@ -41,4 +43,69 @@
 #define LIO_CN23XX_VF_VID	0x9712
 
 #define LIO_DEVICE_NAME_LEN		32
+
+/* Routines for reading and writing CSRs */
+#ifdef RTE_LIBRTE_LIO_DEBUG_REGS
+#define lio_write_csr(lio_dev, reg_off, value)				\
+	do {								\
+		typeof(lio_dev) _dev = lio_dev;				\
+		typeof(reg_off) _reg_off = reg_off;			\
+		typeof(value) _value = value;				\
+		PMD_REGS_LOG(_dev,					\
+			     "Write32: Reg: 0x%08lx Val: 0x%08lx\n",	\
+			     (unsigned long)_reg_off,			\
+			     (unsigned long)_value);			\
+		rte_write32(_value, _dev->hw_addr + _reg_off);		\
+	} while (0)
+
+#define lio_write_csr64(lio_dev, reg_off, val64)			\
+	do {								\
+		typeof(lio_dev) _dev = lio_dev;				\
+		typeof(reg_off) _reg_off = reg_off;			\
+		typeof(val64) _val64 = val64;				\
+		PMD_REGS_LOG(						\
+		    _dev,						\
+		    "Write64: Reg: 0x%08lx Val: 0x%016llx\n",		\
+		    (unsigned long)_reg_off,				\
+		    (unsigned long long)_val64);			\
+		rte_write64(_val64, _dev->hw_addr + _reg_off);		\
+	} while (0)
+
+#define lio_read_csr(lio_dev, reg_off)					\
+	({								\
+		typeof(lio_dev) _dev = lio_dev;				\
+		typeof(reg_off) _reg_off = reg_off;			\
+		uint32_t val = rte_read32(_dev->hw_addr + _reg_off);	\
+		PMD_REGS_LOG(_dev,					\
+			     "Read32: Reg: 0x%08lx Val: 0x%08lx\n",	\
+			     (unsigned long)_reg_off,			\
+			     (unsigned long)val);			\
+		val;							\
+	})
+
+#define lio_read_csr64(lio_dev, reg_off)				\
+	({								\
+		typeof(lio_dev) _dev = lio_dev;				\
+		typeof(reg_off) _reg_off = reg_off;			\
+		uint64_t val64 = rte_read64(_dev->hw_addr + _reg_off);	\
+		PMD_REGS_LOG(						\
+		    _dev,						\
+		    "Read64: Reg: 0x%08lx Val: 0x%016llx\n",	\
+		    (unsigned long)_reg_off,				\
+		    (unsigned long long)val64);				\
+		val64;							\
+	})
+#else
+#define lio_write_csr(lio_dev, reg_off, value)				\
+	rte_write32(value, (lio_dev)->hw_addr + (reg_off))
+
+#define lio_write_csr64(lio_dev, reg_off, val64)			\
+	rte_write64(val64, (lio_dev)->hw_addr + (reg_off))
+
+#define lio_read_csr(lio_dev, reg_off)					\
+	rte_read32((lio_dev)->hw_addr + (reg_off))
+
+#define lio_read_csr64(lio_dev, reg_off)				\
+	rte_read64((lio_dev)->hw_addr + (reg_off))
+#endif
 #endif /* _LIO_HW_DEFS_H_ */
-- 
1.8.3.1

[PATCH 09/50] net/liquidio: liquidio device init

[Shijith Thotton]

Default device configuration and initialization code.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/Makefile           |  1 +
 drivers/net/liquidio/base/lio_23xx_vf.c | 67 ++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h | 84 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_hw_defs.h | 34 +++++++++++++
 drivers/net/liquidio/lio_ethdev.c       | 61 +++++++++++++++++++++++-
 drivers/net/liquidio/lio_struct.h       | 70 +++++++++++++++++++++++++++
 6 files changed, 316 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.c
 create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.h

diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
index 25685a7..8880a10 100644
--- a/drivers/net/liquidio/Makefile
+++ b/drivers/net/liquidio/Makefile
@@ -51,6 +51,7 @@ VPATH += $(RTE_SDK)/drivers/net/liquidio/base
 # all source are stored in SRCS-y
 #
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_ethdev.c
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_23xx_vf.c
 
 # this lib depends upon:
 DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_eal lib/librte_ether
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
new file mode 100644
index 0000000..dd5e3a6
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -0,0 +1,67 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+
+#include "lio_logs.h"
+#include "lio_23xx_vf.h"
+#include "lio_23xx_reg.h"
+
+int
+cn23xx_vf_setup_device(struct lio_device *lio_dev)
+{
+	uint64_t reg_val;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* INPUT_CONTROL[RPVF] gives the VF IOq count */
+	reg_val = lio_read_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(0));
+
+	lio_dev->pf_num = (reg_val >> CN23XX_PKT_INPUT_CTL_PF_NUM_POS) &
+				CN23XX_PKT_INPUT_CTL_PF_NUM_MASK;
+	lio_dev->vf_num = (reg_val >> CN23XX_PKT_INPUT_CTL_VF_NUM_POS) &
+				CN23XX_PKT_INPUT_CTL_VF_NUM_MASK;
+
+	reg_val = reg_val >> CN23XX_PKT_INPUT_CTL_RPVF_POS;
+
+	lio_dev->sriov_info.rings_per_vf =
+				reg_val & CN23XX_PKT_INPUT_CTL_RPVF_MASK;
+
+	lio_dev->default_config = lio_get_conf(lio_dev);
+	if (lio_dev->default_config == NULL)
+		return -1;
+
+	return 0;
+}
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.h b/drivers/net/liquidio/base/lio_23xx_vf.h
new file mode 100644
index 0000000..1c234bf
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_23xx_vf.h
@@ -0,0 +1,84 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_23XX_VF_H_
+#define _LIO_23XX_VF_H_
+
+#include <stdio.h>
+
+#include "lio_struct.h"
+
+static const struct lio_config default_cn23xx_conf	= {
+	.card_type				= LIO_23XX,
+	.card_name				= LIO_23XX_NAME,
+	/** IQ attributes */
+	.iq					= {
+		.max_iqs			= CN23XX_CFG_IO_QUEUES,
+		.pending_list_size		=
+			(CN23XX_MAX_IQ_DESCRIPTORS * CN23XX_CFG_IO_QUEUES),
+		.instr_type			= OCTEON_64BYTE_INSTR,
+	},
+
+	/** OQ attributes */
+	.oq					= {
+		.max_oqs			= CN23XX_CFG_IO_QUEUES,
+		.info_ptr			= OCTEON_OQ_INFOPTR_MODE,
+		.refill_threshold		= CN23XX_OQ_REFIL_THRESHOLD,
+	},
+
+	.num_nic_ports				= CN23XX_DEFAULT_NUM_PORTS,
+	.num_def_rx_descs			= CN23XX_MAX_OQ_DESCRIPTORS,
+	.num_def_tx_descs			= CN23XX_MAX_IQ_DESCRIPTORS,
+	.def_rx_buf_size			= CN23XX_OQ_BUF_SIZE,
+};
+
+static inline const struct lio_config *
+lio_get_conf(struct lio_device *lio_dev)
+{
+	const struct lio_config *default_lio_conf = NULL;
+
+	/* check the LIO Device model & return the corresponding lio
+	 * configuration
+	 */
+	default_lio_conf = &default_cn23xx_conf;
+
+	if (default_lio_conf == NULL) {
+		lio_dev_err(lio_dev, "Configuration verification failed\n");
+		return NULL;
+	}
+
+	return default_lio_conf;
+}
+
+int cn23xx_vf_setup_device(struct lio_device  *lio_dev);
+#endif /* _LIO_23XX_VF_H_  */
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 673c2d6..ba51d7a 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -42,6 +42,40 @@
 
 #define LIO_CN23XX_VF_VID	0x9712
 
+/* --------------------------CONFIG VALUES------------------------ */
+
+/* CN23xx IQ configuration macros */
+#define CN23XX_MAX_RINGS_PER_PF			64
+#define CN23XX_MAX_RINGS_PER_VF			8
+
+#define CN23XX_MAX_INPUT_QUEUES			CN23XX_MAX_RINGS_PER_PF
+#define CN23XX_MAX_IQ_DESCRIPTORS		512
+#define CN23XX_MIN_IQ_DESCRIPTORS		128
+
+#define CN23XX_MAX_OUTPUT_QUEUES		CN23XX_MAX_RINGS_PER_PF
+#define CN23XX_MAX_OQ_DESCRIPTORS		512
+#define CN23XX_MIN_OQ_DESCRIPTORS		128
+#define CN23XX_OQ_BUF_SIZE			1536
+
+#define CN23XX_OQ_REFIL_THRESHOLD		16
+
+#define CN23XX_DEFAULT_NUM_PORTS		1
+
+#define CN23XX_CFG_IO_QUEUES			CN23XX_MAX_RINGS_PER_PF
+
+/* common OCTEON configuration macros */
+#define OCTEON_64BYTE_INSTR			64
+#define OCTEON_OQ_INFOPTR_MODE			1
+
+/* Max IOQs per LIO Link */
+#define LIO_MAX_IOQS_PER_IF			64
+
+enum lio_card_type {
+	LIO_23XX /* 23xx */
+};
+
+#define LIO_23XX_NAME "23xx"
+
 #define LIO_DEVICE_NAME_LEN		32
 
 /* Routines for reading and writing CSRs */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index b808a55..12061d6 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -37,9 +37,63 @@
 #include <rte_alarm.h>
 
 #include "lio_logs.h"
-#include "lio_struct.h"
+#include "lio_23xx_vf.h"
 #include "lio_ethdev.h"
 
+/**
+ * \brief Identify the LIO device and to map the BAR address space
+ * @param lio_dev lio device
+ */
+static int
+lio_chip_specific_setup(struct lio_device *lio_dev)
+{
+	struct rte_pci_device *pdev = lio_dev->pci_dev;
+	uint32_t dev_id = pdev->id.device_id;
+	const char *s;
+	int ret = 1;
+
+	switch (dev_id) {
+	case LIO_CN23XX_VF_VID:
+		lio_dev->chip_id = LIO_CN23XX_VF_VID;
+		ret = cn23xx_vf_setup_device(lio_dev);
+		s = "CN23XX VF";
+		break;
+	default:
+		s = "?";
+		lio_dev_err(lio_dev, "Unsupported Chip\n");
+	}
+
+	if (!ret)
+		lio_dev_info(lio_dev, "DEVICE : %s\n", s);
+
+	return ret;
+}
+
+static int
+lio_first_time_init(struct lio_device *lio_dev,
+		    struct rte_pci_device *pdev)
+{
+	int dpdk_queues;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* set dpdk specific pci device pointer */
+	lio_dev->pci_dev = pdev;
+
+	/* Identify the LIO type and set device ops */
+	if (lio_chip_specific_setup(lio_dev)) {
+		lio_dev_err(lio_dev, "Chip specific setup failed\n");
+		return -1;
+	}
+
+	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
+
+	lio_dev->max_tx_queues = dpdk_queues;
+	lio_dev->max_rx_queues = dpdk_queues;
+
+	return 0;
+}
+
 static int
 lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
 {
@@ -85,6 +139,11 @@
 
 	lio_dev->port_id = eth_dev->data->port_id;
 
+	if (lio_first_time_init(lio_dev, pdev)) {
+		lio_dev_err(lio_dev, "Device init failed\n");
+		return -EINVAL;
+	}
+
 	mac_addr_size = ETHER_ADDR_LEN;
 
 	eth_dev->data->mac_addrs = rte_zmalloc("lio", mac_addr_size, 0);
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index dcf99ce..a1203e4 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -43,16 +43,86 @@
 
 #include "lio_hw_defs.h"
 
+struct lio_sriov_info {
+	/** Number of rings assigned to VF */
+	uint32_t rings_per_vf;
+
+	/** Number of VF devices enabled */
+	uint32_t num_vfs;
+};
+
+/* Structure to define the configuration attributes for each Input queue. */
+struct lio_iq_config {
+	/* Max number of IQs available */
+	uint8_t max_iqs;
+
+	/** Pending list size (usually set to the sum of the size of all Input
+	 *  queues)
+	 */
+	uint32_t pending_list_size;
+
+	/** Command size - 32 or 64 bytes */
+	uint32_t instr_type;
+};
+
+/* Structure to define the configuration attributes for each Output queue. */
+struct lio_oq_config {
+	/* Max number of OQs available */
+	uint8_t max_oqs;
+
+	/** If set, the Output queue uses info-pointer mode. (Default: 1 ) */
+	uint32_t info_ptr;
+
+	/** The number of buffers that were consumed during packet processing by
+	 *  the driver on this Output queue before the driver attempts to
+	 *  replenish the descriptor ring with new buffers.
+	 */
+	uint32_t refill_threshold;
+};
+
+/* Structure to define the configuration. */
+struct lio_config {
+	uint16_t card_type;
+	const char *card_name;
+
+	/** Input Queue attributes. */
+	struct lio_iq_config iq;
+
+	/** Output Queue attributes. */
+	struct lio_oq_config oq;
+
+	int num_nic_ports;
+
+	int num_def_tx_descs;
+
+	/* Num of desc for rx rings */
+	int num_def_rx_descs;
+
+	int def_rx_buf_size;
+};
+
 /* -----------------------  THE LIO DEVICE  --------------------------- */
 /** The lio device.
  *  Each lio device has this structure to represent all its
  *  components.
  */
 struct lio_device {
+	/** PCI device pointer */
+	struct rte_pci_device *pci_dev;
+
+	/** Octeon Chip type */
+	uint16_t chip_id;
+	uint16_t pf_num;
+	uint16_t vf_num;
+
 	uint8_t *hw_addr;
 
+	struct lio_sriov_info sriov_info;
+
 	char dev_string[LIO_DEVICE_NAME_LEN]; /* Device print string */
 
+	const struct lio_config *default_config;
+
 	struct rte_eth_dev      *eth_dev;
 
 	uint8_t max_rx_queues;
-- 
1.8.3.1

[PATCH 10/50] net/liquidio: add API to disable io queues

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 49 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h |  7 +++++
 drivers/net/liquidio/lio_ethdev.c       |  5 ++++
 3 files changed, 61 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index dd5e3a6..d9b9e2a 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -65,3 +65,52 @@
 
 	return 0;
 }
+
+int
+cn23xx_vf_set_io_queues_off(struct lio_device *lio_dev)
+{
+	uint32_t loop = CN23XX_VF_BUSY_READING_REG_LOOP_COUNT;
+	uint64_t q_no;
+
+	/* Disable the i/p and o/p queues for this Octeon.
+	 * IOQs will already be in reset.
+	 * If RST bit is set, wait for Quiet bit to be set
+	 * Once Quiet bit is set, clear the RST bit
+	 */
+	PMD_INIT_FUNC_TRACE();
+
+	for (q_no = 0; q_no < lio_dev->sriov_info.rings_per_vf; q_no++) {
+		volatile uint64_t reg_val;
+
+		reg_val = lio_read_csr64(lio_dev,
+					 CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		while ((reg_val & CN23XX_PKT_INPUT_CTL_RST) && !(reg_val &
+					 CN23XX_PKT_INPUT_CTL_QUIET) && loop) {
+			reg_val = lio_read_csr64(
+					lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+			loop = loop - 1;
+		}
+
+		if (loop == 0) {
+			lio_dev_err(lio_dev,
+				    "clearing the reset reg failed or setting the quiet reg failed for qno %lu\n",
+				    (unsigned long)q_no);
+			return -1;
+		}
+
+		reg_val = reg_val & ~CN23XX_PKT_INPUT_CTL_RST;
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+				reg_val);
+
+		reg_val = lio_read_csr64(lio_dev,
+					 CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		if (reg_val & CN23XX_PKT_INPUT_CTL_RST) {
+			lio_dev_err(lio_dev, "unable to reset qno %lu\n",
+				    (unsigned long)q_no);
+			return -1;
+		}
+	}
+
+	return 0;
+}
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.h b/drivers/net/liquidio/base/lio_23xx_vf.h
index 1c234bf..1af09d0 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.h
+++ b/drivers/net/liquidio/base/lio_23xx_vf.h
@@ -80,5 +80,12 @@
 	return default_lio_conf;
 }
 
+/** Turns off the input and output queues for the device
+ *  @param lio_dev which device io queues to disable
+ */
+int cn23xx_vf_set_io_queues_off(struct lio_device *lio_dev);
+
+#define CN23XX_VF_BUSY_READING_REG_LOOP_COUNT	100000
+
 int cn23xx_vf_setup_device(struct lio_device  *lio_dev);
 #endif /* _LIO_23XX_VF_H_  */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 12061d6..bd6196b 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -86,6 +86,11 @@
 		return -1;
 	}
 
+	if (cn23xx_vf_set_io_queues_off(lio_dev)) {
+		lio_dev_err(lio_dev, "Setting io queues off failed\n");
+		return -1;
+	}
+
 	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
 
 	lio_dev->max_tx_queues = dpdk_queues;
-- 
1.8.3.1

[PATCH 11/50] net/liquidio: add API to setup io queue registers

[Shijith Thotton]

Set default configuration values for io queue registers.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 160 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.c       |   5 +
 drivers/net/liquidio/lio_struct.h       |   7 ++
 3 files changed, 172 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index d9b9e2a..f61f185 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -39,6 +39,164 @@
 #include "lio_23xx_vf.h"
 #include "lio_23xx_reg.h"
 
+static int
+cn23xx_vf_reset_io_queues(struct lio_device *lio_dev, uint32_t num_queues)
+{
+	uint32_t loop = CN23XX_VF_BUSY_READING_REG_LOOP_COUNT;
+	uint64_t d64, q_no;
+	int ret_val = 0;
+
+	PMD_INIT_FUNC_TRACE();
+
+	for (q_no = 0; q_no < num_queues; q_no++) {
+		/* set RST bit to 1. This bit applies to both IQ and OQ */
+		d64 = lio_read_csr64(lio_dev,
+				     CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		d64 = d64 | CN23XX_PKT_INPUT_CTL_RST;
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+				d64);
+	}
+
+	/* wait until the RST bit is clear or the RST and QUIET bits are set */
+	for (q_no = 0; q_no < num_queues; q_no++) {
+		volatile uint64_t reg_val;
+
+		reg_val	= lio_read_csr64(lio_dev,
+					 CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		while ((reg_val & CN23XX_PKT_INPUT_CTL_RST) &&
+				!(reg_val & CN23XX_PKT_INPUT_CTL_QUIET) &&
+				loop) {
+			reg_val = lio_read_csr64(
+					lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+			loop = loop - 1;
+		}
+
+		if (loop == 0) {
+			lio_dev_err(lio_dev,
+				    "clearing the reset reg failed or setting the quiet reg failed for qno: %lu\n",
+				    (unsigned long)q_no);
+			return -1;
+		}
+
+		reg_val = reg_val & ~CN23XX_PKT_INPUT_CTL_RST;
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+				reg_val);
+
+		reg_val = lio_read_csr64(
+		    lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		if (reg_val & CN23XX_PKT_INPUT_CTL_RST) {
+			lio_dev_err(lio_dev,
+				    "clearing the reset failed for qno: %lu\n",
+				    (unsigned long)q_no);
+			ret_val = -1;
+		}
+	}
+
+	return ret_val;
+}
+
+static int
+cn23xx_vf_setup_global_input_regs(struct lio_device *lio_dev)
+{
+	uint64_t q_no;
+	uint64_t d64;
+
+	PMD_INIT_FUNC_TRACE();
+
+	if (cn23xx_vf_reset_io_queues(lio_dev,
+				      lio_dev->sriov_info.rings_per_vf))
+		return -1;
+
+	for (q_no = 0; q_no < (lio_dev->sriov_info.rings_per_vf); q_no++) {
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_DOORBELL(q_no),
+				0xFFFFFFFF);
+
+		d64 = lio_read_csr64(lio_dev,
+				     CN23XX_SLI_IQ_INSTR_COUNT64(q_no));
+
+		d64 &= 0xEFFFFFFFFFFFFFFFL;
+
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_INSTR_COUNT64(q_no),
+				d64);
+
+		/* Select ES, RO, NS, RDSIZE,DPTR Fomat#0 for
+		 * the Input Queues
+		 */
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+				CN23XX_PKT_INPUT_CTL_MASK);
+	}
+
+	return 0;
+}
+
+static void
+cn23xx_vf_setup_global_output_regs(struct lio_device *lio_dev)
+{
+	uint32_t reg_val;
+	uint32_t q_no;
+
+	PMD_INIT_FUNC_TRACE();
+
+	for (q_no = 0; q_no < lio_dev->sriov_info.rings_per_vf; q_no++) {
+		lio_write_csr(lio_dev, CN23XX_SLI_OQ_PKTS_CREDIT(q_no),
+			      0xFFFFFFFF);
+
+		reg_val =
+		    lio_read_csr(lio_dev, CN23XX_SLI_OQ_PKTS_SENT(q_no));
+
+		reg_val &= 0xEFFFFFFFFFFFFFFFL;
+
+		reg_val =
+		    lio_read_csr(lio_dev, CN23XX_SLI_OQ_PKT_CONTROL(q_no));
+
+		/* set IPTR & DPTR */
+		reg_val |=
+		    (CN23XX_PKT_OUTPUT_CTL_IPTR | CN23XX_PKT_OUTPUT_CTL_DPTR);
+
+		/* reset BMODE */
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_BMODE);
+
+		/* No Relaxed Ordering, No Snoop, 64-bit Byte swap
+		 * for Output Queue Scatter List
+		 * reset ROR_P, NSR_P
+		 */
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_ROR_P);
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_NSR_P);
+
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_ES_P);
+#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		reg_val |= (CN23XX_PKT_OUTPUT_CTL_ES_P);
+#endif
+		/* No Relaxed Ordering, No Snoop, 64-bit Byte swap
+		 * for Output Queue Data
+		 * reset ROR, NSR
+		 */
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_ROR);
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_NSR);
+		/* set the ES bit */
+		reg_val |= (CN23XX_PKT_OUTPUT_CTL_ES);
+
+		/* write all the selected settings */
+		lio_write_csr(lio_dev, CN23XX_SLI_OQ_PKT_CONTROL(q_no),
+			      reg_val);
+	}
+}
+
+static int
+cn23xx_vf_setup_device_regs(struct lio_device *lio_dev)
+{
+	PMD_INIT_FUNC_TRACE();
+
+	if (cn23xx_vf_setup_global_input_regs(lio_dev))
+		return -1;
+
+	cn23xx_vf_setup_global_output_regs(lio_dev);
+
+	return 0;
+}
+
 int
 cn23xx_vf_setup_device(struct lio_device *lio_dev)
 {
@@ -63,6 +221,8 @@
 	if (lio_dev->default_config == NULL)
 		return -1;
 
+	lio_dev->fn_list.setup_device_regs	= cn23xx_vf_setup_device_regs;
+
 	return 0;
 }
 
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index bd6196b..daf7fce 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -91,6 +91,11 @@
 		return -1;
 	}
 
+	if (lio_dev->fn_list.setup_device_regs(lio_dev)) {
+		lio_dev_err(lio_dev, "Failed to configure device registers\n");
+		return -1;
+	}
+
 	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
 
 	lio_dev->max_tx_queues = dpdk_queues;
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index a1203e4..577ea49 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -43,6 +43,11 @@
 
 #include "lio_hw_defs.h"
 
+struct lio_device;
+struct lio_fn_list {
+	int (*setup_device_regs)(struct lio_device *);
+};
+
 struct lio_sriov_info {
 	/** Number of rings assigned to VF */
 	uint32_t rings_per_vf;
@@ -117,6 +122,8 @@ struct lio_device {
 
 	uint8_t *hw_addr;
 
+	struct lio_fn_list fn_list;
+
 	struct lio_sriov_info sriov_info;
 
 	char dev_string[LIO_DEVICE_NAME_LEN]; /* Device print string */
-- 
1.8.3.1

[PATCH 12/50] net/liquidio: add mbox APIs for PF/VF communication

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/Makefile        |   1 +
 drivers/net/liquidio/base/lio_mbox.c | 275 +++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_mbox.h | 129 ++++++++++++++++
 drivers/net/liquidio/lio_struct.h    |   3 +
 4 files changed, 408 insertions(+)
 create mode 100644 drivers/net/liquidio/base/lio_mbox.c
 create mode 100644 drivers/net/liquidio/base/lio_mbox.h

diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
index 8880a10..451f49d 100644
--- a/drivers/net/liquidio/Makefile
+++ b/drivers/net/liquidio/Makefile
@@ -52,6 +52,7 @@ VPATH += $(RTE_SDK)/drivers/net/liquidio/base
 #
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_ethdev.c
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_23xx_vf.c
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_mbox.c
 
 # this lib depends upon:
 DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_eal lib/librte_ether
diff --git a/drivers/net/liquidio/base/lio_mbox.c b/drivers/net/liquidio/base/lio_mbox.c
new file mode 100644
index 0000000..b4abc62
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_mbox.c
@@ -0,0 +1,275 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+
+#include "lio_logs.h"
+#include "lio_struct.h"
+#include "lio_mbox.h"
+
+/**
+ * lio_mbox_read:
+ * @mbox: Pointer mailbox
+ *
+ * Reads the 8-bytes of data from the mbox register
+ * Writes back the acknowledgment indicating completion of read
+ */
+int
+lio_mbox_read(struct lio_mbox *mbox)
+{
+	union lio_mbox_message msg;
+	int ret = 0;
+
+	msg.mbox_msg64 = rte_read64(mbox->mbox_read_reg);
+
+	if ((msg.mbox_msg64 == LIO_PFVFACK) || (msg.mbox_msg64 == LIO_PFVFSIG))
+		return 0;
+
+	if (mbox->state & LIO_MBOX_STATE_REQ_RECEIVING) {
+		mbox->mbox_req.data[mbox->mbox_req.recv_len - 1] =
+					msg.mbox_msg64;
+		mbox->mbox_req.recv_len++;
+	} else {
+		if (mbox->state & LIO_MBOX_STATE_RES_RECEIVING) {
+			mbox->mbox_resp.data[mbox->mbox_resp.recv_len - 1] =
+					msg.mbox_msg64;
+			mbox->mbox_resp.recv_len++;
+		} else {
+			if ((mbox->state & LIO_MBOX_STATE_IDLE) &&
+					(msg.s.type == LIO_MBOX_REQUEST)) {
+				mbox->state &= ~LIO_MBOX_STATE_IDLE;
+				mbox->state |= LIO_MBOX_STATE_REQ_RECEIVING;
+				mbox->mbox_req.msg.mbox_msg64 = msg.mbox_msg64;
+				mbox->mbox_req.q_no = mbox->q_no;
+				mbox->mbox_req.recv_len = 1;
+			} else {
+				if ((mbox->state &
+				     LIO_MBOX_STATE_RES_PENDING) &&
+				    (msg.s.type == LIO_MBOX_RESPONSE)) {
+					mbox->state &=
+						~LIO_MBOX_STATE_RES_PENDING;
+					mbox->state |=
+						LIO_MBOX_STATE_RES_RECEIVING;
+					mbox->mbox_resp.msg.mbox_msg64 =
+								msg.mbox_msg64;
+					mbox->mbox_resp.q_no = mbox->q_no;
+					mbox->mbox_resp.recv_len = 1;
+				} else {
+					rte_write64(LIO_PFVFERR,
+						    mbox->mbox_read_reg);
+					mbox->state |= LIO_MBOX_STATE_ERROR;
+					return -1;
+				}
+			}
+		}
+	}
+
+	if (mbox->state & LIO_MBOX_STATE_REQ_RECEIVING) {
+		if (mbox->mbox_req.recv_len < msg.s.len) {
+			ret = 0;
+		} else {
+			mbox->state &= ~LIO_MBOX_STATE_REQ_RECEIVING;
+			mbox->state |= LIO_MBOX_STATE_REQ_RECEIVED;
+			ret = 1;
+		}
+	} else {
+		if (mbox->state & LIO_MBOX_STATE_RES_RECEIVING) {
+			if (mbox->mbox_resp.recv_len < msg.s.len) {
+				ret = 0;
+			} else {
+				mbox->state &= ~LIO_MBOX_STATE_RES_RECEIVING;
+				mbox->state |= LIO_MBOX_STATE_RES_RECEIVED;
+				ret = 1;
+			}
+		} else {
+			RTE_ASSERT(0);
+		}
+	}
+
+	rte_write64(LIO_PFVFACK, mbox->mbox_read_reg);
+
+	return ret;
+}
+
+/**
+ * lio_mbox_write:
+ * @lio_dev: Pointer lio device
+ * @mbox_cmd: Cmd to send to mailbox.
+ *
+ * Populates the queue specific mbox structure
+ * with cmd information.
+ * Write the cmd to mbox register
+ */
+int
+lio_mbox_write(struct lio_device *lio_dev,
+	       struct lio_mbox_cmd *mbox_cmd)
+{
+	struct lio_mbox *mbox = lio_dev->mbox[mbox_cmd->q_no];
+	uint32_t count, i, ret = LIO_MBOX_STATUS_SUCCESS;
+
+	if ((mbox_cmd->msg.s.type == LIO_MBOX_RESPONSE) &&
+			!(mbox->state & LIO_MBOX_STATE_REQ_RECEIVED))
+		return LIO_MBOX_STATUS_FAILED;
+
+	if ((mbox_cmd->msg.s.type == LIO_MBOX_REQUEST) &&
+			!(mbox->state & LIO_MBOX_STATE_IDLE))
+		return LIO_MBOX_STATUS_BUSY;
+
+	if (mbox_cmd->msg.s.type == LIO_MBOX_REQUEST) {
+		rte_memcpy(&mbox->mbox_resp, mbox_cmd,
+			   sizeof(struct lio_mbox_cmd));
+		mbox->state = LIO_MBOX_STATE_RES_PENDING;
+	}
+
+	count = 0;
+
+	while (rte_read64(mbox->mbox_write_reg) != LIO_PFVFSIG) {
+		rte_delay_ms(1);
+		if (count++ == 1000) {
+			ret = LIO_MBOX_STATUS_FAILED;
+			break;
+		}
+	}
+
+	if (ret == LIO_MBOX_STATUS_SUCCESS) {
+		rte_write64(mbox_cmd->msg.mbox_msg64, mbox->mbox_write_reg);
+		for (i = 0; i < (uint32_t)(mbox_cmd->msg.s.len - 1); i++) {
+			count = 0;
+			while (rte_read64(mbox->mbox_write_reg) !=
+					LIO_PFVFACK) {
+				rte_delay_ms(1);
+				if (count++ == 1000) {
+					ret = LIO_MBOX_STATUS_FAILED;
+					break;
+				}
+			}
+			rte_write64(mbox_cmd->data[i], mbox->mbox_write_reg);
+		}
+	}
+
+	if (mbox_cmd->msg.s.type == LIO_MBOX_RESPONSE) {
+		mbox->state = LIO_MBOX_STATE_IDLE;
+		rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+	} else {
+		if ((!mbox_cmd->msg.s.resp_needed) ||
+				(ret == LIO_MBOX_STATUS_FAILED)) {
+			mbox->state &= ~LIO_MBOX_STATE_RES_PENDING;
+			if (!(mbox->state & (LIO_MBOX_STATE_REQ_RECEIVING |
+					     LIO_MBOX_STATE_REQ_RECEIVED)))
+				mbox->state = LIO_MBOX_STATE_IDLE;
+		}
+	}
+
+	return ret;
+}
+
+/**
+ * lio_mbox_process_cmd:
+ * @mbox: Pointer mailbox
+ * @mbox_cmd: Pointer to command received
+ *
+ * Process the cmd received in mbox
+ */
+static int
+lio_mbox_process_cmd(struct lio_mbox *mbox,
+		     struct lio_mbox_cmd *mbox_cmd)
+{
+	struct lio_device *lio_dev = mbox->lio_dev;
+
+	if (mbox_cmd->msg.s.cmd == LIO_CORES_CRASHED)
+		lio_dev_err(lio_dev, "Octeon core(s) crashed or got stuck!\n");
+
+	return 0;
+}
+
+/**
+ * Process the received mbox message.
+ */
+int
+lio_mbox_process_message(struct lio_mbox *mbox)
+{
+	struct lio_mbox_cmd mbox_cmd;
+
+	if (mbox->state & LIO_MBOX_STATE_ERROR) {
+		if (mbox->state & (LIO_MBOX_STATE_RES_PENDING |
+				   LIO_MBOX_STATE_RES_RECEIVING)) {
+			rte_memcpy(&mbox_cmd, &mbox->mbox_resp,
+				   sizeof(struct lio_mbox_cmd));
+			mbox->state = LIO_MBOX_STATE_IDLE;
+			rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+			mbox_cmd.recv_status = 1;
+			if (mbox_cmd.fn)
+				mbox_cmd.fn(mbox->lio_dev, &mbox_cmd,
+					    mbox_cmd.fn_arg);
+
+			return 0;
+		}
+
+		mbox->state = LIO_MBOX_STATE_IDLE;
+		rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+
+		return 0;
+	}
+
+	if (mbox->state & LIO_MBOX_STATE_RES_RECEIVED) {
+		rte_memcpy(&mbox_cmd, &mbox->mbox_resp,
+			   sizeof(struct lio_mbox_cmd));
+		mbox->state = LIO_MBOX_STATE_IDLE;
+		rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+		mbox_cmd.recv_status = 0;
+		if (mbox_cmd.fn)
+			mbox_cmd.fn(mbox->lio_dev, &mbox_cmd, mbox_cmd.fn_arg);
+
+		return 0;
+	}
+
+	if (mbox->state & LIO_MBOX_STATE_REQ_RECEIVED) {
+		rte_memcpy(&mbox_cmd, &mbox->mbox_req,
+			   sizeof(struct lio_mbox_cmd));
+		if (!mbox_cmd.msg.s.resp_needed) {
+			mbox->state &= ~LIO_MBOX_STATE_REQ_RECEIVED;
+			if (!(mbox->state & LIO_MBOX_STATE_RES_PENDING))
+				mbox->state = LIO_MBOX_STATE_IDLE;
+			rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+		}
+
+		lio_mbox_process_cmd(mbox, &mbox_cmd);
+
+		return 0;
+	}
+
+	RTE_ASSERT(0);
+
+	return 0;
+}
diff --git a/drivers/net/liquidio/base/lio_mbox.h b/drivers/net/liquidio/base/lio_mbox.h
new file mode 100644
index 0000000..28c9e1a
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_mbox.h
@@ -0,0 +1,129 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_MBOX_H_
+#define _LIO_MBOX_H_
+
+#include <stdint.h>
+
+#include <rte_spinlock.h>
+
+/* Macros for Mail Box Communication */
+
+#define LIO_MBOX_DATA_MAX			32
+
+#define LIO_CORES_CRASHED			0x3
+
+/* Macro for Read acknowledgment */
+#define LIO_PFVFACK				0xffffffffffffffff
+#define LIO_PFVFSIG				0x1122334455667788
+#define LIO_PFVFERR				0xDEADDEADDEADDEAD
+
+enum lio_mbox_cmd_status {
+	LIO_MBOX_STATUS_SUCCESS		= 0,
+	LIO_MBOX_STATUS_FAILED		= 1,
+	LIO_MBOX_STATUS_BUSY		= 2
+};
+
+enum lio_mbox_message_type {
+	LIO_MBOX_REQUEST	= 0,
+	LIO_MBOX_RESPONSE	= 1
+};
+
+union lio_mbox_message {
+	uint64_t mbox_msg64;
+	struct {
+		uint16_t type : 1;
+		uint16_t resp_needed : 1;
+		uint16_t cmd : 6;
+		uint16_t len : 8;
+		uint8_t params[6];
+	} s;
+};
+
+typedef void (*lio_mbox_callback)(void *, void *, void *);
+
+struct lio_mbox_cmd {
+	union lio_mbox_message msg;
+	uint64_t data[LIO_MBOX_DATA_MAX];
+	uint32_t q_no;
+	uint32_t recv_len;
+	uint32_t recv_status;
+	lio_mbox_callback fn;
+	void *fn_arg;
+};
+
+enum lio_mbox_state {
+	LIO_MBOX_STATE_IDLE		= 1,
+	LIO_MBOX_STATE_REQ_RECEIVING	= 2,
+	LIO_MBOX_STATE_REQ_RECEIVED	= 4,
+	LIO_MBOX_STATE_RES_PENDING	= 8,
+	LIO_MBOX_STATE_RES_RECEIVING	= 16,
+	LIO_MBOX_STATE_RES_RECEIVED	= 16,
+	LIO_MBOX_STATE_ERROR		= 32
+};
+
+struct lio_mbox {
+	/* A spinlock to protect access to this q_mbox. */
+	rte_spinlock_t lock;
+
+	struct lio_device *lio_dev;
+
+	uint32_t q_no;
+
+	enum lio_mbox_state state;
+
+	/* SLI_MAC_PF_MBOX_INT for PF, SLI_PKT_MBOX_INT for VF. */
+	void *mbox_int_reg;
+
+	/* SLI_PKT_PF_VF_MBOX_SIG(0) for PF,
+	 * SLI_PKT_PF_VF_MBOX_SIG(1) for VF.
+	 */
+	void *mbox_write_reg;
+
+	/* SLI_PKT_PF_VF_MBOX_SIG(1) for PF,
+	 * SLI_PKT_PF_VF_MBOX_SIG(0) for VF.
+	 */
+	void *mbox_read_reg;
+
+	struct lio_mbox_cmd mbox_req;
+
+	struct lio_mbox_cmd mbox_resp;
+
+};
+
+int lio_mbox_read(struct lio_mbox *mbox);
+int lio_mbox_write(struct lio_device *lio_dev,
+		   struct lio_mbox_cmd *mbox_cmd);
+int lio_mbox_process_message(struct lio_mbox *mbox);
+#endif	/* _LIO_MBOX_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 577ea49..0af4fe3 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -126,6 +126,9 @@ struct lio_device {
 
 	struct lio_sriov_info sriov_info;
 
+	/** Mail Box details of each lio queue. */
+	struct lio_mbox **mbox;
+
 	char dev_string[LIO_DEVICE_NAME_LEN]; /* Device print string */
 
 	const struct lio_config *default_config;
-- 
1.8.3.1

[PATCH 13/50] net/liquidio: add API to setup mbox registers

[Shijith Thotton]

Map and initialize mbox registers.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 61 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.c       | 15 ++++++--
 drivers/net/liquidio/lio_struct.h       |  3 ++
 3 files changed, 77 insertions(+), 2 deletions(-)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index f61f185..70faa9b 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -38,6 +38,7 @@
 #include "lio_logs.h"
 #include "lio_23xx_vf.h"
 #include "lio_23xx_reg.h"
+#include "lio_mbox.h"
 
 static int
 cn23xx_vf_reset_io_queues(struct lio_device *lio_dev, uint32_t num_queues)
@@ -197,6 +198,63 @@
 	return 0;
 }
 
+static void
+cn23xx_vf_free_mbox(struct lio_device *lio_dev)
+{
+	PMD_INIT_FUNC_TRACE();
+
+	rte_free(lio_dev->mbox[0]);
+	lio_dev->mbox[0] = NULL;
+
+	rte_free(lio_dev->mbox);
+	lio_dev->mbox = NULL;
+}
+
+static int
+cn23xx_vf_setup_mbox(struct lio_device *lio_dev)
+{
+	struct lio_mbox *mbox;
+
+	PMD_INIT_FUNC_TRACE();
+
+	if (lio_dev->mbox == NULL) {
+		lio_dev->mbox = rte_zmalloc(NULL, sizeof(void *), 0);
+		if (lio_dev->mbox == NULL)
+			return -ENOMEM;
+	}
+
+	mbox = rte_zmalloc(NULL, sizeof(struct lio_mbox), 0);
+	if (mbox == NULL) {
+		rte_free(lio_dev->mbox);
+		lio_dev->mbox = NULL;
+		return -ENOMEM;
+	}
+
+	rte_spinlock_init(&mbox->lock);
+
+	mbox->lio_dev = lio_dev;
+
+	mbox->q_no = 0;
+
+	mbox->state = LIO_MBOX_STATE_IDLE;
+
+	/* VF mbox interrupt reg */
+	mbox->mbox_int_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_VF_SLI_PKT_MBOX_INT(0);
+	/* VF reads from SIG0 reg */
+	mbox->mbox_read_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_SLI_PKT_PF_VF_MBOX_SIG(0, 0);
+	/* VF writes into SIG1 reg */
+	mbox->mbox_write_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_SLI_PKT_PF_VF_MBOX_SIG(0, 1);
+
+	lio_dev->mbox[0] = mbox;
+
+	rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+
+	return 0;
+}
+
 int
 cn23xx_vf_setup_device(struct lio_device *lio_dev)
 {
@@ -221,6 +279,9 @@
 	if (lio_dev->default_config == NULL)
 		return -1;
 
+	lio_dev->fn_list.setup_mbox		= cn23xx_vf_setup_mbox;
+	lio_dev->fn_list.free_mbox		= cn23xx_vf_free_mbox;
+
 	lio_dev->fn_list.setup_device_regs	= cn23xx_vf_setup_device_regs;
 
 	return 0;
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index daf7fce..dd5e3b7 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -86,14 +86,19 @@
 		return -1;
 	}
 
+	if (lio_dev->fn_list.setup_mbox(lio_dev)) {
+		lio_dev_err(lio_dev, "Mailbox setup failed\n");
+		goto error;
+	}
+
 	if (cn23xx_vf_set_io_queues_off(lio_dev)) {
 		lio_dev_err(lio_dev, "Setting io queues off failed\n");
-		return -1;
+		goto error;
 	}
 
 	if (lio_dev->fn_list.setup_device_regs(lio_dev)) {
 		lio_dev_err(lio_dev, "Failed to configure device registers\n");
-		return -1;
+		goto error;
 	}
 
 	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
@@ -102,6 +107,12 @@
 	lio_dev->max_rx_queues = dpdk_queues;
 
 	return 0;
+
+error:
+	if (lio_dev->mbox[0])
+		lio_dev->fn_list.free_mbox(lio_dev);
+
+	return -1;
 }
 
 static int
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 0af4fe3..01b5716 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -45,6 +45,9 @@
 
 struct lio_device;
 struct lio_fn_list {
+	int (*setup_mbox)(struct lio_device *);
+	void (*free_mbox)(struct lio_device *);
+
 	int (*setup_device_regs)(struct lio_device *);
 };
 
-- 
1.8.3.1

[PATCH 14/50] net/liquidio: add API for VF/PF handshake

[Shijith Thotton]

Handshakes with PF kernel driver to verify driver version
compatibility.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 96 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h |  4 ++
 drivers/net/liquidio/base/lio_hw_defs.h |  3 ++
 drivers/net/liquidio/base/lio_mbox.h    |  1 +
 drivers/net/liquidio/lio_ethdev.c       | 21 ++++++++
 drivers/net/liquidio/lio_struct.h       | 45 ++++++++++++++++
 6 files changed, 170 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index 70faa9b..6270af5 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -255,6 +255,102 @@
 	return 0;
 }
 
+static void
+cn23xx_pfvf_hs_callback(struct lio_device *lio_dev,
+			struct lio_mbox_cmd *cmd, void *arg)
+{
+	uint32_t major = 0;
+
+	PMD_INIT_FUNC_TRACE();
+
+	rte_memcpy((uint8_t *)&lio_dev->pfvf_hsword, cmd->msg.s.params, 6);
+	if (cmd->recv_len > 1) {
+		struct lio_version *lio_ver = (struct lio_version *)cmd->data;
+
+		major = lio_ver->major;
+		major = major << 16;
+	}
+
+	rte_atomic64_set((rte_atomic64_t *)arg, major | 1);
+}
+
+int
+cn23xx_pfvf_handshake(struct lio_device *lio_dev)
+{
+	struct lio_mbox_cmd mbox_cmd;
+	struct lio_version *lio_ver = (struct lio_version *)&mbox_cmd.data[0];
+	rte_atomic64_t status;
+	uint32_t count = 0;
+	uint32_t pfmajor;
+	uint32_t vfmajor;
+	uint32_t ret;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* Sending VF_ACTIVE indication to the PF driver */
+	lio_dev_dbg(lio_dev, "requesting info from PF\n");
+
+	mbox_cmd.msg.mbox_msg64 = 0;
+	mbox_cmd.msg.s.type = LIO_MBOX_REQUEST;
+	mbox_cmd.msg.s.resp_needed = 1;
+	mbox_cmd.msg.s.cmd = LIO_VF_ACTIVE;
+	mbox_cmd.msg.s.len = 2;
+	mbox_cmd.data[0] = 0;
+	lio_ver->major = LIO_BASE_MAJOR_VERSION;
+	lio_ver->minor = LIO_BASE_MINOR_VERSION;
+	lio_ver->micro = LIO_BASE_MICRO_VERSION;
+	mbox_cmd.q_no = 0;
+	mbox_cmd.recv_len = 0;
+	mbox_cmd.recv_status = 0;
+	mbox_cmd.fn = (lio_mbox_callback)cn23xx_pfvf_hs_callback;
+	mbox_cmd.fn_arg = (void *)&status;
+
+	if (lio_mbox_write(lio_dev, &mbox_cmd)) {
+		lio_dev_err(lio_dev, "Write to mailbox failed\n");
+		return -1;
+	}
+
+	rte_atomic64_set(&status, 0);
+
+	do {
+		rte_delay_ms(1);
+	} while ((rte_atomic64_read(&status) == 0) && (count++ < 10000));
+
+	ret = rte_atomic64_read(&status);
+	if (ret == 0) {
+		lio_dev_err(lio_dev, "cn23xx_pfvf_handshake timeout\n");
+		return -1;
+	}
+
+	vfmajor = LIO_BASE_MAJOR_VERSION;
+	pfmajor = ret >> 16;
+	if (pfmajor != vfmajor) {
+		lio_dev_err(lio_dev,
+			    "VF LiquidIO driver (major version %d) is not compatible with LiquidIO PF driver (major version %d)\n",
+			    vfmajor, pfmajor);
+		ret = -EPERM;
+	} else {
+		lio_dev_dbg(lio_dev,
+			    "VF LiquidIO driver (major version %d), LiquidIO PF driver (major version %d)\n",
+			    vfmajor, pfmajor);
+		ret = 0;
+	}
+
+	return ret;
+}
+
+void
+cn23xx_vf_handle_mbox(struct lio_device *lio_dev)
+{
+	uint64_t mbox_int_val;
+
+	/* read and clear by writing 1 */
+	mbox_int_val = rte_read64(lio_dev->mbox[0]->mbox_int_reg);
+	rte_write64(mbox_int_val, lio_dev->mbox[0]->mbox_int_reg);
+	if (lio_mbox_read(lio_dev->mbox[0]))
+		lio_mbox_process_message(lio_dev->mbox[0]);
+}
+
 int
 cn23xx_vf_setup_device(struct lio_device *lio_dev)
 {
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.h b/drivers/net/liquidio/base/lio_23xx_vf.h
index 1af09d0..83dc053 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.h
+++ b/drivers/net/liquidio/base/lio_23xx_vf.h
@@ -87,5 +87,9 @@
 
 #define CN23XX_VF_BUSY_READING_REG_LOOP_COUNT	100000
 
+int cn23xx_pfvf_handshake(struct lio_device *lio_dev);
+
 int cn23xx_vf_setup_device(struct lio_device  *lio_dev);
+
+void cn23xx_vf_handle_mbox(struct lio_device *lio_dev);
 #endif /* _LIO_23XX_VF_H_  */
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index ba51d7a..890ef17 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -77,6 +77,9 @@ enum lio_card_type {
 #define LIO_23XX_NAME "23xx"
 
 #define LIO_DEVICE_NAME_LEN		32
+#define LIO_BASE_MAJOR_VERSION		1
+#define LIO_BASE_MINOR_VERSION		5
+#define LIO_BASE_MICRO_VERSION		1
 
 /* Routines for reading and writing CSRs */
 #ifdef RTE_LIBRTE_LIO_DEBUG_REGS
diff --git a/drivers/net/liquidio/base/lio_mbox.h b/drivers/net/liquidio/base/lio_mbox.h
index 28c9e1a..f1c5b8e 100644
--- a/drivers/net/liquidio/base/lio_mbox.h
+++ b/drivers/net/liquidio/base/lio_mbox.h
@@ -42,6 +42,7 @@
 
 #define LIO_MBOX_DATA_MAX			32
 
+#define LIO_VF_ACTIVE				0x1
 #define LIO_CORES_CRASHED			0x3
 
 /* Macro for Read acknowledgment */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index dd5e3b7..7318bf5 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -40,6 +40,20 @@
 #include "lio_23xx_vf.h"
 #include "lio_ethdev.h"
 
+static void
+lio_check_pf_hs_response(void *lio_dev)
+{
+	struct lio_device *dev = lio_dev;
+
+	/* check till response arrives */
+	if (dev->pfvf_hsword.coproc_tics_per_us)
+		return;
+
+	cn23xx_vf_handle_mbox(dev);
+
+	rte_eal_alarm_set(1, lio_check_pf_hs_response, lio_dev);
+}
+
 /**
  * \brief Identify the LIO device and to map the BAR address space
  * @param lio_dev lio device
@@ -91,6 +105,13 @@
 		goto error;
 	}
 
+	/* Check PF response */
+	lio_check_pf_hs_response((void *)lio_dev);
+
+	/* Do handshake and exit if incompatible PF driver */
+	if (cn23xx_pfvf_handshake(lio_dev))
+		goto error;
+
 	if (cn23xx_vf_set_io_queues_off(lio_dev)) {
 		lio_dev_err(lio_dev, "Setting io queues off failed\n");
 		goto error;
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 01b5716..e8b6e1d 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -43,6 +43,13 @@
 
 #include "lio_hw_defs.h"
 
+struct lio_version {
+	uint16_t major;
+	uint16_t minor;
+	uint16_t micro;
+	uint16_t reserved;
+};
+
 struct lio_device;
 struct lio_fn_list {
 	int (*setup_mbox)(struct lio_device *);
@@ -51,6 +58,42 @@ struct lio_fn_list {
 	int (*setup_device_regs)(struct lio_device *);
 };
 
+struct lio_pf_vf_hs_word {
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	/** PKIND value assigned for the DPI interface */
+	uint64_t pkind : 8;
+
+	/** OCTEON core clock multiplier */
+	uint64_t core_tics_per_us : 16;
+
+	/** OCTEON coprocessor clock multiplier */
+	uint64_t coproc_tics_per_us : 16;
+
+	/** app that currently running on OCTEON */
+	uint64_t app_mode : 8;
+
+	/** RESERVED */
+	uint64_t reserved : 16;
+
+#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+
+	/** RESERVED */
+	uint64_t reserved : 16;
+
+	/** app that currently running on OCTEON */
+	uint64_t app_mode : 8;
+
+	/** OCTEON coprocessor clock multiplier */
+	uint64_t coproc_tics_per_us : 16;
+
+	/** OCTEON core clock multiplier */
+	uint64_t core_tics_per_us : 16;
+
+	/** PKIND value assigned for the DPI interface */
+	uint64_t pkind : 8;
+#endif
+};
+
 struct lio_sriov_info {
 	/** Number of rings assigned to VF */
 	uint32_t rings_per_vf;
@@ -129,6 +172,8 @@ struct lio_device {
 
 	struct lio_sriov_info sriov_info;
 
+	struct lio_pf_vf_hs_word pfvf_hsword;
+
 	/** Mail Box details of each lio queue. */
 	struct lio_mbox **mbox;
 
-- 
1.8.3.1

[PATCH 15/50] net/liquidio: add API for VF FLR

[Shijith Thotton]

Add API to perform Function Level Reset. VF sends FLR request to PF
using mbox and PF does the reset.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 19 +++++++++++++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h |  2 ++
 drivers/net/liquidio/base/lio_mbox.h    |  1 +
 drivers/net/liquidio/lio_ethdev.c       |  5 +++++
 4 files changed, 27 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index 6270af5..ed5b830 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -255,6 +255,25 @@
 	return 0;
 }
 
+void
+cn23xx_vf_ask_pf_to_do_flr(struct lio_device *lio_dev)
+{
+	struct lio_mbox_cmd mbox_cmd;
+
+	mbox_cmd.msg.mbox_msg64 = 0;
+	mbox_cmd.msg.s.type = LIO_MBOX_REQUEST;
+	mbox_cmd.msg.s.resp_needed = 0;
+	mbox_cmd.msg.s.cmd = LIO_VF_FLR_REQUEST;
+	mbox_cmd.msg.s.len = 1;
+	mbox_cmd.q_no = 0;
+	mbox_cmd.recv_len = 0;
+	mbox_cmd.recv_status = 0;
+	mbox_cmd.fn = NULL;
+	mbox_cmd.fn_arg = 0;
+
+	lio_mbox_write(lio_dev, &mbox_cmd);
+}
+
 static void
 cn23xx_pfvf_hs_callback(struct lio_device *lio_dev,
 			struct lio_mbox_cmd *cmd, void *arg)
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.h b/drivers/net/liquidio/base/lio_23xx_vf.h
index 83dc053..ad8db0d 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.h
+++ b/drivers/net/liquidio/base/lio_23xx_vf.h
@@ -87,6 +87,8 @@
 
 #define CN23XX_VF_BUSY_READING_REG_LOOP_COUNT	100000
 
+void cn23xx_vf_ask_pf_to_do_flr(struct lio_device *lio_dev);
+
 int cn23xx_pfvf_handshake(struct lio_device *lio_dev);
 
 int cn23xx_vf_setup_device(struct lio_device  *lio_dev);
diff --git a/drivers/net/liquidio/base/lio_mbox.h b/drivers/net/liquidio/base/lio_mbox.h
index f1c5b8e..b0875d6 100644
--- a/drivers/net/liquidio/base/lio_mbox.h
+++ b/drivers/net/liquidio/base/lio_mbox.h
@@ -43,6 +43,7 @@
 #define LIO_MBOX_DATA_MAX			32
 
 #define LIO_VF_ACTIVE				0x1
+#define LIO_VF_FLR_REQUEST			0x2
 #define LIO_CORES_CRASHED			0x3
 
 /* Macro for Read acknowledgment */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 7318bf5..23dfa43 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -112,6 +112,11 @@
 	if (cn23xx_pfvf_handshake(lio_dev))
 		goto error;
 
+	/* Initial reset */
+	cn23xx_vf_ask_pf_to_do_flr(lio_dev);
+	/* Wait for FLR for 100ms per SRIOV specification */
+	rte_delay_ms(100);
+
 	if (cn23xx_vf_set_io_queues_off(lio_dev)) {
 		lio_dev_err(lio_dev, "Setting io queues off failed\n");
 		goto error;
-- 
1.8.3.1

[PATCH 16/50] net/liquidio: add APIs to allocate and free IQ

[Shijith Thotton]

Add APIs to allocate and free instruction queue. Allocates instruction
queue 0 initially to send device configurations commands and later re-
allocates as per application requirement during tx queue setup.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/Makefile           |   1 +
 drivers/net/liquidio/base/lio_hw_defs.h |  12 ++
 drivers/net/liquidio/lio_ethdev.c       |   8 ++
 drivers/net/liquidio/lio_rxtx.c         | 206 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         |  57 +++++++++
 drivers/net/liquidio/lio_struct.h       | 112 ++++++++++++++++-
 6 files changed, 395 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/liquidio/lio_rxtx.c
 create mode 100644 drivers/net/liquidio/lio_rxtx.h

diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
index 451f49d..de2ef9b 100644
--- a/drivers/net/liquidio/Makefile
+++ b/drivers/net/liquidio/Makefile
@@ -51,6 +51,7 @@ VPATH += $(RTE_SDK)/drivers/net/liquidio/base
 # all source are stored in SRCS-y
 #
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_ethdev.c
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_rxtx.c
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_23xx_vf.c
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_mbox.c
 
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 890ef17..2262de3 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -76,6 +76,18 @@ enum lio_card_type {
 
 #define LIO_23XX_NAME "23xx"
 
+#define LIO_NUM_DEF_TX_DESCS_CFG(cfg)					\
+		((cfg)->default_config->num_def_tx_descs)
+
+#define LIO_IQ_INSTR_TYPE(cfg)		((cfg)->default_config->iq.instr_type)
+
+/* The following config values are fixed and should not be modified. */
+
+/* Maximum number of Instruction queues */
+#define LIO_MAX_INSTR_QUEUES(lio_dev)		CN23XX_MAX_RINGS_PER_VF
+
+#define LIO_MAX_POSSIBLE_INSTR_QUEUES		CN23XX_MAX_INPUT_QUEUES
+
 #define LIO_DEVICE_NAME_LEN		32
 #define LIO_BASE_MAJOR_VERSION		1
 #define LIO_BASE_MINOR_VERSION		5
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 23dfa43..3b11993 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -39,6 +39,7 @@
 #include "lio_logs.h"
 #include "lio_23xx_vf.h"
 #include "lio_ethdev.h"
+#include "lio_rxtx.h"
 
 static void
 lio_check_pf_hs_response(void *lio_dev)
@@ -127,6 +128,11 @@
 		goto error;
 	}
 
+	if (lio_setup_instr_queue0(lio_dev)) {
+		lio_dev_err(lio_dev, "Failed to setup instruction queue 0\n");
+		goto error;
+	}
+
 	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
 
 	lio_dev->max_tx_queues = dpdk_queues;
@@ -137,6 +143,8 @@
 error:
 	if (lio_dev->mbox[0])
 		lio_dev->fn_list.free_mbox(lio_dev);
+	if (lio_dev->instr_queue[0])
+		lio_free_instr_queue0(lio_dev);
 
 	return -1;
 }
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
new file mode 100644
index 0000000..de98fc6
--- /dev/null
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -0,0 +1,206 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+
+#include "lio_logs.h"
+#include "lio_struct.h"
+#include "lio_ethdev.h"
+#include "lio_rxtx.h"
+
+static void
+lio_dma_zone_free(struct lio_device *lio_dev, const struct rte_memzone *mz)
+{
+	const struct rte_memzone *mz_tmp;
+	int ret = 0;
+
+	if (mz == NULL) {
+		lio_dev_err(lio_dev, "Memzone NULL\n");
+		return;
+	}
+
+	mz_tmp = rte_memzone_lookup(mz->name);
+	if (mz_tmp == NULL) {
+		lio_dev_err(lio_dev, "Memzone %s Not Found\n", mz->name);
+		return;
+	}
+
+	ret = rte_memzone_free(mz);
+	if (ret)
+		lio_dev_err(lio_dev, "Memzone free Failed ret %d\n", ret);
+}
+
+/**
+ *  lio_init_instr_queue()
+ *  @param lio_dev	- pointer to the lio device structure.
+ *  @param txpciq	- queue to be initialized.
+ *
+ *  Called at driver init time for each input queue. iq_conf has the
+ *  configuration parameters for the queue.
+ *
+ *  @return  Success: 0	Failure: -1
+ */
+static int
+lio_init_instr_queue(struct lio_device *lio_dev,
+		     union octeon_txpciq txpciq,
+		     uint32_t num_descs, unsigned int socket_id)
+{
+	uint32_t iq_no = (uint32_t)txpciq.s.q_no;
+	struct lio_instr_queue *iq;
+	uint32_t instr_type;
+	uint32_t q_size;
+
+	instr_type = LIO_IQ_INSTR_TYPE(lio_dev);
+
+	q_size = instr_type * num_descs;
+	iq = lio_dev->instr_queue[iq_no];
+	iq->iq_mz = rte_eth_dma_zone_reserve(lio_dev->eth_dev,
+					     "instr_queue", iq_no, q_size,
+					     RTE_CACHE_LINE_SIZE,
+					     socket_id);
+	if (iq->iq_mz == NULL) {
+		lio_dev_err(lio_dev, "Cannot allocate memory for instr queue %d\n",
+			    iq_no);
+		return -1;
+	}
+
+	iq->base_addr_dma = iq->iq_mz->phys_addr;
+	iq->base_addr = (uint8_t *)iq->iq_mz->addr;
+
+	iq->max_count = num_descs;
+
+	/* Initialize a list to holds requests that have been posted to Octeon
+	 * but has yet to be fetched by octeon
+	 */
+	iq->request_list = rte_zmalloc_socket("request_list",
+					      sizeof(*iq->request_list) *
+							num_descs,
+					      RTE_CACHE_LINE_SIZE,
+					      socket_id);
+	if (iq->request_list == NULL) {
+		lio_dev_err(lio_dev, "Alloc failed for IQ[%d] nr free list\n",
+			    iq_no);
+		lio_dma_zone_free(lio_dev, iq->iq_mz);
+		return -1;
+	}
+
+	lio_dev_dbg(lio_dev, "IQ[%d]: base: %p basedma: %lx count: %d\n",
+		    iq_no, iq->base_addr, (unsigned long)iq->base_addr_dma,
+		    iq->max_count);
+
+	iq->lio_dev = lio_dev;
+	iq->txpciq.txpciq64 = txpciq.txpciq64;
+	iq->fill_cnt = 0;
+	iq->host_write_index = 0;
+	iq->lio_read_index = 0;
+	iq->flush_index = 0;
+
+	rte_atomic64_set(&iq->instr_pending, 0);
+
+	/* Initialize the spinlock for this instruction queue */
+	rte_spinlock_init(&iq->lock);
+	rte_spinlock_init(&iq->post_lock);
+
+	rte_atomic64_clear(&iq->iq_flush_running);
+
+	lio_dev->io_qmask.iq |= (1ULL << iq_no);
+
+	/* Set the 32B/64B mode for each input queue */
+	lio_dev->io_qmask.iq64B |= ((instr_type == 64) << iq_no);
+	iq->iqcmd_64B = (instr_type == 64);
+
+	return 0;
+}
+
+int
+lio_setup_instr_queue0(struct lio_device *lio_dev)
+{
+	union octeon_txpciq txpciq;
+	uint32_t num_descs = 0;
+	uint32_t iq_no = 0;
+
+	num_descs = LIO_NUM_DEF_TX_DESCS_CFG(lio_dev);
+
+	lio_dev->num_iqs = 0;
+
+	lio_dev->instr_queue[0] = rte_zmalloc(NULL,
+					sizeof(struct lio_instr_queue), 0);
+	if (lio_dev->instr_queue[0] == NULL)
+		return -ENOMEM;
+
+	lio_dev->instr_queue[0]->q_index = 0;
+	lio_dev->instr_queue[0]->app_ctx = (void *)(size_t)0;
+	txpciq.txpciq64 = 0;
+	txpciq.s.q_no = iq_no;
+	txpciq.s.pkind = lio_dev->pfvf_hsword.pkind;
+	txpciq.s.use_qpg = 0;
+	txpciq.s.qpg = 0;
+	if (lio_init_instr_queue(lio_dev, txpciq, num_descs, SOCKET_ID_ANY)) {
+		rte_free(lio_dev->instr_queue[0]);
+		lio_dev->instr_queue[0] = NULL;
+		return -1;
+	}
+
+	lio_dev->num_iqs++;
+
+	return 0;
+}
+
+/**
+ *  lio_delete_instr_queue()
+ *  @param lio_dev	- pointer to the lio device structure.
+ *  @param iq_no	- queue to be deleted.
+ *
+ *  Called at driver unload time for each input queue. Deletes all
+ *  allocated resources for the input queue.
+ */
+static void
+lio_delete_instr_queue(struct lio_device *lio_dev, uint32_t iq_no)
+{
+	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
+
+	rte_free(iq->request_list);
+	iq->request_list = NULL;
+	lio_dma_zone_free(lio_dev, iq->iq_mz);
+}
+
+void
+lio_free_instr_queue0(struct lio_device *lio_dev)
+{
+	lio_delete_instr_queue(lio_dev, 0);
+	rte_free(lio_dev->instr_queue[0]);
+	lio_dev->instr_queue[0] = NULL;
+	lio_dev->num_iqs--;
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
new file mode 100644
index 0000000..33f178b
--- /dev/null
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -0,0 +1,57 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_RXTX_H_
+#define _LIO_RXTX_H_
+
+#include <stdio.h>
+#include <stdint.h>
+
+#include <rte_spinlock.h>
+#include <rte_memory.h>
+
+#include "lio_struct.h"
+
+struct lio_request_list {
+	uint32_t reqtype;
+	void *buf;
+};
+
+/** Setup instruction queue zero for the device
+ *  @param lio_dev which lio device to setup
+ *
+ *  @return 0 if success. -1 if fails
+ */
+int lio_setup_instr_queue0(struct lio_device *lio_dev);
+void lio_free_instr_queue0(struct lio_device *lio_dev);
+#endif	/* _LIO_RXTX_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index e8b6e1d..29059a5 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -50,7 +50,110 @@ struct lio_version {
 	uint16_t reserved;
 };
 
-struct lio_device;
+/** The txpciq info passed to host from the firmware */
+union octeon_txpciq {
+	uint64_t txpciq64;
+
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t q_no : 8;
+		uint64_t port : 8;
+		uint64_t pkind : 6;
+		uint64_t use_qpg : 1;
+		uint64_t qpg : 11;
+		uint64_t aura_num : 10;
+		uint64_t reserved : 20;
+#else
+		uint64_t reserved : 20;
+		uint64_t aura_num : 10;
+		uint64_t qpg : 11;
+		uint64_t use_qpg : 1;
+		uint64_t pkind : 6;
+		uint64_t port : 8;
+		uint64_t q_no : 8;
+#endif
+	} s;
+};
+
+/** The instruction (input) queue.
+ *  The input queue is used to post raw (instruction) mode data or packet
+ *  data to Octeon device from the host. Each input queue for
+ *  a LIO device has one such structure to represent it.
+ */
+struct lio_instr_queue {
+	/** A spinlock to protect access to the input ring.  */
+	rte_spinlock_t lock;
+
+	rte_spinlock_t post_lock;
+
+	struct lio_device *lio_dev;
+
+	uint32_t pkt_in_done;
+
+	rte_atomic64_t iq_flush_running;
+
+	/** Flag that indicates if the queue uses 64 byte commands. */
+	uint32_t iqcmd_64B:1;
+
+	/** Queue info. */
+	union octeon_txpciq txpciq;
+
+	uint32_t rsvd:17;
+
+	uint32_t status:8;
+
+	/** Maximum no. of instructions in this queue. */
+	uint32_t max_count;
+
+	/** Index in input ring where the driver should write the next packet */
+	uint32_t host_write_index;
+
+	/** Index in input ring where Octeon is expected to read the next
+	 *  packet.
+	 */
+	uint32_t lio_read_index;
+
+	/** This index aids in finding the window in the queue where Octeon
+	 *  has read the commands.
+	 */
+	uint32_t flush_index;
+
+	/** This field keeps track of the instructions pending in this queue. */
+	rte_atomic64_t instr_pending;
+
+	/** Pointer to the Virtual Base addr of the input ring. */
+	uint8_t *base_addr;
+
+	struct lio_request_list *request_list;
+
+	/** Octeon doorbell register for the ring. */
+	void *doorbell_reg;
+
+	/** Octeon instruction count register for this ring. */
+	void *inst_cnt_reg;
+
+	/** Number of instructions pending to be posted to Octeon. */
+	uint32_t fill_cnt;
+
+	/** DMA mapped base address of the input descriptor ring. */
+	uint64_t base_addr_dma;
+
+	/** Application context */
+	void *app_ctx;
+
+	/* network stack queue index */
+	int q_index;
+
+	/* Memory zone */
+	const struct rte_memzone *iq_mz;
+};
+
+struct lio_io_enable {
+	uint64_t iq;
+	uint64_t oq;
+	uint64_t iq64B;
+};
+
 struct lio_fn_list {
 	int (*setup_mbox)(struct lio_device *);
 	void (*free_mbox)(struct lio_device *);
@@ -170,6 +273,13 @@ struct lio_device {
 
 	struct lio_fn_list fn_list;
 
+	uint32_t num_iqs;
+
+	/** The input instruction queues */
+	struct lio_instr_queue *instr_queue[LIO_MAX_POSSIBLE_INSTR_QUEUES];
+
+	struct lio_io_enable io_qmask;
+
 	struct lio_sriov_info sriov_info;
 
 	struct lio_pf_vf_hs_word pfvf_hsword;
-- 
1.8.3.1

Re: [PATCH 16/50] net/liquidio: add APIs to allocate and free IQ

[Ferruh Yigit]

On 2/21/2017 9:26 AM, Shijith Thotton wrote:
> Add APIs to allocate and free instruction queue. Allocates instruction
> queue 0 initially to send device configurations commands and later re-
> allocates as per application requirement during tx queue setup.

Can you please describe the relevant entity briefly in commit log, even
one sentences is enough.

For this patch: What is Instruction Queue (IQ)

For patch 18/50: What is "soft command pool" ?

etc...


> 
> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
> ---
<...>

Re: [PATCH 16/50] net/liquidio: add APIs to allocate and free IQ

[Shijith Thotton]

On Thursday 23 February 2017 08:00 PM, Ferruh Yigit wrote:
> On 2/21/2017 9:26 AM, Shijith Thotton wrote:
>> Add APIs to allocate and free instruction queue. Allocates instruction
>> queue 0 initially to send device configurations commands and later re-
>> allocates as per application requirement during tx queue setup.
> 
> Can you please describe the relevant entity briefly in commit log, even
> one sentences is enough.
> 
> For this patch: What is Instruction Queue (IQ)
> 
> For patch 18/50: What is "soft command pool" ?
> 
> etc...

Sure. Will make commit logs more clear.

> 
> 
>>
>> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
>> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
>> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
>> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
>> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
>> ---
> <...>
> 

[PATCH 17/50] net/liquidio: add API to setup instruction queue

[Shijith Thotton]

Maps instruction(input) queue registers and sets queue size.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 44 ++++++++++++++++++++++++++++++++-
 drivers/net/liquidio/lio_rxtx.c         |  2 ++
 drivers/net/liquidio/lio_struct.h       |  2 ++
 3 files changed, 47 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index ed5b830..181f830 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -199,6 +199,40 @@
 }
 
 static void
+cn23xx_vf_setup_iq_regs(struct lio_device *lio_dev, uint32_t iq_no)
+{
+	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
+	uint64_t pkt_in_done = 0;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* Write the start of the input queue's ring and its size */
+	lio_write_csr64(lio_dev, CN23XX_SLI_IQ_BASE_ADDR64(iq_no),
+			iq->base_addr_dma);
+	lio_write_csr(lio_dev, CN23XX_SLI_IQ_SIZE(iq_no), iq->max_count);
+
+	/* Remember the doorbell & instruction count register addr
+	 * for this queue
+	 */
+	iq->doorbell_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_SLI_IQ_DOORBELL(iq_no);
+	iq->inst_cnt_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_SLI_IQ_INSTR_COUNT64(iq_no);
+	lio_dev_dbg(lio_dev, "InstQ[%d]:dbell reg @ 0x%p instcnt_reg @ 0x%p\n",
+		    iq_no, iq->doorbell_reg, iq->inst_cnt_reg);
+
+	/* Store the current instruction counter (used in flush_iq
+	 * calculation)
+	 */
+	pkt_in_done = rte_read64(iq->inst_cnt_reg);
+
+	/* Clear the count by writing back what we read, but don't
+	 * enable data traffic here
+	 */
+	rte_write64(pkt_in_done, iq->inst_cnt_reg);
+}
+
+static void
 cn23xx_vf_free_mbox(struct lio_device *lio_dev)
 {
 	PMD_INIT_FUNC_TRACE();
@@ -298,8 +332,8 @@
 {
 	struct lio_mbox_cmd mbox_cmd;
 	struct lio_version *lio_ver = (struct lio_version *)&mbox_cmd.data[0];
+	uint32_t q_no, count = 0;
 	rte_atomic64_t status;
-	uint32_t count = 0;
 	uint32_t pfmajor;
 	uint32_t vfmajor;
 	uint32_t ret;
@@ -341,6 +375,10 @@
 		return -1;
 	}
 
+	for (q_no = 0; q_no < lio_dev->num_iqs; q_no++)
+		lio_dev->instr_queue[q_no]->txpciq.s.pkind =
+						lio_dev->pfvf_hsword.pkind;
+
 	vfmajor = LIO_BASE_MAJOR_VERSION;
 	pfmajor = ret >> 16;
 	if (pfmajor != vfmajor) {
@@ -355,6 +393,9 @@
 		ret = 0;
 	}
 
+	lio_dev_dbg(lio_dev, "got data from PF pkind is %d\n",
+		    lio_dev->pfvf_hsword.pkind);
+
 	return ret;
 }
 
@@ -394,6 +435,7 @@
 	if (lio_dev->default_config == NULL)
 		return -1;
 
+	lio_dev->fn_list.setup_iq_regs		= cn23xx_vf_setup_iq_regs;
 	lio_dev->fn_list.setup_mbox		= cn23xx_vf_setup_mbox;
 	lio_dev->fn_list.free_mbox		= cn23xx_vf_free_mbox;
 
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index de98fc6..4a687d8 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -141,6 +141,8 @@
 	lio_dev->io_qmask.iq64B |= ((instr_type == 64) << iq_no);
 	iq->iqcmd_64B = (instr_type == 64);
 
+	lio_dev->fn_list.setup_iq_regs(lio_dev, iq_no);
+
 	return 0;
 }
 
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 29059a5..2806c37 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -155,6 +155,8 @@ struct lio_io_enable {
 };
 
 struct lio_fn_list {
+	void (*setup_iq_regs)(struct lio_device *, uint32_t);
+
 	int (*setup_mbox)(struct lio_device *);
 	void (*free_mbox)(struct lio_device *);
 
-- 
1.8.3.1

[PATCH 18/50] net/liquidio: add API to allocate and free command pool

[Shijith Thotton]

Allocate soft command pool. It is used to allocate control packet
buffers.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 12 ++++++++++++
 drivers/net/liquidio/lio_rxtx.c   | 21 +++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  9 +++++++++
 drivers/net/liquidio/lio_struct.h |  3 +++
 4 files changed, 45 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 3b11993..3702c61 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -101,6 +101,12 @@
 		return -1;
 	}
 
+	/* Initialize soft command buffer pool */
+	if (lio_setup_sc_buffer_pool(lio_dev)) {
+		lio_dev_err(lio_dev, "sc buffer pool allocation failed\n");
+		return -1;
+	}
+
 	if (lio_dev->fn_list.setup_mbox(lio_dev)) {
 		lio_dev_err(lio_dev, "Mailbox setup failed\n");
 		goto error;
@@ -141,6 +147,7 @@
 	return 0;
 
 error:
+	lio_free_sc_buffer_pool(lio_dev);
 	if (lio_dev->mbox[0])
 		lio_dev->fn_list.free_mbox(lio_dev);
 	if (lio_dev->instr_queue[0])
@@ -152,11 +159,16 @@
 static int
 lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
 {
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
+	/* lio_free_sc_buffer_pool */
+	lio_free_sc_buffer_pool(lio_dev);
+
 	rte_free(eth_dev->data->mac_addrs);
 	eth_dev->data->mac_addrs = NULL;
 
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 4a687d8..1c6ce59 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -206,3 +206,24 @@
 	lio_dev->instr_queue[0] = NULL;
 	lio_dev->num_iqs--;
 }
+
+int
+lio_setup_sc_buffer_pool(struct lio_device *lio_dev)
+{
+	char sc_pool_name[RTE_MEMPOOL_NAMESIZE];
+	uint16_t buf_size;
+
+	buf_size = LIO_SOFT_COMMAND_BUFFER_SIZE + RTE_PKTMBUF_HEADROOM;
+	snprintf(sc_pool_name, sizeof(sc_pool_name),
+		 "lio_sc_pool_%u", lio_dev->port_id);
+	lio_dev->sc_buf_pool = rte_pktmbuf_pool_create(sc_pool_name,
+						LIO_MAX_SOFT_COMMAND_BUFFERS,
+						0, 0, buf_size, SOCKET_ID_ANY);
+	return 0;
+}
+
+void
+lio_free_sc_buffer_pool(struct lio_device *lio_dev)
+{
+	rte_mempool_free(lio_dev->sc_buf_pool);
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 33f178b..b308211 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -47,6 +47,15 @@ struct lio_request_list {
 	void *buf;
 };
 
+/** The size of each buffer in soft command buffer pool */
+#define LIO_SOFT_COMMAND_BUFFER_SIZE	1536
+
+/** Maximum number of buffers to allocate into soft command buffer pool */
+#define LIO_MAX_SOFT_COMMAND_BUFFERS	255
+
+int lio_setup_sc_buffer_pool(struct lio_device *lio_dev);
+void lio_free_sc_buffer_pool(struct lio_device *lio_dev);
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 2806c37..992ad39 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -277,6 +277,9 @@ struct lio_device {
 
 	uint32_t num_iqs;
 
+	/* The pool containing pre allocated buffers used for soft commands */
+	struct rte_mempool *sc_buf_pool;
+
 	/** The input instruction queues */
 	struct lio_instr_queue *instr_queue[LIO_MAX_POSSIBLE_INSTR_QUEUES];
 
-- 
1.8.3.1

[PATCH 19/50] net/liquidio: add API to allocate and free soft command

[Shijith Thotton]

Add API to create command using buffers from soft command buffer pool.
Buffers are freed to the pool once the command reaches device.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_rxtx.c   | 66 +++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   | 38 ++++++++++++++++++++++
 drivers/net/liquidio/lio_struct.h |  6 ++++
 3 files changed, 110 insertions(+)

diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 1c6ce59..cfec96d 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -227,3 +227,69 @@
 {
 	rte_mempool_free(lio_dev->sc_buf_pool);
 }
+
+struct lio_soft_command *
+lio_alloc_soft_command(struct lio_device *lio_dev, uint32_t datasize,
+		       uint32_t rdatasize, uint32_t ctxsize)
+{
+	uint32_t offset = sizeof(struct lio_soft_command);
+	struct lio_soft_command *sc;
+	struct rte_mbuf *m;
+	uint64_t dma_addr;
+
+	RTE_ASSERT((offset + datasize + rdatasize + ctxsize) <=
+		   LIO_SOFT_COMMAND_BUFFER_SIZE);
+
+	m = rte_pktmbuf_alloc(lio_dev->sc_buf_pool);
+	if (m == NULL) {
+		lio_dev_err(lio_dev, "Cannot allocate mbuf for sc\n");
+		return NULL;
+	}
+
+	/* set rte_mbuf data size and there is only 1 segment */
+	m->pkt_len = LIO_SOFT_COMMAND_BUFFER_SIZE;
+	m->data_len = LIO_SOFT_COMMAND_BUFFER_SIZE;
+
+	/* use rte_mbuf buffer for soft command */
+	sc = rte_pktmbuf_mtod(m, struct lio_soft_command *);
+	memset(sc, 0, LIO_SOFT_COMMAND_BUFFER_SIZE);
+	sc->size = LIO_SOFT_COMMAND_BUFFER_SIZE;
+	sc->dma_addr = rte_mbuf_data_dma_addr(m);
+	sc->mbuf = m;
+
+	dma_addr = sc->dma_addr;
+
+	if (ctxsize) {
+		sc->ctxptr = (uint8_t *)sc + offset;
+		sc->ctxsize = ctxsize;
+	}
+
+	/* Start data at 128 byte boundary */
+	offset = (offset + ctxsize + 127) & 0xffffff80;
+
+	if (datasize) {
+		sc->virtdptr = (uint8_t *)sc + offset;
+		sc->dmadptr = dma_addr + offset;
+		sc->datasize = datasize;
+	}
+
+	/* Start rdata at 128 byte boundary */
+	offset = (offset + datasize + 127) & 0xffffff80;
+
+	if (rdatasize) {
+		RTE_ASSERT(rdatasize >= 16);
+		sc->virtrptr = (uint8_t *)sc + offset;
+		sc->dmarptr = dma_addr + offset;
+		sc->rdatasize = rdatasize;
+		sc->status_word = (uint64_t *)((uint8_t *)(sc->virtrptr) +
+					       rdatasize - 8);
+	}
+
+	return sc;
+}
+
+void
+lio_free_soft_command(struct lio_soft_command *sc)
+{
+	rte_pktmbuf_free(sc->mbuf);
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index b308211..8dab73e 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -53,9 +53,47 @@ struct lio_request_list {
 /** Maximum number of buffers to allocate into soft command buffer pool */
 #define LIO_MAX_SOFT_COMMAND_BUFFERS	255
 
+struct lio_soft_command {
+	/** Soft command buffer info. */
+	struct lio_clist_node node;
+	uint64_t dma_addr;
+	uint32_t size;
+
+#define LIO_COMPLETION_WORD_INIT	0xffffffffffffffffULL
+	uint64_t *status_word;
+
+	/** Data buffer info */
+	void *virtdptr;
+	uint64_t dmadptr;
+	uint32_t datasize;
+
+	/** Return buffer info */
+	void *virtrptr;
+	uint64_t dmarptr;
+	uint32_t rdatasize;
+
+	/** Context buffer info */
+	void *ctxptr;
+	uint32_t ctxsize;
+
+	/** Time out and callback */
+	size_t wait_time;
+	size_t timeout;
+	uint32_t iq_no;
+	void (*callback)(uint32_t, void *);
+	void *callback_arg;
+	struct rte_mbuf *mbuf;
+};
+
 int lio_setup_sc_buffer_pool(struct lio_device *lio_dev);
 void lio_free_sc_buffer_pool(struct lio_device *lio_dev);
 
+struct lio_soft_command *
+lio_alloc_soft_command(struct lio_device *lio_dev,
+		       uint32_t datasize, uint32_t rdatasize,
+		       uint32_t ctxsize);
+void lio_free_soft_command(struct lio_soft_command *sc);
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 992ad39..a06726b 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -43,6 +43,12 @@
 
 #include "lio_hw_defs.h"
 
+struct lio_clist_node {
+	CIRCLEQ_ENTRY(lio_clist_node) entries;
+};
+
+CIRCLEQ_HEAD(lio_clist_head, lio_clist_node);
+
 struct lio_version {
 	uint16_t major;
 	uint16_t minor;
-- 
1.8.3.1

[PATCH 20/50] net/liquidio: add APIs for response list

[Shijith Thotton]

Add API to setup and process response list. It is used to wait for
response from device.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c |   5 ++
 drivers/net/liquidio/lio_rxtx.c   | 106 ++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  73 ++++++++++++++++++++++++++
 drivers/net/liquidio/lio_struct.h |  14 +++++
 4 files changed, 198 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 3702c61..a6813e0 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -107,6 +107,11 @@
 		return -1;
 	}
 
+	/* Initialize lists to manage the requests of different types that
+	 * arrive from applications for this lio device.
+	 */
+	lio_setup_response_list(lio_dev);
+
 	if (lio_dev->fn_list.setup_mbox(lio_dev)) {
 		lio_dev_err(lio_dev, "Mailbox setup failed\n");
 		goto error;
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index cfec96d..515ba79 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -293,3 +293,109 @@ struct lio_soft_command *
 {
 	rte_pktmbuf_free(sc->mbuf);
 }
+
+void
+lio_setup_response_list(struct lio_device *lio_dev)
+{
+	CIRCLEQ_INIT(&lio_dev->response_list.head);
+	rte_spinlock_init(&lio_dev->response_list.lock);
+	rte_atomic64_set(&lio_dev->response_list.pending_req_count, 0);
+}
+
+int
+lio_process_ordered_list(struct lio_device *lio_dev)
+{
+	int resp_to_process = LIO_MAX_ORD_REQS_TO_PROCESS;
+	struct lio_response_list *ordered_sc_list;
+	struct lio_soft_command *sc;
+	int request_complete = 0;
+	uint64_t status64;
+	uint32_t status;
+
+	ordered_sc_list = &lio_dev->response_list;
+
+	do {
+		rte_spinlock_lock(&ordered_sc_list->lock);
+
+		if (CIRCLEQ_EMPTY(&ordered_sc_list->head)) {
+			/* ordered_sc_list is empty; there is
+			 * nothing to process
+			 */
+			rte_spinlock_unlock(&ordered_sc_list->lock);
+			return -1;
+		}
+
+		sc = LIO_CLIST_FIRST_ENTRY(&ordered_sc_list->head,
+					   struct lio_soft_command, node);
+
+		status = LIO_REQUEST_PENDING;
+
+		/* check if octeon has finished DMA'ing a response
+		 * to where rptr is pointing to
+		 */
+		status64 = *sc->status_word;
+
+		if (status64 != LIO_COMPLETION_WORD_INIT) {
+			/* This logic ensures that all 64b have been written.
+			 * 1. check byte 0 for non-FF
+			 * 2. if non-FF, then swap result from BE to host order
+			 * 3. check byte 7 (swapped to 0) for non-FF
+			 * 4. if non-FF, use the low 32-bit status code
+			 * 5. if either byte 0 or byte 7 is FF, don't use status
+			 */
+			if ((status64 & 0xff) != 0xff) {
+				lio_swap_8B_data(&status64, 1);
+				if (((status64 & 0xff) != 0xff)) {
+					/* retrieve 16-bit firmware status */
+					status = (uint32_t)(status64 &
+							    0xffffULL);
+					if (status) {
+						status =
+						LIO_FIRMWARE_STATUS_CODE(
+									status);
+					} else {
+						/* i.e. no error */
+						status = LIO_REQUEST_DONE;
+					}
+				}
+			}
+		} else if ((sc->timeout && lio_check_timeout(lio_uptime,
+							     sc->timeout))) {
+			lio_dev_err(lio_dev,
+				    "cmd failed, timeout (%ld, %ld)\n",
+				    (long)lio_uptime, (long)sc->timeout);
+			status = LIO_REQUEST_TIMEOUT;
+		}
+
+		if (status != LIO_REQUEST_PENDING) {
+			/* we have received a response or we have timed out.
+			 * remove node from linked list
+			 */
+			CIRCLEQ_REMOVE(&ordered_sc_list->head,
+				       &sc->node, entries);
+			rte_atomic64_dec(
+			    &lio_dev->response_list.pending_req_count);
+			rte_spinlock_unlock(&ordered_sc_list->lock);
+
+			if (sc->callback)
+				sc->callback(status, sc->callback_arg);
+			request_complete++;
+
+		} else {
+			/* no response yet */
+			request_complete = 0;
+			rte_spinlock_unlock(&ordered_sc_list->lock);
+		}
+
+		/* If we hit the Max Ordered requests to process every loop,
+		 * we quit and let this function be invoked the next time
+		 * the poll thread runs to process the remaining requests.
+		 * This function can take up the entire CPU if there is
+		 * no upper limit to the requests processed.
+		 */
+		if (request_complete >= resp_to_process)
+			break;
+	} while (request_complete);
+
+	return 0;
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 8dab73e..58e2a6e 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -42,6 +42,14 @@
 
 #include "lio_struct.h"
 
+#define LIO_CLIST_FIRST_ENTRY(ptr, type, elem)	\
+	(type *)((char *)((ptr)->cqh_first) - offsetof(type, elem))
+
+#define lio_check_timeout(cur_time, chk_time) ((cur_time) > (chk_time))
+
+#define lio_uptime		\
+	(size_t)(rte_get_timer_cycles() / rte_get_timer_hz())
+
 struct lio_request_list {
 	uint32_t reqtype;
 	void *buf;
@@ -94,6 +102,71 @@ struct lio_soft_command *
 		       uint32_t ctxsize);
 void lio_free_soft_command(struct lio_soft_command *sc);
 
+/** Maximum ordered requests to process in every invocation of
+ *  lio_process_ordered_list(). The function will continue to process requests
+ *  as long as it can find one that has finished processing. If it keeps
+ *  finding requests that have completed, the function can run for ever. The
+ *  value defined here sets an upper limit on the number of requests it can
+ *  process before it returns control to the poll thread.
+ */
+#define LIO_MAX_ORD_REQS_TO_PROCESS	4096
+
+/** Error codes used in Octeon Host-Core communication.
+ *
+ *   31		16 15		0
+ *   ----------------------------
+ * |		|		|
+ *   ----------------------------
+ *   Error codes are 32-bit wide. The upper 16-bits, called Major Error Number,
+ *   are reserved to identify the group to which the error code belongs. The
+ *   lower 16-bits, called Minor Error Number, carry the actual code.
+ *
+ *   So error codes are (MAJOR NUMBER << 16)| MINOR_NUMBER.
+ */
+/** Status for a request.
+ *  If the request is successfully queued, the driver will return
+ *  a LIO_REQUEST_PENDING status. LIO_REQUEST_TIMEOUT is only returned by
+ *  the driver if the response for request failed to arrive before a
+ *  time-out period or if the request processing * got interrupted due to
+ *  a signal respectively.
+ */
+enum {
+	/** A value of 0x00000000 indicates no error i.e. success */
+	LIO_REQUEST_DONE	= 0x00000000,
+	/** (Major number: 0x0000; Minor Number: 0x0001) */
+	LIO_REQUEST_PENDING	= 0x00000001,
+	LIO_REQUEST_TIMEOUT	= 0x00000003,
+
+};
+
+/*------ Error codes used by firmware (bits 15..0 set by firmware */
+#define LIO_FIRMWARE_MAJOR_ERROR_CODE	 0x0001
+#define LIO_FIRMWARE_STATUS_CODE(status) \
+	((LIO_FIRMWARE_MAJOR_ERROR_CODE << 16) | (status))
+
+/** Initialize the response lists. The number of response lists to create is
+ *  given by count.
+ *  @param lio_dev - the lio device structure.
+ */
+void lio_setup_response_list(struct lio_device *lio_dev);
+
+/** Check the status of first entry in the ordered list. If the instruction at
+ *  that entry finished processing or has timed-out, the entry is cleaned.
+ *  @param lio_dev - the lio device structure.
+ *  @return 1 if the ordered list is empty, 0 otherwise.
+ */
+int lio_process_ordered_list(struct lio_device *lio_dev);
+
+static inline void
+lio_swap_8B_data(uint64_t *data, uint32_t blocks)
+{
+	while (blocks) {
+		*data = rte_cpu_to_be_64(*data);
+		blocks--;
+		data++;
+	}
+}
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index a06726b..ad67ed1 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -213,6 +213,17 @@ struct lio_sriov_info {
 	uint32_t num_vfs;
 };
 
+/* Head of a response list */
+struct lio_response_list {
+	/** List structure to add delete pending entries to */
+	struct lio_clist_head head;
+
+	/** A lock for this response list */
+	rte_spinlock_t lock;
+
+	rte_atomic64_t pending_req_count;
+};
+
 /* Structure to define the configuration attributes for each Input queue. */
 struct lio_iq_config {
 	/* Max number of IQs available */
@@ -289,6 +300,9 @@ struct lio_device {
 	/** The input instruction queues */
 	struct lio_instr_queue *instr_queue[LIO_MAX_POSSIBLE_INSTR_QUEUES];
 
+	/** The circular-linked list of instruction response */
+	struct lio_response_list response_list;
+
 	struct lio_io_enable io_qmask;
 
 	struct lio_sriov_info sriov_info;
-- 
1.8.3.1

[PATCH 21/50] net/liquidio: add APIs to send packet to device

[Shijith Thotton]

API to send control and data packets to device. Request list keeps
track of host buffers to be freed till it reaches device.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  11 ++
 drivers/net/liquidio/lio_rxtx.c         | 180 ++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         | 291 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_struct.h       |   6 +
 4 files changed, 488 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 2262de3..b581f44 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -76,6 +76,8 @@ enum lio_card_type {
 
 #define LIO_23XX_NAME "23xx"
 
+#define LIO_DEV_RUNNING		0xc
+
 #define LIO_NUM_DEF_TX_DESCS_CFG(cfg)					\
 		((cfg)->default_config->num_def_tx_descs)
 
@@ -93,6 +95,15 @@ enum lio_card_type {
 #define LIO_BASE_MINOR_VERSION		5
 #define LIO_BASE_MICRO_VERSION		1
 
+/** Tag types used by Octeon cores in its work. */
+enum octeon_tag_type {
+	OCTEON_ORDERED_TAG	= 0,
+	OCTEON_ATOMIC_TAG	= 1,
+};
+
+/* pre-defined host->NIC tag values */
+#define LIO_CONTROL	(0x11111110)
+
 /* Routines for reading and writing CSRs */
 #ifdef RTE_LIBRTE_LIO_DEBUG_REGS
 #define lio_write_csr(lio_dev, reg_off, value)				\
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 515ba79..a1fadfa 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -207,6 +207,186 @@
 	lio_dev->num_iqs--;
 }
 
+static inline void
+lio_ring_doorbell(struct lio_device *lio_dev,
+		  struct lio_instr_queue *iq)
+{
+	if (rte_atomic64_read(&lio_dev->status) == LIO_DEV_RUNNING) {
+		rte_write32(iq->fill_cnt, iq->doorbell_reg);
+		/* make sure doorbell write goes through */
+		rte_wmb();
+		iq->fill_cnt = 0;
+	}
+}
+
+static inline void
+copy_cmd_into_iq(struct lio_instr_queue *iq, uint8_t *cmd)
+{
+	uint8_t *iqptr, cmdsize;
+
+	cmdsize = ((iq->iqcmd_64B) ? 64 : 32);
+	iqptr = iq->base_addr + (cmdsize * iq->host_write_index);
+
+	rte_memcpy(iqptr, cmd, cmdsize);
+}
+
+static inline struct lio_iq_post_status
+post_command2(struct lio_instr_queue *iq, uint8_t *cmd)
+{
+	struct lio_iq_post_status st;
+
+	st.status = LIO_IQ_SEND_OK;
+
+	/* This ensures that the read index does not wrap around to the same
+	 * position if queue gets full before Octeon could fetch any instr.
+	 */
+	if (rte_atomic64_read(&iq->instr_pending) >=
+			(int32_t)(iq->max_count - 1)) {
+		st.status = LIO_IQ_SEND_FAILED;
+		st.index = -1;
+		return st;
+	}
+
+	if (rte_atomic64_read(&iq->instr_pending) >=
+			(int32_t)(iq->max_count - 2))
+		st.status = LIO_IQ_SEND_STOP;
+
+	copy_cmd_into_iq(iq, cmd);
+
+	/* "index" is returned, host_write_index is modified. */
+	st.index = iq->host_write_index;
+	iq->host_write_index = lio_incr_index(iq->host_write_index, 1,
+					      iq->max_count);
+	iq->fill_cnt++;
+
+	/* Flush the command into memory. We need to be sure the data is in
+	 * memory before indicating that the instruction is pending.
+	 */
+	rte_wmb();
+
+	rte_atomic64_inc(&iq->instr_pending);
+
+	return st;
+}
+
+static inline void
+lio_add_to_request_list(struct lio_instr_queue *iq,
+			int idx, void *buf, int reqtype)
+{
+	iq->request_list[idx].buf = buf;
+	iq->request_list[idx].reqtype = reqtype;
+}
+
+static int
+lio_send_command(struct lio_device *lio_dev, uint32_t iq_no, void *cmd,
+		 void *buf, uint32_t datasize __rte_unused, uint32_t reqtype)
+{
+	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
+	struct lio_iq_post_status st;
+
+	rte_spinlock_lock(&iq->post_lock);
+
+	st = post_command2(iq, cmd);
+
+	if (st.status != LIO_IQ_SEND_FAILED) {
+		lio_add_to_request_list(iq, st.index, buf, reqtype);
+		lio_ring_doorbell(lio_dev, iq);
+	}
+
+	rte_spinlock_unlock(&iq->post_lock);
+
+	return st.status;
+}
+
+void
+lio_prepare_soft_command(struct lio_device *lio_dev,
+			 struct lio_soft_command *sc, uint8_t opcode,
+			 uint8_t subcode, uint32_t irh_ossp, uint64_t ossp0,
+			 uint64_t ossp1)
+{
+	struct octeon_instr_pki_ih3 *pki_ih3;
+	struct octeon_instr_ih3 *ih3;
+	struct octeon_instr_irh *irh;
+	struct octeon_instr_rdp *rdp;
+
+	RTE_ASSERT(opcode <= 15);
+	RTE_ASSERT(subcode <= 127);
+
+	ih3	  = (struct octeon_instr_ih3 *)&sc->cmd.cmd3.ih3;
+
+	ih3->pkind = lio_dev->instr_queue[sc->iq_no]->txpciq.s.pkind;
+
+	pki_ih3 = (struct octeon_instr_pki_ih3 *)&sc->cmd.cmd3.pki_ih3;
+
+	pki_ih3->w	= 1;
+	pki_ih3->raw	= 1;
+	pki_ih3->utag	= 1;
+	pki_ih3->uqpg	= lio_dev->instr_queue[sc->iq_no]->txpciq.s.use_qpg;
+	pki_ih3->utt	= 1;
+
+	pki_ih3->tag	= LIO_CONTROL;
+	pki_ih3->tagtype = OCTEON_ATOMIC_TAG;
+	pki_ih3->qpg	= lio_dev->instr_queue[sc->iq_no]->txpciq.s.qpg;
+	pki_ih3->pm	= 0x7;
+	pki_ih3->sl	= 8;
+
+	if (sc->datasize)
+		ih3->dlengsz = sc->datasize;
+
+	irh		= (struct octeon_instr_irh *)&sc->cmd.cmd3.irh;
+	irh->opcode	= opcode;
+	irh->subcode	= subcode;
+
+	/* opcode/subcode specific parameters (ossp) */
+	irh->ossp = irh_ossp;
+	sc->cmd.cmd3.ossp[0] = ossp0;
+	sc->cmd.cmd3.ossp[1] = ossp1;
+
+	if (sc->rdatasize) {
+		rdp = (struct octeon_instr_rdp *)&sc->cmd.cmd3.rdp;
+		rdp->pcie_port = lio_dev->pcie_port;
+		rdp->rlen      = sc->rdatasize;
+		irh->rflag = 1;
+		/* PKI IH3 */
+		ih3->fsz    = OCTEON_SOFT_CMD_RESP_IH3;
+	} else {
+		irh->rflag = 0;
+		/* PKI IH3 */
+		ih3->fsz    = OCTEON_PCI_CMD_O3;
+	}
+}
+
+int
+lio_send_soft_command(struct lio_device *lio_dev,
+		      struct lio_soft_command *sc)
+{
+	struct octeon_instr_ih3 *ih3;
+	struct octeon_instr_irh *irh;
+	uint32_t len = 0;
+
+	ih3 = (struct octeon_instr_ih3 *)&sc->cmd.cmd3.ih3;
+	if (ih3->dlengsz) {
+		RTE_ASSERT(sc->dmadptr);
+		sc->cmd.cmd3.dptr = sc->dmadptr;
+	}
+
+	irh = (struct octeon_instr_irh *)&sc->cmd.cmd3.irh;
+	if (irh->rflag) {
+		RTE_ASSERT(sc->dmarptr);
+		RTE_ASSERT(sc->status_word != NULL);
+		*sc->status_word = LIO_COMPLETION_WORD_INIT;
+		sc->cmd.cmd3.rptr = sc->dmarptr;
+	}
+
+	len = (uint32_t)ih3->dlengsz;
+
+	if (sc->wait_time)
+		sc->timeout = lio_uptime + sc->wait_time;
+
+	return lio_send_command(lio_dev, sc->iq_no, &sc->cmd, sc, len,
+				LIO_REQTYPE_SOFT_COMMAND);
+}
+
 int
 lio_setup_sc_buffer_pool(struct lio_device *lio_dev)
 {
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 58e2a6e..d7b7194 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -50,11 +50,53 @@
 #define lio_uptime		\
 	(size_t)(rte_get_timer_cycles() / rte_get_timer_hz())
 
+#define LIO_IQ_SEND_OK		0
+#define LIO_IQ_SEND_STOP	1
+#define LIO_IQ_SEND_FAILED	-1
+
+/* conditions */
+#define LIO_REQTYPE_NONE		0
+#define LIO_REQTYPE_NORESP_NET		1
+#define LIO_REQTYPE_NORESP_NET_SG	2
+#define LIO_REQTYPE_SOFT_COMMAND	3
+
 struct lio_request_list {
 	uint32_t reqtype;
 	void *buf;
 };
 
+/*----------------------  INSTRUCTION FORMAT ----------------------------*/
+
+struct lio_instr3_64B {
+	/** Pointer where the input data is available. */
+	uint64_t dptr;
+
+	/** Instruction Header. */
+	uint64_t ih3;
+
+	/** Instruction Header. */
+	uint64_t pki_ih3;
+
+	/** Input Request Header. */
+	uint64_t irh;
+
+	/** opcode/subcode specific parameters */
+	uint64_t ossp[2];
+
+	/** Return Data Parameters */
+	uint64_t rdp;
+
+	/** Pointer where the response for a RAW mode packet will be written
+	 *  by Octeon.
+	 */
+	uint64_t rptr;
+
+};
+
+union lio_instr_64B {
+	struct lio_instr3_64B cmd3;
+};
+
 /** The size of each buffer in soft command buffer pool */
 #define LIO_SOFT_COMMAND_BUFFER_SIZE	1536
 
@@ -67,6 +109,9 @@ struct lio_soft_command {
 	uint64_t dma_addr;
 	uint32_t size;
 
+	/** Command and return status */
+	union lio_instr_64B cmd;
+
 #define LIO_COMPLETION_WORD_INIT	0xffffffffffffffffULL
 	uint64_t *status_word;
 
@@ -93,6 +138,230 @@ struct lio_soft_command {
 	struct rte_mbuf *mbuf;
 };
 
+struct lio_iq_post_status {
+	int status;
+	int index;
+};
+
+/*   wqe
+ *  ---------------  0
+ * |  wqe  word0-3 |
+ *  ---------------  32
+ * |    PCI IH     |
+ *  ---------------  40
+ * |     RPTR      |
+ *  ---------------  48
+ * |    PCI IRH    |
+ *  ---------------  56
+ * |    OCTEON_CMD |
+ *  ---------------  64
+ * | Addtl 8-BData |
+ * |               |
+ *  ---------------
+ */
+
+union octeon_cmd {
+	uint64_t cmd64;
+
+	struct	{
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t cmd : 5;
+
+		uint64_t more : 6; /* How many udd words follow the command */
+
+		uint64_t reserved : 29;
+
+		uint64_t param1 : 16;
+
+		uint64_t param2 : 8;
+
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+
+		uint64_t param2 : 8;
+
+		uint64_t param1 : 16;
+
+		uint64_t reserved : 29;
+
+		uint64_t more : 6;
+
+		uint64_t cmd : 5;
+
+#endif
+	} s;
+};
+
+#define OCTEON_CMD_SIZE (sizeof(union octeon_cmd))
+
+/* Instruction Header */
+struct octeon_instr_ih3 {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+
+	/** Reserved3 */
+	uint64_t reserved3 : 1;
+
+	/** Gather indicator 1=gather*/
+	uint64_t gather : 1;
+
+	/** Data length OR no. of entries in gather list */
+	uint64_t dlengsz : 14;
+
+	/** Front Data size */
+	uint64_t fsz : 6;
+
+	/** Reserved2 */
+	uint64_t reserved2 : 4;
+
+	/** PKI port kind - PKIND */
+	uint64_t pkind : 6;
+
+	/** Reserved1 */
+	uint64_t reserved1 : 32;
+
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	/** Reserved1 */
+	uint64_t reserved1 : 32;
+
+	/** PKI port kind - PKIND */
+	uint64_t pkind : 6;
+
+	/** Reserved2 */
+	uint64_t reserved2 : 4;
+
+	/** Front Data size */
+	uint64_t fsz : 6;
+
+	/** Data length OR no. of entries in gather list */
+	uint64_t dlengsz : 14;
+
+	/** Gather indicator 1=gather*/
+	uint64_t gather : 1;
+
+	/** Reserved3 */
+	uint64_t reserved3 : 1;
+
+#endif
+};
+
+/* PKI Instruction Header(PKI IH) */
+struct octeon_instr_pki_ih3 {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+
+	/** Wider bit */
+	uint64_t w : 1;
+
+	/** Raw mode indicator 1 = RAW */
+	uint64_t raw : 1;
+
+	/** Use Tag */
+	uint64_t utag : 1;
+
+	/** Use QPG */
+	uint64_t uqpg : 1;
+
+	/** Reserved2 */
+	uint64_t reserved2 : 1;
+
+	/** Parse Mode */
+	uint64_t pm : 3;
+
+	/** Skip Length */
+	uint64_t sl : 8;
+
+	/** Use Tag Type */
+	uint64_t utt : 1;
+
+	/** Tag type */
+	uint64_t tagtype : 2;
+
+	/** Reserved1 */
+	uint64_t reserved1 : 2;
+
+	/** QPG Value */
+	uint64_t qpg : 11;
+
+	/** Tag Value */
+	uint64_t tag : 32;
+
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+
+	/** Tag Value */
+	uint64_t tag : 32;
+
+	/** QPG Value */
+	uint64_t qpg : 11;
+
+	/** Reserved1 */
+	uint64_t reserved1 : 2;
+
+	/** Tag type */
+	uint64_t tagtype : 2;
+
+	/** Use Tag Type */
+	uint64_t utt : 1;
+
+	/** Skip Length */
+	uint64_t sl : 8;
+
+	/** Parse Mode */
+	uint64_t pm : 3;
+
+	/** Reserved2 */
+	uint64_t reserved2 : 1;
+
+	/** Use QPG */
+	uint64_t uqpg : 1;
+
+	/** Use Tag */
+	uint64_t utag : 1;
+
+	/** Raw mode indicator 1 = RAW */
+	uint64_t raw : 1;
+
+	/** Wider bit */
+	uint64_t w : 1;
+#endif
+};
+
+/** Input Request Header */
+struct octeon_instr_irh {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	uint64_t opcode : 4;
+	uint64_t rflag : 1;
+	uint64_t subcode : 7;
+	uint64_t vlan : 12;
+	uint64_t priority : 3;
+	uint64_t reserved : 5;
+	uint64_t ossp : 32; /* opcode/subcode specific parameters */
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	uint64_t ossp : 32; /* opcode/subcode specific parameters */
+	uint64_t reserved : 5;
+	uint64_t priority : 3;
+	uint64_t vlan : 12;
+	uint64_t subcode : 7;
+	uint64_t rflag : 1;
+	uint64_t opcode : 4;
+#endif
+};
+
+/* pkiih3 + irh + ossp[0] + ossp[1] + rdp + rptr = 40 bytes */
+#define OCTEON_SOFT_CMD_RESP_IH3	(40 + 8)
+/* pki_h3 + irh + ossp[0] + ossp[1] = 32 bytes */
+#define OCTEON_PCI_CMD_O3		(24 + 8)
+
+/** Return Data Parameters */
+struct octeon_instr_rdp {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	uint64_t reserved : 49;
+	uint64_t pcie_port : 3;
+	uint64_t rlen : 12;
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	uint64_t rlen : 12;
+	uint64_t pcie_port : 3;
+	uint64_t reserved : 49;
+#endif
+};
+
 int lio_setup_sc_buffer_pool(struct lio_device *lio_dev);
 void lio_free_sc_buffer_pool(struct lio_device *lio_dev);
 
@@ -100,6 +369,13 @@ struct lio_soft_command *
 lio_alloc_soft_command(struct lio_device *lio_dev,
 		       uint32_t datasize, uint32_t rdatasize,
 		       uint32_t ctxsize);
+void lio_prepare_soft_command(struct lio_device *lio_dev,
+			      struct lio_soft_command *sc,
+			      uint8_t opcode, uint8_t subcode,
+			      uint32_t irh_ossp, uint64_t ossp0,
+			      uint64_t ossp1);
+int lio_send_soft_command(struct lio_device *lio_dev,
+			  struct lio_soft_command *sc);
 void lio_free_soft_command(struct lio_soft_command *sc);
 
 /** Maximum ordered requests to process in every invocation of
@@ -167,6 +443,21 @@ enum {
 	}
 }
 
+/* Macro to increment index.
+ * Index is incremented by count; if the sum exceeds
+ * max, index is wrapped-around to the start.
+ */
+static inline uint32_t
+lio_incr_index(uint32_t index, uint32_t count, uint32_t max)
+{
+	if ((index + count) >= max)
+		index = index + count - max;
+	else
+		index += count;
+
+	return index;
+}
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index ad67ed1..7bbb0ba 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -288,6 +288,12 @@ struct lio_device {
 	uint16_t pf_num;
 	uint16_t vf_num;
 
+	/** This device's PCIe port used for traffic. */
+	uint16_t pcie_port;
+
+	/** The state of this device */
+	rte_atomic64_t status;
+
 	uint8_t *hw_addr;
 
 	struct lio_fn_list fn_list;
-- 
1.8.3.1

[PATCH 22/50] net/liquidio: add API to configure device

[Shijith Thotton]

And API to configure device and initialize ethernet device
operations.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  14 +++
 drivers/net/liquidio/lio_ethdev.c       | 169 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  36 +++++++
 drivers/net/liquidio/lio_struct.h       |  72 ++++++++++++++
 4 files changed, 291 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index b581f44..fd2d1b0 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -95,6 +95,8 @@ enum lio_card_type {
 #define LIO_BASE_MINOR_VERSION		5
 #define LIO_BASE_MICRO_VERSION		1
 
+#define LIO_FW_VERSION_LENGTH		32
+
 /** Tag types used by Octeon cores in its work. */
 enum octeon_tag_type {
 	OCTEON_ORDERED_TAG	= 0,
@@ -104,6 +106,18 @@ enum octeon_tag_type {
 /* pre-defined host->NIC tag values */
 #define LIO_CONTROL	(0x11111110)
 
+/* used for NIC operations */
+#define LIO_OPCODE	1
+
+/** LIO_OPCODE subcodes */
+/* This subcode is sent by core PCI driver to indicate cores are ready. */
+#define LIO_OPCODE_IF_CFG		0x09
+
+/* Interface flags communicated between host driver and core app. */
+enum lio_ifflags {
+	LIO_IFFLAG_UNICAST	= 0x10
+};
+
 /* Routines for reading and writing CSRs */
 #ifdef RTE_LIBRTE_LIO_DEBUG_REGS
 #define lio_write_csr(lio_dev, reg_off, value)				\
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index a6813e0..ff80909 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -41,6 +41,166 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+static uint64_t
+lio_hweight64(uint64_t w)
+{
+	uint64_t res = w - ((w >> 1) & 0x5555555555555555ul);
+
+	res =
+	    (res & 0x3333333333333333ul) + ((res >> 2) & 0x3333333333333333ul);
+	res = (res + (res >> 4)) & 0x0F0F0F0F0F0F0F0Ful;
+	res = res + (res >> 8);
+	res = res + (res >> 16);
+
+	return (res + (res >> 32)) & 0x00000000000000FFul;
+}
+
+static int lio_dev_configure(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
+	int retval, num_iqueues, num_oqueues;
+	uint8_t mac[ETHER_ADDR_LEN], i;
+	struct lio_if_cfg_resp *resp;
+	struct lio_soft_command *sc;
+	union lio_if_cfg if_cfg;
+	uint32_t resp_size;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* Re-configuring firmware not supported.
+	 * Can't change tx/rx queues per port from initial value.
+	 */
+	if (lio_dev->port_configured) {
+		if ((lio_dev->nb_rx_queues != eth_dev->data->nb_rx_queues) ||
+		    (lio_dev->nb_tx_queues != eth_dev->data->nb_tx_queues)) {
+			lio_dev_err(lio_dev,
+				    "rxq/txq re-conf not supported. Restart application with new value.\n");
+			return -ENOTSUP;
+		}
+		return 0;
+	}
+
+	lio_dev->nb_rx_queues = eth_dev->data->nb_rx_queues;
+	lio_dev->nb_tx_queues = eth_dev->data->nb_tx_queues;
+
+	resp_size = sizeof(struct lio_if_cfg_resp);
+	sc = lio_alloc_soft_command(lio_dev, 0, resp_size, 0);
+	if (sc == NULL)
+		return -ENOMEM;
+
+	resp = (struct lio_if_cfg_resp *)sc->virtrptr;
+
+	/* Firmware doesn't have capability to reconfigure the queues,
+	 * Claim all queues, and use as many required
+	 */
+	if_cfg.if_cfg64 = 0;
+	if_cfg.s.num_iqueues = lio_dev->nb_tx_queues;
+	if_cfg.s.num_oqueues = lio_dev->nb_rx_queues;
+	if_cfg.s.base_queue = 0;
+
+	if_cfg.s.gmx_port_id = lio_dev->pf_num;
+
+	lio_prepare_soft_command(lio_dev, sc, LIO_OPCODE,
+				 LIO_OPCODE_IF_CFG, 0,
+				 if_cfg.if_cfg64, 0);
+
+	/* Setting wait time in seconds */
+	sc->wait_time = LIO_MAX_CMD_TIMEOUT / 1000;
+
+	retval = lio_send_soft_command(lio_dev, sc);
+	if (retval == LIO_IQ_SEND_FAILED) {
+		lio_dev_err(lio_dev, "iq/oq config failed status: %x\n",
+			    retval);
+		/* Soft instr is freed by driver in case of failure. */
+		goto nic_config_fail;
+	}
+
+	/* Sleep on a wait queue till the cond flag indicates that the
+	 * response arrived or timed-out.
+	 */
+	while ((*sc->status_word == LIO_COMPLETION_WORD_INIT) && --timeout) {
+		lio_process_ordered_list(lio_dev);
+		rte_delay_ms(1);
+	}
+
+	retval = resp->status;
+	if (retval) {
+		lio_dev_err(lio_dev, "iq/oq config failed\n");
+		goto nic_config_fail;
+	}
+
+	lio_swap_8B_data((uint64_t *)(&resp->cfg_info),
+			 sizeof(struct octeon_if_cfg_info) >> 3);
+
+	num_iqueues = lio_hweight64(resp->cfg_info.iqmask);
+	num_oqueues = lio_hweight64(resp->cfg_info.oqmask);
+
+	if (!(num_iqueues) || !(num_oqueues)) {
+		lio_dev_err(lio_dev,
+			    "Got bad iqueues (%016lx) or oqueues (%016lx) from firmware.\n",
+			    (unsigned long)resp->cfg_info.iqmask,
+			    (unsigned long)resp->cfg_info.oqmask);
+		goto nic_config_fail;
+	}
+
+	lio_dev_dbg(lio_dev,
+		    "interface %d, iqmask %016lx, oqmask %016lx, numiqueues %d, numoqueues %d\n",
+		    eth_dev->data->port_id,
+		    (unsigned long)resp->cfg_info.iqmask,
+		    (unsigned long)resp->cfg_info.oqmask,
+		    num_iqueues, num_oqueues);
+
+	lio_dev->linfo.num_rxpciq = num_oqueues;
+	lio_dev->linfo.num_txpciq = num_iqueues;
+
+	for (i = 0; i < num_oqueues; i++) {
+		lio_dev->linfo.rxpciq[i].rxpciq64 =
+		    resp->cfg_info.linfo.rxpciq[i].rxpciq64;
+		lio_dev_dbg(lio_dev, "index %d OQ %d\n",
+			    i, lio_dev->linfo.rxpciq[i].s.q_no);
+	}
+
+	for (i = 0; i < num_iqueues; i++) {
+		lio_dev->linfo.txpciq[i].txpciq64 =
+		    resp->cfg_info.linfo.txpciq[i].txpciq64;
+		lio_dev_dbg(lio_dev, "index %d IQ %d\n",
+			    i, lio_dev->linfo.txpciq[i].s.q_no);
+	}
+
+	lio_dev->linfo.hw_addr = resp->cfg_info.linfo.hw_addr;
+	lio_dev->linfo.gmxport = resp->cfg_info.linfo.gmxport;
+	lio_dev->linfo.link.link_status64 =
+			resp->cfg_info.linfo.link.link_status64;
+
+	/* 64-bit swap required on LE machines */
+	lio_swap_8B_data(&lio_dev->linfo.hw_addr, 1);
+	for (i = 0; i < ETHER_ADDR_LEN; i++)
+		mac[i] = *((uint8_t *)(((uint8_t *)&lio_dev->linfo.hw_addr) +
+				       2 + i));
+
+	/* Copy the permanent MAC address */
+	ether_addr_copy((struct ether_addr *)mac, &eth_dev->data->mac_addrs[0]);
+
+	lio_dev->port_configured = 1;
+
+	lio_free_soft_command(sc);
+
+	return 0;
+
+nic_config_fail:
+	lio_dev_err(lio_dev, "Failed retval %d\n", retval);
+	lio_free_soft_command(sc);
+	lio_free_instr_queue0(lio_dev);
+
+	return -ENODEV;
+}
+
+/* Define our ethernet definitions */
+static const struct eth_dev_ops liovf_eth_dev_ops = {
+	.dev_configure		= lio_dev_configure,
+};
+
 static void
 lio_check_pf_hs_response(void *lio_dev)
 {
@@ -216,15 +376,24 @@
 		return -EINVAL;
 	}
 
+	eth_dev->dev_ops = &liovf_eth_dev_ops;
 	mac_addr_size = ETHER_ADDR_LEN;
 
 	eth_dev->data->mac_addrs = rte_zmalloc("lio", mac_addr_size, 0);
 	if (eth_dev->data->mac_addrs == NULL) {
 		lio_dev_err(lio_dev,
 			    "MAC addresses memory allocation failed\n");
+		eth_dev->dev_ops = NULL;
 		return -ENOMEM;
 	}
 
+	rte_atomic64_set(&lio_dev->status, LIO_DEV_RUNNING);
+	rte_wmb();
+
+	lio_dev->port_configured = 0;
+	/* Always allow unicast packets */
+	lio_dev->ifflags |= LIO_IFFLAG_UNICAST;
+
 	return 0;
 }
 
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 76c9072..22e3d83 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -36,5 +36,41 @@
 
 #include <stdint.h>
 
+#include "lio_struct.h"
+
+#define LIO_MAX_CMD_TIMEOUT     10000 /* 10000ms (10s) */
+
 #define LIO_DEV(_eth_dev)		((_eth_dev)->data->dev_private)
+
+struct octeon_if_cfg_info {
+	uint64_t iqmask;	/** mask for IQs enabled for the port */
+	uint64_t oqmask;	/** mask for OQs enabled for the port */
+	struct octeon_link_info linfo; /** initial link information */
+	char lio_firmware_version[LIO_FW_VERSION_LENGTH];
+};
+
+union lio_if_cfg {
+	uint64_t if_cfg64;
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t base_queue : 16;
+		uint64_t num_iqueues : 16;
+		uint64_t num_oqueues : 16;
+		uint64_t gmx_port_id : 8;
+		uint64_t vf_id : 8;
+#else
+		uint64_t vf_id : 8;
+		uint64_t gmx_port_id : 8;
+		uint64_t num_oqueues : 16;
+		uint64_t num_iqueues : 16;
+		uint64_t base_queue : 16;
+#endif
+	} s;
+};
+
+struct lio_if_cfg_resp {
+	uint64_t rh;
+	struct octeon_if_cfg_info cfg_info;
+	uint64_t status;
+};
 #endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 7bbb0ba..69a08db 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -274,6 +274,75 @@ struct lio_config {
 	int def_rx_buf_size;
 };
 
+/** Status of a RGMII Link on Octeon as seen by core driver. */
+union octeon_link_status {
+	uint64_t link_status64;
+
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t duplex : 8;
+		uint64_t mtu : 16;
+		uint64_t speed : 16;
+		uint64_t link_up : 1;
+		uint64_t autoneg : 1;
+		uint64_t if_mode : 5;
+		uint64_t pause : 1;
+		uint64_t flashing : 1;
+		uint64_t reserved : 15;
+#else
+		uint64_t reserved : 15;
+		uint64_t flashing : 1;
+		uint64_t pause : 1;
+		uint64_t if_mode : 5;
+		uint64_t autoneg : 1;
+		uint64_t link_up : 1;
+		uint64_t speed : 16;
+		uint64_t mtu : 16;
+		uint64_t duplex : 8;
+#endif
+	} s;
+};
+
+/** The rxpciq info passed to host from the firmware */
+union octeon_rxpciq {
+	uint64_t rxpciq64;
+
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t q_no : 8;
+		uint64_t reserved : 56;
+#else
+		uint64_t reserved : 56;
+		uint64_t q_no : 8;
+#endif
+	} s;
+};
+
+/** Information for a OCTEON ethernet interface shared between core & host. */
+struct octeon_link_info {
+	union octeon_link_status link;
+	uint64_t hw_addr;
+
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	uint64_t gmxport : 16;
+	uint64_t macaddr_is_admin_assigned : 1;
+	uint64_t vlan_is_admin_assigned : 1;
+	uint64_t rsvd : 30;
+	uint64_t num_txpciq : 8;
+	uint64_t num_rxpciq : 8;
+#else
+	uint64_t num_rxpciq : 8;
+	uint64_t num_txpciq : 8;
+	uint64_t rsvd : 30;
+	uint64_t vlan_is_admin_assigned : 1;
+	uint64_t macaddr_is_admin_assigned : 1;
+	uint64_t gmxport : 16;
+#endif
+
+	union octeon_txpciq txpciq[LIO_MAX_IOQS_PER_IF];
+	union octeon_rxpciq rxpciq[LIO_MAX_IOQS_PER_IF];
+};
+
 /* -----------------------  THE LIO DEVICE  --------------------------- */
 /** The lio device.
  *  Each lio device has this structure to represent all its
@@ -294,6 +363,8 @@ struct lio_device {
 	/** The state of this device */
 	rte_atomic64_t status;
 
+	struct octeon_link_info linfo;
+
 	uint8_t *hw_addr;
 
 	struct lio_fn_list fn_list;
@@ -324,6 +395,7 @@ struct lio_device {
 
 	struct rte_eth_dev      *eth_dev;
 
+	uint64_t ifflags;
 	uint8_t max_rx_queues;
 	uint8_t max_tx_queues;
 	uint8_t nb_rx_queues;
-- 
1.8.3.1

[PATCH 23/50] net/liquidio: add API to setup Rx queue

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |   3 +
 drivers/net/liquidio/lio_ethdev.c       |  67 +++++++++++++
 drivers/net/liquidio/lio_rxtx.c         | 168 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         |  56 +++++++++++
 drivers/net/liquidio/lio_struct.h       | 149 ++++++++++++++++++++++++++++
 5 files changed, 443 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index fd2d1b0..a3ac954 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -78,6 +78,8 @@ enum lio_card_type {
 
 #define LIO_DEV_RUNNING		0xc
 
+#define LIO_OQ_REFILL_THRESHOLD_CFG(cfg)				\
+		((cfg)->default_config->oq.refill_threshold)
 #define LIO_NUM_DEF_TX_DESCS_CFG(cfg)					\
 		((cfg)->default_config->num_def_tx_descs)
 
@@ -89,6 +91,7 @@ enum lio_card_type {
 #define LIO_MAX_INSTR_QUEUES(lio_dev)		CN23XX_MAX_RINGS_PER_VF
 
 #define LIO_MAX_POSSIBLE_INSTR_QUEUES		CN23XX_MAX_INPUT_QUEUES
+#define LIO_MAX_POSSIBLE_OUTPUT_QUEUES		CN23XX_MAX_OUTPUT_QUEUES
 
 #define LIO_DEVICE_NAME_LEN		32
 #define LIO_BASE_MAJOR_VERSION		1
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index ff80909..29424c6 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -55,6 +55,72 @@
 	return (res + (res >> 32)) & 0x00000000000000FFul;
 }
 
+/**
+ * Setup our receive queue/ringbuffer. This is the
+ * queue the Octeon uses to send us packets and
+ * responses. We are given a memory pool for our
+ * packet buffers that are used to populate the receive
+ * queue.
+ *
+ * @param eth_dev
+ *    Pointer to the structure rte_eth_dev
+ * @param q_no
+ *    Queue number
+ * @param num_rx_descs
+ *    Number of entries in the queue
+ * @param socket_id
+ *    Where to allocate memory
+ * @param rx_conf
+ *    Pointer to the struction rte_eth_rxconf
+ * @param mp
+ *    Pointer to the packet pool
+ *
+ * @return
+ *    - On success, return 0
+ *    - On failure, return -1
+ */
+static int
+lio_dev_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t q_no,
+		       uint16_t num_rx_descs, unsigned int socket_id,
+		       const struct rte_eth_rxconf *rx_conf __rte_unused,
+		       struct rte_mempool *mp)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct rte_pktmbuf_pool_private *mbp_priv;
+	uint32_t fw_mapped_oq;
+	uint16_t buf_size;
+
+	if (q_no >= lio_dev->nb_rx_queues) {
+		lio_dev_err(lio_dev, "Invalid rx queue number %u\n", q_no);
+		return -EINVAL;
+	}
+
+	lio_dev_dbg(lio_dev, "setting up rx queue %u\n", q_no);
+
+	fw_mapped_oq = lio_dev->linfo.rxpciq[q_no].s.q_no;
+
+	if ((lio_dev->droq[fw_mapped_oq]) &&
+	    (num_rx_descs != lio_dev->droq[fw_mapped_oq]->max_count)) {
+		lio_dev_err(lio_dev,
+			    "Reconfiguring Rx descs not supported. Configure descs to same value %u or restart application\n",
+			    lio_dev->droq[fw_mapped_oq]->max_count);
+		return -ENOTSUP;
+	}
+
+	mbp_priv = rte_mempool_get_priv(mp);
+	buf_size = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM;
+
+	if (lio_setup_droq(lio_dev, fw_mapped_oq, num_rx_descs, buf_size, mp,
+			   socket_id)) {
+		lio_dev_err(lio_dev, "droq allocation failed\n");
+		return -1;
+	}
+
+	eth_dev->data->rx_queues[q_no] = lio_dev->droq[fw_mapped_oq];
+
+	return 0;
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -199,6 +265,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 /* Define our ethernet definitions */
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
+	.rx_queue_setup		= lio_dev_rx_queue_setup,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index a1fadfa..60c895d 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -63,6 +63,174 @@
 }
 
 /**
+ *  Frees the space for descriptor ring for the droq.
+ *
+ *  @param lio_dev	- pointer to the lio device structure
+ *  @param q_no		- droq no.
+ */
+static void
+lio_delete_droq(struct lio_device *lio_dev, uint32_t q_no)
+{
+	struct lio_droq *droq = lio_dev->droq[q_no];
+
+	lio_dev_dbg(lio_dev, "OQ[%d]\n", q_no);
+
+	rte_free(droq->recv_buf_list);
+	droq->recv_buf_list = NULL;
+	lio_dma_zone_free(lio_dev, droq->info_mz);
+	lio_dma_zone_free(lio_dev, droq->desc_ring_mz);
+
+	memset(droq, 0, LIO_DROQ_SIZE);
+}
+
+static void *
+lio_alloc_info_buffer(struct lio_device *lio_dev,
+		      struct lio_droq *droq, unsigned int socket_id)
+{
+	droq->info_mz = rte_eth_dma_zone_reserve(lio_dev->eth_dev,
+						 "info_list", droq->q_no,
+						 (droq->max_count *
+							LIO_DROQ_INFO_SIZE),
+						 RTE_CACHE_LINE_SIZE,
+						 socket_id);
+
+	if (droq->info_mz == NULL)
+		return NULL;
+
+	droq->info_list_dma = droq->info_mz->phys_addr;
+	droq->info_alloc_size = droq->info_mz->len;
+	droq->info_base_addr = (size_t)droq->info_mz->addr;
+
+	return droq->info_mz->addr;
+}
+
+/**
+ *  Allocates space for the descriptor ring for the droq and
+ *  sets the base addr, num desc etc in Octeon registers.
+ *
+ * @param lio_dev	- pointer to the lio device structure
+ * @param q_no		- droq no.
+ * @param app_ctx	- pointer to application context
+ * @return Success: 0	Failure: -1
+ */
+static int
+lio_init_droq(struct lio_device *lio_dev, uint32_t q_no,
+	      uint32_t num_descs, uint32_t desc_size,
+	      struct rte_mempool *mpool, unsigned int socket_id)
+{
+	uint32_t c_refill_threshold;
+	uint32_t desc_ring_size;
+	struct lio_droq *droq;
+
+	lio_dev_dbg(lio_dev, "OQ[%d]\n", q_no);
+
+	droq = lio_dev->droq[q_no];
+	droq->lio_dev = lio_dev;
+	droq->q_no = q_no;
+	droq->mpool = mpool;
+
+	c_refill_threshold = LIO_OQ_REFILL_THRESHOLD_CFG(lio_dev);
+
+	droq->max_count = num_descs;
+	droq->buffer_size = desc_size;
+
+	desc_ring_size = droq->max_count * LIO_DROQ_DESC_SIZE;
+	droq->desc_ring_mz = rte_eth_dma_zone_reserve(lio_dev->eth_dev,
+						      "droq", q_no,
+						      desc_ring_size,
+						      RTE_CACHE_LINE_SIZE,
+						      socket_id);
+
+	if (droq->desc_ring_mz == NULL) {
+		lio_dev_err(lio_dev,
+			    "Output queue %d ring alloc failed\n", q_no);
+		return -1;
+	}
+
+	droq->desc_ring_dma = droq->desc_ring_mz->phys_addr;
+	droq->desc_ring = (struct lio_droq_desc *)droq->desc_ring_mz->addr;
+
+	lio_dev_dbg(lio_dev, "droq[%d]: desc_ring: virt: 0x%p, dma: %lx\n",
+		    q_no, droq->desc_ring, (unsigned long)droq->desc_ring_dma);
+	lio_dev_dbg(lio_dev, "droq[%d]: num_desc: %d\n", q_no,
+		    droq->max_count);
+
+	droq->info_list = lio_alloc_info_buffer(lio_dev, droq, socket_id);
+	if (droq->info_list == NULL) {
+		lio_dev_err(lio_dev, "Cannot allocate memory for info list.\n");
+		goto init_droq_fail;
+	}
+
+	droq->recv_buf_list = rte_zmalloc_socket("recv_buf_list",
+						 (droq->max_count *
+							LIO_DROQ_RECVBUF_SIZE),
+						 RTE_CACHE_LINE_SIZE,
+						 socket_id);
+	if (droq->recv_buf_list == NULL) {
+		lio_dev_err(lio_dev,
+			    "Output queue recv buf list alloc failed\n");
+		goto init_droq_fail;
+	}
+
+	droq->refill_threshold = c_refill_threshold;
+
+	rte_spinlock_init(&droq->lock);
+
+	lio_dev->io_qmask.oq |= (1ULL << q_no);
+
+	return 0;
+
+init_droq_fail:
+	lio_delete_droq(lio_dev, q_no);
+
+	return -1;
+}
+
+int
+lio_setup_droq(struct lio_device *lio_dev, int oq_no, int num_descs,
+	       int desc_size, struct rte_mempool *mpool, unsigned int socket_id)
+{
+	struct lio_droq *droq;
+
+	PMD_INIT_FUNC_TRACE();
+
+	if (lio_dev->droq[oq_no]) {
+		lio_dev_dbg(lio_dev, "Droq %d in use\n", oq_no);
+		return 0;
+	}
+
+	/* Allocate the DS for the new droq. */
+	droq = rte_zmalloc_socket("ethdev RX queue", sizeof(*droq),
+				  RTE_CACHE_LINE_SIZE, socket_id);
+	if (droq == NULL)
+		return -ENOMEM;
+
+	lio_dev->droq[oq_no] = droq;
+
+	/* Initialize the Droq */
+	if (lio_init_droq(lio_dev, oq_no, num_descs, desc_size, mpool,
+			  socket_id)) {
+		lio_dev_err(lio_dev, "Droq[%u] Initialization Failed\n", oq_no);
+		rte_free(lio_dev->droq[oq_no]);
+		lio_dev->droq[oq_no] = NULL;
+		return -ENOMEM;
+	}
+
+	lio_dev->num_oqs++;
+
+	lio_dev_dbg(lio_dev, "Total number of OQ: %d\n", lio_dev->num_oqs);
+
+	/* Send credit for octeon output queues. credits are always
+	 * sent after the output queue is enabled.
+	 */
+	rte_write32(lio_dev->droq[oq_no]->max_count,
+		    lio_dev->droq[oq_no]->pkts_credit_reg);
+	rte_wmb();
+
+	return 0;
+}
+
+/**
  *  lio_init_instr_queue()
  *  @param lio_dev	- pointer to the lio device structure.
  *  @param txpciq	- queue to be initialized.
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index d7b7194..938bf56 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -50,6 +50,58 @@
 #define lio_uptime		\
 	(size_t)(rte_get_timer_cycles() / rte_get_timer_hz())
 
+/** Descriptor format.
+ *  The descriptor ring is made of descriptors which have 2 64-bit values:
+ *  -# Physical (bus) address of the data buffer.
+ *  -# Physical (bus) address of a lio_droq_info structure.
+ *  The device DMA's incoming packets and its information at the address
+ *  given by these descriptor fields.
+ */
+struct lio_droq_desc {
+	/** The buffer pointer */
+	uint64_t buffer_ptr;
+
+	/** The Info pointer */
+	uint64_t info_ptr;
+};
+
+#define LIO_DROQ_DESC_SIZE	(sizeof(struct lio_droq_desc))
+
+/** Information about packet DMA'ed by Octeon.
+ *  The format of the information available at Info Pointer after Octeon
+ *  has posted a packet. Not all descriptors have valid information. Only
+ *  the Info field of the first descriptor for a packet has information
+ *  about the packet.
+ */
+struct lio_droq_info {
+	/** The Output Receive Header. */
+	union octeon_rh rh;
+
+	/** The Length of the packet. */
+	uint64_t length;
+};
+
+#define LIO_DROQ_INFO_SIZE	(sizeof(struct lio_droq_info))
+
+/** Pointer to data buffer.
+ *  Driver keeps a pointer to the data buffer that it made available to
+ *  the Octeon device. Since the descriptor ring keeps physical (bus)
+ *  addresses, this field is required for the driver to keep track of
+ *  the virtual address pointers.
+ */
+struct lio_recv_buffer {
+	/** Packet buffer, including meta data. */
+	void *buffer;
+
+	/** Data in the packet buffer. */
+	uint8_t *data;
+
+};
+
+#define LIO_DROQ_RECVBUF_SIZE	(sizeof(struct lio_recv_buffer))
+
+#define LIO_DROQ_SIZE		(sizeof(struct lio_droq))
+
 #define LIO_IQ_SEND_OK		0
 #define LIO_IQ_SEND_STOP	1
 #define LIO_IQ_SEND_FAILED	-1
@@ -458,6 +510,10 @@ enum {
 	return index;
 }
 
+int lio_setup_droq(struct lio_device *lio_dev, int q_no, int num_descs,
+		   int desc_size, struct rte_mempool *mpool,
+		   unsigned int socket_id);
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 69a08db..a2ba50d 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -56,6 +56,150 @@ struct lio_version {
 	uint16_t reserved;
 };
 
+/** The Descriptor Ring Output Queue structure.
+ *  This structure has all the information required to implement a
+ *  DROQ.
+ */
+struct lio_droq {
+	/** A spinlock to protect access to this ring. */
+	rte_spinlock_t lock;
+
+	uint32_t q_no;
+
+	uint32_t pkt_count;
+
+	struct lio_device *lio_dev;
+
+	/** The 8B aligned descriptor ring starts at this address. */
+	struct lio_droq_desc *desc_ring;
+
+	/** Index in the ring where the driver should read the next packet */
+	uint32_t read_idx;
+
+	/** Index in the ring where Octeon will write the next packet */
+	uint32_t write_idx;
+
+	/** Index in the ring where the driver will refill the descriptor's
+	 * buffer
+	 */
+	uint32_t refill_idx;
+
+	/** Packets pending to be processed */
+	rte_atomic64_t pkts_pending;
+
+	/** Number of  descriptors in this ring. */
+	uint32_t max_count;
+
+	/** The number of descriptors pending refill. */
+	uint32_t refill_count;
+
+	uint32_t refill_threshold;
+
+	/** The 8B aligned info ptrs begin from this address. */
+	struct lio_droq_info *info_list;
+
+	/** The receive buffer list. This list has the virtual addresses of the
+	 *  buffers.
+	 */
+	struct lio_recv_buffer *recv_buf_list;
+
+	/** The size of each buffer pointed by the buffer pointer. */
+	uint32_t buffer_size;
+
+	/** Pointer to the mapped packet credit register.
+	 *  Host writes number of info/buffer ptrs available to this register
+	 */
+	void *pkts_credit_reg;
+
+	/** Pointer to the mapped packet sent register.
+	 *  Octeon writes the number of packets DMA'ed to host memory
+	 *  in this register.
+	 */
+	void *pkts_sent_reg;
+
+	/** DMA mapped address of the DROQ descriptor ring. */
+	size_t desc_ring_dma;
+
+	/** Info ptr list are allocated at this virtual address. */
+	size_t info_base_addr;
+
+	/** DMA mapped address of the info list */
+	size_t info_list_dma;
+
+	/** Allocated size of info list. */
+	uint32_t info_alloc_size;
+
+	/** Memory zone **/
+	const struct rte_memzone *desc_ring_mz;
+	const struct rte_memzone *info_mz;
+	struct rte_mempool *mpool;
+};
+
+/** Receive Header */
+union octeon_rh {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	uint64_t rh64;
+	struct	{
+		uint64_t opcode : 4;
+		uint64_t subcode : 8;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t reserved : 17;
+		uint64_t ossp : 32; /** opcode/subcode specific parameters */
+	} r;
+	struct	{
+		uint64_t opcode : 4;
+		uint64_t subcode : 8;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t extra : 28;
+		uint64_t vlan : 12;
+		uint64_t priority : 3;
+		uint64_t csum_verified : 3; /** checksum verified. */
+		uint64_t has_hwtstamp : 1; /** Has hardware timestamp.1 = yes.*/
+		uint64_t encap_on : 1;
+		uint64_t has_hash : 1; /** Has hash (rth or rss). 1 = yes. */
+	} r_dh;
+	struct {
+		uint64_t opcode : 4;
+		uint64_t subcode : 8;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t reserved : 8;
+		uint64_t extra : 25;
+		uint64_t gmxport : 16;
+	} r_nic_info;
+#else
+	uint64_t rh64;
+	struct {
+		uint64_t ossp : 32; /** opcode/subcode specific parameters */
+		uint64_t reserved : 17;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t subcode : 8;
+		uint64_t opcode : 4;
+	} r;
+	struct {
+		uint64_t has_hash : 1; /** Has hash (rth or rss). 1 = yes. */
+		uint64_t encap_on : 1;
+		uint64_t has_hwtstamp : 1;  /** 1 = has hwtstamp */
+		uint64_t csum_verified : 3; /** checksum verified. */
+		uint64_t priority : 3;
+		uint64_t vlan : 12;
+		uint64_t extra : 28;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t subcode : 8;
+		uint64_t opcode : 4;
+	} r_dh;
+	struct {
+		uint64_t gmxport : 16;
+		uint64_t extra : 25;
+		uint64_t reserved : 8;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t subcode : 8;
+		uint64_t opcode : 4;
+	} r_nic_info;
+#endif
+};
+
+#define OCTEON_RH_SIZE (sizeof(union octeon_rh))
+
 /** The txpciq info passed to host from the firmware */
 union octeon_txpciq {
 	uint64_t txpciq64;
@@ -380,6 +524,11 @@ struct lio_device {
 	/** The circular-linked list of instruction response */
 	struct lio_response_list response_list;
 
+	uint32_t num_oqs;
+
+	/** The DROQ output queues  */
+	struct lio_droq *droq[LIO_MAX_POSSIBLE_OUTPUT_QUEUES];
+
 	struct lio_io_enable io_qmask;
 
 	struct lio_sriov_info sriov_info;
-- 
1.8.3.1

[PATCH 24/50] net/liquidio: initialize Rx queue

[Shijith Thotton]

Initialize Rx queue registers and allocate packet buffers in Rx
queue.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 22 ++++++++
 drivers/net/liquidio/base/lio_hw_defs.h |  2 +
 drivers/net/liquidio/lio_rxtx.c         | 96 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         | 20 +++++++
 drivers/net/liquidio/lio_struct.h       |  1 +
 5 files changed, 141 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index 181f830..44d90c0 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -233,6 +233,27 @@
 }
 
 static void
+cn23xx_vf_setup_oq_regs(struct lio_device *lio_dev, uint32_t oq_no)
+{
+	struct lio_droq *droq = lio_dev->droq[oq_no];
+
+	PMD_INIT_FUNC_TRACE();
+
+	lio_write_csr64(lio_dev, CN23XX_SLI_OQ_BASE_ADDR64(oq_no),
+			droq->desc_ring_dma);
+	lio_write_csr(lio_dev, CN23XX_SLI_OQ_SIZE(oq_no), droq->max_count);
+
+	lio_write_csr(lio_dev, CN23XX_SLI_OQ_BUFF_INFO_SIZE(oq_no),
+		      (droq->buffer_size | (OCTEON_RH_SIZE << 16)));
+
+	/* Get the mapped address of the pkt_sent and pkts_credit regs */
+	droq->pkts_sent_reg = (uint8_t *)lio_dev->hw_addr +
+					CN23XX_SLI_OQ_PKTS_SENT(oq_no);
+	droq->pkts_credit_reg = (uint8_t *)lio_dev->hw_addr +
+					CN23XX_SLI_OQ_PKTS_CREDIT(oq_no);
+}
+
+static void
 cn23xx_vf_free_mbox(struct lio_device *lio_dev)
 {
 	PMD_INIT_FUNC_TRACE();
@@ -436,6 +457,7 @@
 		return -1;
 
 	lio_dev->fn_list.setup_iq_regs		= cn23xx_vf_setup_iq_regs;
+	lio_dev->fn_list.setup_oq_regs		= cn23xx_vf_setup_oq_regs;
 	lio_dev->fn_list.setup_mbox		= cn23xx_vf_setup_mbox;
 	lio_dev->fn_list.free_mbox		= cn23xx_vf_free_mbox;
 
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index a3ac954..912b8b9 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -116,6 +116,8 @@ enum octeon_tag_type {
 /* This subcode is sent by core PCI driver to indicate cores are ready. */
 #define LIO_OPCODE_IF_CFG		0x09
 
+#define LIO_MAX_RX_PKTLEN		(64 * 1024)
+
 /* Interface flags communicated between host driver and core app. */
 enum lio_ifflags {
 	LIO_IFFLAG_UNICAST	= 0x10
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 60c895d..506a1db 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -41,6 +41,96 @@
 #include "lio_rxtx.h"
 
 static void
+lio_droq_compute_max_packet_bufs(struct lio_droq *droq)
+{
+	uint32_t count = 0;
+
+	do {
+		count += droq->buffer_size;
+	} while (count < LIO_MAX_RX_PKTLEN);
+}
+
+static void
+lio_droq_reset_indices(struct lio_droq *droq)
+{
+	droq->read_idx	= 0;
+	droq->write_idx	= 0;
+	droq->refill_idx = 0;
+	droq->refill_count = 0;
+	rte_atomic64_set(&droq->pkts_pending, 0);
+}
+
+static void
+lio_droq_destroy_ring_buffers(struct lio_droq *droq)
+{
+	uint32_t i;
+
+	for (i = 0; i < droq->max_count; i++) {
+		if (droq->recv_buf_list[i].buffer) {
+			rte_pktmbuf_free((struct rte_mbuf *)
+					 droq->recv_buf_list[i].buffer);
+			droq->recv_buf_list[i].buffer = NULL;
+		}
+	}
+
+	lio_droq_reset_indices(droq);
+}
+
+static void *
+lio_recv_buffer_alloc(struct lio_device *lio_dev, int q_no)
+{
+	struct lio_droq *droq = lio_dev->droq[q_no];
+	struct rte_mempool *mpool = droq->mpool;
+	struct rte_mbuf *m;
+
+	m = rte_pktmbuf_alloc(mpool);
+	if (m == NULL) {
+		lio_dev_err(lio_dev, "Cannot allocate\n");
+		return NULL;
+	}
+
+	rte_mbuf_refcnt_set(m, 1);
+	m->next = NULL;
+	m->data_off = RTE_PKTMBUF_HEADROOM;
+	m->nb_segs = 1;
+	m->pool = mpool;
+
+	return m;
+}
+
+static int
+lio_droq_setup_ring_buffers(struct lio_device *lio_dev,
+			    struct lio_droq *droq)
+{
+	struct lio_droq_desc *desc_ring = droq->desc_ring;
+	uint32_t i;
+	void *buf;
+
+	for (i = 0; i < droq->max_count; i++) {
+		buf = lio_recv_buffer_alloc(lio_dev, droq->q_no);
+		if (buf == NULL) {
+			lio_dev_err(lio_dev, "buffer alloc failed\n");
+			lio_droq_destroy_ring_buffers(droq);
+			return -ENOMEM;
+		}
+
+		droq->recv_buf_list[i].buffer = buf;
+		droq->info_list[i].length = 0;
+
+		/* map ring buffers into memory */
+		desc_ring[i].info_ptr = lio_map_ring_info(droq, i);
+		desc_ring[i].buffer_ptr =
+			lio_map_ring(droq->recv_buf_list[i].buffer);
+	}
+
+	lio_droq_reset_indices(droq);
+
+	lio_droq_compute_max_packet_bufs(droq);
+
+	return 0;
+}
+
+static void
 lio_dma_zone_free(struct lio_device *lio_dev, const struct rte_memzone *mz)
 {
 	const struct rte_memzone *mz_tmp;
@@ -75,6 +165,7 @@
 
 	lio_dev_dbg(lio_dev, "OQ[%d]\n", q_no);
 
+	lio_droq_destroy_ring_buffers(droq);
 	rte_free(droq->recv_buf_list);
 	droq->recv_buf_list = NULL;
 	lio_dma_zone_free(lio_dev, droq->info_mz);
@@ -172,10 +263,15 @@
 		goto init_droq_fail;
 	}
 
+	if (lio_droq_setup_ring_buffers(lio_dev, droq))
+		goto init_droq_fail;
+
 	droq->refill_threshold = c_refill_threshold;
 
 	rte_spinlock_init(&droq->lock);
 
+	lio_dev->fn_list.setup_oq_regs(lio_dev, q_no);
+
 	lio_dev->io_qmask.oq |= (1ULL << q_no);
 
 	return 0;
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 938bf56..4d742de 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -495,6 +495,26 @@ enum {
 	}
 }
 
+static inline uint64_t
+lio_map_ring(void *buf)
+{
+	phys_addr_t dma_addr;
+
+	dma_addr = rte_mbuf_data_dma_addr_default(((struct rte_mbuf *)buf));
+
+	return (uint64_t)dma_addr;
+}
+
+static inline uint64_t
+lio_map_ring_info(struct lio_droq *droq, uint32_t i)
+{
+	phys_addr_t dma_addr;
+
+	dma_addr = droq->info_list_dma + (i * LIO_DROQ_INFO_SIZE);
+
+	return (uint64_t)dma_addr;
+}
+
 /* Macro to increment index.
  * Index is incremented by count; if the sum exceeds
  * max, index is wrapped-around to the start.
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index a2ba50d..bb4618f 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -306,6 +306,7 @@ struct lio_io_enable {
 
 struct lio_fn_list {
 	void (*setup_iq_regs)(struct lio_device *, uint32_t);
+	void (*setup_oq_regs)(struct lio_device *, uint32_t);
 
 	int (*setup_mbox)(struct lio_device *);
 	void (*free_mbox)(struct lio_device *);
-- 
1.8.3.1

[PATCH 25/50] net/liquidio: add Rx data path

[Shijith Thotton]

Add APIs to receive packets and re-fill ring buffers.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  12 +
 drivers/net/liquidio/lio_ethdev.c       |   5 +
 drivers/net/liquidio/lio_rxtx.c         | 380 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         |  13 ++
 4 files changed, 410 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 912b8b9..94a21ad 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -112,12 +112,24 @@ enum octeon_tag_type {
 /* used for NIC operations */
 #define LIO_OPCODE	1
 
+/* Subcodes are used by host driver/apps to identify the sub-operation
+ * for the core. They only need to by unique for a given subsystem.
+ */
+#define LIO_OPCODE_SUBCODE(op, sub)		\
+		((((op) & 0x0f) << 8) | ((sub) & 0x7f))
+
 /** LIO_OPCODE subcodes */
 /* This subcode is sent by core PCI driver to indicate cores are ready. */
+#define LIO_OPCODE_NW_DATA		0x02 /* network packet data */
 #define LIO_OPCODE_IF_CFG		0x09
 
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
 
+/* RX(packets coming from wire) Checksum verification flags */
+/* TCP/UDP csum */
+#define LIO_L4_CSUM_VERIFIED		0x1
+#define LIO_IP_CSUM_VERIFIED		0x2
+
 /* Interface flags communicated between host driver and core app. */
 enum lio_ifflags {
 	LIO_IFFLAG_UNICAST	= 0x10
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 29424c6..300baee 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -404,6 +404,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	rte_free(eth_dev->data->mac_addrs);
 	eth_dev->data->mac_addrs = NULL;
 
+	eth_dev->rx_pkt_burst = NULL;
+
 	return 0;
 }
 
@@ -416,6 +418,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 
 	PMD_INIT_FUNC_TRACE();
 
+	eth_dev->rx_pkt_burst = &lio_dev_recv_pkts;
+
 	/* Primary does the initialization. */
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
@@ -451,6 +455,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 		lio_dev_err(lio_dev,
 			    "MAC addresses memory allocation failed\n");
 		eth_dev->dev_ops = NULL;
+		eth_dev->rx_pkt_burst = NULL;
 		return -ENOMEM;
 	}
 
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 506a1db..7c6b446 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -326,6 +326,386 @@
 	return 0;
 }
 
+static inline uint32_t
+lio_droq_get_bufcount(uint32_t buf_size, uint32_t total_len)
+{
+	uint32_t buf_cnt = 0;
+
+	while (total_len > (buf_size * buf_cnt))
+		buf_cnt++;
+
+	return buf_cnt;
+}
+
+/* If we were not able to refill all buffers, try to move around
+ * the buffers that were not dispatched.
+ */
+static inline uint32_t
+lio_droq_refill_pullup_descs(struct lio_droq *droq,
+			     struct lio_droq_desc *desc_ring)
+{
+	uint32_t refill_index = droq->refill_idx;
+	uint32_t desc_refilled = 0;
+
+	while (refill_index != droq->read_idx) {
+		if (droq->recv_buf_list[refill_index].buffer) {
+			droq->recv_buf_list[droq->refill_idx].buffer =
+				droq->recv_buf_list[refill_index].buffer;
+			desc_ring[droq->refill_idx].buffer_ptr =
+				desc_ring[refill_index].buffer_ptr;
+			droq->recv_buf_list[refill_index].buffer = NULL;
+			desc_ring[refill_index].buffer_ptr = 0;
+			do {
+				droq->refill_idx = lio_incr_index(
+							droq->refill_idx, 1,
+							droq->max_count);
+				desc_refilled++;
+				droq->refill_count--;
+			} while (droq->recv_buf_list[droq->refill_idx].buffer);
+		}
+		refill_index = lio_incr_index(refill_index, 1,
+					      droq->max_count);
+	}	/* while */
+
+	return desc_refilled;
+}
+
+/* lio_droq_refill
+ *
+ * @param lio_dev	- pointer to the lio device structure
+ * @param droq		- droq in which descriptors require new buffers.
+ *
+ * Description:
+ *  Called during normal DROQ processing in interrupt mode or by the poll
+ *  thread to refill the descriptors from which buffers were dispatched
+ *  to upper layers. Attempts to allocate new buffers. If that fails, moves
+ *  up buffers (that were not dispatched) to form a contiguous ring.
+ *
+ * Returns:
+ *  No of descriptors refilled.
+ *
+ * Locks:
+ * This routine is called with droq->lock held.
+ */
+static uint32_t
+lio_droq_refill(struct lio_device *lio_dev, struct lio_droq *droq)
+{
+	struct lio_droq_desc *desc_ring;
+	uint32_t desc_refilled = 0;
+	void *buf = NULL;
+
+	desc_ring = droq->desc_ring;
+
+	while (droq->refill_count && (desc_refilled < droq->max_count)) {
+		/* If a valid buffer exists (happens if there is no dispatch),
+		 * reuse the buffer, else allocate.
+		 */
+		if (droq->recv_buf_list[droq->refill_idx].buffer == NULL) {
+			buf = lio_recv_buffer_alloc(lio_dev, droq->q_no);
+			/* If a buffer could not be allocated, no point in
+			 * continuing
+			 */
+			if (buf == NULL)
+				break;
+
+			droq->recv_buf_list[droq->refill_idx].buffer = buf;
+		}
+
+		desc_ring[droq->refill_idx].buffer_ptr =
+		    lio_map_ring(droq->recv_buf_list[droq->refill_idx].buffer);
+		/* Reset any previous values in the length field. */
+		droq->info_list[droq->refill_idx].length = 0;
+
+		droq->refill_idx = lio_incr_index(droq->refill_idx, 1,
+						  droq->max_count);
+		desc_refilled++;
+		droq->refill_count--;
+	}
+
+	if (droq->refill_count)
+		desc_refilled += lio_droq_refill_pullup_descs(droq, desc_ring);
+
+	/* if droq->refill_count
+	 * The refill count would not change in pass two. We only moved buffers
+	 * to close the gap in the ring, but we would still have the same no. of
+	 * buffers to refill.
+	 */
+	return desc_refilled;
+}
+
+static int
+lio_droq_fast_process_packet(struct lio_device *lio_dev,
+			     struct lio_droq *droq,
+			     struct rte_mbuf **rx_pkts)
+{
+	struct rte_mbuf *nicbuf = NULL;
+	struct lio_droq_info *info;
+	uint32_t total_len = 0;
+	int data_total_len = 0;
+	uint32_t pkt_len = 0;
+	union octeon_rh *rh;
+	int data_pkts = 0;
+
+	info = &droq->info_list[droq->read_idx];
+	lio_swap_8B_data((uint64_t *)info, 2);
+
+	if (!info->length)
+		return -1;
+
+	/* Len of resp hdr in included in the received data len. */
+	info->length -= OCTEON_RH_SIZE;
+	rh = &info->rh;
+
+	total_len += (uint32_t)info->length;
+
+	if (lio_opcode_slow_path(rh)) {
+		uint32_t buf_cnt;
+
+		buf_cnt = lio_droq_get_bufcount(droq->buffer_size,
+						(uint32_t)info->length);
+		droq->read_idx = lio_incr_index(droq->read_idx, buf_cnt,
+						droq->max_count);
+		droq->refill_count += buf_cnt;
+	} else {
+		if (info->length <= droq->buffer_size) {
+			if (rh->r_dh.has_hash)
+				pkt_len = (uint32_t)(info->length - 8);
+			else
+				pkt_len = (uint32_t)info->length;
+
+			nicbuf = droq->recv_buf_list[droq->read_idx].buffer;
+			droq->recv_buf_list[droq->read_idx].buffer = NULL;
+			droq->read_idx = lio_incr_index(
+						droq->read_idx, 1,
+						droq->max_count);
+			droq->refill_count++;
+
+			if (likely(nicbuf != NULL)) {
+				nicbuf->data_off = RTE_PKTMBUF_HEADROOM;
+				nicbuf->nb_segs = 1;
+				nicbuf->next = NULL;
+				/* We don't have a way to pass flags yet */
+				nicbuf->ol_flags = 0;
+				if (rh->r_dh.has_hash) {
+					uint64_t *hash_ptr;
+
+					nicbuf->ol_flags |= PKT_RX_RSS_HASH;
+					hash_ptr = rte_pktmbuf_mtod(nicbuf,
+								    uint64_t *);
+					lio_swap_8B_data(hash_ptr, 1);
+					nicbuf->hash.rss = (uint32_t)*hash_ptr;
+					nicbuf->data_off += 8;
+				}
+
+				nicbuf->pkt_len = pkt_len;
+				nicbuf->data_len = pkt_len;
+				nicbuf->port = lio_dev->port_id;
+				/* Store the mbuf */
+				rx_pkts[data_pkts++] = nicbuf;
+				data_total_len += pkt_len;
+			}
+
+			/* Prefetch buffer pointers when on a cache line
+			 * boundary
+			 */
+			if ((droq->read_idx & 3) == 0) {
+				rte_prefetch0(
+				    &droq->recv_buf_list[droq->read_idx]);
+				rte_prefetch0(
+				    &droq->info_list[droq->read_idx]);
+			}
+		} else {
+			struct rte_mbuf *first_buf = NULL;
+			struct rte_mbuf *last_buf = NULL;
+
+			while (pkt_len < info->length) {
+				int cpy_len = 0;
+
+				cpy_len = ((pkt_len + droq->buffer_size) >
+						info->length)
+						? ((uint32_t)info->length -
+							pkt_len)
+						: droq->buffer_size;
+
+				nicbuf =
+				    droq->recv_buf_list[droq->read_idx].buffer;
+				droq->recv_buf_list[droq->read_idx].buffer =
+				    NULL;
+
+				if (likely(nicbuf != NULL)) {
+					/* Note the first seg */
+					if (!pkt_len)
+						first_buf = nicbuf;
+
+					nicbuf->data_off = RTE_PKTMBUF_HEADROOM;
+					nicbuf->nb_segs = 1;
+					nicbuf->next = NULL;
+					nicbuf->port = lio_dev->port_id;
+					/* We don't have a way to pass
+					 * flags yet
+					 */
+					nicbuf->ol_flags = 0;
+					if ((!pkt_len) && (rh->r_dh.has_hash)) {
+						uint64_t *hash_ptr;
+
+						nicbuf->ol_flags |=
+						    PKT_RX_RSS_HASH;
+						hash_ptr = rte_pktmbuf_mtod(
+						    nicbuf, uint64_t *);
+						lio_swap_8B_data(hash_ptr, 1);
+						nicbuf->hash.rss =
+						    (uint32_t)*hash_ptr;
+						nicbuf->data_off += 8;
+						nicbuf->pkt_len = cpy_len - 8;
+						nicbuf->data_len = cpy_len - 8;
+					} else {
+						nicbuf->pkt_len = cpy_len;
+						nicbuf->data_len = cpy_len;
+					}
+
+					if (pkt_len)
+						first_buf->nb_segs++;
+
+					if (last_buf)
+						last_buf->next = nicbuf;
+
+					last_buf = nicbuf;
+				} else {
+					PMD_RX_LOG(lio_dev, ERR, "no buf\n");
+				}
+
+				pkt_len += cpy_len;
+				droq->read_idx = lio_incr_index(
+							droq->read_idx,
+							1, droq->max_count);
+				droq->refill_count++;
+
+				/* Prefetch buffer pointers when on a
+				 * cache line boundary
+				 */
+				if ((droq->read_idx & 3) == 0) {
+					rte_prefetch0(&droq->recv_buf_list
+							      [droq->read_idx]);
+
+					rte_prefetch0(
+					    &droq->info_list[droq->read_idx]);
+				}
+			}
+			rx_pkts[data_pkts++] = first_buf;
+			if (rh->r_dh.has_hash)
+				data_total_len += (pkt_len - 8);
+			else
+				data_total_len += pkt_len;
+		}
+
+		/* Inform upper layer about packet checksum verification */
+		struct rte_mbuf *m = rx_pkts[data_pkts - 1];
+
+		if (rh->r_dh.csum_verified & LIO_IP_CSUM_VERIFIED)
+			m->ol_flags |= PKT_RX_IP_CKSUM_GOOD;
+
+		if (rh->r_dh.csum_verified & LIO_L4_CSUM_VERIFIED)
+			m->ol_flags |= PKT_RX_L4_CKSUM_GOOD;
+	}
+
+	if (droq->refill_count >= droq->refill_threshold) {
+		int desc_refilled = lio_droq_refill(lio_dev, droq);
+
+		/* Flush the droq descriptor data to memory to be sure
+		 * that when we update the credits the data in memory is
+		 * accurate.
+		 */
+		rte_wmb();
+		rte_write32(desc_refilled, droq->pkts_credit_reg);
+		/* make sure mmio write completes */
+		rte_wmb();
+	}
+
+	info->length = 0;
+	info->rh.rh64 = 0;
+
+	return data_pkts;
+}
+
+static uint32_t
+lio_droq_fast_process_packets(struct lio_device *lio_dev,
+			      struct lio_droq *droq,
+			      struct rte_mbuf **rx_pkts,
+			      uint32_t pkts_to_process)
+{
+	int ret, data_pkts = 0;
+	uint32_t pkt;
+
+	for (pkt = 0; pkt < pkts_to_process; pkt++) {
+		ret = lio_droq_fast_process_packet(lio_dev, droq,
+						   &rx_pkts[data_pkts]);
+		if (ret < 0) {
+			lio_dev_err(lio_dev, "Port[%d] DROQ[%d] idx: %d len:0, pkt_cnt: %d\n",
+				    lio_dev->port_id, droq->q_no,
+				    droq->read_idx, pkts_to_process);
+			break;
+		}
+		data_pkts += ret;
+	}
+
+	rte_atomic64_sub(&droq->pkts_pending, pkt);
+
+	return data_pkts;
+}
+
+static inline uint32_t
+lio_droq_check_hw_for_pkts(struct lio_droq *droq)
+{
+	uint32_t last_count;
+	uint32_t pkt_count;
+
+	pkt_count = rte_read32(droq->pkts_sent_reg);
+
+	last_count = pkt_count - droq->pkt_count;
+	droq->pkt_count = pkt_count;
+
+	if (last_count)
+		rte_atomic64_add(&droq->pkts_pending, last_count);
+
+	return last_count;
+}
+
+uint16_t
+lio_dev_recv_pkts(void *rx_queue,
+		  struct rte_mbuf **rx_pkts,
+		  uint16_t budget)
+{
+	struct lio_droq *droq = rx_queue;
+	struct lio_device *lio_dev = droq->lio_dev;
+	uint32_t pkts_processed = 0;
+	uint32_t pkt_count = 0;
+
+	lio_droq_check_hw_for_pkts(droq);
+
+	pkt_count = rte_atomic64_read(&droq->pkts_pending);
+	if (!pkt_count)
+		return 0;
+
+	if (pkt_count > budget)
+		pkt_count = budget;
+
+	/* Grab the lock */
+	rte_spinlock_lock(&droq->lock);
+	pkts_processed = lio_droq_fast_process_packets(lio_dev,
+						       droq, rx_pkts,
+						       pkt_count);
+
+	if (droq->pkt_count) {
+		rte_write32(droq->pkt_count, droq->pkts_sent_reg);
+		droq->pkt_count = 0;
+	}
+
+	/* Release the spin lock */
+	rte_spinlock_unlock(&droq->lock);
+
+	return pkts_processed;
+}
+
 /**
  *  lio_init_instr_queue()
  *  @param lio_dev	- pointer to the lio device structure.
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 4d742de..ccf9ca3 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -515,6 +515,17 @@ enum {
 	return (uint64_t)dma_addr;
 }
 
+static inline int
+lio_opcode_slow_path(union octeon_rh *rh)
+{
+	uint16_t subcode1, subcode2;
+
+	subcode1 = LIO_OPCODE_SUBCODE(rh->r.opcode, rh->r.subcode);
+	subcode2 = LIO_OPCODE_SUBCODE(LIO_OPCODE, LIO_OPCODE_NW_DATA);
+
+	return subcode2 != subcode1;
+}
+
 /* Macro to increment index.
  * Index is incremented by count; if the sum exceeds
  * max, index is wrapped-around to the start.
@@ -533,6 +544,8 @@ enum {
 int lio_setup_droq(struct lio_device *lio_dev, int q_no, int num_descs,
 		   int desc_size, struct rte_mempool *mpool,
 		   unsigned int socket_id);
+uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+			   uint16_t budget);
 
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
-- 
1.8.3.1

[PATCH 26/50] net/liquidio: add API to release Rx queue

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 28 ++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.c   | 10 ++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  1 +
 3 files changed, 39 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 300baee..9778b3a 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -121,6 +121,33 @@
 	return 0;
 }
 
+/**
+ * Release the receive queue/ringbuffer. Called by
+ * the upper layers.
+ *
+ * @param rxq
+ *    Opaque pointer to the receive queue to release
+ *
+ * @return
+ *    - nothing
+ */
+static void
+lio_dev_rx_queue_release(void *rxq)
+{
+	struct lio_droq *droq = rxq;
+	struct lio_device *lio_dev = droq->lio_dev;
+	int oq_no;
+
+	/* Run time queue deletion not supported */
+	if (lio_dev->port_configured)
+		return;
+
+	if (droq != NULL) {
+		oq_no = droq->q_no;
+		lio_delete_droq_queue(droq->lio_dev, oq_no);
+	}
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -266,6 +293,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
+	.rx_queue_release	= lio_dev_rx_queue_release,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 7c6b446..b7687a1 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -706,6 +706,16 @@
 	return pkts_processed;
 }
 
+void
+lio_delete_droq_queue(struct lio_device *lio_dev,
+		      int oq_no)
+{
+	lio_delete_droq(lio_dev, oq_no);
+	lio_dev->num_oqs--;
+	rte_free(lio_dev->droq[oq_no]);
+	lio_dev->droq[oq_no] = NULL;
+}
+
 /**
  *  lio_init_instr_queue()
  *  @param lio_dev	- pointer to the lio device structure.
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index ccf9ca3..41c351a 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -546,6 +546,7 @@ int lio_setup_droq(struct lio_device *lio_dev, int q_no, int num_descs,
 		   unsigned int socket_id);
 uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			   uint16_t budget);
+void lio_delete_droq_queue(struct lio_device *lio_dev, int oq_no);
 
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
-- 
1.8.3.1

[PATCH 27/50] net/liquidio: add API to setup Tx queue

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 60 +++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.c   | 39 +++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  3 ++
 3 files changed, 102 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 9778b3a..d096edc 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -148,6 +148,65 @@
 	}
 }
 
+/**
+ * Allocate and initialize SW ring. Initialize associated HW registers.
+ *
+ * @param eth_dev
+ *   Pointer to structure rte_eth_dev
+ *
+ * @param q_no
+ *   Queue number
+ *
+ * @param num_tx_descs
+ *   Number of ringbuffer descriptors
+ *
+ * @param socket_id
+ *   NUMA socket id, used for memory allocations
+ *
+ * @param tx_conf
+ *   Pointer to the structure rte_eth_txconf
+ *
+ * @return
+ *   - On success, return 0
+ *   - On failure, return -errno value
+ */
+static int
+lio_dev_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t q_no,
+		       uint16_t num_tx_descs, unsigned int socket_id,
+		       const struct rte_eth_txconf *tx_conf __rte_unused)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	int fw_mapped_iq = lio_dev->linfo.txpciq[q_no].s.q_no;
+	int retval;
+
+	if (q_no >= lio_dev->nb_tx_queues) {
+		lio_dev_err(lio_dev, "Invalid tx queue number %u\n", q_no);
+		return -EINVAL;
+	}
+
+	lio_dev_dbg(lio_dev, "setting up tx queue %u\n", q_no);
+
+	if ((lio_dev->instr_queue[fw_mapped_iq] != NULL) &&
+	    (num_tx_descs != lio_dev->instr_queue[fw_mapped_iq]->max_count)) {
+		lio_dev_err(lio_dev,
+			    "Reconfiguring Tx descs not supported. Configure descs to same value %u or restart application\n",
+			    lio_dev->instr_queue[fw_mapped_iq]->max_count);
+		return -ENOTSUP;
+	}
+
+	retval = lio_setup_iq(lio_dev, q_no, lio_dev->linfo.txpciq[q_no],
+			      num_tx_descs, lio_dev, socket_id);
+
+	if (retval) {
+		lio_dev_err(lio_dev, "Runtime IQ(TxQ) creation failed.\n");
+		return retval;
+	}
+
+	eth_dev->data->tx_queues[q_no] = lio_dev->instr_queue[fw_mapped_iq];
+
+	return 0;
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -294,6 +353,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.dev_configure		= lio_dev_configure,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
+	.tx_queue_setup		= lio_dev_tx_queue_setup,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index b7687a1..11319b2 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -861,6 +861,45 @@
 	lio_dev->num_iqs--;
 }
 
+/* Return 0 on success, -1 on failure */
+int
+lio_setup_iq(struct lio_device *lio_dev, int q_index,
+	     union octeon_txpciq txpciq, uint32_t num_descs, void *app_ctx,
+	     unsigned int socket_id)
+{
+	uint32_t iq_no = (uint32_t)txpciq.s.q_no;
+
+	if (lio_dev->instr_queue[iq_no]) {
+		lio_dev_dbg(lio_dev, "IQ is in use. Cannot create the IQ: %d again\n",
+			    iq_no);
+		lio_dev->instr_queue[iq_no]->txpciq.txpciq64 = txpciq.txpciq64;
+		lio_dev->instr_queue[iq_no]->app_ctx = app_ctx;
+		return 0;
+	}
+
+	lio_dev->instr_queue[iq_no] = rte_zmalloc_socket("ethdev TX queue",
+						sizeof(struct lio_instr_queue),
+						RTE_CACHE_LINE_SIZE, socket_id);
+	if (lio_dev->instr_queue[iq_no] == NULL)
+		return -1;
+
+	lio_dev->instr_queue[iq_no]->q_index = q_index;
+	lio_dev->instr_queue[iq_no]->app_ctx = app_ctx;
+
+	if (lio_init_instr_queue(lio_dev, txpciq, num_descs, socket_id))
+		goto release_lio_iq;
+
+	lio_dev->num_iqs++;
+
+	return 0;
+
+release_lio_iq:
+	rte_free(lio_dev->instr_queue[iq_no]);
+	lio_dev->instr_queue[iq_no] = NULL;
+
+	return -1;
+}
+
 static inline void
 lio_ring_doorbell(struct lio_device *lio_dev,
 		  struct lio_instr_queue *iq)
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 41c351a..6f8ec75 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -548,6 +548,9 @@ uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			   uint16_t budget);
 void lio_delete_droq_queue(struct lio_device *lio_dev, int oq_no);
 
+int lio_setup_iq(struct lio_device *lio_dev, int q_index,
+		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
+		 unsigned int socket_id);
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
-- 
1.8.3.1

[PATCH 28/50] net/liquidio: add APIs for sg list

[Shijith Thotton]

SG list is used while sending packets with multiple segments.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c |  23 ++++++++
 drivers/net/liquidio/lio_rxtx.c   | 107 ++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |   7 +++
 drivers/net/liquidio/lio_struct.h |  40 ++++++++++++++
 4 files changed, 177 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index d096edc..8488342 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -202,6 +202,15 @@
 		return retval;
 	}
 
+	retval = lio_setup_sglists(lio_dev, q_no, fw_mapped_iq,
+				lio_dev->instr_queue[fw_mapped_iq]->max_count,
+				socket_id);
+
+	if (retval) {
+		lio_delete_instruction_queue(lio_dev, fw_mapped_iq);
+		return retval;
+	}
+
 	eth_dev->data->tx_queues[q_no] = lio_dev->instr_queue[fw_mapped_iq];
 
 	return 0;
@@ -334,6 +343,20 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	/* Copy the permanent MAC address */
 	ether_addr_copy((struct ether_addr *)mac, &eth_dev->data->mac_addrs[0]);
 
+	lio_dev->glist_lock =
+	    rte_zmalloc(NULL, sizeof(*lio_dev->glist_lock) * num_iqueues, 0);
+	if (lio_dev->glist_lock == NULL)
+		return -ENOMEM;
+
+	lio_dev->glist_head =
+		rte_zmalloc(NULL, sizeof(*lio_dev->glist_head) * num_iqueues,
+			    0);
+	if (lio_dev->glist_head == NULL) {
+		rte_free(lio_dev->glist_lock);
+		lio_dev->glist_lock = NULL;
+		return -ENOMEM;
+	}
+
 	lio_dev->port_configured = 1;
 
 	lio_free_soft_command(sc);
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 11319b2..ec9e184 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -40,6 +40,8 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+#define LIO_MAX_SG 12
+
 static void
 lio_droq_compute_max_packet_bufs(struct lio_droq *droq)
 {
@@ -1272,3 +1274,108 @@ struct lio_soft_command *
 
 	return 0;
 }
+
+static inline struct lio_clist_node *
+list_delete_head(struct lio_clist_head *head)
+{
+	struct lio_clist_node *node;
+
+	if (CIRCLEQ_EMPTY(head))
+		node = NULL;
+	else
+		node = CIRCLEQ_FIRST(head);
+
+	if (node)
+		CIRCLEQ_REMOVE(head, node, entries);
+
+	return node;
+}
+
+static void
+lio_delete_sglist(struct lio_instr_queue *txq)
+{
+	struct lio_device *lio_dev = txq->lio_dev;
+	int iq_no = txq->q_index;
+	struct lio_gather *g;
+
+	if (lio_dev->glist_head == NULL)
+		return;
+
+	do {
+		g = (struct lio_gather *)list_delete_head(
+						&lio_dev->glist_head[iq_no]);
+		if (g) {
+			if (g->sg)
+				rte_free(
+				    (void *)((unsigned long)g->sg - g->adjust));
+			rte_free(g);
+		}
+	} while (g);
+}
+
+/**
+ * \brief Setup gather lists
+ * @param lio per-network private data
+ */
+int
+lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
+		  int fw_mapped_iq, int num_descs, unsigned int socket_id)
+{
+	struct lio_gather *g;
+	int i;
+
+	rte_spinlock_init(&lio_dev->glist_lock[iq_no]);
+
+	CIRCLEQ_INIT(&lio_dev->glist_head[iq_no]);
+
+	for (i = 0; i < num_descs; i++) {
+		g = rte_zmalloc_socket(NULL, sizeof(*g), RTE_CACHE_LINE_SIZE,
+				       socket_id);
+		if (g == NULL) {
+			lio_dev_err(lio_dev,
+				    "lio_gather memory allocation failed for qno %d\n",
+				    iq_no);
+			break;
+		}
+
+		g->sg_size =
+		    ((ROUNDUP4(LIO_MAX_SG) >> 2) * LIO_SG_ENTRY_SIZE);
+
+		g->sg = rte_zmalloc_socket(NULL, g->sg_size + 8,
+					   RTE_CACHE_LINE_SIZE, socket_id);
+		if (g->sg == NULL) {
+			lio_dev_err(lio_dev,
+				    "sg list memory allocation failed for qno %d\n",
+				    iq_no);
+			rte_free(g);
+			break;
+		}
+
+		/* The gather component should be aligned on 64-bit boundary */
+		if (((unsigned long)g->sg) & 7) {
+			g->adjust = 8 - (((unsigned long)g->sg) & 7);
+			g->sg =
+			    (struct lio_sg_entry *)((unsigned long)g->sg +
+						       g->adjust);
+		}
+
+		CIRCLEQ_INSERT_TAIL(&lio_dev->glist_head[iq_no], &g->list,
+				    entries);
+	}
+
+	if (i != num_descs) {
+		lio_delete_sglist(lio_dev->instr_queue[fw_mapped_iq]);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+void
+lio_delete_instruction_queue(struct lio_device *lio_dev, int iq_no)
+{
+	lio_delete_instr_queue(lio_dev, iq_no);
+	rte_free(lio_dev->instr_queue[iq_no]);
+	lio_dev->instr_queue[iq_no] = NULL;
+	lio_dev->num_iqs--;
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 6f8ec75..3fe178a 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -42,6 +42,10 @@
 
 #include "lio_struct.h"
 
+#ifndef ROUNDUP4
+#define ROUNDUP4(val) (((val) + 3) & 0xfffffffc)
+#endif
+
 #define LIO_CLIST_FIRST_ENTRY(ptr, type, elem)	\
 	(type *)((char *)((ptr)->cqh_first) - offsetof(type, elem))
 
@@ -548,9 +552,12 @@ uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			   uint16_t budget);
 void lio_delete_droq_queue(struct lio_device *lio_dev, int oq_no);
 
+int lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
+		      int fw_mapped_iq, int num_descs, unsigned int socket_id);
 int lio_setup_iq(struct lio_device *lio_dev, int q_index,
 		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
 		 unsigned int socket_id);
+void lio_delete_instruction_queue(struct lio_device *lio_dev, int iq_no);
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index bb4618f..1280785 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -298,6 +298,41 @@ struct lio_instr_queue {
 	const struct rte_memzone *iq_mz;
 };
 
+/* The Scatter-Gather List Entry. The scatter or gather component used with
+ * input instruction has this format.
+ */
+struct lio_sg_entry {
+	/** The first 64 bit gives the size of data in each dptr. */
+	union {
+		uint16_t size[4];
+		uint64_t size64;
+	} u;
+
+	/** The 4 dptr pointers for this entry. */
+	uint64_t ptr[4];
+};
+
+#define LIO_SG_ENTRY_SIZE	(sizeof(struct lio_sg_entry))
+
+/** Structure of a node in list of gather components maintained by
+ *  driver for each network device.
+ */
+struct lio_gather {
+	/** List manipulation. Next and prev pointers. */
+	struct lio_clist_node list;
+
+	/** Size of the gather component at sg in bytes. */
+	int sg_size;
+
+	/** Number of bytes that sg was adjusted to make it 8B-aligned. */
+	int adjust;
+
+	/** Gather component that can accommodate max sized fragment list
+	 *  received from the IP layer.
+	 */
+	struct lio_sg_entry *sg;
+};
+
 struct lio_io_enable {
 	uint64_t iq;
 	uint64_t oq;
@@ -516,6 +551,11 @@ struct lio_device {
 
 	uint32_t num_iqs;
 
+	/** Guards each glist */
+	rte_spinlock_t *glist_lock;
+	/** Array of gather component linked lists */
+	struct lio_clist_head *glist_head;
+
 	/* The pool containing pre allocated buffers used for soft commands */
 	struct rte_mempool *sc_buf_pool;
 
-- 
1.8.3.1

Re: [PATCH 28/50] net/liquidio: add APIs for sg list

[Ferruh Yigit]

On 2/21/2017 9:26 AM, Shijith Thotton wrote:
> SG list is used while sending packets with multiple segments.

May be good to use "Scatter-Gather List" in commit log to make it clear,
and since it is an abbreviation, uppercase "SG" can be used in patch title.

> 
> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
> ---
<...>

[PATCH 29/50] net/liquidio: add API to enable and disable IO queues

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 70 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.c       | 13 ++++++
 drivers/net/liquidio/lio_rxtx.c         |  5 +++
 drivers/net/liquidio/lio_struct.h       |  2 +
 4 files changed, 90 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index 44d90c0..6ff5b69 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -310,6 +310,73 @@
 	return 0;
 }
 
+static int
+cn23xx_vf_enable_io_queues(struct lio_device *lio_dev)
+{
+	uint32_t q_no;
+
+	PMD_INIT_FUNC_TRACE();
+
+	for (q_no = 0; q_no < lio_dev->num_iqs; q_no++) {
+		uint64_t reg_val;
+
+		/* set the corresponding IQ IS_64B bit */
+		if (lio_dev->io_qmask.iq64B & (1ULL << q_no)) {
+			reg_val = lio_read_csr64(
+					lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+			reg_val = reg_val | CN23XX_PKT_INPUT_CTL_IS_64B;
+			lio_write_csr64(lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+					reg_val);
+		}
+
+		/* set the corresponding IQ ENB bit */
+		if (lio_dev->io_qmask.iq & (1ULL << q_no)) {
+			reg_val = lio_read_csr64(
+					lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+			reg_val = reg_val | CN23XX_PKT_INPUT_CTL_RING_ENB;
+			lio_write_csr64(lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+					reg_val);
+		}
+	}
+	for (q_no = 0; q_no < lio_dev->num_oqs; q_no++) {
+		uint32_t reg_val;
+
+		/* set the corresponding OQ ENB bit */
+		if (lio_dev->io_qmask.oq & (1ULL << q_no)) {
+			reg_val = lio_read_csr(
+					lio_dev,
+					CN23XX_SLI_OQ_PKT_CONTROL(q_no));
+			reg_val = reg_val | CN23XX_PKT_OUTPUT_CTL_RING_ENB;
+			lio_write_csr(lio_dev,
+				      CN23XX_SLI_OQ_PKT_CONTROL(q_no),
+				      reg_val);
+		}
+	}
+
+	return 0;
+}
+
+static void
+cn23xx_vf_disable_io_queues(struct lio_device *lio_dev)
+{
+	uint32_t num_queues;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* per HRM, rings can only be disabled via reset operation,
+	 * NOT via SLI_PKT()_INPUT/OUTPUT_CONTROL[ENB]
+	 */
+	num_queues = lio_dev->num_iqs;
+	if (num_queues < lio_dev->num_oqs)
+		num_queues = lio_dev->num_oqs;
+
+	cn23xx_vf_reset_io_queues(lio_dev, num_queues);
+}
+
 void
 cn23xx_vf_ask_pf_to_do_flr(struct lio_device *lio_dev)
 {
@@ -463,6 +530,9 @@
 
 	lio_dev->fn_list.setup_device_regs	= cn23xx_vf_setup_device_regs;
 
+	lio_dev->fn_list.enable_io_queues	= cn23xx_vf_enable_io_queues;
+	lio_dev->fn_list.disable_io_queues	= cn23xx_vf_disable_io_queues;
+
 	return 0;
 }
 
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 8488342..ff1fdad 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -361,6 +361,15 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 
 	lio_free_soft_command(sc);
 
+	/* Disable iq_0 for reconf */
+	lio_dev->fn_list.disable_io_queues(lio_dev);
+
+	/* Reset ioq regs */
+	lio_dev->fn_list.setup_device_regs(lio_dev);
+
+	/* Free iq_0 used during init */
+	lio_free_instr_queue0(lio_dev);
+
 	return 0;
 
 nic_config_fail:
@@ -487,6 +496,10 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	lio_dev->max_tx_queues = dpdk_queues;
 	lio_dev->max_rx_queues = dpdk_queues;
 
+	/* Enable input and output queues for this device */
+	if (lio_dev->fn_list.enable_io_queues(lio_dev))
+		goto error;
+
 	return 0;
 
 error:
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index ec9e184..9ed0030 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -892,9 +892,14 @@
 		goto release_lio_iq;
 
 	lio_dev->num_iqs++;
+	if (lio_dev->fn_list.enable_io_queues(lio_dev))
+		goto delete_lio_iq;
 
 	return 0;
 
+delete_lio_iq:
+	lio_delete_instr_queue(lio_dev, iq_no);
+	lio_dev->num_iqs--;
 release_lio_iq:
 	rte_free(lio_dev->instr_queue[iq_no]);
 	lio_dev->instr_queue[iq_no] = NULL;
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 1280785..be3bf79 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -347,6 +347,8 @@ struct lio_fn_list {
 	void (*free_mbox)(struct lio_device *);
 
 	int (*setup_device_regs)(struct lio_device *);
+	int (*enable_io_queues)(struct lio_device *);
+	void (*disable_io_queues)(struct lio_device *);
 };
 
 struct lio_pf_vf_hs_word {
-- 
1.8.3.1

[PATCH 30/50] net/liquidio: add Tx data path for single segment

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |   1 +
 drivers/net/liquidio/lio_ethdev.c       |   3 +
 drivers/net/liquidio/lio_rxtx.c         |  89 ++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         | 138 ++++++++++++++++++++++++++++++++
 4 files changed, 231 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 94a21ad..ed6d90c 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -108,6 +108,7 @@ enum octeon_tag_type {
 
 /* pre-defined host->NIC tag values */
 #define LIO_CONTROL	(0x11111110)
+#define LIO_DATA(i)	(0x11111111 + (i))
 
 /* used for NIC operations */
 #define LIO_OPCODE	1
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index ff1fdad..a5acce8 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -529,6 +529,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	eth_dev->data->mac_addrs = NULL;
 
 	eth_dev->rx_pkt_burst = NULL;
+	eth_dev->tx_pkt_burst = NULL;
 
 	return 0;
 }
@@ -543,6 +544,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	PMD_INIT_FUNC_TRACE();
 
 	eth_dev->rx_pkt_burst = &lio_dev_recv_pkts;
+	eth_dev->tx_pkt_burst = &lio_dev_xmit_pkts;
 
 	/* Primary does the initialization. */
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
@@ -580,6 +582,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 			    "MAC addresses memory allocation failed\n");
 		eth_dev->dev_ops = NULL;
 		eth_dev->rx_pkt_burst = NULL;
+		eth_dev->tx_pkt_burst = NULL;
 		return -ENOMEM;
 	}
 
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 9ed0030..9ec4c12 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1384,3 +1384,92 @@ struct lio_soft_command *
 	lio_dev->instr_queue[iq_no] = NULL;
 	lio_dev->num_iqs--;
 }
+
+/** Send data packet to the device
+ *  @param lio_dev - lio device pointer
+ *  @param ndata   - control structure with queueing, and buffer information
+ *
+ *  @returns IQ_FAILED if it failed to add to the input queue. IQ_STOP if it the
+ *  queue should be stopped, and LIO_IQ_SEND_OK if it sent okay.
+ */
+static inline int
+lio_send_data_pkt(struct lio_device *lio_dev, struct lio_data_pkt *ndata)
+{
+	return lio_send_command(lio_dev, ndata->q_no, &ndata->cmd,
+				ndata->buf, ndata->datasize, ndata->reqtype);
+}
+
+uint16_t
+lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts, uint16_t nb_pkts)
+{
+	struct lio_instr_queue *txq = tx_queue;
+	union lio_cmd_setup cmdsetup;
+	struct lio_device *lio_dev;
+	struct lio_data_pkt ndata;
+	int i, processed = 0;
+	struct rte_mbuf *m;
+	uint32_t tag = 0;
+	int status = 0;
+	int iq_no;
+
+	lio_dev = txq->lio_dev;
+	iq_no = txq->txpciq.s.q_no;
+
+	if (!lio_dev->linfo.link.s.link_up) {
+		PMD_TX_LOG(lio_dev, ERR, "Transmit failed link_status : %d\n",
+			   lio_dev->linfo.link.s.link_up);
+		goto xmit_failed;
+	}
+
+	for (i = 0; i < nb_pkts; i++) {
+		uint32_t pkt_len = 0;
+
+		m = pkts[i];
+
+		/* Prepare the attributes for the data to be passed to BASE. */
+		memset(&ndata, 0, sizeof(struct lio_data_pkt));
+
+		ndata.buf = m;
+
+		ndata.q_no = iq_no;
+
+		cmdsetup.cmd_setup64 = 0;
+		cmdsetup.s.iq_no = iq_no;
+
+		/* check checksum offload flags to form cmd */
+		if (m->ol_flags & PKT_TX_IP_CKSUM)
+			cmdsetup.s.ip_csum = 1;
+
+		if ((m->ol_flags & PKT_TX_TCP_CKSUM) ||
+				(m->ol_flags & PKT_TX_UDP_CKSUM))
+			cmdsetup.s.transport_csum = 1;
+
+		if (m->nb_segs == 1) {
+			pkt_len = rte_pktmbuf_data_len(m);
+			cmdsetup.s.u.datasize = pkt_len;
+			lio_prepare_pci_cmd(lio_dev, &ndata.cmd,
+					    &cmdsetup, tag);
+			ndata.cmd.cmd3.dptr = rte_mbuf_data_dma_addr(m);
+			ndata.reqtype = LIO_REQTYPE_NORESP_NET;
+		}
+
+		ndata.datasize = pkt_len;
+
+		status = lio_send_data_pkt(lio_dev, &ndata);
+
+		if (unlikely(status == LIO_IQ_SEND_FAILED)) {
+			PMD_TX_LOG(lio_dev, ERR, "send failed\n");
+			break;
+		}
+
+		if (unlikely(status == LIO_IQ_SEND_STOP))
+			PMD_TX_LOG(lio_dev, DEBUG, "iq full\n");
+
+		processed++;
+	}
+
+xmit_failed:
+
+	return processed;
+}
+
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 3fe178a..1eff46f 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -249,6 +249,50 @@ struct lio_iq_post_status {
 
 #define OCTEON_CMD_SIZE (sizeof(union octeon_cmd))
 
+/** Structure of data information passed by driver to the BASE
+ *  layer when forwarding data to Octeon device software.
+ */
+struct lio_data_pkt {
+	/** Pointer to information maintained by NIC module for this packet. The
+	 *  BASE layer passes this as-is to the driver.
+	 */
+	void *buf;
+
+	/** Type of buffer passed in "buf" above. */
+	uint32_t reqtype;
+
+	/** Total data bytes to be transferred in this command. */
+	uint32_t datasize;
+
+	/** Command to be passed to the Octeon device software. */
+	union lio_instr_64B cmd;
+
+	/** Input queue to use to send this command. */
+	uint32_t q_no;
+};
+
+/** Structure passed by driver to BASE layer to prepare a command to send
+ *  network data to Octeon.
+ */
+union lio_cmd_setup {
+	struct {
+		uint32_t iq_no : 8;
+		uint32_t gather : 1;
+		uint32_t timestamp : 1;
+		uint32_t ip_csum : 1;
+		uint32_t transport_csum : 1;
+		uint32_t tnl_csum : 1;
+		uint32_t rsvd : 19;
+
+		union {
+			uint32_t datasize;
+			uint32_t gatherptrs;
+		} u;
+	} s;
+
+	uint64_t cmd_setup64;
+};
+
 /* Instruction Header */
 struct octeon_instr_ih3 {
 #if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
@@ -418,6 +462,98 @@ struct octeon_instr_rdp {
 #endif
 };
 
+union octeon_packet_params {
+	uint32_t pkt_params32;
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint32_t reserved : 24;
+		uint32_t ip_csum : 1; /* Perform IP header checksum(s) */
+		/* Perform Outer transport header checksum */
+		uint32_t transport_csum : 1;
+		/* Find tunnel, and perform transport csum. */
+		uint32_t tnl_csum : 1;
+		uint32_t tsflag : 1;   /* Timestamp this packet */
+		uint32_t ipsec_ops : 4; /* IPsec operation */
+#else
+		uint32_t ipsec_ops : 4;
+		uint32_t tsflag : 1;
+		uint32_t tnl_csum : 1;
+		uint32_t transport_csum : 1;
+		uint32_t ip_csum : 1;
+		uint32_t reserved : 7;
+#endif
+	} s;
+};
+
+/** Utility function to prepare a 64B NIC instruction based on a setup command
+ * @param cmd - pointer to instruction to be filled in.
+ * @param setup - pointer to the setup structure
+ * @param q_no - which queue for back pressure
+ *
+ * Assumes the cmd instruction is pre-allocated, but no fields are filled in.
+ */
+static inline void
+lio_prepare_pci_cmd(struct lio_device *lio_dev,
+		    union lio_instr_64B *cmd,
+		    union lio_cmd_setup *setup,
+		    uint32_t tag)
+{
+	union octeon_packet_params packet_params;
+	struct octeon_instr_pki_ih3 *pki_ih3;
+	struct octeon_instr_irh *irh;
+	struct octeon_instr_ih3 *ih3;
+	int port;
+
+	memset(cmd, 0, sizeof(union lio_instr_64B));
+
+	ih3 = (struct octeon_instr_ih3 *)&cmd->cmd3.ih3;
+	pki_ih3 = (struct octeon_instr_pki_ih3 *)&cmd->cmd3.pki_ih3;
+
+	/* assume that rflag is cleared so therefore front data will only have
+	 * irh and ossp[1] and ossp[2] for a total of 24 bytes
+	 */
+	ih3->pkind = lio_dev->instr_queue[setup->s.iq_no]->txpciq.s.pkind;
+	/* PKI IH */
+	ih3->fsz = OCTEON_PCI_CMD_O3;
+
+	if (!setup->s.gather) {
+		ih3->dlengsz = setup->s.u.datasize;
+	} else {
+		ih3->gather = 1;
+		ih3->dlengsz = setup->s.u.gatherptrs;
+	}
+
+	pki_ih3->w = 1;
+	pki_ih3->raw = 0;
+	pki_ih3->utag = 0;
+	pki_ih3->utt = 1;
+	pki_ih3->uqpg = lio_dev->instr_queue[setup->s.iq_no]->txpciq.s.use_qpg;
+
+	port = (int)lio_dev->instr_queue[setup->s.iq_no]->txpciq.s.port;
+
+	if (tag)
+		pki_ih3->tag = tag;
+	else
+		pki_ih3->tag = LIO_DATA(port);
+
+	pki_ih3->tagtype = OCTEON_ORDERED_TAG;
+	pki_ih3->qpg = lio_dev->instr_queue[setup->s.iq_no]->txpciq.s.qpg;
+	pki_ih3->pm = 0x0; /* parse from L2 */
+	pki_ih3->sl = 32;  /* sl will be sizeof(pki_ih3) + irh + ossp0 + ossp1*/
+
+	irh = (struct octeon_instr_irh *)&cmd->cmd3.irh;
+
+	irh->opcode = LIO_OPCODE;
+	irh->subcode = LIO_OPCODE_NW_DATA;
+
+	packet_params.pkt_params32 = 0;
+	packet_params.s.ip_csum = setup->s.ip_csum;
+	packet_params.s.transport_csum = setup->s.transport_csum;
+	packet_params.s.tsflag = setup->s.timestamp;
+
+	irh->ossp = packet_params.pkt_params32;
+}
+
 int lio_setup_sc_buffer_pool(struct lio_device *lio_dev);
 void lio_free_sc_buffer_pool(struct lio_device *lio_dev);
 
@@ -554,6 +690,8 @@ uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 
 int lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
 		      int fw_mapped_iq, int num_descs, unsigned int socket_id);
+uint16_t lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts,
+			   uint16_t nb_pkts);
 int lio_setup_iq(struct lio_device *lio_dev, int q_index,
 		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
 		 unsigned int socket_id);
-- 
1.8.3.1

Re: [PATCH 30/50] net/liquidio: add Tx data path for single segment

[Ferruh Yigit]

On 2/21/2017 9:26 AM, Shijith Thotton wrote:
> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
> ---

<...>

> @@ -543,6 +544,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
>  	PMD_INIT_FUNC_TRACE();
>  
>  	eth_dev->rx_pkt_burst = &lio_dev_recv_pkts;
> +	eth_dev->tx_pkt_burst = &lio_dev_xmit_pkts;

Not directly related to this patch, but double check, if tx_prep
required for this driver.

Since it is relatively new feature, it may be good to verify.

[PATCH 31/50] net/liquidio: add Tx data path for multiple segments

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_rxtx.c   | 62 +++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   | 11 +++++++
 drivers/net/liquidio/lio_struct.h | 24 +++++++++++++++
 3 files changed, 97 insertions(+)

diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 9ec4c12..32ef0fd 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1451,6 +1451,68 @@ struct lio_soft_command *
 					    &cmdsetup, tag);
 			ndata.cmd.cmd3.dptr = rte_mbuf_data_dma_addr(m);
 			ndata.reqtype = LIO_REQTYPE_NORESP_NET;
+		} else {
+			struct lio_buf_free_info *finfo;
+			struct lio_gather *g;
+			phys_addr_t phyaddr;
+			int i, frags;
+
+			finfo = (struct lio_buf_free_info *)rte_malloc(NULL,
+							sizeof(*finfo), 0);
+			if (finfo == NULL) {
+				PMD_TX_LOG(lio_dev, ERR,
+					   "free buffer alloc failed\n");
+				goto xmit_failed;
+			}
+
+			rte_spinlock_lock(&lio_dev->glist_lock[iq_no]);
+			g = (struct lio_gather *)list_delete_head(
+						&lio_dev->glist_head[iq_no]);
+			rte_spinlock_unlock(&lio_dev->glist_lock[iq_no]);
+			if (g == NULL) {
+				PMD_TX_LOG(lio_dev, ERR,
+					   "Transmit scatter gather: glist null!\n");
+				goto xmit_failed;
+			}
+
+			cmdsetup.s.gather = 1;
+			cmdsetup.s.u.gatherptrs = m->nb_segs;
+			lio_prepare_pci_cmd(lio_dev, &ndata.cmd,
+					    &cmdsetup, tag);
+
+			memset(g->sg, 0, g->sg_size);
+			g->sg[0].ptr[0] = rte_mbuf_data_dma_addr(m);
+			lio_add_sg_size(&g->sg[0], m->data_len, 0);
+			pkt_len = m->data_len;
+			finfo->mbuf = m;
+
+			/* First seg taken care above */
+			frags = m->nb_segs - 1;
+			i = 1;
+			m = m->next;
+			while (frags--) {
+				g->sg[(i >> 2)].ptr[(i & 3)] =
+						rte_mbuf_data_dma_addr(m);
+				lio_add_sg_size(&g->sg[(i >> 2)],
+						m->data_len, (i & 3));
+				pkt_len += m->data_len;
+				i++;
+				m = m->next;
+			}
+
+			phyaddr = rte_mem_virt2phy(g->sg);
+			if (phyaddr == RTE_BAD_PHYS_ADDR) {
+				PMD_TX_LOG(lio_dev, ERR, "bad phys addr\n");
+				goto xmit_failed;
+			}
+
+			ndata.cmd.cmd3.dptr = phyaddr;
+			ndata.reqtype = LIO_REQTYPE_NORESP_NET_SG;
+
+			finfo->g = g;
+			finfo->lio_dev = lio_dev;
+			finfo->iq_no = (uint64_t)iq_no;
+			ndata.buf = finfo;
 		}
 
 		ndata.datasize = pkt_len;
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 1eff46f..dbf8685 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -666,6 +666,17 @@ enum {
 	return subcode2 != subcode1;
 }
 
+static inline void
+lio_add_sg_size(struct lio_sg_entry *sg_entry,
+		uint16_t size, uint32_t pos)
+{
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	sg_entry->u.size[pos] = size;
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	sg_entry->u.size[3 - pos] = size;
+#endif
+}
+
 /* Macro to increment index.
  * Index is incremented by count; if the sum exceeds
  * max, index is wrapped-around to the start.
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index be3bf79..de4c1f3 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -298,6 +298,30 @@ struct lio_instr_queue {
 	const struct rte_memzone *iq_mz;
 };
 
+/** This structure is used by driver to store information required
+ *  to free the mbuff when the packet has been fetched by Octeon.
+ *  Bytes offset below assume worst-case of a 64-bit system.
+ */
+struct lio_buf_free_info {
+	/** Bytes 1-8. Pointer to network device private structure. */
+	struct lio_device *lio_dev;
+
+	/** Bytes 9-16. Pointer to mbuff. */
+	struct rte_mbuf *mbuf;
+
+	/** Bytes 17-24. Pointer to gather list. */
+	struct lio_gather *g;
+
+	/** Bytes 25-32. Physical address of mbuf->data or gather list. */
+	uint64_t dptr;
+
+	/** Bytes 33-47. Piggybacked soft command, if any */
+	struct lio_soft_command *sc;
+
+	/** Bytes 48-63. iq no */
+	uint64_t iq_no;
+};
+
 /* The Scatter-Gather List Entry. The scatter or gather component used with
  * input instruction has this format.
  */
-- 
1.8.3.1

[PATCH 32/50] net/liquidio: add APIs to flush IQ and free buffers

[Shijith Thotton]

Add support for instruction queue flush. Check how many packets
reached device from host and free host buffers accordingly from request
list.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c |   1 +
 drivers/net/liquidio/lio_rxtx.c   | 187 +++++++++++++++++++++++++++++++++++++-
 drivers/net/liquidio/lio_rxtx.h   |   1 +
 3 files changed, 188 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index a5acce8..bf85089 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -281,6 +281,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	 * response arrived or timed-out.
 	 */
 	while ((*sc->status_word == LIO_COMPLETION_WORD_INIT) && --timeout) {
+		lio_flush_iq(lio_dev, lio_dev->instr_queue[sc->iq_no]);
 		lio_process_ordered_list(lio_dev);
 		rte_delay_ms(1);
 	}
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 32ef0fd..448c8b9 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -41,6 +41,9 @@
 #include "lio_rxtx.h"
 
 #define LIO_MAX_SG 12
+/* Flush iq if available tx_desc fall below LIO_FLUSH_WM */
+#define LIO_FLUSH_WM(_iq) ((_iq)->max_count / 2)
+#define LIO_PKT_IN_DONE_CNT_MASK 0x00000000FFFFFFFFULL
 
 static void
 lio_droq_compute_max_packet_bufs(struct lio_droq *droq)
@@ -977,6 +980,146 @@
 	iq->request_list[idx].reqtype = reqtype;
 }
 
+static inline void
+lio_free_netsgbuf(void *buf)
+{
+	struct lio_buf_free_info *finfo = buf;
+	struct lio_device *lio_dev = finfo->lio_dev;
+	struct rte_mbuf *m = finfo->mbuf;
+	struct lio_gather *g = finfo->g;
+	uint8_t iq = finfo->iq_no;
+
+	/* This will take care of multiple segments also */
+	rte_pktmbuf_free(m);
+
+	rte_spinlock_lock(&lio_dev->glist_lock[iq]);
+	CIRCLEQ_INSERT_TAIL(&lio_dev->glist_head[iq], &g->list, entries);
+	rte_spinlock_unlock(&lio_dev->glist_lock[iq]);
+	rte_free(finfo);
+}
+
+/* Can only run in process context */
+static int
+lio_process_iq_request_list(struct lio_device *lio_dev,
+			    struct lio_instr_queue *iq)
+{
+	struct octeon_instr_irh *irh = NULL;
+	uint32_t old = iq->flush_index;
+	struct lio_soft_command *sc;
+	uint32_t inst_count = 0;
+	int reqtype;
+	void *buf;
+
+	while (old != iq->lio_read_index) {
+		reqtype = iq->request_list[old].reqtype;
+		buf     = iq->request_list[old].buf;
+
+		if (reqtype == LIO_REQTYPE_NONE)
+			goto skip_this;
+
+		switch (reqtype) {
+		case LIO_REQTYPE_NORESP_NET:
+			rte_pktmbuf_free((struct rte_mbuf *)buf);
+			break;
+		case LIO_REQTYPE_NORESP_NET_SG:
+			lio_free_netsgbuf(buf);
+			break;
+		case LIO_REQTYPE_SOFT_COMMAND:
+			sc = buf;
+			irh = (struct octeon_instr_irh *)&sc->cmd.cmd3.irh;
+			if (irh->rflag) {
+				/* We're expecting a response from Octeon.
+				 * It's up to lio_process_ordered_list() to
+				 * process sc. Add sc to the ordered soft
+				 * command response list because we expect
+				 * a response from Octeon.
+				 */
+				rte_spinlock_lock(&lio_dev->response_list.lock);
+				rte_atomic64_inc(
+				    &lio_dev->response_list.pending_req_count);
+				CIRCLEQ_INSERT_TAIL(
+					&lio_dev->response_list.head,
+					&sc->node, entries);
+				rte_spinlock_unlock(
+						&lio_dev->response_list.lock);
+			} else {
+				if (sc->callback) {
+					/* This callback must not sleep */
+					sc->callback(LIO_REQUEST_DONE,
+						     sc->callback_arg);
+				}
+			}
+			break;
+		default:
+			lio_dev_err(lio_dev,
+				    "Unknown reqtype: %d buf: %p at idx %d\n",
+				    reqtype, buf, old);
+		}
+
+		iq->request_list[old].buf = NULL;
+		iq->request_list[old].reqtype = 0;
+
+ skip_this:
+		inst_count++;
+		old = lio_incr_index(old, 1, iq->max_count);
+	}
+
+	iq->flush_index = old;
+
+	return inst_count;
+}
+
+static void
+lio_update_read_index(struct lio_instr_queue *iq)
+{
+	uint32_t pkt_in_done = rte_read32(iq->inst_cnt_reg);
+	uint32_t last_done;
+
+	last_done = pkt_in_done - iq->pkt_in_done;
+	iq->pkt_in_done = pkt_in_done;
+
+	/* Add last_done and modulo with the IQ size to get new index */
+	iq->lio_read_index = (iq->lio_read_index +
+			(uint32_t)(last_done & LIO_PKT_IN_DONE_CNT_MASK)) %
+			iq->max_count;
+}
+
+int
+lio_flush_iq(struct lio_device *lio_dev, struct lio_instr_queue *iq)
+{
+	uint32_t tot_inst_processed = 0;
+	uint32_t inst_processed = 0;
+	int tx_done = 1;
+
+	if (rte_atomic64_test_and_set(&iq->iq_flush_running) == 0)
+		return tx_done;
+
+	rte_spinlock_lock(&iq->lock);
+
+	lio_update_read_index(iq);
+
+	do {
+		/* Process any outstanding IQ packets. */
+		if (iq->flush_index == iq->lio_read_index)
+			break;
+
+		inst_processed = lio_process_iq_request_list(lio_dev, iq);
+
+		if (inst_processed)
+			rte_atomic64_sub(&iq->instr_pending, inst_processed);
+
+		tot_inst_processed += inst_processed;
+		inst_processed = 0;
+
+	} while (1);
+
+	rte_spinlock_unlock(&iq->lock);
+
+	rte_atomic64_clear(&iq->iq_flush_running);
+
+	return tx_done;
+}
+
 static int
 lio_send_command(struct lio_device *lio_dev, uint32_t iq_no, void *cmd,
 		 void *buf, uint32_t datasize __rte_unused, uint32_t reqtype)
@@ -1385,6 +1528,35 @@ struct lio_soft_command *
 	lio_dev->num_iqs--;
 }
 
+static inline uint32_t
+lio_iq_get_available(struct lio_device *lio_dev, uint32_t q_no)
+{
+	return ((lio_dev->instr_queue[q_no]->max_count - 1) -
+		(uint32_t)rte_atomic64_read(
+				&lio_dev->instr_queue[q_no]->instr_pending));
+}
+
+static inline int
+lio_iq_is_full(struct lio_device *lio_dev, uint32_t q_no)
+{
+	return ((uint32_t)rte_atomic64_read(
+				&lio_dev->instr_queue[q_no]->instr_pending) >=
+				(lio_dev->instr_queue[q_no]->max_count - 2));
+}
+
+static int
+lio_dev_cleanup_iq(struct lio_device *lio_dev, int iq_no)
+{
+	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
+	uint32_t count = 10000;
+
+	while ((lio_iq_get_available(lio_dev, iq_no) < LIO_FLUSH_WM(iq)) &&
+			--count)
+		lio_flush_iq(lio_dev, iq);
+
+	return count ? 0 : 1;
+}
+
 /** Send data packet to the device
  *  @param lio_dev - lio device pointer
  *  @param ndata   - control structure with queueing, and buffer information
@@ -1421,6 +1593,8 @@ struct lio_soft_command *
 		goto xmit_failed;
 	}
 
+	lio_dev_cleanup_iq(lio_dev, iq_no);
+
 	for (i = 0; i < nb_pkts; i++) {
 		uint32_t pkt_len = 0;
 
@@ -1432,6 +1606,14 @@ struct lio_soft_command *
 		ndata.buf = m;
 
 		ndata.q_no = iq_no;
+		if (lio_iq_is_full(lio_dev, ndata.q_no)) {
+			if (lio_dev_cleanup_iq(lio_dev, iq_no)) {
+				PMD_TX_LOG(lio_dev, ERR,
+					   "Transmit failed iq:%d full\n",
+					   ndata.q_no);
+				break;
+			}
+		}
 
 		cmdsetup.cmd_setup64 = 0;
 		cmdsetup.s.iq_no = iq_no;
@@ -1524,8 +1706,11 @@ struct lio_soft_command *
 			break;
 		}
 
-		if (unlikely(status == LIO_IQ_SEND_STOP))
+		if (unlikely(status == LIO_IQ_SEND_STOP)) {
 			PMD_TX_LOG(lio_dev, DEBUG, "iq full\n");
+			/* create space as iq is full */
+			lio_dev_cleanup_iq(lio_dev, iq_no);
+		}
 
 		processed++;
 	}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index dbf8685..9b94fbc 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -706,6 +706,7 @@ uint16_t lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts,
 int lio_setup_iq(struct lio_device *lio_dev, int q_index,
 		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
 		 unsigned int socket_id);
+int lio_flush_iq(struct lio_device *lio_dev, struct lio_instr_queue *iq);
 void lio_delete_instruction_queue(struct lio_device *lio_dev, int iq_no);
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
-- 
1.8.3.1

[PATCH 33/50] net/liquidio: add API to release Tx queue

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 31 +++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.c   |  2 +-
 drivers/net/liquidio/lio_rxtx.h   |  1 +
 3 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index bf85089..e997e40 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -216,6 +216,36 @@
 	return 0;
 }
 
+/**
+ * Release the transmit queue/ringbuffer. Called by
+ * the upper layers.
+ *
+ * @param txq
+ *    Opaque pointer to the transmit queue to release
+ *
+ * @return
+ *    - nothing
+ */
+static void
+lio_dev_tx_queue_release(void *txq)
+{
+	struct lio_instr_queue *tq = txq;
+	struct lio_device *lio_dev = tq->lio_dev;
+	uint32_t fw_mapped_iq_no;
+
+	/* Run time queue deletion not supported */
+	if (lio_dev->port_configured)
+		return;
+
+	if (tq != NULL) {
+		/* Free sg_list */
+		lio_delete_sglist(tq);
+
+		fw_mapped_iq_no = tq->txpciq.s.q_no;
+		lio_delete_instruction_queue(tq->lio_dev, fw_mapped_iq_no);
+	}
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -387,6 +417,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
+	.tx_queue_release	= lio_dev_tx_queue_release,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 448c8b9..edb1577 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1439,7 +1439,7 @@ struct lio_soft_command *
 	return node;
 }
 
-static void
+void
 lio_delete_sglist(struct lio_instr_queue *txq)
 {
 	struct lio_device *lio_dev = txq->lio_dev;
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 9b94fbc..2461fef 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -699,6 +699,7 @@ uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			   uint16_t budget);
 void lio_delete_droq_queue(struct lio_device *lio_dev, int oq_no);
 
+void lio_delete_sglist(struct lio_instr_queue *txq);
 int lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
 		      int fw_mapped_iq, int num_descs, unsigned int socket_id);
 uint16_t lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts,
-- 
1.8.3.1

[PATCH 34/50] net/liquidio: add API to start device and check link

[Shijith Thotton]

Add API to start device and periodic check of link state.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |   1 +
 drivers/net/liquidio/lio_ethdev.c       | 108 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |   8 +++
 drivers/net/liquidio/lio_rxtx.c         |   2 +-
 drivers/net/liquidio/lio_struct.h       |   2 +
 5 files changed, 120 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index ed6d90c..302a512 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -122,6 +122,7 @@ enum octeon_tag_type {
 /** LIO_OPCODE subcodes */
 /* This subcode is sent by core PCI driver to indicate cores are ready. */
 #define LIO_OPCODE_NW_DATA		0x02 /* network packet data */
+#define LIO_OPCODE_INFO			0x04
 #define LIO_OPCODE_IF_CFG		0x09
 
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index e997e40..256e6b8 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -246,6 +246,113 @@
 	}
 }
 
+/**
+ * Api to check link state.
+ */
+static void
+lio_dev_get_link_status(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
+	struct lio_link_status_resp *resp;
+	union octeon_link_status *ls;
+	struct lio_soft_command *sc;
+	uint32_t resp_size;
+
+	if (!lio_dev->intf_open)
+		return;
+
+	resp_size = sizeof(struct lio_link_status_resp);
+	sc = lio_alloc_soft_command(lio_dev, 0, resp_size, 0);
+	if (sc == NULL)
+		return;
+
+	resp = (struct lio_link_status_resp *)sc->virtrptr;
+	lio_prepare_soft_command(lio_dev, sc, LIO_OPCODE,
+				 LIO_OPCODE_INFO, 0, 0, 0);
+
+	/* Setting wait time in seconds */
+	sc->wait_time = LIO_MAX_CMD_TIMEOUT / 1000;
+
+	if (lio_send_soft_command(lio_dev, sc) == LIO_IQ_SEND_FAILED)
+		goto get_status_fail;
+
+	while ((*sc->status_word == LIO_COMPLETION_WORD_INIT) && --timeout) {
+		lio_flush_iq(lio_dev, lio_dev->instr_queue[sc->iq_no]);
+		rte_delay_ms(1);
+	}
+
+	if (resp->status)
+		goto get_status_fail;
+
+	ls = &resp->link_info.link;
+
+	lio_swap_8B_data((uint64_t *)ls, sizeof(union octeon_link_status) >> 3);
+
+	if (lio_dev->linfo.link.link_status64 != ls->link_status64)
+		lio_dev->linfo.link.link_status64 = ls->link_status64;
+
+	lio_free_soft_command(sc);
+
+	return;
+
+get_status_fail:
+	lio_free_soft_command(sc);
+}
+
+/* This function will be invoked every LSC_TIMEOUT ns (100ms)
+ * and will update link state if it changes.
+ */
+static void
+lio_sync_link_state_check(void *eth_dev)
+{
+	struct lio_device *lio_dev =
+		(((struct rte_eth_dev *)eth_dev)->data->dev_private);
+
+	if (lio_dev->port_configured)
+		lio_dev_get_link_status(eth_dev);
+
+	/* Schedule periodic link status check.
+	 * Stop check if interface is close and start again while opening.
+	 */
+	if (lio_dev->intf_open)
+		rte_eal_alarm_set(LIO_LSC_TIMEOUT, lio_sync_link_state_check,
+				  eth_dev);
+}
+
+static int
+lio_dev_start(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	int ret = 0;
+
+	lio_dev_info(lio_dev, "Starting port %d\n", eth_dev->data->port_id);
+
+	if (lio_dev->fn_list.enable_io_queues(lio_dev))
+		return -1;
+
+	/* Ready for link status updates */
+	lio_dev->intf_open = 1;
+	rte_mb();
+
+	/* start polling for lsc */
+	ret = rte_eal_alarm_set(LIO_LSC_TIMEOUT,
+				lio_sync_link_state_check,
+				eth_dev);
+	if (ret) {
+		lio_dev_err(lio_dev,
+			    "link state check handler creation failed\n");
+		goto dev_lsc_handle_error;
+	}
+
+	return 0;
+
+dev_lsc_handle_error:
+	lio_dev->intf_open = 0;
+
+	return ret;
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -414,6 +521,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 /* Define our ethernet definitions */
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
+	.dev_start		= lio_dev_start,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 22e3d83..c7d3336 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -38,6 +38,8 @@
 
 #include "lio_struct.h"
 
+/* timeout to check link state updates from firmware in us */
+#define LIO_LSC_TIMEOUT		100000 /* 100000us (100ms) */
 #define LIO_MAX_CMD_TIMEOUT     10000 /* 10000ms (10s) */
 
 #define LIO_DEV(_eth_dev)		((_eth_dev)->data->dev_private)
@@ -73,4 +75,10 @@ struct lio_if_cfg_resp {
 	struct octeon_if_cfg_info cfg_info;
 	uint64_t status;
 };
+
+struct lio_link_status_resp {
+	uint64_t rh;
+	struct octeon_link_info link_info;
+	uint64_t status;
+};
 #endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index edb1577..fe9a7f1 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1587,7 +1587,7 @@ struct lio_soft_command *
 	lio_dev = txq->lio_dev;
 	iq_no = txq->txpciq.s.q_no;
 
-	if (!lio_dev->linfo.link.s.link_up) {
+	if (!lio_dev->intf_open || !lio_dev->linfo.link.s.link_up) {
 		PMD_TX_LOG(lio_dev, ERR, "Transmit failed link_status : %d\n",
 			   lio_dev->linfo.link.s.link_up);
 		goto xmit_failed;
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index de4c1f3..91a8fd2 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -569,6 +569,8 @@ struct lio_device {
 	/** The state of this device */
 	rte_atomic64_t status;
 
+	uint8_t intf_open;
+
 	struct octeon_link_info linfo;
 
 	uint8_t *hw_addr;
-- 
1.8.3.1

[PATCH 35/50] net/liquidio: add API for link update

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 76 ++++++++++++++++++++++++++++++++++++++-
 drivers/net/liquidio/lio_ethdev.h |  5 +++
 2 files changed, 80 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 256e6b8..299d28b 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -41,6 +41,32 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+/**
+ * Atomically writes the link status information into global
+ * structure rte_eth_dev.
+ *
+ * @param eth_dev
+ *   - Pointer to the structure rte_eth_dev to read from.
+ *   - Pointer to the buffer to be saved with the link status.
+ *
+ * @return
+ *   - On success, zero.
+ *   - On failure, negative value.
+ */
+static inline int
+lio_dev_atomic_write_link_status(struct rte_eth_dev *eth_dev,
+				 struct rte_eth_link *link)
+{
+	struct rte_eth_link *dst = &eth_dev->data->dev_link;
+	struct rte_eth_link *src = link;
+
+	if (rte_atomic64_cmpset((uint64_t *)dst, *(uint64_t *)dst,
+				*(uint64_t *)src) == 0)
+		return -1;
+
+	return 0;
+}
+
 static uint64_t
 lio_hweight64(uint64_t w)
 {
@@ -55,6 +81,49 @@
 	return (res + (res >> 32)) & 0x00000000000000FFul;
 }
 
+static int
+lio_dev_link_update(struct rte_eth_dev *eth_dev,
+		    int wait_to_complete __rte_unused)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct rte_eth_link link, old;
+
+	/* Initialize */
+	link.link_status = ETH_LINK_DOWN;
+	link.link_speed = ETH_SPEED_NUM_NONE;
+	link.link_duplex = ETH_LINK_HALF_DUPLEX;
+	memset(&old, 0, sizeof(old));
+
+	/* Return what we found */
+	if (lio_dev->linfo.link.s.link_up == 0) {
+		/* Interface is down */
+		if (lio_dev_atomic_write_link_status(eth_dev, &link))
+			return -1;
+		if (link.link_status == old.link_status)
+			return -1;
+		return 0;
+	}
+
+	link.link_status = ETH_LINK_UP; /* Interface is up */
+	link.link_duplex = ETH_LINK_FULL_DUPLEX;
+	switch (lio_dev->linfo.link.s.speed) {
+	case LIO_LINK_SPEED_10000:
+		link.link_speed = ETH_SPEED_NUM_10G;
+		break;
+	default:
+		link.link_speed = ETH_SPEED_NUM_NONE;
+		link.link_duplex = ETH_LINK_HALF_DUPLEX;
+	}
+
+	if (lio_dev_atomic_write_link_status(eth_dev, &link))
+		return -1;
+
+	if (link.link_status == old.link_status)
+		return -1;
+
+	return 0;
+}
+
 /**
  * Setup our receive queue/ringbuffer. This is the
  * queue the Octeon uses to send us packets and
@@ -289,8 +358,10 @@
 
 	lio_swap_8B_data((uint64_t *)ls, sizeof(union octeon_link_status) >> 3);
 
-	if (lio_dev->linfo.link.link_status64 != ls->link_status64)
+	if (lio_dev->linfo.link.link_status64 != ls->link_status64) {
 		lio_dev->linfo.link.link_status64 = ls->link_status64;
+		lio_dev_link_update(eth_dev, 0);
+	}
 
 	lio_free_soft_command(sc);
 
@@ -495,6 +566,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
+	lio_dev_link_update(eth_dev, 0);
+
 	lio_dev->port_configured = 1;
 
 	lio_free_soft_command(sc);
@@ -522,6 +595,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
 	.dev_start		= lio_dev_start,
+	.link_update		= lio_dev_link_update,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index c7d3336..98ff493 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -44,6 +44,11 @@
 
 #define LIO_DEV(_eth_dev)		((_eth_dev)->data->dev_private)
 
+enum lio_bus_speed {
+	LIO_LINK_SPEED_UNKNOWN  = 0,
+	LIO_LINK_SPEED_10000    = 10000
+};
+
 struct octeon_if_cfg_info {
 	uint64_t iqmask;	/** mask for IQs enabled for the port */
 	uint64_t oqmask;	/** mask for OQs enabled for the port */
-- 
1.8.3.1

[PATCH 36/50] net/liquidio: add API to alloc and send command

[Shijith Thotton]

Add APIs to allocate and send control command to device.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  1 +
 drivers/net/liquidio/lio_ethdev.h       |  6 +++
 drivers/net/liquidio/lio_rxtx.c         | 82 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         | 44 ++++++++++++++++++
 4 files changed, 133 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 302a512..3943438 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -122,6 +122,7 @@ enum octeon_tag_type {
 /** LIO_OPCODE subcodes */
 /* This subcode is sent by core PCI driver to indicate cores are ready. */
 #define LIO_OPCODE_NW_DATA		0x02 /* network packet data */
+#define LIO_OPCODE_CMD			0x03
 #define LIO_OPCODE_INFO			0x04
 #define LIO_OPCODE_IF_CFG		0x09
 
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 98ff493..7b5343a 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -44,6 +44,12 @@
 
 #define LIO_DEV(_eth_dev)		((_eth_dev)->data->dev_private)
 
+/* LIO Response condition variable */
+struct lio_dev_ctrl_cmd {
+	struct rte_eth_dev *eth_dev;
+	uint64_t cond;
+};
+
 enum lio_bus_speed {
 	LIO_LINK_SPEED_UNKNOWN  = 0,
 	LIO_LINK_SPEED_10000    = 10000
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index fe9a7f1..d26aa6f 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1557,6 +1557,88 @@ struct lio_soft_command *
 	return count ? 0 : 1;
 }
 
+static void
+lio_ctrl_cmd_callback(uint32_t status __rte_unused, void *sc_ptr)
+{
+	struct lio_soft_command *sc = sc_ptr;
+	struct lio_dev_ctrl_cmd *ctrl_cmd;
+	struct lio_ctrl_pkt *ctrl_pkt;
+
+	ctrl_pkt = (struct lio_ctrl_pkt *)sc->ctxptr;
+	ctrl_cmd = ctrl_pkt->ctrl_cmd;
+	ctrl_cmd->cond = 1;
+
+	lio_free_soft_command(sc);
+}
+
+static inline struct lio_soft_command *
+lio_alloc_ctrl_pkt_sc(struct lio_device *lio_dev,
+		      struct lio_ctrl_pkt *ctrl_pkt)
+{
+	struct lio_soft_command *sc = NULL;
+	uint32_t uddsize, datasize;
+	uint32_t rdatasize;
+	uint8_t *data;
+
+	uddsize = (uint32_t)(ctrl_pkt->ncmd.s.more * 8);
+
+	datasize = OCTEON_CMD_SIZE + uddsize;
+	rdatasize = (ctrl_pkt->wait_time) ? 16 : 0;
+
+	sc = lio_alloc_soft_command(lio_dev, datasize,
+				    rdatasize, sizeof(struct lio_ctrl_pkt));
+	if (sc == NULL)
+		return NULL;
+
+	rte_memcpy(sc->ctxptr, ctrl_pkt, sizeof(struct lio_ctrl_pkt));
+
+	data = (uint8_t *)sc->virtdptr;
+
+	rte_memcpy(data, &ctrl_pkt->ncmd, OCTEON_CMD_SIZE);
+
+	lio_swap_8B_data((uint64_t *)data, OCTEON_CMD_SIZE >> 3);
+
+	if (uddsize) {
+		/* Endian-Swap for UDD should have been done by caller. */
+		rte_memcpy(data + OCTEON_CMD_SIZE, ctrl_pkt->udd, uddsize);
+	}
+
+	sc->iq_no = (uint32_t)ctrl_pkt->iq_no;
+
+	lio_prepare_soft_command(lio_dev, sc,
+				 LIO_OPCODE, LIO_OPCODE_CMD,
+				 0, 0, 0);
+
+	sc->callback = lio_ctrl_cmd_callback;
+	sc->callback_arg = sc;
+	sc->wait_time = ctrl_pkt->wait_time;
+
+	return sc;
+}
+
+int
+lio_send_ctrl_pkt(struct lio_device *lio_dev, struct lio_ctrl_pkt *ctrl_pkt)
+{
+	struct lio_soft_command *sc = NULL;
+	int retval;
+
+	sc = lio_alloc_ctrl_pkt_sc(lio_dev, ctrl_pkt);
+	if (sc == NULL) {
+		lio_dev_err(lio_dev, "soft command allocation failed\n");
+		return -1;
+	}
+
+	retval = lio_send_soft_command(lio_dev, sc);
+	if (retval == LIO_IQ_SEND_FAILED) {
+		lio_free_soft_command(sc);
+		lio_dev_err(lio_dev, "Port: %d soft command: %d send failed status: %x\n",
+			    lio_dev->port_id, ctrl_pkt->ncmd.s.cmd, retval);
+		return -1;
+	}
+
+	return retval;
+}
+
 /** Send data packet to the device
  *  @param lio_dev - lio device pointer
  *  @param ndata   - control structure with queueing, and buffer information
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 2461fef..e846d30 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -249,6 +249,40 @@ struct lio_iq_post_status {
 
 #define OCTEON_CMD_SIZE (sizeof(union octeon_cmd))
 
+/* Maximum number of 8-byte words can be
+ * sent in a NIC control message.
+ */
+#define LIO_MAX_NCTRL_UDD	32
+
+/* Structure of control information passed by driver to the BASE
+ * layer when sending control commands to Octeon device software.
+ */
+struct lio_ctrl_pkt {
+	/** Command to be passed to the Octeon device software. */
+	union octeon_cmd ncmd;
+
+	/** Send buffer */
+	void *data;
+	uint64_t dmadata;
+
+	/** Response buffer */
+	void *rdata;
+	uint64_t dmardata;
+
+	/** Additional data that may be needed by some commands. */
+	uint64_t udd[LIO_MAX_NCTRL_UDD];
+
+	/** Input queue to use to send this command. */
+	uint64_t iq_no;
+
+	/** Time to wait for Octeon software to respond to this control command.
+	 *  If wait_time is 0, BASE assumes no response is expected.
+	 */
+	size_t wait_time;
+
+	struct lio_dev_ctrl_cmd *ctrl_cmd;
+};
+
 /** Structure of data information passed by driver to the BASE
  *  layer when forwarding data to Octeon device software.
  */
@@ -570,6 +604,16 @@ int lio_send_soft_command(struct lio_device *lio_dev,
 			  struct lio_soft_command *sc);
 void lio_free_soft_command(struct lio_soft_command *sc);
 
+/** Send control packet to the device
+ *  @param lio_dev - lio device pointer
+ *  @param nctrl   - control structure with command, timeout, and callback info
+ *
+ *  @returns IQ_FAILED if it failed to add to the input queue. IQ_STOP if it the
+ *  queue should be stopped, and LIO_IQ_SEND_OK if it sent okay.
+ */
+int lio_send_ctrl_pkt(struct lio_device *lio_dev,
+		      struct lio_ctrl_pkt *ctrl_pkt);
+
 /** Maximum ordered requests to process in every invocation of
  *  lio_process_ordered_list(). The function will continue to process requests
  *  as long as it can find one that has finished processing. If it keeps
-- 
1.8.3.1

Re: [PATCH 36/50] net/liquidio: add API to alloc and send command

[Ferruh Yigit]

On 2/21/2017 9:26 AM, Shijith Thotton wrote:

Can it be "control command"?

add API to alloc and send _control command_ ?


> Add APIs to allocate and send control command to device.
> 
> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
<...>

Re: [PATCH 36/50] net/liquidio: add API to alloc and send command

[Shijith Thotton]

On Thursday 23 February 2017 08:03 PM, Ferruh Yigit wrote:
> On 2/21/2017 9:26 AM, Shijith Thotton wrote:
> 
> Can it be "control command"?
> 
> add API to alloc and send _control command_ ?
> 
> 

Yes. Will modify commit title.

>> Add APIs to allocate and send control command to device.
>>
>> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
>> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
>> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
>> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
>> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
> <...>
> 

[PATCH 37/50] net/liquidio: add API to control Rx

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  3 ++
 drivers/net/liquidio/lio_ethdev.c       | 59 +++++++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 3943438..8a0a60c 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -128,6 +128,9 @@ enum octeon_tag_type {
 
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
 
+/* NIC Command types */
+#define LIO_CMD_RX_CTL			0x4
+
 /* RX(packets coming from wire) Checksum verification flags */
 /* TCP/UDP csum */
 #define LIO_L4_CSUM_VERIFIED		0x1
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 299d28b..0c2cdd8 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -41,6 +41,61 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+/* Wait for control command to reach nic. */
+static uint16_t
+lio_wait_for_ctrl_cmd(struct lio_device *lio_dev,
+		      struct lio_dev_ctrl_cmd *ctrl_cmd)
+{
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
+
+	while ((ctrl_cmd->cond == 0) && --timeout) {
+		lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+		rte_delay_ms(1);
+	}
+
+	return !timeout;
+}
+
+/**
+ * \brief Send Rx control command
+ * @param eth_dev Pointer to the structure rte_eth_dev
+ * @param start_stop whether to start or stop
+ */
+static int
+lio_send_rx_ctrl_cmd(struct rte_eth_dev *eth_dev, int start_stop)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_RX_CTL;
+	ctrl_pkt.ncmd.s.param1 = start_stop;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send RX Control message\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "RX Control command timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Atomically writes the link status information into global
  * structure rte_eth_dev.
@@ -402,6 +457,9 @@
 	if (lio_dev->fn_list.enable_io_queues(lio_dev))
 		return -1;
 
+	if (lio_send_rx_ctrl_cmd(eth_dev, 1))
+		return -1;
+
 	/* Ready for link status updates */
 	lio_dev->intf_open = 1;
 	rte_mb();
@@ -420,6 +478,7 @@
 
 dev_lsc_handle_error:
 	lio_dev->intf_open = 0;
+	lio_send_rx_ctrl_cmd(eth_dev, 0);
 
 	return ret;
 }
-- 
1.8.3.1

[PATCH 38/50] net/liquidio: add RSS support

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  26 +++
 drivers/net/liquidio/lio_ethdev.c       | 336 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  21 ++
 drivers/net/liquidio/lio_struct.h       |  16 ++
 4 files changed, 399 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 8a0a60c..d0210e5 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -130,12 +130,38 @@ enum octeon_tag_type {
 
 /* NIC Command types */
 #define LIO_CMD_RX_CTL			0x4
+#define LIO_CMD_SET_RSS			0xD
 
 /* RX(packets coming from wire) Checksum verification flags */
 /* TCP/UDP csum */
 #define LIO_L4_CSUM_VERIFIED		0x1
 #define LIO_IP_CSUM_VERIFIED		0x2
 
+/* RSS */
+#define LIO_RSS_PARAM_DISABLE_RSS		0x10
+#define LIO_RSS_PARAM_HASH_KEY_UNCHANGED	0x08
+#define LIO_RSS_PARAM_ITABLE_UNCHANGED		0x04
+#define LIO_RSS_PARAM_HASH_INFO_UNCHANGED	0x02
+
+#define LIO_RSS_HASH_IPV4			0x100
+#define LIO_RSS_HASH_TCP_IPV4			0x200
+#define LIO_RSS_HASH_IPV6			0x400
+#define LIO_RSS_HASH_TCP_IPV6			0x1000
+#define LIO_RSS_HASH_IPV6_EX			0x800
+#define LIO_RSS_HASH_TCP_IPV6_EX		0x2000
+
+#define LIO_RSS_OFFLOAD_ALL (		\
+		LIO_RSS_HASH_IPV4 |	\
+		LIO_RSS_HASH_TCP_IPV4 |	\
+		LIO_RSS_HASH_IPV6 |	\
+		LIO_RSS_HASH_TCP_IPV6 |	\
+		LIO_RSS_HASH_IPV6_EX |	\
+		LIO_RSS_HASH_TCP_IPV6_EX)
+
+#define LIO_RSS_MAX_TABLE_SZ		128
+#define LIO_RSS_MAX_KEY_SZ		40
+#define LIO_RSS_PARAM_SIZE		16
+
 /* Interface flags communicated between host driver and core app. */
 enum lio_ifflags {
 	LIO_IFFLAG_UNICAST	= 0x10
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 0c2cdd8..70b4c45 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -41,6 +41,15 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+/* Default RSS key in use */
+static uint8_t lio_rss_key[40] = {
+	0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2,
+	0x41, 0x67, 0x25, 0x3D, 0x43, 0xA3, 0x8F, 0xB0,
+	0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+	0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C,
+	0x6A, 0x42, 0xB7, 0x3B, 0xBE, 0xAC, 0x01, 0xFA,
+};
+
 /* Wait for control command to reach nic. */
 static uint16_t
 lio_wait_for_ctrl_cmd(struct lio_device *lio_dev,
@@ -96,6 +105,267 @@
 	return 0;
 }
 
+static int
+lio_dev_rss_reta_update(struct rte_eth_dev *eth_dev,
+			struct rte_eth_rss_reta_entry64 *reta_conf,
+			uint16_t reta_size)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	struct lio_rss_set *rss_param;
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+	int i, j, index;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't update reta\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	if (reta_size != LIO_RSS_MAX_TABLE_SZ) {
+		lio_dev_err(lio_dev,
+			    "The size of hash lookup table configured (%d) doesn't match the number hardware can supported (%d)\n",
+			    reta_size, LIO_RSS_MAX_TABLE_SZ);
+		return -EINVAL;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	rss_param = (struct lio_rss_set *)&ctrl_pkt.udd[0];
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_SET_RSS;
+	ctrl_pkt.ncmd.s.more = sizeof(struct lio_rss_set) >> 3;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	rss_param->param.flags = 0xF;
+	rss_param->param.flags &= ~LIO_RSS_PARAM_ITABLE_UNCHANGED;
+	rss_param->param.itablesize = LIO_RSS_MAX_TABLE_SZ;
+
+	for (i = 0; i < (reta_size / RTE_RETA_GROUP_SIZE); i++) {
+		for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) {
+			if ((reta_conf[i].mask) & ((uint64_t)1 << j)) {
+				index = (i * RTE_RETA_GROUP_SIZE) + j;
+				rss_state->itable[index] = reta_conf[i].reta[j];
+			}
+		}
+	}
+
+	rss_state->itable_size = LIO_RSS_MAX_TABLE_SZ;
+	memcpy(rss_param->itable, rss_state->itable, rss_state->itable_size);
+
+	lio_swap_8B_data((uint64_t *)rss_param, LIO_RSS_PARAM_SIZE >> 3);
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to set rss hash\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "Set rss hash timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+static int
+lio_dev_rss_reta_query(struct rte_eth_dev *eth_dev,
+		       struct rte_eth_rss_reta_entry64 *reta_conf,
+		       uint16_t reta_size)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	int i, num;
+
+	if (reta_size != LIO_RSS_MAX_TABLE_SZ) {
+		lio_dev_err(lio_dev,
+			    "The size of hash lookup table configured (%d) doesn't match the number hardware can supported (%d)\n",
+			    reta_size, LIO_RSS_MAX_TABLE_SZ);
+		return -EINVAL;
+	}
+
+	num = reta_size / RTE_RETA_GROUP_SIZE;
+
+	for (i = 0; i < num; i++) {
+		memcpy(reta_conf->reta,
+		       &rss_state->itable[i * RTE_RETA_GROUP_SIZE],
+		       RTE_RETA_GROUP_SIZE);
+		reta_conf++;
+	}
+
+	return 0;
+}
+
+static int
+lio_dev_rss_hash_conf_get(struct rte_eth_dev *eth_dev,
+			  struct rte_eth_rss_conf *rss_conf)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	uint8_t *hash_key = NULL;
+	uint64_t rss_hf = 0;
+
+	if (rss_state->hash_disable) {
+		lio_dev_info(lio_dev, "RSS disabled in nic\n");
+		rss_conf->rss_hf = 0;
+		return 0;
+	}
+
+	/* Get key value */
+	hash_key = rss_conf->rss_key;
+	if (hash_key != NULL)
+		memcpy(hash_key, rss_state->hash_key, rss_state->hash_key_size);
+
+	if (rss_state->ip)
+		rss_hf |= ETH_RSS_IPV4;
+	if (rss_state->tcp_hash)
+		rss_hf |= ETH_RSS_NONFRAG_IPV4_TCP;
+	if (rss_state->ipv6)
+		rss_hf |= ETH_RSS_IPV6;
+	if (rss_state->ipv6_tcp_hash)
+		rss_hf |= ETH_RSS_NONFRAG_IPV6_TCP;
+	if (rss_state->ipv6_ex)
+		rss_hf |= ETH_RSS_IPV6_EX;
+	if (rss_state->ipv6_tcp_ex_hash)
+		rss_hf |= ETH_RSS_IPV6_TCP_EX;
+
+	rss_conf->rss_hf = rss_hf;
+
+	return 0;
+}
+
+static int
+lio_dev_rss_hash_update(struct rte_eth_dev *eth_dev,
+			struct rte_eth_rss_conf *rss_conf)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	struct lio_rss_set *rss_param;
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't update hash\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	rss_param = (struct lio_rss_set *)&ctrl_pkt.udd[0];
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_SET_RSS;
+	ctrl_pkt.ncmd.s.more = sizeof(struct lio_rss_set) >> 3;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	rss_param->param.flags = 0xF;
+
+	if (rss_conf->rss_key) {
+		rss_param->param.flags &= ~LIO_RSS_PARAM_HASH_KEY_UNCHANGED;
+		rss_state->hash_key_size = LIO_RSS_MAX_KEY_SZ;
+		rss_param->param.hashkeysize = LIO_RSS_MAX_KEY_SZ;
+		memcpy(rss_state->hash_key, rss_conf->rss_key,
+		       rss_state->hash_key_size);
+		memcpy(rss_param->key, rss_state->hash_key,
+		       rss_state->hash_key_size);
+	}
+
+	if ((rss_conf->rss_hf & LIO_RSS_OFFLOAD_ALL) == 0) {
+		/* Can't disable rss through hash flags,
+		 * if it is enabled by default during init
+		 */
+		if (!rss_state->hash_disable)
+			return -EINVAL;
+
+		/* This is for --disable-rss during testpmd launch */
+		rss_param->param.flags |= LIO_RSS_PARAM_DISABLE_RSS;
+	} else {
+		uint32_t hashinfo = 0;
+
+		/* Can't enable rss if disabled by default during init */
+		if (rss_state->hash_disable)
+			return -EINVAL;
+
+		if (rss_conf->rss_hf & ETH_RSS_IPV4) {
+			hashinfo |= LIO_RSS_HASH_IPV4;
+			rss_state->ip = 1;
+		} else {
+			rss_state->ip = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_NONFRAG_IPV4_TCP) {
+			hashinfo |= LIO_RSS_HASH_TCP_IPV4;
+			rss_state->tcp_hash = 1;
+		} else {
+			rss_state->tcp_hash = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_IPV6) {
+			hashinfo |= LIO_RSS_HASH_IPV6;
+			rss_state->ipv6 = 1;
+		} else {
+			rss_state->ipv6 = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_NONFRAG_IPV6_TCP) {
+			hashinfo |= LIO_RSS_HASH_TCP_IPV6;
+			rss_state->ipv6_tcp_hash = 1;
+		} else {
+			rss_state->ipv6_tcp_hash = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_IPV6_EX) {
+			hashinfo |= LIO_RSS_HASH_IPV6_EX;
+			rss_state->ipv6_ex = 1;
+		} else {
+			rss_state->ipv6_ex = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_IPV6_TCP_EX) {
+			hashinfo |= LIO_RSS_HASH_TCP_IPV6_EX;
+			rss_state->ipv6_tcp_ex_hash = 1;
+		} else {
+			rss_state->ipv6_tcp_ex_hash = 0;
+		}
+
+		rss_param->param.flags &= ~LIO_RSS_PARAM_HASH_INFO_UNCHANGED;
+		rss_param->param.hashinfo = hashinfo;
+	}
+
+	lio_swap_8B_data((uint64_t *)rss_param, LIO_RSS_PARAM_SIZE >> 3);
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to set rss hash\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "Set rss hash timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Atomically writes the link status information into global
  * structure rte_eth_dev.
@@ -179,6 +449,65 @@
 	return 0;
 }
 
+static void
+lio_dev_rss_configure(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	struct rte_eth_rss_reta_entry64 reta_conf[8];
+	struct rte_eth_rss_conf rss_conf;
+	uint16_t i;
+
+	/* Configure the RSS key and the RSS protocols used to compute
+	 * the RSS hash of input packets.
+	 */
+	rss_conf = eth_dev->data->dev_conf.rx_adv_conf.rss_conf;
+	if ((rss_conf.rss_hf & LIO_RSS_OFFLOAD_ALL) == 0) {
+		rss_state->hash_disable = 1;
+		lio_dev_rss_hash_update(eth_dev, &rss_conf);
+		return;
+	}
+
+	if (rss_conf.rss_key == NULL)
+		rss_conf.rss_key = lio_rss_key; /* Default hash key */
+
+	lio_dev_rss_hash_update(eth_dev, &rss_conf);
+
+	memset(reta_conf, 0, sizeof(reta_conf));
+	for (i = 0; i < LIO_RSS_MAX_TABLE_SZ; i++) {
+		uint8_t q_idx, conf_idx, reta_idx;
+
+		q_idx = (uint8_t)((eth_dev->data->nb_rx_queues > 1) ?
+				  i % eth_dev->data->nb_rx_queues : 0);
+		conf_idx = i / RTE_RETA_GROUP_SIZE;
+		reta_idx = i % RTE_RETA_GROUP_SIZE;
+		reta_conf[conf_idx].reta[reta_idx] = q_idx;
+		reta_conf[conf_idx].mask |= ((uint64_t)1 << reta_idx);
+	}
+
+	lio_dev_rss_reta_update(eth_dev, reta_conf, LIO_RSS_MAX_TABLE_SZ);
+}
+
+static void
+lio_dev_mq_rx_configure(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	struct rte_eth_rss_conf rss_conf;
+
+	switch (eth_dev->data->dev_conf.rxmode.mq_mode) {
+	case ETH_MQ_RX_RSS:
+		lio_dev_rss_configure(eth_dev);
+		break;
+	case ETH_MQ_RX_NONE:
+	/* if mq_mode is none, disable rss mode. */
+	default:
+		memset(&rss_conf, 0, sizeof(rss_conf));
+		rss_state->hash_disable = 1;
+		lio_dev_rss_hash_update(eth_dev, &rss_conf);
+	}
+}
+
 /**
  * Setup our receive queue/ringbuffer. This is the
  * queue the Octeon uses to send us packets and
@@ -464,6 +793,9 @@
 	lio_dev->intf_open = 1;
 	rte_mb();
 
+	/* Configure RSS if device configured with multiple RX queues. */
+	lio_dev_mq_rx_configure(eth_dev);
+
 	/* start polling for lsc */
 	ret = rte_eal_alarm_set(LIO_LSC_TIMEOUT,
 				lio_sync_link_state_check,
@@ -659,6 +991,10 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
 	.tx_queue_release	= lio_dev_tx_queue_release,
+	.reta_update		= lio_dev_rss_reta_update,
+	.reta_query		= lio_dev_rss_reta_query,
+	.rss_hash_conf_get	= lio_dev_rss_hash_conf_get,
+	.rss_hash_update	= lio_dev_rss_hash_update,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 7b5343a..6543061 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -92,4 +92,25 @@ struct lio_link_status_resp {
 	struct octeon_link_info link_info;
 	uint64_t status;
 };
+
+struct lio_rss_set {
+	struct param {
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+		uint64_t flags : 16;
+		uint64_t hashinfo : 32;
+		uint64_t itablesize : 16;
+		uint64_t hashkeysize : 16;
+		uint64_t reserved : 48;
+#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t itablesize : 16;
+		uint64_t hashinfo : 32;
+		uint64_t flags : 16;
+		uint64_t reserved : 48;
+		uint64_t hashkeysize : 16;
+#endif
+	} param;
+
+	uint8_t itable[LIO_RSS_MAX_TABLE_SZ];
+	uint8_t key[LIO_RSS_MAX_KEY_SZ];
+};
 #endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 91a8fd2..bdcc301 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -357,6 +357,21 @@ struct lio_gather {
 	struct lio_sg_entry *sg;
 };
 
+struct lio_rss_ctx {
+	uint16_t hash_key_size;
+	uint8_t  hash_key[LIO_RSS_MAX_KEY_SZ];
+	/* Ideally a factor of number of queues */
+	uint8_t  itable[LIO_RSS_MAX_TABLE_SZ];
+	uint8_t  itable_size;
+	uint8_t  ip;
+	uint8_t  tcp_hash;
+	uint8_t  ipv6;
+	uint8_t  ipv6_tcp_hash;
+	uint8_t  ipv6_ex;
+	uint8_t  ipv6_tcp_ex_hash;
+	uint8_t  hash_disable;
+};
+
 struct lio_io_enable {
 	uint64_t iq;
 	uint64_t oq;
@@ -619,6 +634,7 @@ struct lio_device {
 	uint8_t nb_rx_queues;
 	uint8_t nb_tx_queues;
 	uint8_t port_configured;
+	struct lio_rss_ctx rss_state;
 	uint8_t port_id;
 };
 #endif /* _LIO_STRUCT_H_ */
-- 
1.8.3.1

[PATCH 39/50] net/liquidio: add API to get device info

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  1 +
 drivers/net/liquidio/lio_ethdev.c       | 47 +++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index d0210e5..d4be699 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -126,6 +126,7 @@ enum octeon_tag_type {
 #define LIO_OPCODE_INFO			0x04
 #define LIO_OPCODE_IF_CFG		0x09
 
+#define LIO_MIN_RX_BUF_SIZE		64
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
 
 /* NIC Command types */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 70b4c45..ad827f8 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -50,6 +50,18 @@
 	0x6A, 0x42, 0xB7, 0x3B, 0xBE, 0xAC, 0x01, 0xFA,
 };
 
+static const struct rte_eth_desc_lim lio_rx_desc_lim = {
+	.nb_max		= CN23XX_MAX_OQ_DESCRIPTORS,
+	.nb_min		= CN23XX_MIN_OQ_DESCRIPTORS,
+	.nb_align	= 1,
+};
+
+static const struct rte_eth_desc_lim lio_tx_desc_lim = {
+	.nb_max		= CN23XX_MAX_IQ_DESCRIPTORS,
+	.nb_min		= CN23XX_MIN_IQ_DESCRIPTORS,
+	.nb_align	= 1,
+};
+
 /* Wait for control command to reach nic. */
 static uint16_t
 lio_wait_for_ctrl_cmd(struct lio_device *lio_dev,
@@ -105,6 +117,40 @@
 	return 0;
 }
 
+static void
+lio_dev_info_get(struct rte_eth_dev *eth_dev,
+		 struct rte_eth_dev_info *devinfo)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	devinfo->max_rx_queues = lio_dev->max_rx_queues;
+	devinfo->max_tx_queues = lio_dev->max_tx_queues;
+
+	devinfo->min_rx_bufsize = LIO_MIN_RX_BUF_SIZE;
+	devinfo->max_rx_pktlen = LIO_MAX_RX_PKTLEN;
+
+	devinfo->max_mac_addrs = 1;
+
+	devinfo->rx_offload_capa = (DEV_RX_OFFLOAD_IPV4_CKSUM		|
+				    DEV_RX_OFFLOAD_UDP_CKSUM		|
+				    DEV_RX_OFFLOAD_TCP_CKSUM);
+	devinfo->tx_offload_capa = (DEV_TX_OFFLOAD_IPV4_CKSUM		|
+				    DEV_TX_OFFLOAD_UDP_CKSUM		|
+				    DEV_TX_OFFLOAD_TCP_CKSUM);
+
+	devinfo->rx_desc_lim = lio_rx_desc_lim;
+	devinfo->tx_desc_lim = lio_tx_desc_lim;
+
+	devinfo->reta_size = LIO_RSS_MAX_TABLE_SZ;
+	devinfo->hash_key_size = LIO_RSS_MAX_KEY_SZ;
+	devinfo->flow_type_rss_offloads = (ETH_RSS_IPV4			|
+					   ETH_RSS_NONFRAG_IPV4_TCP	|
+					   ETH_RSS_IPV6			|
+					   ETH_RSS_NONFRAG_IPV6_TCP	|
+					   ETH_RSS_IPV6_EX		|
+					   ETH_RSS_IPV6_TCP_EX);
+}
+
 static int
 lio_dev_rss_reta_update(struct rte_eth_dev *eth_dev,
 			struct rte_eth_rss_reta_entry64 *reta_conf,
@@ -987,6 +1033,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.dev_configure		= lio_dev_configure,
 	.dev_start		= lio_dev_start,
 	.link_update		= lio_dev_link_update,
+	.dev_infos_get		= lio_dev_info_get,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
-- 
1.8.3.1

[PATCH 40/50] net/liquidio: add API to set MTU

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 48 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index ad827f8..d82a46b 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -152,6 +152,33 @@
 }
 
 static int
+lio_dev_change_vf_mtu(struct rte_eth_dev *eth_dev, uint16_t new_mtu)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	PMD_INIT_FUNC_TRACE();
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't change MTU\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	/* Limit the MTU to make sure the ethernet packets are between
+	 * ETHER_MIN_MTU bytes and PF's MTU
+	 */
+	if ((new_mtu < ETHER_MIN_MTU) ||
+			(new_mtu > lio_dev->linfo.link.s.mtu)) {
+		lio_dev_err(lio_dev, "Invalid MTU: %d\n", new_mtu);
+		lio_dev_err(lio_dev, "Valid range %d and %d\n",
+			    ETHER_MIN_MTU, lio_dev->linfo.link.s.mtu);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int
 lio_dev_rss_reta_update(struct rte_eth_dev *eth_dev,
 			struct rte_eth_rss_reta_entry64 *reta_conf,
 			uint16_t reta_size)
@@ -824,7 +851,9 @@
 static int
 lio_dev_start(struct rte_eth_dev *eth_dev)
 {
+	uint16_t mtu = eth_dev->data->dev_conf.rxmode.max_rx_pkt_len;
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
 	int ret = 0;
 
 	lio_dev_info(lio_dev, "Starting port %d\n", eth_dev->data->port_id);
@@ -852,8 +881,26 @@
 		goto dev_lsc_handle_error;
 	}
 
+	while ((lio_dev->linfo.link.link_status64 == 0) && (--timeout))
+		rte_delay_ms(1);
+
+	if (lio_dev->linfo.link.link_status64 == 0) {
+		ret = -1;
+		goto dev_mtu_updation_error;
+	}
+
+	if (lio_dev->linfo.link.s.mtu != mtu) {
+		if (lio_dev_change_vf_mtu(eth_dev, mtu)) {
+			ret = -1;
+			goto dev_mtu_updation_error;
+		}
+	}
+
 	return 0;
 
+dev_mtu_updation_error:
+rte_eal_alarm_cancel(lio_sync_link_state_check, eth_dev);
+
 dev_lsc_handle_error:
 	lio_dev->intf_open = 0;
 	lio_send_rx_ctrl_cmd(eth_dev, 0);
@@ -1034,6 +1081,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.dev_start		= lio_dev_start,
 	.link_update		= lio_dev_link_update,
 	.dev_infos_get		= lio_dev_info_get,
+	.mtu_set		= lio_dev_change_vf_mtu,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
-- 
1.8.3.1

Re: [PATCH 40/50] net/liquidio: add API to set MTU

[Ferruh Yigit]

On 2/21/2017 9:26 AM, Shijith Thotton wrote:
> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
> ---
<...>

>  
> +dev_mtu_updation_error:
> +rte_eal_alarm_cancel(lio_sync_link_state_check, eth_dev);

Wrong indentation.

[PATCH 41/50] net/liquidio: add API to enable and disable multicast

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  2 +
 drivers/net/liquidio/lio_ethdev.c       | 68 +++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index d4be699..54d554a 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -130,6 +130,7 @@ enum octeon_tag_type {
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
 
 /* NIC Command types */
+#define LIO_CMD_CHANGE_DEVFLAGS		0x3
 #define LIO_CMD_RX_CTL			0x4
 #define LIO_CMD_SET_RSS			0xD
 
@@ -165,6 +166,7 @@ enum octeon_tag_type {
 
 /* Interface flags communicated between host driver and core app. */
 enum lio_ifflags {
+	LIO_IFFLAG_ALLMULTI	= 0x02,
 	LIO_IFFLAG_UNICAST	= 0x10
 };
 
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index d82a46b..c534f48 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -522,6 +522,72 @@
 	return 0;
 }
 
+/**
+ * \brief Net device enable, disable allmulticast
+ * @param eth_dev Pointer to the structure rte_eth_dev
+ */
+static void
+lio_change_dev_flag(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	/* Create a ctrl pkt command to be sent to core app. */
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_CHANGE_DEVFLAGS;
+	ctrl_pkt.ncmd.s.param1 = lio_dev->ifflags;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send change flag message\n");
+		return;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd))
+		lio_dev_err(lio_dev, "Change dev flag command timed out\n");
+}
+
+static void
+lio_dev_allmulticast_enable(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't enable multicast\n",
+			    lio_dev->port_id);
+		return;
+	}
+
+	lio_dev->ifflags |= LIO_IFFLAG_ALLMULTI;
+	lio_change_dev_flag(eth_dev);
+}
+
+static void
+lio_dev_allmulticast_disable(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't disable multicast\n",
+			    lio_dev->port_id);
+		return;
+	}
+
+	lio_dev->ifflags &= ~LIO_IFFLAG_ALLMULTI;
+	lio_change_dev_flag(eth_dev);
+}
+
 static void
 lio_dev_rss_configure(struct rte_eth_dev *eth_dev)
 {
@@ -1079,6 +1145,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
 	.dev_start		= lio_dev_start,
+	.allmulticast_enable	= lio_dev_allmulticast_enable,
+	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
 	.dev_infos_get		= lio_dev_info_get,
 	.mtu_set		= lio_dev_change_vf_mtu,
-- 
1.8.3.1

[PATCH 42/50] net/liquidio: add API to set link up and down

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 56 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index c534f48..cbe07f4 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -974,6 +974,60 @@
 	return ret;
 }
 
+static int
+lio_dev_set_link_up(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	if (!lio_dev->intf_open) {
+		lio_dev_info(lio_dev, "Port is stopped, Start the port first\n");
+		return 0;
+	}
+
+	if (lio_dev->linfo.link.s.link_up) {
+		lio_dev_info(lio_dev, "Link is already UP\n");
+		return 0;
+	}
+
+	if (lio_send_rx_ctrl_cmd(eth_dev, 1)) {
+		lio_dev_err(lio_dev, "Unable to set Link UP\n");
+		return -1;
+	}
+
+	lio_dev->linfo.link.s.link_up = 1;
+	eth_dev->data->dev_link.link_status = ETH_LINK_UP;
+
+	return 0;
+}
+
+static int
+lio_dev_set_link_down(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	if (!lio_dev->intf_open) {
+		lio_dev_info(lio_dev, "Port is stopped, Start the port first\n");
+		return 0;
+	}
+
+	if (!lio_dev->linfo.link.s.link_up) {
+		lio_dev_info(lio_dev, "Link is already DOWN\n");
+		return 0;
+	}
+
+	lio_dev->linfo.link.s.link_up = 0;
+	eth_dev->data->dev_link.link_status = ETH_LINK_DOWN;
+
+	if (lio_send_rx_ctrl_cmd(eth_dev, 0)) {
+		lio_dev->linfo.link.s.link_up = 1;
+		eth_dev->data->dev_link.link_status = ETH_LINK_UP;
+		lio_dev_err(lio_dev, "Unable to set Link Down\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -1145,6 +1199,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
 	.dev_start		= lio_dev_start,
+	.dev_set_link_up	= lio_dev_set_link_up,
+	.dev_set_link_down	= lio_dev_set_link_down,
 	.allmulticast_enable	= lio_dev_allmulticast_enable,
 	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
-- 
1.8.3.1

[PATCH 43/50] net/liquidio: add API to configure udp tunnel port

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |   8 ++
 drivers/net/liquidio/lio_ethdev.c       | 191 +++++++++++++++++++++++++++++++-
 drivers/net/liquidio/lio_rxtx.c         |   4 +-
 drivers/net/liquidio/lio_rxtx.h         |   1 +
 4 files changed, 202 insertions(+), 2 deletions(-)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 54d554a..39a3b3d 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -133,6 +133,14 @@ enum octeon_tag_type {
 #define LIO_CMD_CHANGE_DEVFLAGS		0x3
 #define LIO_CMD_RX_CTL			0x4
 #define LIO_CMD_SET_RSS			0xD
+#define LIO_CMD_TNL_RX_CSUM_CTL		0x10
+#define LIO_CMD_TNL_TX_CSUM_CTL		0x11
+#define LIO_CMD_VXLAN_PORT_CONFIG	0x19
+
+#define LIO_CMD_VXLAN_PORT_ADD		0x0
+#define LIO_CMD_VXLAN_PORT_DEL		0x1
+#define LIO_CMD_RXCSUM_ENABLE		0x0
+#define LIO_CMD_TXCSUM_ENABLE		0x0
 
 /* RX(packets coming from wire) Checksum verification flags */
 /* TCP/UDP csum */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index cbe07f4..5f8f1c0 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -136,7 +136,8 @@
 				    DEV_RX_OFFLOAD_TCP_CKSUM);
 	devinfo->tx_offload_capa = (DEV_TX_OFFLOAD_IPV4_CKSUM		|
 				    DEV_TX_OFFLOAD_UDP_CKSUM		|
-				    DEV_TX_OFFLOAD_TCP_CKSUM);
+				    DEV_TX_OFFLOAD_TCP_CKSUM		|
+				    DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM);
 
 	devinfo->rx_desc_lim = lio_rx_desc_lim;
 	devinfo->tx_desc_lim = lio_tx_desc_lim;
@@ -440,6 +441,120 @@
 }
 
 /**
+ * Add vxlan dest udp port for an interface.
+ *
+ * @param eth_dev
+ *  Pointer to the structure rte_eth_dev
+ * @param udp_tnl
+ *  udp tunnel conf
+ *
+ * @return
+ *  On success return 0
+ *  On failure return -1
+ */
+static int
+lio_dev_udp_tunnel_add(struct rte_eth_dev *eth_dev,
+		       struct rte_eth_udp_tunnel *udp_tnl)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (udp_tnl == NULL)
+		return -EINVAL;
+
+	if (udp_tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN) {
+		lio_dev_err(lio_dev, "Unsupported tunnel type\n");
+		return -1;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_VXLAN_PORT_CONFIG;
+	ctrl_pkt.ncmd.s.param1 = udp_tnl->udp_port;
+	ctrl_pkt.ncmd.s.more = LIO_CMD_VXLAN_PORT_ADD;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send VXLAN_PORT_ADD command\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "VXLAN_PORT_ADD command timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+/**
+ * Remove vxlan dest udp port for an interface.
+ *
+ * @param eth_dev
+ *  Pointer to the structure rte_eth_dev
+ * @param udp_tnl
+ *  udp tunnel conf
+ *
+ * @return
+ *  On success return 0
+ *  On failure return -1
+ */
+static int
+lio_dev_udp_tunnel_del(struct rte_eth_dev *eth_dev,
+		       struct rte_eth_udp_tunnel *udp_tnl)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (udp_tnl == NULL)
+		return -EINVAL;
+
+	if (udp_tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN) {
+		lio_dev_err(lio_dev, "Unsupported tunnel type\n");
+		return -1;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_VXLAN_PORT_CONFIG;
+	ctrl_pkt.ncmd.s.param1 = udp_tnl->udp_port;
+	ctrl_pkt.ncmd.s.more = LIO_CMD_VXLAN_PORT_DEL;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send VXLAN_PORT_DEL command\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "VXLAN_PORT_DEL command timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+/**
  * Atomically writes the link status information into global
  * structure rte_eth_dev.
  *
@@ -1028,6 +1143,74 @@
 	return 0;
 }
 
+/**
+ * Enable tunnel rx checksum verification from firmware.
+ */
+static void
+lio_enable_hw_tunnel_rx_checksum(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_TNL_RX_CSUM_CTL;
+	ctrl_pkt.ncmd.s.param1 = LIO_CMD_RXCSUM_ENABLE;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send TNL_RX_CSUM command\n");
+		return;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd))
+		lio_dev_err(lio_dev, "TNL_RX_CSUM command timed out\n");
+}
+
+/**
+ * Enable checksum calculation for inner packet in a tunnel.
+ */
+static void
+lio_enable_hw_tunnel_tx_checksum(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_TNL_TX_CSUM_CTL;
+	ctrl_pkt.ncmd.s.param1 = LIO_CMD_TXCSUM_ENABLE;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send TNL_TX_CSUM command\n");
+		return;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd))
+		lio_dev_err(lio_dev, "TNL_TX_CSUM command timed out\n");
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -1156,6 +1339,10 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	/* Copy the permanent MAC address */
 	ether_addr_copy((struct ether_addr *)mac, &eth_dev->data->mac_addrs[0]);
 
+	/* enable firmware checksum support for tunnel packets */
+	lio_enable_hw_tunnel_rx_checksum(eth_dev);
+	lio_enable_hw_tunnel_tx_checksum(eth_dev);
+
 	lio_dev->glist_lock =
 	    rte_zmalloc(NULL, sizeof(*lio_dev->glist_lock) * num_iqueues, 0);
 	if (lio_dev->glist_lock == NULL)
@@ -1214,6 +1401,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.reta_query		= lio_dev_rss_reta_query,
 	.rss_hash_conf_get	= lio_dev_rss_hash_conf_get,
 	.rss_hash_update	= lio_dev_rss_hash_update,
+	.udp_tunnel_port_add	= lio_dev_udp_tunnel_add,
+	.udp_tunnel_port_del	= lio_dev_udp_tunnel_del,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index d26aa6f..9c587ef 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1704,7 +1704,9 @@ struct lio_soft_command *
 		if (m->ol_flags & PKT_TX_IP_CKSUM)
 			cmdsetup.s.ip_csum = 1;
 
-		if ((m->ol_flags & PKT_TX_TCP_CKSUM) ||
+		if (m->ol_flags & PKT_TX_OUTER_IP_CKSUM)
+			cmdsetup.s.tnl_csum = 1;
+		else if ((m->ol_flags & PKT_TX_TCP_CKSUM) ||
 				(m->ol_flags & PKT_TX_UDP_CKSUM))
 			cmdsetup.s.transport_csum = 1;
 
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index e846d30..655042d 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -583,6 +583,7 @@ struct octeon_instr_rdp {
 	packet_params.pkt_params32 = 0;
 	packet_params.s.ip_csum = setup->s.ip_csum;
 	packet_params.s.transport_csum = setup->s.transport_csum;
+	packet_params.s.tnl_csum = setup->s.tnl_csum;
 	packet_params.s.tsflag = setup->s.timestamp;
 
 	irh->ossp = packet_params.pkt_params32;
-- 
1.8.3.1

[PATCH 44/50] net/liquidio: add support for Rx stats

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 50 +++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.c   | 10 +++++++-
 drivers/net/liquidio/lio_struct.h | 34 ++++++++++++++++++++++++++
 3 files changed, 93 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 5f8f1c0..2bdba09 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -117,6 +117,54 @@
 	return 0;
 }
 
+/* Retrieve the device statistics (# packets in/out, # bytes in/out, etc */
+static void
+lio_dev_stats_get(struct rte_eth_dev *eth_dev,
+		  struct rte_eth_stats *stats)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_droq_stats *oq_stats;
+	struct lio_droq *droq;
+	uint64_t bytes = 0;
+	uint64_t pkts = 0;
+	uint64_t drop = 0;
+	int i, oq_no;
+
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
+		oq_no = lio_dev->linfo.rxpciq[i].s.q_no;
+		droq = lio_dev->droq[oq_no];
+		if (droq != NULL) {
+			oq_stats = &droq->stats;
+			pkts += oq_stats->rx_pkts_received;
+			drop += (oq_stats->rx_dropped +
+					oq_stats->dropped_toomany +
+					oq_stats->dropped_nomem);
+			bytes += oq_stats->rx_bytes_received;
+		}
+	}
+	stats->ibytes = bytes;
+	stats->ipackets = pkts;
+	stats->ierrors = drop;
+}
+
+static void
+lio_dev_stats_reset(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_droq_stats *oq_stats;
+	struct lio_droq *droq;
+	int i, oq_no;
+
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
+		oq_no = lio_dev->linfo.rxpciq[i].s.q_no;
+		droq = lio_dev->droq[oq_no];
+		if (droq != NULL) {
+			oq_stats = &droq->stats;
+			memset(oq_stats, 0, sizeof(struct lio_droq_stats));
+		}
+	}
+}
+
 static void
 lio_dev_info_get(struct rte_eth_dev *eth_dev,
 		 struct rte_eth_dev_info *devinfo)
@@ -1391,6 +1439,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.allmulticast_enable	= lio_dev_allmulticast_enable,
 	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
+	.stats_get		= lio_dev_stats_get,
+	.stats_reset		= lio_dev_stats_reset,
 	.dev_infos_get		= lio_dev_info_get,
 	.mtu_set		= lio_dev_change_vf_mtu,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 9c587ef..794db32 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -115,6 +115,7 @@
 		buf = lio_recv_buffer_alloc(lio_dev, droq->q_no);
 		if (buf == NULL) {
 			lio_dev_err(lio_dev, "buffer alloc failed\n");
+			droq->stats.rx_alloc_failure++;
 			lio_droq_destroy_ring_buffers(droq);
 			return -ENOMEM;
 		}
@@ -410,8 +411,10 @@
 			/* If a buffer could not be allocated, no point in
 			 * continuing
 			 */
-			if (buf == NULL)
+			if (buf == NULL) {
+				droq->stats.rx_alloc_failure++;
 				break;
+			}
 
 			droq->recv_buf_list[droq->refill_idx].buffer = buf;
 		}
@@ -629,6 +632,11 @@
 	info->length = 0;
 	info->rh.rh64 = 0;
 
+	droq->stats.pkts_received++;
+	droq->stats.rx_pkts_received += data_pkts;
+	droq->stats.rx_bytes_received += data_total_len;
+	droq->stats.bytes_received += total_len;
+
 	return data_pkts;
 }
 
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index bdcc301..6788da1 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -56,6 +56,37 @@ struct lio_version {
 	uint16_t reserved;
 };
 
+/** Output Queue statistics. Each output queue has four stats fields. */
+struct lio_droq_stats {
+	/** Number of packets received in this queue. */
+	uint64_t pkts_received;
+
+	/** Bytes received by this queue. */
+	uint64_t bytes_received;
+
+	/** Packets dropped due to no memory available. */
+	uint64_t dropped_nomem;
+
+	/** Packets dropped due to large number of pkts to process. */
+	uint64_t dropped_toomany;
+
+	/** Number of packets  sent to stack from this queue. */
+	uint64_t rx_pkts_received;
+
+	/** Number of Bytes sent to stack from this queue. */
+	uint64_t rx_bytes_received;
+
+	/** Num of Packets dropped due to receive path failures. */
+	uint64_t rx_dropped;
+
+	/** Num of vxlan packets received; */
+	uint64_t rx_vxlan;
+
+	/** Num of failures of lio_recv_buffer_alloc() */
+	uint64_t rx_alloc_failure;
+
+};
+
 /** The Descriptor Ring Output Queue structure.
  *  This structure has all the information required to implement a
  *  DROQ.
@@ -117,6 +148,9 @@ struct lio_droq {
 	 */
 	void *pkts_sent_reg;
 
+	/** Statistics for this DROQ. */
+	struct lio_droq_stats stats;
+
 	/** DMA mapped address of the DROQ descriptor ring. */
 	size_t desc_ring_dma;
 
-- 
1.8.3.1

[PATCH 45/50] net/liquidio: add support for Tx stats

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 36 ++++++++++++++++++++++++++++++++++--
 drivers/net/liquidio/lio_rxtx.c   | 18 ++++++++++++++++--
 drivers/net/liquidio/lio_rxtx.h   |  3 +++
 drivers/net/liquidio/lio_struct.h | 15 +++++++++++++++
 4 files changed, 68 insertions(+), 4 deletions(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 2bdba09..7e6277a 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -124,11 +124,32 @@
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
 	struct lio_droq_stats *oq_stats;
+	struct lio_iq_stats *iq_stats;
+	struct lio_instr_queue *txq;
 	struct lio_droq *droq;
+	int i, iq_no, oq_no;
 	uint64_t bytes = 0;
 	uint64_t pkts = 0;
 	uint64_t drop = 0;
-	int i, oq_no;
+
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
+		iq_no = lio_dev->linfo.txpciq[i].s.q_no;
+		txq = lio_dev->instr_queue[iq_no];
+		if (txq != NULL) {
+			iq_stats = &txq->stats;
+			pkts += iq_stats->tx_done;
+			drop += iq_stats->tx_dropped;
+			bytes += iq_stats->tx_tot_bytes;
+		}
+	}
+
+	stats->opackets = pkts;
+	stats->obytes = bytes;
+	stats->oerrors = drop;
+
+	pkts = 0;
+	drop = 0;
+	bytes = 0;
 
 	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
 		oq_no = lio_dev->linfo.rxpciq[i].s.q_no;
@@ -152,8 +173,19 @@
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
 	struct lio_droq_stats *oq_stats;
+	struct lio_iq_stats *iq_stats;
+	struct lio_instr_queue *txq;
 	struct lio_droq *droq;
-	int i, oq_no;
+	int i, iq_no, oq_no;
+
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
+		iq_no = lio_dev->linfo.txpciq[i].s.q_no;
+		txq = lio_dev->instr_queue[iq_no];
+		if (txq != NULL) {
+			iq_stats = &txq->stats;
+			memset(iq_stats, 0, sizeof(struct lio_iq_stats));
+		}
+	}
 
 	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
 		oq_no = lio_dev->linfo.rxpciq[i].s.q_no;
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 794db32..edd16c6 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1113,8 +1113,10 @@
 
 		inst_processed = lio_process_iq_request_list(lio_dev, iq);
 
-		if (inst_processed)
+		if (inst_processed) {
 			rte_atomic64_sub(&iq->instr_pending, inst_processed);
+			iq->stats.instr_processed += inst_processed;
+		}
 
 		tot_inst_processed += inst_processed;
 		inst_processed = 0;
@@ -1130,7 +1132,7 @@
 
 static int
 lio_send_command(struct lio_device *lio_dev, uint32_t iq_no, void *cmd,
-		 void *buf, uint32_t datasize __rte_unused, uint32_t reqtype)
+		 void *buf, uint32_t datasize, uint32_t reqtype)
 {
 	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
 	struct lio_iq_post_status st;
@@ -1141,7 +1143,13 @@
 
 	if (st.status != LIO_IQ_SEND_FAILED) {
 		lio_add_to_request_list(iq, st.index, buf, reqtype);
+		LIO_INCR_INSTRQUEUE_PKT_COUNT(lio_dev, iq_no, bytes_sent,
+					      datasize);
+		LIO_INCR_INSTRQUEUE_PKT_COUNT(lio_dev, iq_no, instr_posted, 1);
+
 		lio_ring_doorbell(lio_dev, iq);
+	} else {
+		LIO_INCR_INSTRQUEUE_PKT_COUNT(lio_dev, iq_no, instr_dropped, 1);
 	}
 
 	rte_spinlock_unlock(&iq->post_lock);
@@ -1667,6 +1675,7 @@ struct lio_soft_command *
 	struct lio_instr_queue *txq = tx_queue;
 	union lio_cmd_setup cmdsetup;
 	struct lio_device *lio_dev;
+	struct lio_iq_stats *stats;
 	struct lio_data_pkt ndata;
 	int i, processed = 0;
 	struct rte_mbuf *m;
@@ -1676,6 +1685,7 @@ struct lio_soft_command *
 
 	lio_dev = txq->lio_dev;
 	iq_no = txq->txpciq.s.q_no;
+	stats = &lio_dev->instr_queue[iq_no]->stats;
 
 	if (!lio_dev->intf_open || !lio_dev->linfo.link.s.link_up) {
 		PMD_TX_LOG(lio_dev, ERR, "Transmit failed link_status : %d\n",
@@ -1697,6 +1707,7 @@ struct lio_soft_command *
 
 		ndata.q_no = iq_no;
 		if (lio_iq_is_full(lio_dev, ndata.q_no)) {
+			stats->tx_iq_busy++;
 			if (lio_dev_cleanup_iq(lio_dev, iq_no)) {
 				PMD_TX_LOG(lio_dev, ERR,
 					   "Transmit failed iq:%d full\n",
@@ -1804,10 +1815,13 @@ struct lio_soft_command *
 			lio_dev_cleanup_iq(lio_dev, iq_no);
 		}
 
+		stats->tx_done++;
+		stats->tx_tot_bytes += pkt_len;
 		processed++;
 	}
 
 xmit_failed:
+	stats->tx_dropped += (nb_pkts - processed);
 
 	return processed;
 }
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 655042d..6c024e1 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -670,6 +670,9 @@ enum {
  */
 int lio_process_ordered_list(struct lio_device *lio_dev);
 
+#define LIO_INCR_INSTRQUEUE_PKT_COUNT(lio_dev, iq_no, field, count)	\
+	(((lio_dev)->instr_queue[iq_no]->stats.field) += count)
+
 static inline void
 lio_swap_8B_data(uint64_t *data, uint32_t blocks)
 {
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 6788da1..b1fca92 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -56,6 +56,18 @@ struct lio_version {
 	uint16_t reserved;
 };
 
+/** Input Queue statistics. Each input queue has four stats fields. */
+struct lio_iq_stats {
+	uint64_t instr_posted; /**< Instructions posted to this queue. */
+	uint64_t instr_processed; /**< Instructions processed in this queue. */
+	uint64_t instr_dropped; /**< Instructions that could not be processed */
+	uint64_t bytes_sent; /**< Bytes sent through this queue. */
+	uint64_t tx_done; /**< Num of packets sent to network. */
+	uint64_t tx_iq_busy; /**< Num of times this iq was found to be full. */
+	uint64_t tx_dropped; /**< Num of pkts dropped due to xmitpath errors. */
+	uint64_t tx_tot_bytes; /**< Total count of bytes sent to network. */
+};
+
 /** Output Queue statistics. Each output queue has four stats fields. */
 struct lio_droq_stats {
 	/** Number of packets received in this queue. */
@@ -319,6 +331,9 @@ struct lio_instr_queue {
 	/** Number of instructions pending to be posted to Octeon. */
 	uint32_t fill_cnt;
 
+	/** Statistics for this input queue. */
+	struct lio_iq_stats stats;
+
 	/** DMA mapped base address of the input descriptor ring. */
 	uint64_t base_addr_dma;
 
-- 
1.8.3.1

[PATCH 46/50] net/liquidio: add APIs for hardware stats

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |   2 +
 drivers/net/liquidio/lio_ethdev.c       | 194 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  83 ++++++++++++++
 3 files changed, 279 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 39a3b3d..d0a8936 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -124,6 +124,7 @@ enum octeon_tag_type {
 #define LIO_OPCODE_NW_DATA		0x02 /* network packet data */
 #define LIO_OPCODE_CMD			0x03
 #define LIO_OPCODE_INFO			0x04
+#define LIO_OPCODE_PORT_STATS		0x05
 #define LIO_OPCODE_IF_CFG		0x09
 
 #define LIO_MIN_RX_BUF_SIZE		64
@@ -132,6 +133,7 @@ enum octeon_tag_type {
 /* NIC Command types */
 #define LIO_CMD_CHANGE_DEVFLAGS		0x3
 #define LIO_CMD_RX_CTL			0x4
+#define LIO_CMD_CLEAR_STATS		0x6
 #define LIO_CMD_SET_RSS			0xD
 #define LIO_CMD_TNL_RX_CSUM_CTL		0x10
 #define LIO_CMD_TNL_TX_CSUM_CTL		0x11
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 7e6277a..17448fe 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -117,6 +117,197 @@
 	return 0;
 }
 
+/* store statistics names and its offset in stats structure */
+struct rte_lio_xstats_name_off {
+	char name[RTE_ETH_XSTATS_NAME_SIZE];
+	unsigned int offset;
+};
+
+static const struct rte_lio_xstats_name_off rte_lio_stats_strings[] = {
+	{"rx_pkts", offsetof(struct octeon_rx_stats, total_rcvd)},
+	{"rx_bytes", offsetof(struct octeon_rx_stats, bytes_rcvd)},
+	{"rx_broadcast_pkts", offsetof(struct octeon_rx_stats, total_bcst)},
+	{"rx_multicast_pkts", offsetof(struct octeon_rx_stats, total_mcst)},
+	{"rx_flow_ctrl_pkts", offsetof(struct octeon_rx_stats, ctl_rcvd)},
+	{"rx_fifo_err", offsetof(struct octeon_rx_stats, fifo_err)},
+	{"rx_dmac_drop", offsetof(struct octeon_rx_stats, dmac_drop)},
+	{"rx_fcs_err", offsetof(struct octeon_rx_stats, fcs_err)},
+	{"rx_jabber_err", offsetof(struct octeon_rx_stats, jabber_err)},
+	{"rx_l2_err", offsetof(struct octeon_rx_stats, l2_err)},
+	{"rx_vxlan_pkts", offsetof(struct octeon_rx_stats, fw_rx_vxlan)},
+	{"rx_vxlan_err", offsetof(struct octeon_rx_stats, fw_rx_vxlan_err)},
+	{"rx_lro_pkts", offsetof(struct octeon_rx_stats, fw_lro_pkts)},
+	{"tx_pkts", (offsetof(struct octeon_tx_stats, total_pkts_sent)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_bytes", (offsetof(struct octeon_tx_stats, total_bytes_sent)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_broadcast_pkts",
+		(offsetof(struct octeon_tx_stats, bcast_pkts_sent)) +
+			sizeof(struct octeon_rx_stats)},
+	{"tx_multicast_pkts",
+		(offsetof(struct octeon_tx_stats, mcast_pkts_sent)) +
+			sizeof(struct octeon_rx_stats)},
+	{"tx_flow_ctrl_pkts", (offsetof(struct octeon_tx_stats, ctl_sent)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_fifo_err", (offsetof(struct octeon_tx_stats, fifo_err)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_total_collisions", (offsetof(struct octeon_tx_stats,
+					  total_collisions)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_tso", (offsetof(struct octeon_tx_stats, fw_tso)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_vxlan_pkts", (offsetof(struct octeon_tx_stats, fw_tx_vxlan)) +
+						sizeof(struct octeon_rx_stats)},
+};
+
+#define LIO_NB_XSTATS	RTE_DIM(rte_lio_stats_strings)
+
+/* Get hw stats of the port */
+static int
+lio_dev_xstats_get(struct rte_eth_dev *eth_dev, struct rte_eth_xstat *xstats,
+		   unsigned int n)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
+	struct octeon_link_stats *hw_stats;
+	struct lio_link_stats_resp *resp;
+	struct lio_soft_command *sc;
+	uint32_t resp_size;
+	unsigned int i;
+	int retval;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	if (n < LIO_NB_XSTATS)
+		return LIO_NB_XSTATS;
+
+	resp_size = sizeof(struct lio_link_stats_resp);
+	sc = lio_alloc_soft_command(lio_dev, 0, resp_size, 0);
+	if (sc == NULL)
+		return -ENOMEM;
+
+	resp = (struct lio_link_stats_resp *)sc->virtrptr;
+	lio_prepare_soft_command(lio_dev, sc, LIO_OPCODE,
+				 LIO_OPCODE_PORT_STATS, 0, 0, 0);
+
+	/* Setting wait time in seconds */
+	sc->wait_time = LIO_MAX_CMD_TIMEOUT / 1000;
+
+	retval = lio_send_soft_command(lio_dev, sc);
+	if (retval == LIO_IQ_SEND_FAILED) {
+		lio_dev_err(lio_dev, "failed to get port stats from firmware. status: %x\n",
+			    retval);
+		goto get_stats_fail;
+	}
+
+	while ((*sc->status_word == LIO_COMPLETION_WORD_INIT) && --timeout) {
+		lio_flush_iq(lio_dev, lio_dev->instr_queue[sc->iq_no]);
+		lio_process_ordered_list(lio_dev);
+		rte_delay_ms(1);
+	}
+
+	retval = resp->status;
+	if (retval) {
+		lio_dev_err(lio_dev, "failed to get port stats from firmware\n");
+		goto get_stats_fail;
+	}
+
+	lio_swap_8B_data((uint64_t *)(&resp->link_stats),
+			 sizeof(struct octeon_link_stats) >> 3);
+
+	hw_stats = &resp->link_stats;
+
+	for (i = 0; i < LIO_NB_XSTATS; i++) {
+		xstats[i].id = i;
+		xstats[i].value =
+		    *(uint64_t *)(((char *)hw_stats) +
+					rte_lio_stats_strings[i].offset);
+	}
+
+	lio_free_soft_command(sc);
+
+	return LIO_NB_XSTATS;
+
+get_stats_fail:
+	lio_free_soft_command(sc);
+
+	return -1;
+}
+
+static int
+lio_dev_xstats_get_names(struct rte_eth_dev *eth_dev,
+			 struct rte_eth_xstat_name *xstats_names,
+			 unsigned limit __rte_unused)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	unsigned int i;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	if (xstats_names == NULL)
+		return LIO_NB_XSTATS;
+
+	/* Note: limit checked in rte_eth_xstats_names() */
+
+	for (i = 0; i < LIO_NB_XSTATS; i++) {
+		snprintf(xstats_names[i].name, sizeof(xstats_names[i].name),
+			 "%s", rte_lio_stats_strings[i].name);
+	}
+
+	return LIO_NB_XSTATS;
+}
+
+/* Reset hw stats for the port */
+static void
+lio_dev_xstats_reset(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down\n",
+			    lio_dev->port_id);
+		return;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_CLEAR_STATS;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send clear stats command\n");
+		return;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "Clear stats command timed out\n");
+		return;
+	}
+
+	/* clear stored per queue stats */
+	RTE_FUNC_PTR_OR_RET(*eth_dev->dev_ops->stats_reset);
+	(*eth_dev->dev_ops->stats_reset)(eth_dev);
+}
+
 /* Retrieve the device statistics (# packets in/out, # bytes in/out, etc */
 static void
 lio_dev_stats_get(struct rte_eth_dev *eth_dev,
@@ -1472,7 +1663,10 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
 	.stats_get		= lio_dev_stats_get,
+	.xstats_get		= lio_dev_xstats_get,
+	.xstats_get_names	= lio_dev_xstats_get_names,
 	.stats_reset		= lio_dev_stats_reset,
+	.xstats_reset		= lio_dev_xstats_reset,
 	.dev_infos_get		= lio_dev_info_get,
 	.mtu_set		= lio_dev_change_vf_mtu,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 6543061..150e9c9 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -62,6 +62,83 @@ struct octeon_if_cfg_info {
 	char lio_firmware_version[LIO_FW_VERSION_LENGTH];
 };
 
+/** Stats for each NIC port in RX direction. */
+struct octeon_rx_stats {
+	/* link-level stats */
+	uint64_t total_rcvd;
+	uint64_t bytes_rcvd;
+	uint64_t total_bcst;
+	uint64_t total_mcst;
+	uint64_t runts;
+	uint64_t ctl_rcvd;
+	uint64_t fifo_err; /* Accounts for over/under-run of buffers */
+	uint64_t dmac_drop;
+	uint64_t fcs_err;
+	uint64_t jabber_err;
+	uint64_t l2_err;
+	uint64_t frame_err;
+
+	/* firmware stats */
+	uint64_t fw_total_rcvd;
+	uint64_t fw_total_fwd;
+	uint64_t fw_total_fwd_bytes;
+	uint64_t fw_err_pko;
+	uint64_t fw_err_link;
+	uint64_t fw_err_drop;
+	uint64_t fw_rx_vxlan;
+	uint64_t fw_rx_vxlan_err;
+
+	/* LRO */
+	uint64_t fw_lro_pkts;   /* Number of packets that are LROed */
+	uint64_t fw_lro_octs;   /* Number of octets that are LROed */
+	uint64_t fw_total_lro;  /* Number of LRO packets formed */
+	uint64_t fw_lro_aborts; /* Number of times lRO of packet aborted */
+	uint64_t fw_lro_aborts_port;
+	uint64_t fw_lro_aborts_seq;
+	uint64_t fw_lro_aborts_tsval;
+	uint64_t fw_lro_aborts_timer;
+	/* intrmod: packet forward rate */
+	uint64_t fwd_rate;
+};
+
+/** Stats for each NIC port in RX direction. */
+struct octeon_tx_stats {
+	/* link-level stats */
+	uint64_t total_pkts_sent;
+	uint64_t total_bytes_sent;
+	uint64_t mcast_pkts_sent;
+	uint64_t bcast_pkts_sent;
+	uint64_t ctl_sent;
+	uint64_t one_collision_sent;	/* Packets sent after one collision */
+	/* Packets sent after multiple collision */
+	uint64_t multi_collision_sent;
+	/* Packets not sent due to max collisions */
+	uint64_t max_collision_fail;
+	/* Packets not sent due to max deferrals */
+	uint64_t max_deferral_fail;
+	/* Accounts for over/under-run of buffers */
+	uint64_t fifo_err;
+	uint64_t runts;
+	uint64_t total_collisions; /* Total number of collisions detected */
+
+	/* firmware stats */
+	uint64_t fw_total_sent;
+	uint64_t fw_total_fwd;
+	uint64_t fw_total_fwd_bytes;
+	uint64_t fw_err_pko;
+	uint64_t fw_err_link;
+	uint64_t fw_err_drop;
+	uint64_t fw_err_tso;
+	uint64_t fw_tso;     /* number of tso requests */
+	uint64_t fw_tso_fwd; /* number of packets segmented in tso */
+	uint64_t fw_tx_vxlan;
+};
+
+struct octeon_link_stats {
+	struct octeon_rx_stats fromwire;
+	struct octeon_tx_stats fromhost;
+};
+
 union lio_if_cfg {
 	uint64_t if_cfg64;
 	struct {
@@ -87,6 +164,12 @@ struct lio_if_cfg_resp {
 	uint64_t status;
 };
 
+struct lio_link_stats_resp {
+	uint64_t rh;
+	struct octeon_link_stats link_stats;
+	uint64_t status;
+};
+
 struct lio_link_status_resp {
 	uint64_t rh;
 	struct octeon_link_info link_info;
-- 
1.8.3.1

[PATCH 47/50] net/liquidio: add API for dev stop

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 17448fe..d28ded8 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1360,6 +1360,25 @@ struct rte_lio_xstats_name_off {
 	return ret;
 }
 
+/* Stop device and disable input/output functions */
+static void
+lio_dev_stop(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	lio_dev_info(lio_dev, "Stopping port %d\n", eth_dev->data->port_id);
+	lio_dev->intf_open = 0;
+	rte_mb();
+
+	/* Cancel callback if still running. */
+	rte_eal_alarm_cancel(lio_sync_link_state_check, eth_dev);
+
+	lio_send_rx_ctrl_cmd(eth_dev, 0);
+
+	/* Clear recorded link status */
+	lio_dev->linfo.link.link_status64 = 0;
+}
+
 static int
 lio_dev_set_link_up(struct rte_eth_dev *eth_dev)
 {
@@ -1657,6 +1676,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
 	.dev_start		= lio_dev_start,
+	.dev_stop		= lio_dev_stop,
 	.dev_set_link_up	= lio_dev_set_link_up,
 	.dev_set_link_down	= lio_dev_set_link_down,
 	.allmulticast_enable	= lio_dev_allmulticast_enable,
-- 
1.8.3.1

[PATCH 48/50] net/liquidio: add API for dev close

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 68 +++++++++++++++++++++++++++++++++++++--
 drivers/net/liquidio/lio_ethdev.h |  5 +++
 drivers/net/liquidio/lio_rxtx.c   | 57 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  2 ++
 4 files changed, 130 insertions(+), 2 deletions(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index d28ded8..cb2d048 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1109,7 +1109,7 @@ struct rte_lio_xstats_name_off {
  * @return
  *    - nothing
  */
-static void
+void
 lio_dev_rx_queue_release(void *rxq)
 {
 	struct lio_droq *droq = rxq;
@@ -1204,7 +1204,7 @@ struct rte_lio_xstats_name_off {
  * @return
  *    - nothing
  */
-static void
+void
 lio_dev_tx_queue_release(void *txq)
 {
 	struct lio_instr_queue *tq = txq;
@@ -1434,6 +1434,68 @@ struct rte_lio_xstats_name_off {
 }
 
 /**
+ * Reset and stop the device. This occurs on the first
+ * call to this routine. Subsequent calls will simply
+ * return. NB: This will require the NIC to be rebooted.
+ *
+ * @param eth_dev
+ *    Pointer to the structure rte_eth_dev
+ *
+ * @return
+ *    - nothing
+ */
+static void
+lio_dev_close(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint32_t i;
+
+	lio_dev_info(lio_dev, "closing port %d\n", eth_dev->data->port_id);
+
+	if (lio_dev->intf_open)
+		lio_dev_stop(eth_dev);
+
+	lio_wait_for_instr_fetch(lio_dev);
+
+	lio_dev->fn_list.disable_io_queues(lio_dev);
+
+	cn23xx_vf_set_io_queues_off(lio_dev);
+
+	/* Reset iq regs (IQ_DBELL).
+	 * Clear sli_pktx_cnts (OQ_PKTS_SENT).
+	 */
+	for (i = 0; i < lio_dev->nb_rx_queues; i++) {
+		struct lio_droq *droq = lio_dev->droq[i];
+
+		if (droq == NULL)
+			break;
+
+		uint32_t pkt_count = rte_read32(droq->pkts_sent_reg);
+
+		lio_dev_dbg(lio_dev,
+			    "pending oq count %u\n", pkt_count);
+		rte_write32(pkt_count, droq->pkts_sent_reg);
+	}
+
+	/* Do FLR for the VF */
+	cn23xx_vf_ask_pf_to_do_flr(lio_dev);
+
+	/* lio_free_mbox */
+	lio_dev->fn_list.free_mbox(lio_dev);
+
+	/* Free glist resources */
+	rte_free(lio_dev->glist_head);
+	rte_free(lio_dev->glist_lock);
+	lio_dev->glist_head = NULL;
+	lio_dev->glist_lock = NULL;
+
+	lio_dev->port_configured = 0;
+
+	 /* Delete all queues */
+	lio_dev_clear_queues(eth_dev);
+}
+
+/**
  * Enable tunnel rx checksum verification from firmware.
  */
 static void
@@ -1679,6 +1741,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.dev_stop		= lio_dev_stop,
 	.dev_set_link_up	= lio_dev_set_link_up,
 	.dev_set_link_down	= lio_dev_set_link_down,
+	.dev_close		= lio_dev_close,
 	.allmulticast_enable	= lio_dev_allmulticast_enable,
 	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
@@ -1841,6 +1904,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	rte_free(eth_dev->data->mac_addrs);
 	eth_dev->data->mac_addrs = NULL;
 
+	eth_dev->dev_ops = NULL;
 	eth_dev->rx_pkt_burst = NULL;
 	eth_dev->tx_pkt_burst = NULL;
 
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 150e9c9..ee30615 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -196,4 +196,9 @@ struct lio_rss_set {
 	uint8_t itable[LIO_RSS_MAX_TABLE_SZ];
 	uint8_t key[LIO_RSS_MAX_KEY_SZ];
 };
+
+void lio_dev_rx_queue_release(void *rxq);
+
+void lio_dev_tx_queue_release(void *txq);
+
 #endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index edd16c6..2cc40d8 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -918,6 +918,40 @@
 	return -1;
 }
 
+int
+lio_wait_for_instr_fetch(struct lio_device *lio_dev)
+{
+	int pending, instr_cnt;
+	int i, retry = 1000;
+
+	do {
+		instr_cnt = 0;
+
+		for (i = 0; i < LIO_MAX_INSTR_QUEUES(lio_dev); i++) {
+			if (!(lio_dev->io_qmask.iq & (1ULL << i)))
+				continue;
+
+			if (lio_dev->instr_queue[i] == NULL)
+				break;
+
+			pending = rte_atomic64_read(
+			    &lio_dev->instr_queue[i]->instr_pending);
+			if (pending)
+				lio_flush_iq(lio_dev, lio_dev->instr_queue[i]);
+
+			instr_cnt += pending;
+		}
+
+		if (instr_cnt == 0)
+			break;
+
+		rte_delay_ms(1);
+
+	} while (retry-- && instr_cnt);
+
+	return instr_cnt;
+}
+
 static inline void
 lio_ring_doorbell(struct lio_device *lio_dev,
 		  struct lio_instr_queue *iq)
@@ -1826,3 +1860,26 @@ struct lio_soft_command *
 	return processed;
 }
 
+void
+lio_dev_clear_queues(struct rte_eth_dev *eth_dev)
+{
+	struct lio_instr_queue *txq;
+	struct lio_droq *rxq;
+	uint16_t i;
+
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
+		txq = eth_dev->data->tx_queues[i];
+		if (txq != NULL) {
+			lio_dev_tx_queue_release(txq);
+			eth_dev->data->tx_queues[i] = NULL;
+		}
+	}
+
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
+		rxq = eth_dev->data->rx_queues[i];
+		if (rxq != NULL) {
+			lio_dev_rx_queue_release(rxq);
+			eth_dev->data->rx_queues[i] = NULL;
+		}
+	}
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 6c024e1..739e86d 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -752,6 +752,7 @@ int lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
 		      int fw_mapped_iq, int num_descs, unsigned int socket_id);
 uint16_t lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts,
 			   uint16_t nb_pkts);
+int lio_wait_for_instr_fetch(struct lio_device *lio_dev);
 int lio_setup_iq(struct lio_device *lio_dev, int q_index,
 		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
 		 unsigned int socket_id);
@@ -764,4 +765,5 @@ int lio_setup_iq(struct lio_device *lio_dev, int q_index,
  */
 int lio_setup_instr_queue0(struct lio_device *lio_dev);
 void lio_free_instr_queue0(struct lio_device *lio_dev);
+void lio_dev_clear_queues(struct rte_eth_dev *eth_dev);
 #endif	/* _LIO_RXTX_H_ */
-- 
1.8.3.1

[PATCH 49/50] net/liquidio: add API to add and remove VLAN port

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  2 ++
 drivers/net/liquidio/lio_ethdev.c       | 46 ++++++++++++++++++++++++++++++++-
 2 files changed, 47 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index d0a8936..dec12f8 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -137,6 +137,8 @@ enum octeon_tag_type {
 #define LIO_CMD_SET_RSS			0xD
 #define LIO_CMD_TNL_RX_CSUM_CTL		0x10
 #define LIO_CMD_TNL_TX_CSUM_CTL		0x11
+#define LIO_CMD_ADD_VLAN_FILTER		0x17
+#define LIO_CMD_DEL_VLAN_FILTER		0x18
 #define LIO_CMD_VXLAN_PORT_CONFIG	0x19
 
 #define LIO_CMD_VXLAN_PORT_ADD		0x0
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index cb2d048..a003de4 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -404,7 +404,8 @@ struct rte_lio_xstats_name_off {
 
 	devinfo->rx_offload_capa = (DEV_RX_OFFLOAD_IPV4_CKSUM		|
 				    DEV_RX_OFFLOAD_UDP_CKSUM		|
-				    DEV_RX_OFFLOAD_TCP_CKSUM);
+				    DEV_RX_OFFLOAD_TCP_CKSUM		|
+				    DEV_RX_OFFLOAD_VLAN_STRIP);
 	devinfo->tx_offload_capa = (DEV_TX_OFFLOAD_IPV4_CKSUM		|
 				    DEV_TX_OFFLOAD_UDP_CKSUM		|
 				    DEV_TX_OFFLOAD_TCP_CKSUM		|
@@ -825,6 +826,48 @@ struct rte_lio_xstats_name_off {
 	return 0;
 }
 
+static int
+lio_dev_vlan_filter_set(struct rte_eth_dev *eth_dev, uint16_t vlan_id, int on)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+
+	if (lio_dev->linfo.vlan_is_admin_assigned)
+		return -EPERM;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = on ?
+			LIO_CMD_ADD_VLAN_FILTER : LIO_CMD_DEL_VLAN_FILTER;
+	ctrl_pkt.ncmd.s.param1 = vlan_id;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to %s VLAN port\n",
+			    on ? "add" : "remove");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "Command to %s VLAN port timed out\n",
+			    on ? "add" : "remove");
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Atomically writes the link status information into global
  * structure rte_eth_dev.
@@ -1752,6 +1795,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.xstats_reset		= lio_dev_xstats_reset,
 	.dev_infos_get		= lio_dev_info_get,
 	.mtu_set		= lio_dev_change_vf_mtu,
+	.vlan_filter_set	= lio_dev_vlan_filter_set,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
-- 
1.8.3.1

[PATCH 50/50] doc: added documents

[Shijith Thotton]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=y, Size: 10359 bytes --]

Added doc/guides/nics/liquidio.rst and
doc/guides/nics/features/liquidio.ini. Updated release notes.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
---
 doc/guides/nics/features/liquidio.ini  |  29 ++++
 doc/guides/nics/index.rst              |   1 +
 doc/guides/nics/liquidio.rst           | 269 +++++++++++++++++++++++++++++++++
 doc/guides/rel_notes/release_17_05.rst |   3 +
 4 files changed, 302 insertions(+)
 create mode 100644 doc/guides/nics/features/liquidio.ini
 create mode 100644 doc/guides/nics/liquidio.rst

diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
new file mode 100644
index 0000000..eac32ba
--- /dev/null
+++ b/doc/guides/nics/features/liquidio.ini
@@ -0,0 +1,29 @@
+;
+; Supported features of the 'LiquidIO' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+Link status          = Y
+Link status event    = Y
+MTU update           = Y
+Jumbo frame          = Y
+Scattered Rx         = Y
+Allmulticast mode    = Y
+RSS hash             = Y
+RSS key update       = Y
+RSS reta update      = Y
+SR-IOV               = Y
+VLAN filter          = Y
+CRC offload          = Y
+VLAN offload         = P
+L3 checksum offload  = Y
+L4 checksum offload  = Y
+Inner L3 checksum    = Y
+Inner L4 checksum    = Y
+Basic stats          = Y
+Extended stats       = Y
+Linux UIO            = Y
+Linux VFIO           = Y
+x86-64               = Y
+Usage doc            = Y
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 5248625..37e6416 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -47,6 +47,7 @@ Network Interface Controller Drivers
     ixgbe
     intel_vf
     kni
+    liquidio
     mlx4
     mlx5
     nfp
diff --git a/doc/guides/nics/liquidio.rst b/doc/guides/nics/liquidio.rst
new file mode 100644
index 0000000..4bf586b
--- /dev/null
+++ b/doc/guides/nics/liquidio.rst
@@ -0,0 +1,269 @@
+..  BSD LICENSE
+    Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+    All rights reserved.
+
+    Redistribution and use in source and binary forms, with or without
+    modification, are permitted provided that the following conditions
+    are met:
+
+    * Redistributions of source code must retain the above copyright
+    notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright
+    notice, this list of conditions and the following disclaimer in
+    the documentation and/or other materials provided with the
+    distribution.
+    * Neither the name of Cavium, Inc. nor the names of its
+    contributors may be used to endorse or promote products derived
+    from this software without specific prior written permission.
+
+    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+    OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+LiquidIO VF Poll Mode Driver
+============================
+
+The LiquidIO VF PMD library(librte_pmd_lio) provides poll mode driver support for
+Cavium LiquidIO® II server adapter VFs. PF management and VF creation can be
+done using kernel driver.
+
+Supported LiquidIO Adapters
+-----------------------------
+
+- LiquidIO II CN2350 210SV
+
+
+Pre-Installation Configuration
+------------------------------
+
+The following options can be modified in the ``config`` file.
+Please note that enabling debugging options may affect system performance.
+
+- ``CONFIG_RTE_LIBRTE_LIO_PMD`` (default ``y``)
+
+  Toggle compilation of LiquidIO PMD.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_DRIVER`` (default ``n``)
+
+  Toggle display of generic debugging messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_INIT`` (default ``n``)
+
+  Toggle display of initialization related messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_RX`` (default ``n``)
+
+  Toggle display of receive fast path run-time messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_TX`` (default ``n``)
+
+  Toggle display of transmit fast path run-time messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_MBOX`` (default ``n``)
+
+  Toggle display of mailbox messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_REGS`` (default ``n``)
+
+  Toggle display of register reads and writes.
+
+
+.. _lio_driver-compilation:
+
+Driver Compilation
+------------------
+
+To compile LiquidIO PMD for Linux x86_64 gcc target, run the following "make"
+command:
+
+.. code-block:: console
+
+   cd <DPDK-source-directory>
+   make install T=x86_64-native-linuxapp-gcc
+
+
+Sample Application Notes
+------------------------
+
+This section demonstrates how to launch ``testpmd`` with LiquidIO® CN23X0
+device managed by ``librte_pmd_lio`` in Linux operating system.
+
+#. Mount huge pages:
+
+   .. code-block:: console
+
+      mkdir /mnt/huge
+      mount -t hugetlbfs nodev /mnt/huge
+
+#. Request huge pages:
+
+   .. code-block:: console
+
+      echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages/nr_hugepages
+
+#. Load ``vfio-pci`` driver:
+
+   .. code-block:: console
+
+      modprobe vfio-pci
+
+#. Bind the LiquidIO VFs to ``vfio-pci`` loaded in previous step:
+
+   Setup VFIO permissions for regular users and then bind to ``vfio-pci``:
+
+   .. code-block:: console
+
+      sudo chmod a+x /dev/vfio
+
+      sudo chmod 0666 /dev/vfio/*
+
+      ./usertools/dpdk-devbind.py --bind vfio-pci 0000:03:00.3 0000:03:08.3
+
+#. Start ``testpmd`` with basic parameters:
+
+   .. code-block:: console
+
+      ./build/app/testpmd -c 0xf -n 4 -- -i
+
+   Example output:
+
+   .. code-block:: console
+
+      [...]
+      EAL: PCI device 0000:03:00.3 on NUMA socket 0
+      EAL:   probe driver: 177d:9712 net_liovf
+      EAL:   using IOMMU type 1 (Type 1)
+      PMD: net_liovf[03:00.3]INFO: DEVICE : CN23XX VF
+      EAL: PCI device 0000:03:08.3 on NUMA socket 0
+      EAL:   probe driver: 177d:9712 net_liovf
+      PMD: net_liovf[03:08.3]INFO: DEVICE : CN23XX VF
+      Interactive-mode selected
+      USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=171456, size=2176, socket=0
+      Configuring Port 0 (socket 0)
+      PMD: net_liovf[03:00.3]INFO: Starting port 0
+      Port 0: F2:A8:1B:5E:B4:66
+      Configuring Port 1 (socket 0)
+      PMD: net_liovf[03:08.3]INFO: Starting port 1
+      Port 1: 32:76:CC:EE:56:D7
+      Checking link statuses...
+      Port 0 Link Up - speed 10000 Mbps - full-duplex
+      Port 1 Link Up - speed 10000 Mbps - full-duplex
+      Done
+      testpmd>
+
+
+SR-IOV: Prerequisites and Sample Application Notes
+--------------------------------------------------
+
+This section provides instructions to configure SR-IOV with Linux OS.
+
+#. Verify SR-IOV and ARI capabilities are enabled on the adapter using ``lspci``:
+
+   .. code-block:: console
+
+      lspci -s <slot> -vvv
+
+   Example output:
+
+   .. code-block:: console
+
+      [...]
+      Capabilities: [148 v1] Alternative Routing-ID Interpretation (ARI)
+      [...]
+      Capabilities: [178 v1] Single Root I/O Virtualization (SR-IOV)
+      [...]
+      Kernel driver in use: LiquidIO
+
+#. Load the kernel module:
+
+   .. code-block:: console
+
+      modprobe liquidio
+
+#. Bring up the PF ports:
+
+   .. code-block:: console
+
+      ifconfig p4p1 up
+      ifconfig p4p2 up
+
+#. Change PF MTU if required:
+
+   .. code-block:: console
+
+      ifconfig p4p1 mtu 9000
+      ifconfig p4p2 mtu 9000
+
+#. Create VF device(s):
+
+   Echo number of VFs to be created into ``"sriov_numvfs"`` sysfs entry
+   of the parent PF.
+
+   .. code-block:: console
+
+      echo 1 > /sys/bus/pci/devices/0000:03:00.0/sriov_numvfs
+      echo 1 > /sys/bus/pci/devices/0000:03:00.1/sriov_numvfs
+
+
+#. Assign VF MAC address:
+
+   Assign MAC address to the VF using iproute2 utility. The syntax is::
+
+      ip link set <PF iface> vf <VF id> mac <macaddr>
+
+   Example output:
+
+   .. code-block:: console
+
+      ip link set p4p1 vf 0 mac F2:A8:1B:5E:B4:66
+
+
+#. Assign VF(s) to VM.
+
+   The VF devices may be passed through to the guest VM using qemu or
+   virt-manager or virsh etc.
+
+   Example qemu guest launch command:
+
+   .. code-block:: console
+
+      ./qemu-system-x86_64 -name lio-vm -machine accel=kvm \
+      -cpu host -m 4096 -smp 4 \
+      -drive file=<disk_file>,if=none,id=disk1,format=<type> \
+      -device virtio-blk-pci,scsi=off,drive=disk1,id=virtio-disk1,bootindex=1 \
+      -device vfio-pci,host=03:00.3 -device vfio-pci,host=03:08.3
+
+
+#. Running testpmd
+
+   Refer :ref:`notes above <lio_driver-compilation>`
+   to compile and run ``testpmd`` application.
+   Use ``igb_uio`` instead of ``vfio-pci`` in VM.
+
+
+Limitations
+-----------
+
+VF MTU
+~~~~~~
+
+VF MTU is limited by PF MTU. Raise PF value before configuring VF for larger packet size.
+
+VLAN offload
+~~~~~~~~~~~~
+
+Tx VLAN insertion is not supported and consequently VLAN offload feature is
+marked partial.
+
+Ring size
+~~~~~~~~~
+
+Number of descriptors for Rx/Tx ring should be in the range 128 to 512.
diff --git a/doc/guides/rel_notes/release_17_05.rst b/doc/guides/rel_notes/release_17_05.rst
index 48fb5bd..23a4bfe 100644
--- a/doc/guides/rel_notes/release_17_05.rst
+++ b/doc/guides/rel_notes/release_17_05.rst
@@ -41,6 +41,9 @@ New Features
      Also, make sure to start the actual text at the margin.
      =========================================================
 
+* **Added LiquidIO network PMD.**
+
+  Added poll mode driver support for Cavium LiquidIO II server adapter VFs.
 
 Resolved Issues
 ---------------
-- 
1.8.3.1

Re: [PATCH 50/50] doc: added documents

[Ferruh Yigit]

On 2/21/2017 9:27 AM, Shijith Thotton wrote:
> Added doc/guides/nics/liquidio.rst and
> doc/guides/nics/features/liquidio.ini. Updated release notes.
> 
> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> ---
>  doc/guides/nics/features/liquidio.ini  |  29 ++++
>  doc/guides/nics/index.rst              |   1 +
>  doc/guides/nics/liquidio.rst           | 269 +++++++++++++++++++++++++++++++++
>  doc/guides/rel_notes/release_17_05.rst |   3 +

The web page that list the supported NICs [1] also needs to be updated,
in a separate patch.

[1]
http://dpdk.org/doc/nics

>  4 files changed, 302 insertions(+)
>  create mode 100644 doc/guides/nics/features/liquidio.ini
>  create mode 100644 doc/guides/nics/liquidio.rst
> 
> diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
> new file mode 100644
> index 0000000..eac32ba
> --- /dev/null
> +++ b/doc/guides/nics/features/liquidio.ini
> @@ -0,0 +1,29 @@
> +;
> +; Supported features of the 'LiquidIO' network poll mode driver.
> +;
> +; Refer to default.ini for the full list of available PMD features.
> +;
> +[Features]
> +Link status          = Y
> +Link status event    = Y
> +MTU update           = Y
> +Jumbo frame          = Y
> +Scattered Rx         = Y
> +Allmulticast mode    = Y
> +RSS hash             = Y
> +RSS key update       = Y
> +RSS reta update      = Y
> +SR-IOV               = Y
> +VLAN filter          = Y
> +CRC offload          = Y
> +VLAN offload         = P
> +L3 checksum offload  = Y
> +L4 checksum offload  = Y
> +Inner L3 checksum    = Y
> +Inner L4 checksum    = Y
> +Basic stats          = Y
> +Extended stats       = Y
> +Linux UIO            = Y
> +Linux VFIO           = Y
> +x86-64               = Y

No arm support?

> +Usage doc            = Y
> diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
> index 5248625..37e6416 100644
> --- a/doc/guides/nics/index.rst
> +++ b/doc/guides/nics/index.rst
> @@ -47,6 +47,7 @@ Network Interface Controller Drivers
>      ixgbe
>      intel_vf
>      kni
> +    liquidio
>      mlx4
>      mlx5
>      nfp
> diff --git a/doc/guides/nics/liquidio.rst b/doc/guides/nics/liquidio.rst
> new file mode 100644
> index 0000000..4bf586b
> --- /dev/null
> +++ b/doc/guides/nics/liquidio.rst
> @@ -0,0 +1,269 @@
<...>
> +LiquidIO VF Poll Mode Driver
> +============================
> +
> +The LiquidIO VF PMD library(librte_pmd_lio) provides poll mode driver support for
> +Cavium LiquidIO® II server adapter VFs. PF management and VF creation can be
> +done using kernel driver.

Is it possible to provide a link here to point NIC documentation.

Re: [PATCH 50/50] doc: added documents

[Shijith Thotton]

On Thursday 23 February 2017 08:05 PM, Ferruh Yigit wrote:
> On 2/21/2017 9:27 AM, Shijith Thotton wrote:
>> Added doc/guides/nics/liquidio.rst and
>> doc/guides/nics/features/liquidio.ini. Updated release notes.
>>
>> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
>> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
>> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
>> ---
>>  doc/guides/nics/features/liquidio.ini  |  29 ++++
>>  doc/guides/nics/index.rst              |   1 +
>>  doc/guides/nics/liquidio.rst           | 269 +++++++++++++++++++++++++++++++++
>>  doc/guides/rel_notes/release_17_05.rst |   3 +
> 
> The web page that list the supported NICs [1] also needs to be updated,
> in a separate patch.
> 
> [1]
> http://dpdk.org/doc/nics
> 
>>  4 files changed, 302 insertions(+)
>>  create mode 100644 doc/guides/nics/features/liquidio.ini
>>  create mode 100644 doc/guides/nics/liquidio.rst
>>
>> diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
>> new file mode 100644
>> index 0000000..eac32ba
>> --- /dev/null
>> +++ b/doc/guides/nics/features/liquidio.ini
>> @@ -0,0 +1,29 @@
>> +;
>> +; Supported features of the 'LiquidIO' network poll mode driver.
>> +;
>> +; Refer to default.ini for the full list of available PMD features.
>> +;
>> +[Features]
>> +Link status          = Y
>> +Link status event    = Y
>> +MTU update           = Y
>> +Jumbo frame          = Y
>> +Scattered Rx         = Y
>> +Allmulticast mode    = Y
>> +RSS hash             = Y
>> +RSS key update       = Y
>> +RSS reta update      = Y
>> +SR-IOV               = Y
>> +VLAN filter          = Y
>> +CRC offload          = Y
>> +VLAN offload         = P
>> +L3 checksum offload  = Y
>> +L4 checksum offload  = Y
>> +Inner L3 checksum    = Y
>> +Inner L4 checksum    = Y
>> +Basic stats          = Y
>> +Extended stats       = Y
>> +Linux UIO            = Y
>> +Linux VFIO           = Y
>> +x86-64               = Y
> 
> No arm support?
> 
>> +Usage doc            = Y
>> diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
>> index 5248625..37e6416 100644
>> --- a/doc/guides/nics/index.rst
>> +++ b/doc/guides/nics/index.rst
>> @@ -47,6 +47,7 @@ Network Interface Controller Drivers
>>      ixgbe
>>      intel_vf
>>      kni
>> +    liquidio
>>      mlx4
>>      mlx5
>>      nfp
>> diff --git a/doc/guides/nics/liquidio.rst b/doc/guides/nics/liquidio.rst
>> new file mode 100644
>> index 0000000..4bf586b
>> --- /dev/null
>> +++ b/doc/guides/nics/liquidio.rst
>> @@ -0,0 +1,269 @@
> <...>
>> +LiquidIO VF Poll Mode Driver
>> +============================
>> +
>> +The LiquidIO VF PMD library(librte_pmd_lio) provides poll mode driver support for
>> +Cavium LiquidIO® II server adapter VFs. PF management and VF creation can be
>> +done using kernel driver.
> 
> Is it possible to provide a link here to point NIC documentation.
> 

Hi Ferruh,

Thanks a lot for the reviews. We are yet to completely verify ARM
support. Will make changes as suggested.

Shijith

Re: [PATCH 50/50] doc: added documents

[Mcnamara, John]

Hi,

Thanks for the doc. Some comments below.


> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Shijith Thotton
> Sent: Tuesday, February 21, 2017 9:27 AM
> To: dev@dpdk.org
> Cc: Jerin Jacob <jerin.jacob@caviumnetworks.com>; Derek Chickles
> <derek.chickles@caviumnetworks.com>
> Subject: [dpdk-dev] [PATCH 50/50] doc: added documents


A better title would be:

    doc: add docs for liquidio


> 
> Added doc/guides/nics/liquidio.rst and
> doc/guides/nics/features/liquidio.ini. Updated release notes.


>
> +LiquidIO VF Poll Mode Driver
> +============================
> +
> +The LiquidIO VF PMD library(librte_pmd_lio) provides poll mode driver

Missing space after library. Also it would probably be worth adding a
link to the main product page for LiquidIO at the end of this section.


John

Re: [PATCH 00/50] LiquidIO PMD

[Stephen Hemminger]

On Tue, 21 Feb 2017 14:56:15 +0530
Shijith Thotton <shijith.thotton@caviumnetworks.com> wrote:

> The patch series provides initial version of virtual function poll mode
> driver for Cavium LiquidIO II server adapters. This version adds support
> for LiquidIO II CN2350 210SV adapter.
> 
> Patch series includes driver documentation doc/guides/nics/liquidio.rst
> and list of supported features doc/guides/nics/features/liquidio.ini.
> Updated release notes to notify the addition of new PMD.
> 
> Shijith Thotton (50):
>   net/liquidio/base: hardware register definitions
>   config: liquidio PMD configuration
>   net/liquidio: added PMD version map file
>   net/liquidio: definitions for log
>   maintainers: claim responsibility for LiquidIO PMD
>   net/liquidio: liquidio VF PMD Driver registration
>   net/liquidio: added Makefile
>   net/liquidio/base: macros to read and write register
>   net/liquidio: liquidio device init
>   net/liquidio: add API to disable io queues
>   net/liquidio: add API to setup io queue registers
>   net/liquidio: add mbox APIs for PF/VF communication
>   net/liquidio: add API to setup mbox registers
>   net/liquidio: add API for VF/PF handshake
>   net/liquidio: add API for VF FLR
>   net/liquidio: add APIs to allocate and free IQ
>   net/liquidio: add API to setup instruction queue
>   net/liquidio: add API to allocate and free command pool
>   net/liquidio: add API to allocate and free soft command
>   net/liquidio: add APIs for response list
>   net/liquidio: add APIs to send packet to device
>   net/liquidio: add API to configure device
>   net/liquidio: add API to setup Rx queue
>   net/liquidio: initialize Rx queue
>   net/liquidio: add Rx data path
>   net/liquidio: add API to release Rx queue
>   net/liquidio: add API to setup Tx queue
>   net/liquidio: add APIs for sg list
>   net/liquidio: add API to enable and disable IO queues
>   net/liquidio: add Tx data path for single segment
>   net/liquidio: add Tx data path for multiple segments
>   net/liquidio: add APIs to flush IQ and free buffers
>   net/liquidio: add API to release Tx queue
>   net/liquidio: add API to start device and check link
>   net/liquidio: add API for link update
>   net/liquidio: add API to alloc and send command
>   net/liquidio: add API to control Rx
>   net/liquidio: add RSS support
>   net/liquidio: add API to get device info
>   net/liquidio: add API to set MTU
>   net/liquidio: add API to enable and disable multicast
>   net/liquidio: add API to set link up and down
>   net/liquidio: add API to configure udp tunnel port
>   net/liquidio: add support for Rx stats
>   net/liquidio: add support for Tx stats
>   net/liquidio: add APIs for hardware stats
>   net/liquidio: add API for dev stop
>   net/liquidio: add API for dev close
>   net/liquidio: add API to add and remove VLAN port
>   doc: added documents
> 
>  MAINTAINERS                                  |    7 +
>  config/common_base                           |   11 +
>  doc/guides/nics/features/liquidio.ini        |   29 +
>  doc/guides/nics/index.rst                    |    1 +
>  doc/guides/nics/liquidio.rst                 |  269 ++++
>  doc/guides/rel_notes/release_17_05.rst       |    3 +
>  drivers/net/Makefile                         |    1 +
>  drivers/net/liquidio/Makefile                |   62 +
>  drivers/net/liquidio/base/lio_23xx_reg.h     |  194 +++
>  drivers/net/liquidio/base/lio_23xx_vf.c      |  586 ++++++++
>  drivers/net/liquidio/base/lio_23xx_vf.h      |   97 ++
>  drivers/net/liquidio/base/lio_hw_defs.h      |  249 ++++
>  drivers/net/liquidio/base/lio_mbox.c         |  275 ++++
>  drivers/net/liquidio/base/lio_mbox.h         |  131 ++
>  drivers/net/liquidio/lio_ethdev.c            | 2040 ++++++++++++++++++++++++++
>  drivers/net/liquidio/lio_ethdev.h            |  204 +++
>  drivers/net/liquidio/lio_logs.h              |   91 ++
>  drivers/net/liquidio/lio_rxtx.c              | 1885 ++++++++++++++++++++++++
>  drivers/net/liquidio/lio_rxtx.h              |  769 ++++++++++
>  drivers/net/liquidio/lio_struct.h            |  689 +++++++++
>  drivers/net/liquidio/rte_pmd_lio_version.map |    4 +
>  mk/rte.app.mk                                |    1 +
>  22 files changed, 7598 insertions(+)
>  create mode 100644 doc/guides/nics/features/liquidio.ini
>  create mode 100644 doc/guides/nics/liquidio.rst
>  create mode 100644 drivers/net/liquidio/Makefile
>  create mode 100644 drivers/net/liquidio/base/lio_23xx_reg.h
>  create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.c
>  create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.h
>  create mode 100644 drivers/net/liquidio/base/lio_hw_defs.h
>  create mode 100644 drivers/net/liquidio/base/lio_mbox.c
>  create mode 100644 drivers/net/liquidio/base/lio_mbox.h
>  create mode 100644 drivers/net/liquidio/lio_ethdev.c
>  create mode 100644 drivers/net/liquidio/lio_ethdev.h
>  create mode 100644 drivers/net/liquidio/lio_logs.h
>  create mode 100644 drivers/net/liquidio/lio_rxtx.c
>  create mode 100644 drivers/net/liquidio/lio_rxtx.h
>  create mode 100644 drivers/net/liquidio/lio_struct.h
>  create mode 100644 drivers/net/liquidio/rte_pmd_lio_version.map
> 


Splitting into small patches is great.
Is this still bisectable? Does each step build?

Re: [PATCH 00/50] LiquidIO PMD

[Shijith Thotton]

On Wednesday 22 February 2017 01:52 AM, Stephen Hemminger wrote:
> On Tue, 21 Feb 2017 14:56:15 +0530
> Shijith Thotton <shijith.thotton@caviumnetworks.com> wrote:
> 
>> The patch series provides initial version of virtual function poll mode
>> driver for Cavium LiquidIO II server adapters. This version adds support
>> for LiquidIO II CN2350 210SV adapter.
>>
>> Patch series includes driver documentation doc/guides/nics/liquidio.rst
>> and list of supported features doc/guides/nics/features/liquidio.ini.
>> Updated release notes to notify the addition of new PMD.
>>
>> Shijith Thotton (50):
>>   net/liquidio/base: hardware register definitions
>>   config: liquidio PMD configuration
>>   net/liquidio: added PMD version map file
>>   net/liquidio: definitions for log
>>   maintainers: claim responsibility for LiquidIO PMD
>>   net/liquidio: liquidio VF PMD Driver registration
>>   net/liquidio: added Makefile
>>   net/liquidio/base: macros to read and write register
>>   net/liquidio: liquidio device init
>>   net/liquidio: add API to disable io queues
>>   net/liquidio: add API to setup io queue registers
>>   net/liquidio: add mbox APIs for PF/VF communication
>>   net/liquidio: add API to setup mbox registers
>>   net/liquidio: add API for VF/PF handshake
>>   net/liquidio: add API for VF FLR
>>   net/liquidio: add APIs to allocate and free IQ
>>   net/liquidio: add API to setup instruction queue
>>   net/liquidio: add API to allocate and free command pool
>>   net/liquidio: add API to allocate and free soft command
>>   net/liquidio: add APIs for response list
>>   net/liquidio: add APIs to send packet to device
>>   net/liquidio: add API to configure device
>>   net/liquidio: add API to setup Rx queue
>>   net/liquidio: initialize Rx queue
>>   net/liquidio: add Rx data path
>>   net/liquidio: add API to release Rx queue
>>   net/liquidio: add API to setup Tx queue
>>   net/liquidio: add APIs for sg list
>>   net/liquidio: add API to enable and disable IO queues
>>   net/liquidio: add Tx data path for single segment
>>   net/liquidio: add Tx data path for multiple segments
>>   net/liquidio: add APIs to flush IQ and free buffers
>>   net/liquidio: add API to release Tx queue
>>   net/liquidio: add API to start device and check link
>>   net/liquidio: add API for link update
>>   net/liquidio: add API to alloc and send command
>>   net/liquidio: add API to control Rx
>>   net/liquidio: add RSS support
>>   net/liquidio: add API to get device info
>>   net/liquidio: add API to set MTU
>>   net/liquidio: add API to enable and disable multicast
>>   net/liquidio: add API to set link up and down
>>   net/liquidio: add API to configure udp tunnel port
>>   net/liquidio: add support for Rx stats
>>   net/liquidio: add support for Tx stats
>>   net/liquidio: add APIs for hardware stats
>>   net/liquidio: add API for dev stop
>>   net/liquidio: add API for dev close
>>   net/liquidio: add API to add and remove VLAN port
>>   doc: added documents
>>
>>  MAINTAINERS                                  |    7 +
>>  config/common_base                           |   11 +
>>  doc/guides/nics/features/liquidio.ini        |   29 +
>>  doc/guides/nics/index.rst                    |    1 +
>>  doc/guides/nics/liquidio.rst                 |  269 ++++
>>  doc/guides/rel_notes/release_17_05.rst       |    3 +
>>  drivers/net/Makefile                         |    1 +
>>  drivers/net/liquidio/Makefile                |   62 +
>>  drivers/net/liquidio/base/lio_23xx_reg.h     |  194 +++
>>  drivers/net/liquidio/base/lio_23xx_vf.c      |  586 ++++++++
>>  drivers/net/liquidio/base/lio_23xx_vf.h      |   97 ++
>>  drivers/net/liquidio/base/lio_hw_defs.h      |  249 ++++
>>  drivers/net/liquidio/base/lio_mbox.c         |  275 ++++
>>  drivers/net/liquidio/base/lio_mbox.h         |  131 ++
>>  drivers/net/liquidio/lio_ethdev.c            | 2040 ++++++++++++++++++++++++++
>>  drivers/net/liquidio/lio_ethdev.h            |  204 +++
>>  drivers/net/liquidio/lio_logs.h              |   91 ++
>>  drivers/net/liquidio/lio_rxtx.c              | 1885 ++++++++++++++++++++++++
>>  drivers/net/liquidio/lio_rxtx.h              |  769 ++++++++++
>>  drivers/net/liquidio/lio_struct.h            |  689 +++++++++
>>  drivers/net/liquidio/rte_pmd_lio_version.map |    4 +
>>  mk/rte.app.mk                                |    1 +
>>  22 files changed, 7598 insertions(+)
>>  create mode 100644 doc/guides/nics/features/liquidio.ini
>>  create mode 100644 doc/guides/nics/liquidio.rst
>>  create mode 100644 drivers/net/liquidio/Makefile
>>  create mode 100644 drivers/net/liquidio/base/lio_23xx_reg.h
>>  create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.c
>>  create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.h
>>  create mode 100644 drivers/net/liquidio/base/lio_hw_defs.h
>>  create mode 100644 drivers/net/liquidio/base/lio_mbox.c
>>  create mode 100644 drivers/net/liquidio/base/lio_mbox.h
>>  create mode 100644 drivers/net/liquidio/lio_ethdev.c
>>  create mode 100644 drivers/net/liquidio/lio_ethdev.h
>>  create mode 100644 drivers/net/liquidio/lio_logs.h
>>  create mode 100644 drivers/net/liquidio/lio_rxtx.c
>>  create mode 100644 drivers/net/liquidio/lio_rxtx.h
>>  create mode 100644 drivers/net/liquidio/lio_struct.h
>>  create mode 100644 drivers/net/liquidio/rte_pmd_lio_version.map
>>
> 
> 
> Splitting into small patches is great.
> Is this still bisectable? Does each step build?
> 

Hi Stephen,

Each step can be build independently.

Re: [PATCH 00/50] LiquidIO PMD

[Ferruh Yigit]

On 2/22/2017 4:56 AM, Shijith Thotton wrote:
> 
> 
> On Wednesday 22 February 2017 01:52 AM, Stephen Hemminger wrote:
>> On Tue, 21 Feb 2017 14:56:15 +0530
>> Shijith Thotton <shijith.thotton@caviumnetworks.com> wrote:
>>
>>> The patch series provides initial version of virtual function poll mode
>>> driver for Cavium LiquidIO II server adapters. This version adds support
>>> for LiquidIO II CN2350 210SV adapter.
>>>
>>> Patch series includes driver documentation doc/guides/nics/liquidio.rst
>>> and list of supported features doc/guides/nics/features/liquidio.ini.
>>> Updated release notes to notify the addition of new PMD.
>>>
>>> Shijith Thotton (50):
>>>   net/liquidio/base: hardware register definitions
>>>   config: liquidio PMD configuration
>>>   net/liquidio: added PMD version map file
>>>   net/liquidio: definitions for log
>>>   maintainers: claim responsibility for LiquidIO PMD
>>>   net/liquidio: liquidio VF PMD Driver registration
>>>   net/liquidio: added Makefile
>>>   net/liquidio/base: macros to read and write register
>>>   net/liquidio: liquidio device init
>>>   net/liquidio: add API to disable io queues
>>>   net/liquidio: add API to setup io queue registers
>>>   net/liquidio: add mbox APIs for PF/VF communication
>>>   net/liquidio: add API to setup mbox registers
>>>   net/liquidio: add API for VF/PF handshake
>>>   net/liquidio: add API for VF FLR
>>>   net/liquidio: add APIs to allocate and free IQ
>>>   net/liquidio: add API to setup instruction queue
>>>   net/liquidio: add API to allocate and free command pool
>>>   net/liquidio: add API to allocate and free soft command
>>>   net/liquidio: add APIs for response list
>>>   net/liquidio: add APIs to send packet to device
>>>   net/liquidio: add API to configure device
>>>   net/liquidio: add API to setup Rx queue
>>>   net/liquidio: initialize Rx queue
>>>   net/liquidio: add Rx data path
>>>   net/liquidio: add API to release Rx queue
>>>   net/liquidio: add API to setup Tx queue
>>>   net/liquidio: add APIs for sg list
>>>   net/liquidio: add API to enable and disable IO queues
>>>   net/liquidio: add Tx data path for single segment
>>>   net/liquidio: add Tx data path for multiple segments
>>>   net/liquidio: add APIs to flush IQ and free buffers
>>>   net/liquidio: add API to release Tx queue
>>>   net/liquidio: add API to start device and check link
>>>   net/liquidio: add API for link update
>>>   net/liquidio: add API to alloc and send command
>>>   net/liquidio: add API to control Rx
>>>   net/liquidio: add RSS support
>>>   net/liquidio: add API to get device info
>>>   net/liquidio: add API to set MTU
>>>   net/liquidio: add API to enable and disable multicast
>>>   net/liquidio: add API to set link up and down
>>>   net/liquidio: add API to configure udp tunnel port
>>>   net/liquidio: add support for Rx stats
>>>   net/liquidio: add support for Tx stats
>>>   net/liquidio: add APIs for hardware stats
>>>   net/liquidio: add API for dev stop
>>>   net/liquidio: add API for dev close
>>>   net/liquidio: add API to add and remove VLAN port
>>>   doc: added documents
>>>
>>>  MAINTAINERS                                  |    7 +
>>>  config/common_base                           |   11 +
>>>  doc/guides/nics/features/liquidio.ini        |   29 +
>>>  doc/guides/nics/index.rst                    |    1 +
>>>  doc/guides/nics/liquidio.rst                 |  269 ++++
>>>  doc/guides/rel_notes/release_17_05.rst       |    3 +
>>>  drivers/net/Makefile                         |    1 +
>>>  drivers/net/liquidio/Makefile                |   62 +
>>>  drivers/net/liquidio/base/lio_23xx_reg.h     |  194 +++
>>>  drivers/net/liquidio/base/lio_23xx_vf.c      |  586 ++++++++
>>>  drivers/net/liquidio/base/lio_23xx_vf.h      |   97 ++
>>>  drivers/net/liquidio/base/lio_hw_defs.h      |  249 ++++
>>>  drivers/net/liquidio/base/lio_mbox.c         |  275 ++++
>>>  drivers/net/liquidio/base/lio_mbox.h         |  131 ++
>>>  drivers/net/liquidio/lio_ethdev.c            | 2040 ++++++++++++++++++++++++++
>>>  drivers/net/liquidio/lio_ethdev.h            |  204 +++
>>>  drivers/net/liquidio/lio_logs.h              |   91 ++
>>>  drivers/net/liquidio/lio_rxtx.c              | 1885 ++++++++++++++++++++++++
>>>  drivers/net/liquidio/lio_rxtx.h              |  769 ++++++++++
>>>  drivers/net/liquidio/lio_struct.h            |  689 +++++++++
>>>  drivers/net/liquidio/rte_pmd_lio_version.map |    4 +
>>>  mk/rte.app.mk                                |    1 +
>>>  22 files changed, 7598 insertions(+)
>>>  create mode 100644 doc/guides/nics/features/liquidio.ini
>>>  create mode 100644 doc/guides/nics/liquidio.rst
>>>  create mode 100644 drivers/net/liquidio/Makefile
>>>  create mode 100644 drivers/net/liquidio/base/lio_23xx_reg.h
>>>  create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.c
>>>  create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.h
>>>  create mode 100644 drivers/net/liquidio/base/lio_hw_defs.h
>>>  create mode 100644 drivers/net/liquidio/base/lio_mbox.c
>>>  create mode 100644 drivers/net/liquidio/base/lio_mbox.h
>>>  create mode 100644 drivers/net/liquidio/lio_ethdev.c
>>>  create mode 100644 drivers/net/liquidio/lio_ethdev.h
>>>  create mode 100644 drivers/net/liquidio/lio_logs.h
>>>  create mode 100644 drivers/net/liquidio/lio_rxtx.c
>>>  create mode 100644 drivers/net/liquidio/lio_rxtx.h
>>>  create mode 100644 drivers/net/liquidio/lio_struct.h
>>>  create mode 100644 drivers/net/liquidio/rte_pmd_lio_version.map
>>>
>>
>>
>> Splitting into small patches is great.
>> Is this still bisectable? Does each step build?
>>
> 
> Hi Stephen,
> 
> Each step can be build independently.

I confirm overall build and patch by patch build is fine.

[PATCH v2 00/46] LiquidIO PMD

[Shijith Thotton]

The patch series provides initial version of virtual function poll mode
driver for Cavium LiquidIO II server adapters. This version adds support
for LiquidIO II CN23XX 210SV adapters.

Patch series includes driver documentation doc/guides/nics/liquidio.rst
and list of supported features doc/guides/nics/features/liquidio.ini.
Updated release notes to notify the addition of new PMD.

v2 changes:
* Restructured patches as suggested by Ferruh.
 - http://dpdk.org/ml/archives/dev/2017-February/058186.html
* Addressed review comments on driver from Ferruh.
 - http://dpdk.org/ml/archives/dev/2017-February/058188.html
 - http://dpdk.org/ml/archives/dev/2017-February/058194.html
* Modified commit logs as suggested by Ferruh.
 - http://dpdk.org/ml/archives/dev/2017-February/058189.html
 - http://dpdk.org/ml/archives/dev/2017-February/058190.html
 - http://dpdk.org/ml/archives/dev/2017-February/058193.html
* Made changes to documentation as per review comments from John.
 - http://dpdk.org/ml/archives/dev/2017-February/058206.html
* Fixed FreeBSD build failure.
 - http://dpdk.org/ml/archives/test-report/2017-February/011272.html
* Updated driver documentation:
 - Added CN2360 under supported LiquidIO adapters.
 - Added CRC strip under limitations.

Shijith Thotton (46):
  config: add liquidio PMD skeleton
  net/liquidio/base: hardware register definitions
  net/liquidio: definitions for log
  net/liquidio: liquidio VF PMD driver registration
  net/liquidio/base: macros to read and write register
  net/liquidio: liquidio device init
  net/liquidio: add API to disable IO queues
  net/liquidio: add API to setup IO queue registers
  net/liquidio: add mbox APIs for PF VF communication
  net/liquidio: add API to setup mbox registers
  net/liquidio: add API for PF VF handshake
  net/liquidio: add API for VF FLR
  net/liquidio: add APIs to allocate and free IQ
  net/liquidio: add API to setup IQ
  net/liquidio: add APIs to allocate and free SC buffer pool
  net/liquidio: add APIs to allocate and free soft command
  net/liquidio: add APIs for response list
  net/liquidio: add API to send packet to device
  net/liquidio: add API to configure device
  net/liquidio: add API to setup Rx queue
  net/liquidio: initialize Rx queue
  net/liquidio: add Rx data path
  net/liquidio: add API to release Rx queue
  net/liquidio: add API to setup Tx queue
  net/liquidio: add APIs for SG list
  net/liquidio: add APIs to enable and disable IO queues
  net/liquidio: add Tx data path for single segment
  net/liquidio: add Tx data path for multiple segments
  net/liquidio: add API to flush IQ
  net/liquidio: add API to release Tx queue
  net/liquidio: add APIs to start device and update link
  net/liquidio: add APIs to alloc and send control command
  net/liquidio: add API to control Rx
  net/liquidio: add RSS support
  net/liquidio: add API to get device info
  net/liquidio: add API to set MTU
  net/liquidio: add APIs to enable and disable multicast
  net/liquidio: add APIs to set link up and down
  net/liquidio: add API to configure UDP tunnel port
  net/liquidio: add support for Rx stats
  net/liquidio: add support for Tx stats
  net/liquidio: add APIs for hardware stats
  net/liquidio: add API to stop device
  net/liquidio: add API to close device
  net/liquidio: add API to add and remove VLAN port
  doc: add doc for liquidio

 MAINTAINERS                                  |    7 +
 config/common_base                           |   11 +
 doc/guides/nics/features/liquidio.ini        |   29 +
 doc/guides/nics/index.rst                    |    1 +
 doc/guides/nics/liquidio.rst                 |  280 ++++
 doc/guides/rel_notes/release_17_05.rst       |    3 +
 drivers/net/Makefile                         |    1 +
 drivers/net/liquidio/Makefile                |   62 +
 drivers/net/liquidio/base/lio_23xx_reg.h     |  194 +++
 drivers/net/liquidio/base/lio_23xx_vf.c      |  586 ++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h      |   97 ++
 drivers/net/liquidio/base/lio_hw_defs.h      |  249 ++++
 drivers/net/liquidio/base/lio_mbox.c         |  275 ++++
 drivers/net/liquidio/base/lio_mbox.h         |  131 ++
 drivers/net/liquidio/lio_ethdev.c            | 2035 ++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h            |  204 +++
 drivers/net/liquidio/lio_logs.h              |   91 ++
 drivers/net/liquidio/lio_rxtx.c              | 1885 ++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h              |  769 ++++++++++
 drivers/net/liquidio/lio_struct.h            |  689 +++++++++
 drivers/net/liquidio/rte_pmd_lio_version.map |    4 +
 mk/rte.app.mk                                |    1 +
 22 files changed, 7604 insertions(+)
 create mode 100644 doc/guides/nics/features/liquidio.ini
 create mode 100644 doc/guides/nics/liquidio.rst
 create mode 100644 drivers/net/liquidio/Makefile
 create mode 100644 drivers/net/liquidio/base/lio_23xx_reg.h
 create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.c
 create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.h
 create mode 100644 drivers/net/liquidio/base/lio_hw_defs.h
 create mode 100644 drivers/net/liquidio/base/lio_mbox.c
 create mode 100644 drivers/net/liquidio/base/lio_mbox.h
 create mode 100644 drivers/net/liquidio/lio_ethdev.c
 create mode 100644 drivers/net/liquidio/lio_ethdev.h
 create mode 100644 drivers/net/liquidio/lio_logs.h
 create mode 100644 drivers/net/liquidio/lio_rxtx.c
 create mode 100644 drivers/net/liquidio/lio_rxtx.h
 create mode 100644 drivers/net/liquidio/lio_struct.h
 create mode 100644 drivers/net/liquidio/rte_pmd_lio_version.map

-- 
1.8.3.1

[PATCH v2 01/46] config: add liquidio PMD skeleton

[Shijith Thotton]

Add makefile and config file options to compile PMD. Add version map
file and update maintainers file to claim responsibility.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 MAINTAINERS                                  |  5 +++
 config/common_base                           |  5 +++
 drivers/net/Makefile                         |  1 +
 drivers/net/liquidio/Makefile                | 59 ++++++++++++++++++++++++++++
 drivers/net/liquidio/rte_pmd_lio_version.map |  4 ++
 mk/rte.app.mk                                |  1 +
 6 files changed, 75 insertions(+)
 create mode 100644 drivers/net/liquidio/Makefile
 create mode 100644 drivers/net/liquidio/rte_pmd_lio_version.map

diff --git a/MAINTAINERS b/MAINTAINERS
index 306e27a..2e16c5d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -299,6 +299,11 @@ M: Maciej Czekaj <maciej.czekaj@caviumnetworks.com>
 F: drivers/net/thunderx/
 F: doc/guides/nics/thunderx.rst
 
+Cavium LiquidIO
+M: Shijith Thotton <shijith.thotton@cavium.com>
+M: Srisivasubramanian Srinivasan <ssrinivasan@cavium.com>
+F: drivers/net/liquidio/
+
 Chelsio cxgbe
 M: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
 F: drivers/net/cxgbe/
diff --git a/config/common_base b/config/common_base
index e3ce210..23afaea 100644
--- a/config/common_base
+++ b/config/common_base
@@ -287,6 +287,11 @@ CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_DRIVER=n
 CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_MBOX=n
 
 #
+# Compile burst-oriented Cavium LiquidIO PMD driver
+#
+CONFIG_RTE_LIBRTE_LIO_PMD=y
+
+#
 # Compile burst-oriented VIRTIO PMD driver
 #
 CONFIG_RTE_LIBRTE_VIRTIO_PMD=y
diff --git a/drivers/net/Makefile b/drivers/net/Makefile
index ab60cb8..2093e09 100644
--- a/drivers/net/Makefile
+++ b/drivers/net/Makefile
@@ -41,6 +41,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_ENIC_PMD) += enic
 DIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k
 DIRS-$(CONFIG_RTE_LIBRTE_I40E_PMD) += i40e
 DIRS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) += ixgbe
+DIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += liquidio
 DIRS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += mlx4
 DIRS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5
 DIRS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD) += mpipe
diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
new file mode 100644
index 0000000..ce2b7fc
--- /dev/null
+++ b/drivers/net/liquidio/Makefile
@@ -0,0 +1,59 @@
+#
+#   BSD LICENSE
+#
+#   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Cavium, Inc. nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+LIB = librte_pmd_lio.a
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)/base -I$(SRCDIR)
+
+EXPORT_MAP := rte_pmd_lio_version.map
+
+LIBABIVER := 1
+
+VPATH += $(RTE_SDK)/drivers/net/liquidio/base
+
+#
+# all source are stored in SRCS-y
+#
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) +=
+
+# this lib depends upon:
+DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_eal lib/librte_ether
+DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_mempool lib/librte_mbuf
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/liquidio/rte_pmd_lio_version.map b/drivers/net/liquidio/rte_pmd_lio_version.map
new file mode 100644
index 0000000..8591cc0
--- /dev/null
+++ b/drivers/net/liquidio/rte_pmd_lio_version.map
@@ -0,0 +1,4 @@
+DPDK_17.05 {
+
+	local: *;
+};
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 236da9c..837cca5 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -116,6 +116,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD)      += -lrte_pmd_ixgbe
 ifeq ($(CONFIG_RTE_LIBRTE_KNI),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KNI)        += -lrte_pmd_kni
 endif
+_LDLIBS-$(CONFIG_RTE_LIBRTE_LIO_PMD)        += -lrte_pmd_lio
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -lrte_pmd_mlx4 -libverbs
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -lrte_pmd_mlx5 -libverbs
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lrte_pmd_mpipe -lgxio
-- 
1.8.3.1

[PATCH v2 02/46] net/liquidio/base: hardware register definitions

[Shijith Thotton]

Add HW register definitions for LiquidIO II CN23XX adapter.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_reg.h | 194 +++++++++++++++++++++++++++++++
 1 file changed, 194 insertions(+)
 create mode 100644 drivers/net/liquidio/base/lio_23xx_reg.h

diff --git a/drivers/net/liquidio/base/lio_23xx_reg.h b/drivers/net/liquidio/base/lio_23xx_reg.h
new file mode 100644
index 0000000..794bc2c
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_23xx_reg.h
@@ -0,0 +1,194 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_23XX_REG_H_
+#define _LIO_23XX_REG_H_
+
+/* ###################### REQUEST QUEUE ######################### */
+
+/* 64 registers for Input Queues Start Addr - SLI_PKT(0..63)_INSTR_BADDR */
+#define CN23XX_SLI_PKT_INSTR_BADDR_START64	0x10010
+
+/* 64 registers for Input Doorbell - SLI_PKT(0..63)_INSTR_BAOFF_DBELL */
+#define CN23XX_SLI_PKT_INSTR_BADDR_DBELL_START	0x10020
+
+/* 64 registers for Input Queue size - SLI_PKT(0..63)_INSTR_FIFO_RSIZE */
+#define CN23XX_SLI_PKT_INSTR_FIFO_RSIZE_START	0x10030
+
+/* 64 registers for Input Queue Instr Count - SLI_PKT_IN_DONE(0..63)_CNTS */
+#define CN23XX_SLI_PKT_IN_DONE_CNTS_START64	0x10040
+
+/* 64 registers (64-bit) - ES, RO, NS, Arbitration for Input Queue Data &
+ * gather list fetches. SLI_PKT(0..63)_INPUT_CONTROL.
+ */
+#define CN23XX_SLI_PKT_INPUT_CONTROL_START64	0x10000
+
+/* ------- Request Queue Macros --------- */
+
+/* Each Input Queue register is at a 16-byte Offset in BAR0 */
+#define CN23XX_IQ_OFFSET			0x20000
+
+#define CN23XX_SLI_IQ_PKT_CONTROL64(iq)					\
+	(CN23XX_SLI_PKT_INPUT_CONTROL_START64 + ((iq) * CN23XX_IQ_OFFSET))
+
+#define CN23XX_SLI_IQ_BASE_ADDR64(iq)					\
+	(CN23XX_SLI_PKT_INSTR_BADDR_START64 + ((iq) * CN23XX_IQ_OFFSET))
+
+#define CN23XX_SLI_IQ_SIZE(iq)						\
+	(CN23XX_SLI_PKT_INSTR_FIFO_RSIZE_START + ((iq) * CN23XX_IQ_OFFSET))
+
+#define CN23XX_SLI_IQ_DOORBELL(iq)					\
+	(CN23XX_SLI_PKT_INSTR_BADDR_DBELL_START + ((iq) * CN23XX_IQ_OFFSET))
+
+#define CN23XX_SLI_IQ_INSTR_COUNT64(iq)					\
+	(CN23XX_SLI_PKT_IN_DONE_CNTS_START64 + ((iq) * CN23XX_IQ_OFFSET))
+
+/* Number of instructions to be read in one MAC read request.
+ * setting to Max value(4)
+ */
+#define CN23XX_PKT_INPUT_CTL_RDSIZE			(3 << 25)
+#define CN23XX_PKT_INPUT_CTL_IS_64B			(1 << 24)
+#define CN23XX_PKT_INPUT_CTL_RST			(1 << 23)
+#define CN23XX_PKT_INPUT_CTL_QUIET			(1 << 28)
+#define CN23XX_PKT_INPUT_CTL_RING_ENB			(1 << 22)
+#define CN23XX_PKT_INPUT_CTL_DATA_ES_64B_SWAP		(1 << 6)
+#define CN23XX_PKT_INPUT_CTL_USE_CSR			(1 << 4)
+#define CN23XX_PKT_INPUT_CTL_GATHER_ES_64B_SWAP		(2)
+
+/* These bits[47:44] select the Physical function number within the MAC */
+#define CN23XX_PKT_INPUT_CTL_PF_NUM_POS		45
+/* These bits[43:32] select the function number within the PF */
+#define CN23XX_PKT_INPUT_CTL_VF_NUM_POS		32
+
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+#define CN23XX_PKT_INPUT_CTL_MASK			\
+	(CN23XX_PKT_INPUT_CTL_RDSIZE |			\
+	 CN23XX_PKT_INPUT_CTL_DATA_ES_64B_SWAP |	\
+	 CN23XX_PKT_INPUT_CTL_USE_CSR)
+#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+#define CN23XX_PKT_INPUT_CTL_MASK			\
+	(CN23XX_PKT_INPUT_CTL_RDSIZE |			\
+	 CN23XX_PKT_INPUT_CTL_DATA_ES_64B_SWAP |	\
+	 CN23XX_PKT_INPUT_CTL_USE_CSR |			\
+	 CN23XX_PKT_INPUT_CTL_GATHER_ES_64B_SWAP)
+#endif
+
+/* ############################ OUTPUT QUEUE ######################### */
+
+/* 64 registers for Output queue control - SLI_PKT(0..63)_OUTPUT_CONTROL */
+#define CN23XX_SLI_PKT_OUTPUT_CONTROL_START	0x10050
+
+/* 64 registers for Output queue buffer and info size
+ * SLI_PKT(0..63)_OUT_SIZE
+ */
+#define CN23XX_SLI_PKT_OUT_SIZE			0x10060
+
+/* 64 registers for Output Queue Start Addr - SLI_PKT(0..63)_SLIST_BADDR */
+#define CN23XX_SLI_SLIST_BADDR_START64		0x10070
+
+/* 64 registers for Output Queue Packet Credits
+ * SLI_PKT(0..63)_SLIST_BAOFF_DBELL
+ */
+#define CN23XX_SLI_PKT_SLIST_BAOFF_DBELL_START	0x10080
+
+/* 64 registers for Output Queue size - SLI_PKT(0..63)_SLIST_FIFO_RSIZE */
+#define CN23XX_SLI_PKT_SLIST_FIFO_RSIZE_START	0x10090
+
+/* 64 registers for Output Queue Packet Count - SLI_PKT(0..63)_CNTS */
+#define CN23XX_SLI_PKT_CNTS_START		0x100B0
+
+/* Each Output Queue register is at a 16-byte Offset in BAR0 */
+#define CN23XX_OQ_OFFSET			0x20000
+
+/* ------- Output Queue Macros --------- */
+
+#define CN23XX_SLI_OQ_PKT_CONTROL(oq)					\
+	(CN23XX_SLI_PKT_OUTPUT_CONTROL_START + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_BASE_ADDR64(oq)					\
+	(CN23XX_SLI_SLIST_BADDR_START64 + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_SIZE(oq)						\
+	(CN23XX_SLI_PKT_SLIST_FIFO_RSIZE_START + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_BUFF_INFO_SIZE(oq)				\
+	(CN23XX_SLI_PKT_OUT_SIZE + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_PKTS_SENT(oq)					\
+	(CN23XX_SLI_PKT_CNTS_START + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_PKTS_CREDIT(oq)					\
+	(CN23XX_SLI_PKT_SLIST_BAOFF_DBELL_START + ((oq) * CN23XX_OQ_OFFSET))
+
+/* ------------------ Masks ---------------- */
+#define CN23XX_PKT_OUTPUT_CTL_IPTR		(1 << 11)
+#define CN23XX_PKT_OUTPUT_CTL_ES		(1 << 9)
+#define CN23XX_PKT_OUTPUT_CTL_NSR		(1 << 8)
+#define CN23XX_PKT_OUTPUT_CTL_ROR		(1 << 7)
+#define CN23XX_PKT_OUTPUT_CTL_DPTR		(1 << 6)
+#define CN23XX_PKT_OUTPUT_CTL_BMODE		(1 << 5)
+#define CN23XX_PKT_OUTPUT_CTL_ES_P		(1 << 3)
+#define CN23XX_PKT_OUTPUT_CTL_NSR_P		(1 << 2)
+#define CN23XX_PKT_OUTPUT_CTL_ROR_P		(1 << 1)
+#define CN23XX_PKT_OUTPUT_CTL_RING_ENB		(1 << 0)
+
+/* Rings per Virtual Function [RO] */
+#define CN23XX_PKT_INPUT_CTL_RPVF_MASK		0x3F
+#define CN23XX_PKT_INPUT_CTL_RPVF_POS		48
+
+/* These bits[47:44][RO] give the Physical function
+ * number info within the MAC
+ */
+#define CN23XX_PKT_INPUT_CTL_PF_NUM_MASK	0x7
+
+/* These bits[43:32][RO] give the virtual function
+ * number info within the PF
+ */
+#define CN23XX_PKT_INPUT_CTL_VF_NUM_MASK	0x1FFF
+
+/* ######################### Mailbox Reg Macros ######################## */
+#define CN23XX_SLI_PKT_PF_VF_MBOX_SIG_START	0x10200
+#define CN23XX_VF_SLI_PKT_MBOX_INT_START	0x10210
+
+#define CN23XX_SLI_MBOX_OFFSET			0x20000
+#define CN23XX_SLI_MBOX_SIG_IDX_OFFSET		0x8
+
+#define CN23XX_SLI_PKT_PF_VF_MBOX_SIG(q, idx)				\
+	(CN23XX_SLI_PKT_PF_VF_MBOX_SIG_START +				\
+	 ((q) * CN23XX_SLI_MBOX_OFFSET +				\
+	  (idx) * CN23XX_SLI_MBOX_SIG_IDX_OFFSET))
+
+#define CN23XX_VF_SLI_PKT_MBOX_INT(q)					\
+	(CN23XX_VF_SLI_PKT_MBOX_INT_START + ((q) * CN23XX_SLI_MBOX_OFFSET))
+
+#endif /* _LIO_23XX_REG_H_ */
-- 
1.8.3.1

[PATCH v2 03/46] net/liquidio: definitions for log

[Shijith Thotton]

Add debug options to config file. Define macros used for log and make
use of config file options to enable them.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 config/common_base              |  6 +++
 drivers/net/liquidio/lio_logs.h | 91 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 97 insertions(+)
 create mode 100644 drivers/net/liquidio/lio_logs.h

diff --git a/config/common_base b/config/common_base
index 23afaea..109dd4d 100644
--- a/config/common_base
+++ b/config/common_base
@@ -290,6 +290,12 @@ CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_MBOX=n
 # Compile burst-oriented Cavium LiquidIO PMD driver
 #
 CONFIG_RTE_LIBRTE_LIO_PMD=y
+CONFIG_RTE_LIBRTE_LIO_DEBUG_DRIVER=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_INIT=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_RX=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_TX=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_MBOX=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_REGS=n
 
 #
 # Compile burst-oriented VIRTIO PMD driver
diff --git a/drivers/net/liquidio/lio_logs.h b/drivers/net/liquidio/lio_logs.h
new file mode 100644
index 0000000..a4c9ca4
--- /dev/null
+++ b/drivers/net/liquidio/lio_logs.h
@@ -0,0 +1,91 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_LOGS_H_
+#define _LIO_LOGS_H_
+
+#define lio_dev_printf(lio_dev, level, fmt, args...)			\
+	RTE_LOG(level, PMD, "%s" fmt, (lio_dev)->dev_string, ##args)
+
+#define lio_dev_info(lio_dev, fmt, args...)				\
+	lio_dev_printf(lio_dev, INFO, "INFO: " fmt, ##args)
+
+#define lio_dev_err(lio_dev, fmt, args...)				\
+	lio_dev_printf(lio_dev, ERR, "ERROR: %s() " fmt, __func__, ##args)
+
+#define PMD_INIT_LOG(level, fmt, args...) RTE_LOG(level, PMD, fmt, ## args)
+
+/* Enable these through config options */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_INIT
+#define PMD_INIT_FUNC_TRACE() PMD_INIT_LOG(DEBUG, "%s() >>\n", __func__)
+#else /* !RTE_LIBRTE_LIO_DEBUG_INIT */
+#define PMD_INIT_FUNC_TRACE() do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_INIT */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_DRIVER
+#define lio_dev_dbg(lio_dev, fmt, args...)				\
+	lio_dev_printf(lio_dev, DEBUG, "DEBUG: %s() " fmt, __func__, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_DRIVER */
+#define lio_dev_dbg(lio_dev, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_DRIVER */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_RX
+#define PMD_RX_LOG(lio_dev, level, fmt, args...)			\
+	lio_dev_printf(lio_dev, level, "RX: %s() " fmt, __func__, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_RX */
+#define PMD_RX_LOG(lio_dev, level, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_RX */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_TX
+#define PMD_TX_LOG(lio_dev, level, fmt, args...)			\
+	lio_dev_printf(lio_dev, level, "TX: %s() " fmt, __func__, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_TX */
+#define PMD_TX_LOG(lio_dev, level, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_TX */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_MBOX
+#define PMD_MBOX_LOG(lio_dev, level, fmt, args...)			\
+	lio_dev_printf(lio_dev, level, "MBOX: %s() " fmt, __func__, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_MBOX */
+#define PMD_MBOX_LOG(level, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_MBOX */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_REGS
+#define PMD_REGS_LOG(lio_dev, fmt, args...)				\
+	lio_dev_printf(lio_dev, DEBUG, "REGS: " fmt, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_REGS */
+#define PMD_REGS_LOG(level, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_REGS */
+
+#endif  /* _LIO_LOGS_H_ */
-- 
1.8.3.1

[PATCH v2 04/46] net/liquidio: liquidio VF PMD driver registration

[Shijith Thotton]

Register LiquidIO PMD (net_liovf) and define APIs to init and uninit.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/Makefile           |   2 +-
 drivers/net/liquidio/base/lio_hw_defs.h |  44 ++++++++++++
 drivers/net/liquidio/lio_ethdev.c       | 117 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  40 +++++++++++
 drivers/net/liquidio/lio_struct.h       |  65 ++++++++++++++++++
 5 files changed, 267 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/liquidio/base/lio_hw_defs.h
 create mode 100644 drivers/net/liquidio/lio_ethdev.c
 create mode 100644 drivers/net/liquidio/lio_ethdev.h
 create mode 100644 drivers/net/liquidio/lio_struct.h

diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
index ce2b7fc..25685a7 100644
--- a/drivers/net/liquidio/Makefile
+++ b/drivers/net/liquidio/Makefile
@@ -50,7 +50,7 @@ VPATH += $(RTE_SDK)/drivers/net/liquidio/base
 #
 # all source are stored in SRCS-y
 #
-SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) +=
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_ethdev.c
 
 # this lib depends upon:
 DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_eal lib/librte_ether
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
new file mode 100644
index 0000000..db42f3e
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -0,0 +1,44 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_HW_DEFS_H_
+#define _LIO_HW_DEFS_H_
+
+#ifndef PCI_VENDOR_ID_CAVIUM
+#define PCI_VENDOR_ID_CAVIUM	0x177D
+#endif
+
+#define LIO_CN23XX_VF_VID	0x9712
+
+#define LIO_DEVICE_NAME_LEN		32
+#endif /* _LIO_HW_DEFS_H_ */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
new file mode 100644
index 0000000..49efede
--- /dev/null
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -0,0 +1,117 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+#include <rte_alarm.h>
+
+#include "lio_logs.h"
+#include "lio_struct.h"
+#include "lio_ethdev.h"
+
+static int
+lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
+{
+	PMD_INIT_FUNC_TRACE();
+
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return -EPERM;
+
+	rte_free(eth_dev->data->mac_addrs);
+	eth_dev->data->mac_addrs = NULL;
+
+	return 0;
+}
+
+static int
+lio_eth_dev_init(struct rte_eth_dev *eth_dev)
+{
+	struct rte_pci_device *pdev = RTE_DEV_TO_PCI(eth_dev->device);
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* Primary does the initialization. */
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	rte_eth_copy_pci_info(eth_dev, pdev);
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_DETACHABLE;
+
+	if (pdev->mem_resource[0].addr) {
+		lio_dev->hw_addr = pdev->mem_resource[0].addr;
+	} else {
+		PMD_INIT_LOG(ERR, "ERROR: Failed to map BAR0\n");
+		return -ENODEV;
+	}
+
+	lio_dev->eth_dev = eth_dev;
+	/* set lio device print string */
+	snprintf(lio_dev->dev_string, sizeof(lio_dev->dev_string),
+		 "%s[%02x:%02x.%x]", pdev->driver->driver.name,
+		 pdev->addr.bus, pdev->addr.devid, pdev->addr.function);
+
+	lio_dev->port_id = eth_dev->data->port_id;
+
+	eth_dev->data->mac_addrs = rte_zmalloc("lio", ETHER_ADDR_LEN, 0);
+	if (eth_dev->data->mac_addrs == NULL) {
+		lio_dev_err(lio_dev,
+			    "MAC addresses memory allocation failed\n");
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+/* Set of PCI devices this driver supports */
+static const struct rte_pci_id pci_id_liovf_map[] = {
+	{ RTE_PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, LIO_CN23XX_VF_VID) },
+	{ .vendor_id = 0, /* sentinel */ }
+};
+
+static struct eth_driver rte_liovf_pmd = {
+	.pci_drv = {
+		.id_table	= pci_id_liovf_map,
+		.drv_flags      = RTE_PCI_DRV_NEED_MAPPING,
+		.probe		= rte_eth_dev_pci_probe,
+		.remove		= rte_eth_dev_pci_remove,
+	},
+	.eth_dev_init		= lio_eth_dev_init,
+	.eth_dev_uninit		= lio_eth_dev_uninit,
+	.dev_private_size	= sizeof(struct lio_device),
+};
+
+RTE_PMD_REGISTER_PCI(net_liovf, rte_liovf_pmd.pci_drv);
+RTE_PMD_REGISTER_PCI_TABLE(net_liovf, pci_id_liovf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_liovf, "* igb_uio | vfio");
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
new file mode 100644
index 0000000..76c9072
--- /dev/null
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -0,0 +1,40 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_ETHDEV_H_
+#define _LIO_ETHDEV_H_
+
+#include <stdint.h>
+
+#define LIO_DEV(_eth_dev)		((_eth_dev)->data->dev_private)
+#endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
new file mode 100644
index 0000000..dcf99ce
--- /dev/null
+++ b/drivers/net/liquidio/lio_struct.h
@@ -0,0 +1,65 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_STRUCT_H_
+#define _LIO_STRUCT_H_
+
+#include <stdio.h>
+#include <stdint.h>
+#include <sys/queue.h>
+
+#include <rte_spinlock.h>
+#include <rte_atomic.h>
+
+#include "lio_hw_defs.h"
+
+/* -----------------------  THE LIO DEVICE  --------------------------- */
+/** The lio device.
+ *  Each lio device has this structure to represent all its
+ *  components.
+ */
+struct lio_device {
+	uint8_t *hw_addr;
+
+	char dev_string[LIO_DEVICE_NAME_LEN]; /* Device print string */
+
+	struct rte_eth_dev      *eth_dev;
+
+	uint8_t max_rx_queues;
+	uint8_t max_tx_queues;
+	uint8_t nb_rx_queues;
+	uint8_t nb_tx_queues;
+	uint8_t port_configured;
+	uint8_t port_id;
+};
+#endif /* _LIO_STRUCT_H_ */
-- 
1.8.3.1

[PATCH v2 05/46] net/liquidio/base: macros to read and write register

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h | 67 +++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index db42f3e..9a7a894 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -34,6 +34,8 @@
 #ifndef _LIO_HW_DEFS_H_
 #define _LIO_HW_DEFS_H_
 
+#include <rte_io.h>
+
 #ifndef PCI_VENDOR_ID_CAVIUM
 #define PCI_VENDOR_ID_CAVIUM	0x177D
 #endif
@@ -41,4 +43,69 @@
 #define LIO_CN23XX_VF_VID	0x9712
 
 #define LIO_DEVICE_NAME_LEN		32
+
+/* Routines for reading and writing CSRs */
+#ifdef RTE_LIBRTE_LIO_DEBUG_REGS
+#define lio_write_csr(lio_dev, reg_off, value)				\
+	do {								\
+		typeof(lio_dev) _dev = lio_dev;				\
+		typeof(reg_off) _reg_off = reg_off;			\
+		typeof(value) _value = value;				\
+		PMD_REGS_LOG(_dev,					\
+			     "Write32: Reg: 0x%08lx Val: 0x%08lx\n",	\
+			     (unsigned long)_reg_off,			\
+			     (unsigned long)_value);			\
+		rte_write32(_value, _dev->hw_addr + _reg_off);		\
+	} while (0)
+
+#define lio_write_csr64(lio_dev, reg_off, val64)			\
+	do {								\
+		typeof(lio_dev) _dev = lio_dev;				\
+		typeof(reg_off) _reg_off = reg_off;			\
+		typeof(val64) _val64 = val64;				\
+		PMD_REGS_LOG(						\
+		    _dev,						\
+		    "Write64: Reg: 0x%08lx Val: 0x%016llx\n",		\
+		    (unsigned long)_reg_off,				\
+		    (unsigned long long)_val64);			\
+		rte_write64(_val64, _dev->hw_addr + _reg_off);		\
+	} while (0)
+
+#define lio_read_csr(lio_dev, reg_off)					\
+	({								\
+		typeof(lio_dev) _dev = lio_dev;				\
+		typeof(reg_off) _reg_off = reg_off;			\
+		uint32_t val = rte_read32(_dev->hw_addr + _reg_off);	\
+		PMD_REGS_LOG(_dev,					\
+			     "Read32: Reg: 0x%08lx Val: 0x%08lx\n",	\
+			     (unsigned long)_reg_off,			\
+			     (unsigned long)val);			\
+		val;							\
+	})
+
+#define lio_read_csr64(lio_dev, reg_off)				\
+	({								\
+		typeof(lio_dev) _dev = lio_dev;				\
+		typeof(reg_off) _reg_off = reg_off;			\
+		uint64_t val64 = rte_read64(_dev->hw_addr + _reg_off);	\
+		PMD_REGS_LOG(						\
+		    _dev,						\
+		    "Read64: Reg: 0x%08lx Val: 0x%016llx\n",		\
+		    (unsigned long)_reg_off,				\
+		    (unsigned long long)val64);				\
+		val64;							\
+	})
+#else
+#define lio_write_csr(lio_dev, reg_off, value)				\
+	rte_write32(value, (lio_dev)->hw_addr + (reg_off))
+
+#define lio_write_csr64(lio_dev, reg_off, val64)			\
+	rte_write64(val64, (lio_dev)->hw_addr + (reg_off))
+
+#define lio_read_csr(lio_dev, reg_off)					\
+	rte_read32((lio_dev)->hw_addr + (reg_off))
+
+#define lio_read_csr64(lio_dev, reg_off)				\
+	rte_read64((lio_dev)->hw_addr + (reg_off))
+#endif
 #endif /* _LIO_HW_DEFS_H_ */
-- 
1.8.3.1

[PATCH v2 06/46] net/liquidio: liquidio device init

[Shijith Thotton]

Default device configuration and initialization code.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/Makefile           |  1 +
 drivers/net/liquidio/base/lio_23xx_vf.c | 67 ++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h | 84 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_hw_defs.h | 34 +++++++++++++
 drivers/net/liquidio/lio_ethdev.c       | 61 +++++++++++++++++++++++-
 drivers/net/liquidio/lio_struct.h       | 70 +++++++++++++++++++++++++++
 6 files changed, 316 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.c
 create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.h

diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
index 25685a7..8880a10 100644
--- a/drivers/net/liquidio/Makefile
+++ b/drivers/net/liquidio/Makefile
@@ -51,6 +51,7 @@ VPATH += $(RTE_SDK)/drivers/net/liquidio/base
 # all source are stored in SRCS-y
 #
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_ethdev.c
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_23xx_vf.c
 
 # this lib depends upon:
 DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_eal lib/librte_ether
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
new file mode 100644
index 0000000..dd5e3a6
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -0,0 +1,67 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+
+#include "lio_logs.h"
+#include "lio_23xx_vf.h"
+#include "lio_23xx_reg.h"
+
+int
+cn23xx_vf_setup_device(struct lio_device *lio_dev)
+{
+	uint64_t reg_val;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* INPUT_CONTROL[RPVF] gives the VF IOq count */
+	reg_val = lio_read_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(0));
+
+	lio_dev->pf_num = (reg_val >> CN23XX_PKT_INPUT_CTL_PF_NUM_POS) &
+				CN23XX_PKT_INPUT_CTL_PF_NUM_MASK;
+	lio_dev->vf_num = (reg_val >> CN23XX_PKT_INPUT_CTL_VF_NUM_POS) &
+				CN23XX_PKT_INPUT_CTL_VF_NUM_MASK;
+
+	reg_val = reg_val >> CN23XX_PKT_INPUT_CTL_RPVF_POS;
+
+	lio_dev->sriov_info.rings_per_vf =
+				reg_val & CN23XX_PKT_INPUT_CTL_RPVF_MASK;
+
+	lio_dev->default_config = lio_get_conf(lio_dev);
+	if (lio_dev->default_config == NULL)
+		return -1;
+
+	return 0;
+}
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.h b/drivers/net/liquidio/base/lio_23xx_vf.h
new file mode 100644
index 0000000..1c234bf
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_23xx_vf.h
@@ -0,0 +1,84 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_23XX_VF_H_
+#define _LIO_23XX_VF_H_
+
+#include <stdio.h>
+
+#include "lio_struct.h"
+
+static const struct lio_config default_cn23xx_conf	= {
+	.card_type				= LIO_23XX,
+	.card_name				= LIO_23XX_NAME,
+	/** IQ attributes */
+	.iq					= {
+		.max_iqs			= CN23XX_CFG_IO_QUEUES,
+		.pending_list_size		=
+			(CN23XX_MAX_IQ_DESCRIPTORS * CN23XX_CFG_IO_QUEUES),
+		.instr_type			= OCTEON_64BYTE_INSTR,
+	},
+
+	/** OQ attributes */
+	.oq					= {
+		.max_oqs			= CN23XX_CFG_IO_QUEUES,
+		.info_ptr			= OCTEON_OQ_INFOPTR_MODE,
+		.refill_threshold		= CN23XX_OQ_REFIL_THRESHOLD,
+	},
+
+	.num_nic_ports				= CN23XX_DEFAULT_NUM_PORTS,
+	.num_def_rx_descs			= CN23XX_MAX_OQ_DESCRIPTORS,
+	.num_def_tx_descs			= CN23XX_MAX_IQ_DESCRIPTORS,
+	.def_rx_buf_size			= CN23XX_OQ_BUF_SIZE,
+};
+
+static inline const struct lio_config *
+lio_get_conf(struct lio_device *lio_dev)
+{
+	const struct lio_config *default_lio_conf = NULL;
+
+	/* check the LIO Device model & return the corresponding lio
+	 * configuration
+	 */
+	default_lio_conf = &default_cn23xx_conf;
+
+	if (default_lio_conf == NULL) {
+		lio_dev_err(lio_dev, "Configuration verification failed\n");
+		return NULL;
+	}
+
+	return default_lio_conf;
+}
+
+int cn23xx_vf_setup_device(struct lio_device  *lio_dev);
+#endif /* _LIO_23XX_VF_H_  */
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 9a7a894..a2654cd 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -42,6 +42,40 @@
 
 #define LIO_CN23XX_VF_VID	0x9712
 
+/* --------------------------CONFIG VALUES------------------------ */
+
+/* CN23xx IQ configuration macros */
+#define CN23XX_MAX_RINGS_PER_PF			64
+#define CN23XX_MAX_RINGS_PER_VF			8
+
+#define CN23XX_MAX_INPUT_QUEUES			CN23XX_MAX_RINGS_PER_PF
+#define CN23XX_MAX_IQ_DESCRIPTORS		512
+#define CN23XX_MIN_IQ_DESCRIPTORS		128
+
+#define CN23XX_MAX_OUTPUT_QUEUES		CN23XX_MAX_RINGS_PER_PF
+#define CN23XX_MAX_OQ_DESCRIPTORS		512
+#define CN23XX_MIN_OQ_DESCRIPTORS		128
+#define CN23XX_OQ_BUF_SIZE			1536
+
+#define CN23XX_OQ_REFIL_THRESHOLD		16
+
+#define CN23XX_DEFAULT_NUM_PORTS		1
+
+#define CN23XX_CFG_IO_QUEUES			CN23XX_MAX_RINGS_PER_PF
+
+/* common OCTEON configuration macros */
+#define OCTEON_64BYTE_INSTR			64
+#define OCTEON_OQ_INFOPTR_MODE			1
+
+/* Max IOQs per LIO Link */
+#define LIO_MAX_IOQS_PER_IF			64
+
+enum lio_card_type {
+	LIO_23XX /* 23xx */
+};
+
+#define LIO_23XX_NAME "23xx"
+
 #define LIO_DEVICE_NAME_LEN		32
 
 /* Routines for reading and writing CSRs */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 49efede..734f6c4 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -37,9 +37,63 @@
 #include <rte_alarm.h>
 
 #include "lio_logs.h"
-#include "lio_struct.h"
+#include "lio_23xx_vf.h"
 #include "lio_ethdev.h"
 
+/**
+ * \brief Identify the LIO device and to map the BAR address space
+ * @param lio_dev lio device
+ */
+static int
+lio_chip_specific_setup(struct lio_device *lio_dev)
+{
+	struct rte_pci_device *pdev = lio_dev->pci_dev;
+	uint32_t dev_id = pdev->id.device_id;
+	const char *s;
+	int ret = 1;
+
+	switch (dev_id) {
+	case LIO_CN23XX_VF_VID:
+		lio_dev->chip_id = LIO_CN23XX_VF_VID;
+		ret = cn23xx_vf_setup_device(lio_dev);
+		s = "CN23XX VF";
+		break;
+	default:
+		s = "?";
+		lio_dev_err(lio_dev, "Unsupported Chip\n");
+	}
+
+	if (!ret)
+		lio_dev_info(lio_dev, "DEVICE : %s\n", s);
+
+	return ret;
+}
+
+static int
+lio_first_time_init(struct lio_device *lio_dev,
+		    struct rte_pci_device *pdev)
+{
+	int dpdk_queues;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* set dpdk specific pci device pointer */
+	lio_dev->pci_dev = pdev;
+
+	/* Identify the LIO type and set device ops */
+	if (lio_chip_specific_setup(lio_dev)) {
+		lio_dev_err(lio_dev, "Chip specific setup failed\n");
+		return -1;
+	}
+
+	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
+
+	lio_dev->max_tx_queues = dpdk_queues;
+	lio_dev->max_rx_queues = dpdk_queues;
+
+	return 0;
+}
+
 static int
 lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
 {
@@ -84,6 +138,11 @@
 
 	lio_dev->port_id = eth_dev->data->port_id;
 
+	if (lio_first_time_init(lio_dev, pdev)) {
+		lio_dev_err(lio_dev, "Device init failed\n");
+		return -EINVAL;
+	}
+
 	eth_dev->data->mac_addrs = rte_zmalloc("lio", ETHER_ADDR_LEN, 0);
 	if (eth_dev->data->mac_addrs == NULL) {
 		lio_dev_err(lio_dev,
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index dcf99ce..a1203e4 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -43,16 +43,86 @@
 
 #include "lio_hw_defs.h"
 
+struct lio_sriov_info {
+	/** Number of rings assigned to VF */
+	uint32_t rings_per_vf;
+
+	/** Number of VF devices enabled */
+	uint32_t num_vfs;
+};
+
+/* Structure to define the configuration attributes for each Input queue. */
+struct lio_iq_config {
+	/* Max number of IQs available */
+	uint8_t max_iqs;
+
+	/** Pending list size (usually set to the sum of the size of all Input
+	 *  queues)
+	 */
+	uint32_t pending_list_size;
+
+	/** Command size - 32 or 64 bytes */
+	uint32_t instr_type;
+};
+
+/* Structure to define the configuration attributes for each Output queue. */
+struct lio_oq_config {
+	/* Max number of OQs available */
+	uint8_t max_oqs;
+
+	/** If set, the Output queue uses info-pointer mode. (Default: 1 ) */
+	uint32_t info_ptr;
+
+	/** The number of buffers that were consumed during packet processing by
+	 *  the driver on this Output queue before the driver attempts to
+	 *  replenish the descriptor ring with new buffers.
+	 */
+	uint32_t refill_threshold;
+};
+
+/* Structure to define the configuration. */
+struct lio_config {
+	uint16_t card_type;
+	const char *card_name;
+
+	/** Input Queue attributes. */
+	struct lio_iq_config iq;
+
+	/** Output Queue attributes. */
+	struct lio_oq_config oq;
+
+	int num_nic_ports;
+
+	int num_def_tx_descs;
+
+	/* Num of desc for rx rings */
+	int num_def_rx_descs;
+
+	int def_rx_buf_size;
+};
+
 /* -----------------------  THE LIO DEVICE  --------------------------- */
 /** The lio device.
  *  Each lio device has this structure to represent all its
  *  components.
  */
 struct lio_device {
+	/** PCI device pointer */
+	struct rte_pci_device *pci_dev;
+
+	/** Octeon Chip type */
+	uint16_t chip_id;
+	uint16_t pf_num;
+	uint16_t vf_num;
+
 	uint8_t *hw_addr;
 
+	struct lio_sriov_info sriov_info;
+
 	char dev_string[LIO_DEVICE_NAME_LEN]; /* Device print string */
 
+	const struct lio_config *default_config;
+
 	struct rte_eth_dev      *eth_dev;
 
 	uint8_t max_rx_queues;
-- 
1.8.3.1

[PATCH v2 07/46] net/liquidio: add API to disable IO queues

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 49 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h |  7 +++++
 drivers/net/liquidio/lio_ethdev.c       |  5 ++++
 3 files changed, 61 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index dd5e3a6..d9b9e2a 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -65,3 +65,52 @@
 
 	return 0;
 }
+
+int
+cn23xx_vf_set_io_queues_off(struct lio_device *lio_dev)
+{
+	uint32_t loop = CN23XX_VF_BUSY_READING_REG_LOOP_COUNT;
+	uint64_t q_no;
+
+	/* Disable the i/p and o/p queues for this Octeon.
+	 * IOQs will already be in reset.
+	 * If RST bit is set, wait for Quiet bit to be set
+	 * Once Quiet bit is set, clear the RST bit
+	 */
+	PMD_INIT_FUNC_TRACE();
+
+	for (q_no = 0; q_no < lio_dev->sriov_info.rings_per_vf; q_no++) {
+		volatile uint64_t reg_val;
+
+		reg_val = lio_read_csr64(lio_dev,
+					 CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		while ((reg_val & CN23XX_PKT_INPUT_CTL_RST) && !(reg_val &
+					 CN23XX_PKT_INPUT_CTL_QUIET) && loop) {
+			reg_val = lio_read_csr64(
+					lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+			loop = loop - 1;
+		}
+
+		if (loop == 0) {
+			lio_dev_err(lio_dev,
+				    "clearing the reset reg failed or setting the quiet reg failed for qno %lu\n",
+				    (unsigned long)q_no);
+			return -1;
+		}
+
+		reg_val = reg_val & ~CN23XX_PKT_INPUT_CTL_RST;
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+				reg_val);
+
+		reg_val = lio_read_csr64(lio_dev,
+					 CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		if (reg_val & CN23XX_PKT_INPUT_CTL_RST) {
+			lio_dev_err(lio_dev, "unable to reset qno %lu\n",
+				    (unsigned long)q_no);
+			return -1;
+		}
+	}
+
+	return 0;
+}
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.h b/drivers/net/liquidio/base/lio_23xx_vf.h
index 1c234bf..1af09d0 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.h
+++ b/drivers/net/liquidio/base/lio_23xx_vf.h
@@ -80,5 +80,12 @@
 	return default_lio_conf;
 }
 
+/** Turns off the input and output queues for the device
+ *  @param lio_dev which device io queues to disable
+ */
+int cn23xx_vf_set_io_queues_off(struct lio_device *lio_dev);
+
+#define CN23XX_VF_BUSY_READING_REG_LOOP_COUNT	100000
+
 int cn23xx_vf_setup_device(struct lio_device  *lio_dev);
 #endif /* _LIO_23XX_VF_H_  */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 734f6c4..0487f3d 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -86,6 +86,11 @@
 		return -1;
 	}
 
+	if (cn23xx_vf_set_io_queues_off(lio_dev)) {
+		lio_dev_err(lio_dev, "Setting io queues off failed\n");
+		return -1;
+	}
+
 	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
 
 	lio_dev->max_tx_queues = dpdk_queues;
-- 
1.8.3.1

[PATCH v2 08/46] net/liquidio: add API to setup IO queue registers

[Shijith Thotton]

Set default configuration values for input and output queue registers.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 160 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.c       |   5 +
 drivers/net/liquidio/lio_struct.h       |   7 ++
 3 files changed, 172 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index d9b9e2a..f61f185 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -39,6 +39,164 @@
 #include "lio_23xx_vf.h"
 #include "lio_23xx_reg.h"
 
+static int
+cn23xx_vf_reset_io_queues(struct lio_device *lio_dev, uint32_t num_queues)
+{
+	uint32_t loop = CN23XX_VF_BUSY_READING_REG_LOOP_COUNT;
+	uint64_t d64, q_no;
+	int ret_val = 0;
+
+	PMD_INIT_FUNC_TRACE();
+
+	for (q_no = 0; q_no < num_queues; q_no++) {
+		/* set RST bit to 1. This bit applies to both IQ and OQ */
+		d64 = lio_read_csr64(lio_dev,
+				     CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		d64 = d64 | CN23XX_PKT_INPUT_CTL_RST;
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+				d64);
+	}
+
+	/* wait until the RST bit is clear or the RST and QUIET bits are set */
+	for (q_no = 0; q_no < num_queues; q_no++) {
+		volatile uint64_t reg_val;
+
+		reg_val	= lio_read_csr64(lio_dev,
+					 CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		while ((reg_val & CN23XX_PKT_INPUT_CTL_RST) &&
+				!(reg_val & CN23XX_PKT_INPUT_CTL_QUIET) &&
+				loop) {
+			reg_val = lio_read_csr64(
+					lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+			loop = loop - 1;
+		}
+
+		if (loop == 0) {
+			lio_dev_err(lio_dev,
+				    "clearing the reset reg failed or setting the quiet reg failed for qno: %lu\n",
+				    (unsigned long)q_no);
+			return -1;
+		}
+
+		reg_val = reg_val & ~CN23XX_PKT_INPUT_CTL_RST;
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+				reg_val);
+
+		reg_val = lio_read_csr64(
+		    lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		if (reg_val & CN23XX_PKT_INPUT_CTL_RST) {
+			lio_dev_err(lio_dev,
+				    "clearing the reset failed for qno: %lu\n",
+				    (unsigned long)q_no);
+			ret_val = -1;
+		}
+	}
+
+	return ret_val;
+}
+
+static int
+cn23xx_vf_setup_global_input_regs(struct lio_device *lio_dev)
+{
+	uint64_t q_no;
+	uint64_t d64;
+
+	PMD_INIT_FUNC_TRACE();
+
+	if (cn23xx_vf_reset_io_queues(lio_dev,
+				      lio_dev->sriov_info.rings_per_vf))
+		return -1;
+
+	for (q_no = 0; q_no < (lio_dev->sriov_info.rings_per_vf); q_no++) {
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_DOORBELL(q_no),
+				0xFFFFFFFF);
+
+		d64 = lio_read_csr64(lio_dev,
+				     CN23XX_SLI_IQ_INSTR_COUNT64(q_no));
+
+		d64 &= 0xEFFFFFFFFFFFFFFFL;
+
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_INSTR_COUNT64(q_no),
+				d64);
+
+		/* Select ES, RO, NS, RDSIZE,DPTR Fomat#0 for
+		 * the Input Queues
+		 */
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+				CN23XX_PKT_INPUT_CTL_MASK);
+	}
+
+	return 0;
+}
+
+static void
+cn23xx_vf_setup_global_output_regs(struct lio_device *lio_dev)
+{
+	uint32_t reg_val;
+	uint32_t q_no;
+
+	PMD_INIT_FUNC_TRACE();
+
+	for (q_no = 0; q_no < lio_dev->sriov_info.rings_per_vf; q_no++) {
+		lio_write_csr(lio_dev, CN23XX_SLI_OQ_PKTS_CREDIT(q_no),
+			      0xFFFFFFFF);
+
+		reg_val =
+		    lio_read_csr(lio_dev, CN23XX_SLI_OQ_PKTS_SENT(q_no));
+
+		reg_val &= 0xEFFFFFFFFFFFFFFFL;
+
+		reg_val =
+		    lio_read_csr(lio_dev, CN23XX_SLI_OQ_PKT_CONTROL(q_no));
+
+		/* set IPTR & DPTR */
+		reg_val |=
+		    (CN23XX_PKT_OUTPUT_CTL_IPTR | CN23XX_PKT_OUTPUT_CTL_DPTR);
+
+		/* reset BMODE */
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_BMODE);
+
+		/* No Relaxed Ordering, No Snoop, 64-bit Byte swap
+		 * for Output Queue Scatter List
+		 * reset ROR_P, NSR_P
+		 */
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_ROR_P);
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_NSR_P);
+
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_ES_P);
+#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		reg_val |= (CN23XX_PKT_OUTPUT_CTL_ES_P);
+#endif
+		/* No Relaxed Ordering, No Snoop, 64-bit Byte swap
+		 * for Output Queue Data
+		 * reset ROR, NSR
+		 */
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_ROR);
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_NSR);
+		/* set the ES bit */
+		reg_val |= (CN23XX_PKT_OUTPUT_CTL_ES);
+
+		/* write all the selected settings */
+		lio_write_csr(lio_dev, CN23XX_SLI_OQ_PKT_CONTROL(q_no),
+			      reg_val);
+	}
+}
+
+static int
+cn23xx_vf_setup_device_regs(struct lio_device *lio_dev)
+{
+	PMD_INIT_FUNC_TRACE();
+
+	if (cn23xx_vf_setup_global_input_regs(lio_dev))
+		return -1;
+
+	cn23xx_vf_setup_global_output_regs(lio_dev);
+
+	return 0;
+}
+
 int
 cn23xx_vf_setup_device(struct lio_device *lio_dev)
 {
@@ -63,6 +221,8 @@
 	if (lio_dev->default_config == NULL)
 		return -1;
 
+	lio_dev->fn_list.setup_device_regs	= cn23xx_vf_setup_device_regs;
+
 	return 0;
 }
 
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 0487f3d..34b7b54 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -91,6 +91,11 @@
 		return -1;
 	}
 
+	if (lio_dev->fn_list.setup_device_regs(lio_dev)) {
+		lio_dev_err(lio_dev, "Failed to configure device registers\n");
+		return -1;
+	}
+
 	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
 
 	lio_dev->max_tx_queues = dpdk_queues;
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index a1203e4..577ea49 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -43,6 +43,11 @@
 
 #include "lio_hw_defs.h"
 
+struct lio_device;
+struct lio_fn_list {
+	int (*setup_device_regs)(struct lio_device *);
+};
+
 struct lio_sriov_info {
 	/** Number of rings assigned to VF */
 	uint32_t rings_per_vf;
@@ -117,6 +122,8 @@ struct lio_device {
 
 	uint8_t *hw_addr;
 
+	struct lio_fn_list fn_list;
+
 	struct lio_sriov_info sriov_info;
 
 	char dev_string[LIO_DEVICE_NAME_LEN]; /* Device print string */
-- 
1.8.3.1

[PATCH v2 09/46] net/liquidio: add mbox APIs for PF VF communication

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/Makefile        |   1 +
 drivers/net/liquidio/base/lio_mbox.c | 275 +++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_mbox.h | 129 ++++++++++++++++
 drivers/net/liquidio/lio_struct.h    |   3 +
 4 files changed, 408 insertions(+)
 create mode 100644 drivers/net/liquidio/base/lio_mbox.c
 create mode 100644 drivers/net/liquidio/base/lio_mbox.h

diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
index 8880a10..451f49d 100644
--- a/drivers/net/liquidio/Makefile
+++ b/drivers/net/liquidio/Makefile
@@ -52,6 +52,7 @@ VPATH += $(RTE_SDK)/drivers/net/liquidio/base
 #
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_ethdev.c
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_23xx_vf.c
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_mbox.c
 
 # this lib depends upon:
 DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_eal lib/librte_ether
diff --git a/drivers/net/liquidio/base/lio_mbox.c b/drivers/net/liquidio/base/lio_mbox.c
new file mode 100644
index 0000000..b4abc62
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_mbox.c
@@ -0,0 +1,275 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+
+#include "lio_logs.h"
+#include "lio_struct.h"
+#include "lio_mbox.h"
+
+/**
+ * lio_mbox_read:
+ * @mbox: Pointer mailbox
+ *
+ * Reads the 8-bytes of data from the mbox register
+ * Writes back the acknowledgment indicating completion of read
+ */
+int
+lio_mbox_read(struct lio_mbox *mbox)
+{
+	union lio_mbox_message msg;
+	int ret = 0;
+
+	msg.mbox_msg64 = rte_read64(mbox->mbox_read_reg);
+
+	if ((msg.mbox_msg64 == LIO_PFVFACK) || (msg.mbox_msg64 == LIO_PFVFSIG))
+		return 0;
+
+	if (mbox->state & LIO_MBOX_STATE_REQ_RECEIVING) {
+		mbox->mbox_req.data[mbox->mbox_req.recv_len - 1] =
+					msg.mbox_msg64;
+		mbox->mbox_req.recv_len++;
+	} else {
+		if (mbox->state & LIO_MBOX_STATE_RES_RECEIVING) {
+			mbox->mbox_resp.data[mbox->mbox_resp.recv_len - 1] =
+					msg.mbox_msg64;
+			mbox->mbox_resp.recv_len++;
+		} else {
+			if ((mbox->state & LIO_MBOX_STATE_IDLE) &&
+					(msg.s.type == LIO_MBOX_REQUEST)) {
+				mbox->state &= ~LIO_MBOX_STATE_IDLE;
+				mbox->state |= LIO_MBOX_STATE_REQ_RECEIVING;
+				mbox->mbox_req.msg.mbox_msg64 = msg.mbox_msg64;
+				mbox->mbox_req.q_no = mbox->q_no;
+				mbox->mbox_req.recv_len = 1;
+			} else {
+				if ((mbox->state &
+				     LIO_MBOX_STATE_RES_PENDING) &&
+				    (msg.s.type == LIO_MBOX_RESPONSE)) {
+					mbox->state &=
+						~LIO_MBOX_STATE_RES_PENDING;
+					mbox->state |=
+						LIO_MBOX_STATE_RES_RECEIVING;
+					mbox->mbox_resp.msg.mbox_msg64 =
+								msg.mbox_msg64;
+					mbox->mbox_resp.q_no = mbox->q_no;
+					mbox->mbox_resp.recv_len = 1;
+				} else {
+					rte_write64(LIO_PFVFERR,
+						    mbox->mbox_read_reg);
+					mbox->state |= LIO_MBOX_STATE_ERROR;
+					return -1;
+				}
+			}
+		}
+	}
+
+	if (mbox->state & LIO_MBOX_STATE_REQ_RECEIVING) {
+		if (mbox->mbox_req.recv_len < msg.s.len) {
+			ret = 0;
+		} else {
+			mbox->state &= ~LIO_MBOX_STATE_REQ_RECEIVING;
+			mbox->state |= LIO_MBOX_STATE_REQ_RECEIVED;
+			ret = 1;
+		}
+	} else {
+		if (mbox->state & LIO_MBOX_STATE_RES_RECEIVING) {
+			if (mbox->mbox_resp.recv_len < msg.s.len) {
+				ret = 0;
+			} else {
+				mbox->state &= ~LIO_MBOX_STATE_RES_RECEIVING;
+				mbox->state |= LIO_MBOX_STATE_RES_RECEIVED;
+				ret = 1;
+			}
+		} else {
+			RTE_ASSERT(0);
+		}
+	}
+
+	rte_write64(LIO_PFVFACK, mbox->mbox_read_reg);
+
+	return ret;
+}
+
+/**
+ * lio_mbox_write:
+ * @lio_dev: Pointer lio device
+ * @mbox_cmd: Cmd to send to mailbox.
+ *
+ * Populates the queue specific mbox structure
+ * with cmd information.
+ * Write the cmd to mbox register
+ */
+int
+lio_mbox_write(struct lio_device *lio_dev,
+	       struct lio_mbox_cmd *mbox_cmd)
+{
+	struct lio_mbox *mbox = lio_dev->mbox[mbox_cmd->q_no];
+	uint32_t count, i, ret = LIO_MBOX_STATUS_SUCCESS;
+
+	if ((mbox_cmd->msg.s.type == LIO_MBOX_RESPONSE) &&
+			!(mbox->state & LIO_MBOX_STATE_REQ_RECEIVED))
+		return LIO_MBOX_STATUS_FAILED;
+
+	if ((mbox_cmd->msg.s.type == LIO_MBOX_REQUEST) &&
+			!(mbox->state & LIO_MBOX_STATE_IDLE))
+		return LIO_MBOX_STATUS_BUSY;
+
+	if (mbox_cmd->msg.s.type == LIO_MBOX_REQUEST) {
+		rte_memcpy(&mbox->mbox_resp, mbox_cmd,
+			   sizeof(struct lio_mbox_cmd));
+		mbox->state = LIO_MBOX_STATE_RES_PENDING;
+	}
+
+	count = 0;
+
+	while (rte_read64(mbox->mbox_write_reg) != LIO_PFVFSIG) {
+		rte_delay_ms(1);
+		if (count++ == 1000) {
+			ret = LIO_MBOX_STATUS_FAILED;
+			break;
+		}
+	}
+
+	if (ret == LIO_MBOX_STATUS_SUCCESS) {
+		rte_write64(mbox_cmd->msg.mbox_msg64, mbox->mbox_write_reg);
+		for (i = 0; i < (uint32_t)(mbox_cmd->msg.s.len - 1); i++) {
+			count = 0;
+			while (rte_read64(mbox->mbox_write_reg) !=
+					LIO_PFVFACK) {
+				rte_delay_ms(1);
+				if (count++ == 1000) {
+					ret = LIO_MBOX_STATUS_FAILED;
+					break;
+				}
+			}
+			rte_write64(mbox_cmd->data[i], mbox->mbox_write_reg);
+		}
+	}
+
+	if (mbox_cmd->msg.s.type == LIO_MBOX_RESPONSE) {
+		mbox->state = LIO_MBOX_STATE_IDLE;
+		rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+	} else {
+		if ((!mbox_cmd->msg.s.resp_needed) ||
+				(ret == LIO_MBOX_STATUS_FAILED)) {
+			mbox->state &= ~LIO_MBOX_STATE_RES_PENDING;
+			if (!(mbox->state & (LIO_MBOX_STATE_REQ_RECEIVING |
+					     LIO_MBOX_STATE_REQ_RECEIVED)))
+				mbox->state = LIO_MBOX_STATE_IDLE;
+		}
+	}
+
+	return ret;
+}
+
+/**
+ * lio_mbox_process_cmd:
+ * @mbox: Pointer mailbox
+ * @mbox_cmd: Pointer to command received
+ *
+ * Process the cmd received in mbox
+ */
+static int
+lio_mbox_process_cmd(struct lio_mbox *mbox,
+		     struct lio_mbox_cmd *mbox_cmd)
+{
+	struct lio_device *lio_dev = mbox->lio_dev;
+
+	if (mbox_cmd->msg.s.cmd == LIO_CORES_CRASHED)
+		lio_dev_err(lio_dev, "Octeon core(s) crashed or got stuck!\n");
+
+	return 0;
+}
+
+/**
+ * Process the received mbox message.
+ */
+int
+lio_mbox_process_message(struct lio_mbox *mbox)
+{
+	struct lio_mbox_cmd mbox_cmd;
+
+	if (mbox->state & LIO_MBOX_STATE_ERROR) {
+		if (mbox->state & (LIO_MBOX_STATE_RES_PENDING |
+				   LIO_MBOX_STATE_RES_RECEIVING)) {
+			rte_memcpy(&mbox_cmd, &mbox->mbox_resp,
+				   sizeof(struct lio_mbox_cmd));
+			mbox->state = LIO_MBOX_STATE_IDLE;
+			rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+			mbox_cmd.recv_status = 1;
+			if (mbox_cmd.fn)
+				mbox_cmd.fn(mbox->lio_dev, &mbox_cmd,
+					    mbox_cmd.fn_arg);
+
+			return 0;
+		}
+
+		mbox->state = LIO_MBOX_STATE_IDLE;
+		rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+
+		return 0;
+	}
+
+	if (mbox->state & LIO_MBOX_STATE_RES_RECEIVED) {
+		rte_memcpy(&mbox_cmd, &mbox->mbox_resp,
+			   sizeof(struct lio_mbox_cmd));
+		mbox->state = LIO_MBOX_STATE_IDLE;
+		rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+		mbox_cmd.recv_status = 0;
+		if (mbox_cmd.fn)
+			mbox_cmd.fn(mbox->lio_dev, &mbox_cmd, mbox_cmd.fn_arg);
+
+		return 0;
+	}
+
+	if (mbox->state & LIO_MBOX_STATE_REQ_RECEIVED) {
+		rte_memcpy(&mbox_cmd, &mbox->mbox_req,
+			   sizeof(struct lio_mbox_cmd));
+		if (!mbox_cmd.msg.s.resp_needed) {
+			mbox->state &= ~LIO_MBOX_STATE_REQ_RECEIVED;
+			if (!(mbox->state & LIO_MBOX_STATE_RES_PENDING))
+				mbox->state = LIO_MBOX_STATE_IDLE;
+			rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+		}
+
+		lio_mbox_process_cmd(mbox, &mbox_cmd);
+
+		return 0;
+	}
+
+	RTE_ASSERT(0);
+
+	return 0;
+}
diff --git a/drivers/net/liquidio/base/lio_mbox.h b/drivers/net/liquidio/base/lio_mbox.h
new file mode 100644
index 0000000..28c9e1a
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_mbox.h
@@ -0,0 +1,129 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_MBOX_H_
+#define _LIO_MBOX_H_
+
+#include <stdint.h>
+
+#include <rte_spinlock.h>
+
+/* Macros for Mail Box Communication */
+
+#define LIO_MBOX_DATA_MAX			32
+
+#define LIO_CORES_CRASHED			0x3
+
+/* Macro for Read acknowledgment */
+#define LIO_PFVFACK				0xffffffffffffffff
+#define LIO_PFVFSIG				0x1122334455667788
+#define LIO_PFVFERR				0xDEADDEADDEADDEAD
+
+enum lio_mbox_cmd_status {
+	LIO_MBOX_STATUS_SUCCESS		= 0,
+	LIO_MBOX_STATUS_FAILED		= 1,
+	LIO_MBOX_STATUS_BUSY		= 2
+};
+
+enum lio_mbox_message_type {
+	LIO_MBOX_REQUEST	= 0,
+	LIO_MBOX_RESPONSE	= 1
+};
+
+union lio_mbox_message {
+	uint64_t mbox_msg64;
+	struct {
+		uint16_t type : 1;
+		uint16_t resp_needed : 1;
+		uint16_t cmd : 6;
+		uint16_t len : 8;
+		uint8_t params[6];
+	} s;
+};
+
+typedef void (*lio_mbox_callback)(void *, void *, void *);
+
+struct lio_mbox_cmd {
+	union lio_mbox_message msg;
+	uint64_t data[LIO_MBOX_DATA_MAX];
+	uint32_t q_no;
+	uint32_t recv_len;
+	uint32_t recv_status;
+	lio_mbox_callback fn;
+	void *fn_arg;
+};
+
+enum lio_mbox_state {
+	LIO_MBOX_STATE_IDLE		= 1,
+	LIO_MBOX_STATE_REQ_RECEIVING	= 2,
+	LIO_MBOX_STATE_REQ_RECEIVED	= 4,
+	LIO_MBOX_STATE_RES_PENDING	= 8,
+	LIO_MBOX_STATE_RES_RECEIVING	= 16,
+	LIO_MBOX_STATE_RES_RECEIVED	= 16,
+	LIO_MBOX_STATE_ERROR		= 32
+};
+
+struct lio_mbox {
+	/* A spinlock to protect access to this q_mbox. */
+	rte_spinlock_t lock;
+
+	struct lio_device *lio_dev;
+
+	uint32_t q_no;
+
+	enum lio_mbox_state state;
+
+	/* SLI_MAC_PF_MBOX_INT for PF, SLI_PKT_MBOX_INT for VF. */
+	void *mbox_int_reg;
+
+	/* SLI_PKT_PF_VF_MBOX_SIG(0) for PF,
+	 * SLI_PKT_PF_VF_MBOX_SIG(1) for VF.
+	 */
+	void *mbox_write_reg;
+
+	/* SLI_PKT_PF_VF_MBOX_SIG(1) for PF,
+	 * SLI_PKT_PF_VF_MBOX_SIG(0) for VF.
+	 */
+	void *mbox_read_reg;
+
+	struct lio_mbox_cmd mbox_req;
+
+	struct lio_mbox_cmd mbox_resp;
+
+};
+
+int lio_mbox_read(struct lio_mbox *mbox);
+int lio_mbox_write(struct lio_device *lio_dev,
+		   struct lio_mbox_cmd *mbox_cmd);
+int lio_mbox_process_message(struct lio_mbox *mbox);
+#endif	/* _LIO_MBOX_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 577ea49..0af4fe3 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -126,6 +126,9 @@ struct lio_device {
 
 	struct lio_sriov_info sriov_info;
 
+	/** Mail Box details of each lio queue. */
+	struct lio_mbox **mbox;
+
 	char dev_string[LIO_DEVICE_NAME_LEN]; /* Device print string */
 
 	const struct lio_config *default_config;
-- 
1.8.3.1

[PATCH v2 10/46] net/liquidio: add API to setup mbox registers

[Shijith Thotton]

Map and initialize mbox registers.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 61 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.c       | 15 ++++++--
 drivers/net/liquidio/lio_struct.h       |  3 ++
 3 files changed, 77 insertions(+), 2 deletions(-)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index f61f185..70faa9b 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -38,6 +38,7 @@
 #include "lio_logs.h"
 #include "lio_23xx_vf.h"
 #include "lio_23xx_reg.h"
+#include "lio_mbox.h"
 
 static int
 cn23xx_vf_reset_io_queues(struct lio_device *lio_dev, uint32_t num_queues)
@@ -197,6 +198,63 @@
 	return 0;
 }
 
+static void
+cn23xx_vf_free_mbox(struct lio_device *lio_dev)
+{
+	PMD_INIT_FUNC_TRACE();
+
+	rte_free(lio_dev->mbox[0]);
+	lio_dev->mbox[0] = NULL;
+
+	rte_free(lio_dev->mbox);
+	lio_dev->mbox = NULL;
+}
+
+static int
+cn23xx_vf_setup_mbox(struct lio_device *lio_dev)
+{
+	struct lio_mbox *mbox;
+
+	PMD_INIT_FUNC_TRACE();
+
+	if (lio_dev->mbox == NULL) {
+		lio_dev->mbox = rte_zmalloc(NULL, sizeof(void *), 0);
+		if (lio_dev->mbox == NULL)
+			return -ENOMEM;
+	}
+
+	mbox = rte_zmalloc(NULL, sizeof(struct lio_mbox), 0);
+	if (mbox == NULL) {
+		rte_free(lio_dev->mbox);
+		lio_dev->mbox = NULL;
+		return -ENOMEM;
+	}
+
+	rte_spinlock_init(&mbox->lock);
+
+	mbox->lio_dev = lio_dev;
+
+	mbox->q_no = 0;
+
+	mbox->state = LIO_MBOX_STATE_IDLE;
+
+	/* VF mbox interrupt reg */
+	mbox->mbox_int_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_VF_SLI_PKT_MBOX_INT(0);
+	/* VF reads from SIG0 reg */
+	mbox->mbox_read_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_SLI_PKT_PF_VF_MBOX_SIG(0, 0);
+	/* VF writes into SIG1 reg */
+	mbox->mbox_write_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_SLI_PKT_PF_VF_MBOX_SIG(0, 1);
+
+	lio_dev->mbox[0] = mbox;
+
+	rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+
+	return 0;
+}
+
 int
 cn23xx_vf_setup_device(struct lio_device *lio_dev)
 {
@@ -221,6 +279,9 @@
 	if (lio_dev->default_config == NULL)
 		return -1;
 
+	lio_dev->fn_list.setup_mbox		= cn23xx_vf_setup_mbox;
+	lio_dev->fn_list.free_mbox		= cn23xx_vf_free_mbox;
+
 	lio_dev->fn_list.setup_device_regs	= cn23xx_vf_setup_device_regs;
 
 	return 0;
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 34b7b54..5ee1bb5 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -86,14 +86,19 @@
 		return -1;
 	}
 
+	if (lio_dev->fn_list.setup_mbox(lio_dev)) {
+		lio_dev_err(lio_dev, "Mailbox setup failed\n");
+		goto error;
+	}
+
 	if (cn23xx_vf_set_io_queues_off(lio_dev)) {
 		lio_dev_err(lio_dev, "Setting io queues off failed\n");
-		return -1;
+		goto error;
 	}
 
 	if (lio_dev->fn_list.setup_device_regs(lio_dev)) {
 		lio_dev_err(lio_dev, "Failed to configure device registers\n");
-		return -1;
+		goto error;
 	}
 
 	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
@@ -102,6 +107,12 @@
 	lio_dev->max_rx_queues = dpdk_queues;
 
 	return 0;
+
+error:
+	if (lio_dev->mbox[0])
+		lio_dev->fn_list.free_mbox(lio_dev);
+
+	return -1;
 }
 
 static int
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 0af4fe3..01b5716 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -45,6 +45,9 @@
 
 struct lio_device;
 struct lio_fn_list {
+	int (*setup_mbox)(struct lio_device *);
+	void (*free_mbox)(struct lio_device *);
+
 	int (*setup_device_regs)(struct lio_device *);
 };
 
-- 
1.8.3.1

[PATCH v2 11/46] net/liquidio: add API for PF VF handshake

[Shijith Thotton]

Handshake with PF kernel driver to check driver version compatibility.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 96 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h |  4 ++
 drivers/net/liquidio/base/lio_hw_defs.h |  3 ++
 drivers/net/liquidio/base/lio_mbox.h    |  1 +
 drivers/net/liquidio/lio_ethdev.c       | 21 ++++++++
 drivers/net/liquidio/lio_struct.h       | 45 ++++++++++++++++
 6 files changed, 170 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index 70faa9b..6270af5 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -255,6 +255,102 @@
 	return 0;
 }
 
+static void
+cn23xx_pfvf_hs_callback(struct lio_device *lio_dev,
+			struct lio_mbox_cmd *cmd, void *arg)
+{
+	uint32_t major = 0;
+
+	PMD_INIT_FUNC_TRACE();
+
+	rte_memcpy((uint8_t *)&lio_dev->pfvf_hsword, cmd->msg.s.params, 6);
+	if (cmd->recv_len > 1) {
+		struct lio_version *lio_ver = (struct lio_version *)cmd->data;
+
+		major = lio_ver->major;
+		major = major << 16;
+	}
+
+	rte_atomic64_set((rte_atomic64_t *)arg, major | 1);
+}
+
+int
+cn23xx_pfvf_handshake(struct lio_device *lio_dev)
+{
+	struct lio_mbox_cmd mbox_cmd;
+	struct lio_version *lio_ver = (struct lio_version *)&mbox_cmd.data[0];
+	rte_atomic64_t status;
+	uint32_t count = 0;
+	uint32_t pfmajor;
+	uint32_t vfmajor;
+	uint32_t ret;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* Sending VF_ACTIVE indication to the PF driver */
+	lio_dev_dbg(lio_dev, "requesting info from PF\n");
+
+	mbox_cmd.msg.mbox_msg64 = 0;
+	mbox_cmd.msg.s.type = LIO_MBOX_REQUEST;
+	mbox_cmd.msg.s.resp_needed = 1;
+	mbox_cmd.msg.s.cmd = LIO_VF_ACTIVE;
+	mbox_cmd.msg.s.len = 2;
+	mbox_cmd.data[0] = 0;
+	lio_ver->major = LIO_BASE_MAJOR_VERSION;
+	lio_ver->minor = LIO_BASE_MINOR_VERSION;
+	lio_ver->micro = LIO_BASE_MICRO_VERSION;
+	mbox_cmd.q_no = 0;
+	mbox_cmd.recv_len = 0;
+	mbox_cmd.recv_status = 0;
+	mbox_cmd.fn = (lio_mbox_callback)cn23xx_pfvf_hs_callback;
+	mbox_cmd.fn_arg = (void *)&status;
+
+	if (lio_mbox_write(lio_dev, &mbox_cmd)) {
+		lio_dev_err(lio_dev, "Write to mailbox failed\n");
+		return -1;
+	}
+
+	rte_atomic64_set(&status, 0);
+
+	do {
+		rte_delay_ms(1);
+	} while ((rte_atomic64_read(&status) == 0) && (count++ < 10000));
+
+	ret = rte_atomic64_read(&status);
+	if (ret == 0) {
+		lio_dev_err(lio_dev, "cn23xx_pfvf_handshake timeout\n");
+		return -1;
+	}
+
+	vfmajor = LIO_BASE_MAJOR_VERSION;
+	pfmajor = ret >> 16;
+	if (pfmajor != vfmajor) {
+		lio_dev_err(lio_dev,
+			    "VF LiquidIO driver (major version %d) is not compatible with LiquidIO PF driver (major version %d)\n",
+			    vfmajor, pfmajor);
+		ret = -EPERM;
+	} else {
+		lio_dev_dbg(lio_dev,
+			    "VF LiquidIO driver (major version %d), LiquidIO PF driver (major version %d)\n",
+			    vfmajor, pfmajor);
+		ret = 0;
+	}
+
+	return ret;
+}
+
+void
+cn23xx_vf_handle_mbox(struct lio_device *lio_dev)
+{
+	uint64_t mbox_int_val;
+
+	/* read and clear by writing 1 */
+	mbox_int_val = rte_read64(lio_dev->mbox[0]->mbox_int_reg);
+	rte_write64(mbox_int_val, lio_dev->mbox[0]->mbox_int_reg);
+	if (lio_mbox_read(lio_dev->mbox[0]))
+		lio_mbox_process_message(lio_dev->mbox[0]);
+}
+
 int
 cn23xx_vf_setup_device(struct lio_device *lio_dev)
 {
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.h b/drivers/net/liquidio/base/lio_23xx_vf.h
index 1af09d0..83dc053 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.h
+++ b/drivers/net/liquidio/base/lio_23xx_vf.h
@@ -87,5 +87,9 @@
 
 #define CN23XX_VF_BUSY_READING_REG_LOOP_COUNT	100000
 
+int cn23xx_pfvf_handshake(struct lio_device *lio_dev);
+
 int cn23xx_vf_setup_device(struct lio_device  *lio_dev);
+
+void cn23xx_vf_handle_mbox(struct lio_device *lio_dev);
 #endif /* _LIO_23XX_VF_H_  */
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index a2654cd..9282068 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -77,6 +77,9 @@ enum lio_card_type {
 #define LIO_23XX_NAME "23xx"
 
 #define LIO_DEVICE_NAME_LEN		32
+#define LIO_BASE_MAJOR_VERSION		1
+#define LIO_BASE_MINOR_VERSION		5
+#define LIO_BASE_MICRO_VERSION		1
 
 /* Routines for reading and writing CSRs */
 #ifdef RTE_LIBRTE_LIO_DEBUG_REGS
diff --git a/drivers/net/liquidio/base/lio_mbox.h b/drivers/net/liquidio/base/lio_mbox.h
index 28c9e1a..f1c5b8e 100644
--- a/drivers/net/liquidio/base/lio_mbox.h
+++ b/drivers/net/liquidio/base/lio_mbox.h
@@ -42,6 +42,7 @@
 
 #define LIO_MBOX_DATA_MAX			32
 
+#define LIO_VF_ACTIVE				0x1
 #define LIO_CORES_CRASHED			0x3
 
 /* Macro for Read acknowledgment */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 5ee1bb5..bebe0e8 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -40,6 +40,20 @@
 #include "lio_23xx_vf.h"
 #include "lio_ethdev.h"
 
+static void
+lio_check_pf_hs_response(void *lio_dev)
+{
+	struct lio_device *dev = lio_dev;
+
+	/* check till response arrives */
+	if (dev->pfvf_hsword.coproc_tics_per_us)
+		return;
+
+	cn23xx_vf_handle_mbox(dev);
+
+	rte_eal_alarm_set(1, lio_check_pf_hs_response, lio_dev);
+}
+
 /**
  * \brief Identify the LIO device and to map the BAR address space
  * @param lio_dev lio device
@@ -91,6 +105,13 @@
 		goto error;
 	}
 
+	/* Check PF response */
+	lio_check_pf_hs_response((void *)lio_dev);
+
+	/* Do handshake and exit if incompatible PF driver */
+	if (cn23xx_pfvf_handshake(lio_dev))
+		goto error;
+
 	if (cn23xx_vf_set_io_queues_off(lio_dev)) {
 		lio_dev_err(lio_dev, "Setting io queues off failed\n");
 		goto error;
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 01b5716..e8b6e1d 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -43,6 +43,13 @@
 
 #include "lio_hw_defs.h"
 
+struct lio_version {
+	uint16_t major;
+	uint16_t minor;
+	uint16_t micro;
+	uint16_t reserved;
+};
+
 struct lio_device;
 struct lio_fn_list {
 	int (*setup_mbox)(struct lio_device *);
@@ -51,6 +58,42 @@ struct lio_fn_list {
 	int (*setup_device_regs)(struct lio_device *);
 };
 
+struct lio_pf_vf_hs_word {
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	/** PKIND value assigned for the DPI interface */
+	uint64_t pkind : 8;
+
+	/** OCTEON core clock multiplier */
+	uint64_t core_tics_per_us : 16;
+
+	/** OCTEON coprocessor clock multiplier */
+	uint64_t coproc_tics_per_us : 16;
+
+	/** app that currently running on OCTEON */
+	uint64_t app_mode : 8;
+
+	/** RESERVED */
+	uint64_t reserved : 16;
+
+#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+
+	/** RESERVED */
+	uint64_t reserved : 16;
+
+	/** app that currently running on OCTEON */
+	uint64_t app_mode : 8;
+
+	/** OCTEON coprocessor clock multiplier */
+	uint64_t coproc_tics_per_us : 16;
+
+	/** OCTEON core clock multiplier */
+	uint64_t core_tics_per_us : 16;
+
+	/** PKIND value assigned for the DPI interface */
+	uint64_t pkind : 8;
+#endif
+};
+
 struct lio_sriov_info {
 	/** Number of rings assigned to VF */
 	uint32_t rings_per_vf;
@@ -129,6 +172,8 @@ struct lio_device {
 
 	struct lio_sriov_info sriov_info;
 
+	struct lio_pf_vf_hs_word pfvf_hsword;
+
 	/** Mail Box details of each lio queue. */
 	struct lio_mbox **mbox;
 
-- 
1.8.3.1

[PATCH v2 12/46] net/liquidio: add API for VF FLR

[Shijith Thotton]

VF sends Function Level Reset request to PF using mbox and PF does the
reset.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 19 +++++++++++++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h |  2 ++
 drivers/net/liquidio/base/lio_mbox.h    |  1 +
 drivers/net/liquidio/lio_ethdev.c       |  5 +++++
 4 files changed, 27 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index 6270af5..ed5b830 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -255,6 +255,25 @@
 	return 0;
 }
 
+void
+cn23xx_vf_ask_pf_to_do_flr(struct lio_device *lio_dev)
+{
+	struct lio_mbox_cmd mbox_cmd;
+
+	mbox_cmd.msg.mbox_msg64 = 0;
+	mbox_cmd.msg.s.type = LIO_MBOX_REQUEST;
+	mbox_cmd.msg.s.resp_needed = 0;
+	mbox_cmd.msg.s.cmd = LIO_VF_FLR_REQUEST;
+	mbox_cmd.msg.s.len = 1;
+	mbox_cmd.q_no = 0;
+	mbox_cmd.recv_len = 0;
+	mbox_cmd.recv_status = 0;
+	mbox_cmd.fn = NULL;
+	mbox_cmd.fn_arg = 0;
+
+	lio_mbox_write(lio_dev, &mbox_cmd);
+}
+
 static void
 cn23xx_pfvf_hs_callback(struct lio_device *lio_dev,
 			struct lio_mbox_cmd *cmd, void *arg)
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.h b/drivers/net/liquidio/base/lio_23xx_vf.h
index 83dc053..ad8db0d 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.h
+++ b/drivers/net/liquidio/base/lio_23xx_vf.h
@@ -87,6 +87,8 @@
 
 #define CN23XX_VF_BUSY_READING_REG_LOOP_COUNT	100000
 
+void cn23xx_vf_ask_pf_to_do_flr(struct lio_device *lio_dev);
+
 int cn23xx_pfvf_handshake(struct lio_device *lio_dev);
 
 int cn23xx_vf_setup_device(struct lio_device  *lio_dev);
diff --git a/drivers/net/liquidio/base/lio_mbox.h b/drivers/net/liquidio/base/lio_mbox.h
index f1c5b8e..b0875d6 100644
--- a/drivers/net/liquidio/base/lio_mbox.h
+++ b/drivers/net/liquidio/base/lio_mbox.h
@@ -43,6 +43,7 @@
 #define LIO_MBOX_DATA_MAX			32
 
 #define LIO_VF_ACTIVE				0x1
+#define LIO_VF_FLR_REQUEST			0x2
 #define LIO_CORES_CRASHED			0x3
 
 /* Macro for Read acknowledgment */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index bebe0e8..5aae105 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -112,6 +112,11 @@
 	if (cn23xx_pfvf_handshake(lio_dev))
 		goto error;
 
+	/* Initial reset */
+	cn23xx_vf_ask_pf_to_do_flr(lio_dev);
+	/* Wait for FLR for 100ms per SRIOV specification */
+	rte_delay_ms(100);
+
 	if (cn23xx_vf_set_io_queues_off(lio_dev)) {
 		lio_dev_err(lio_dev, "Setting io queues off failed\n");
 		goto error;
-- 
1.8.3.1

[PATCH v2 13/46] net/liquidio: add APIs to allocate and free IQ

[Shijith Thotton]

Instruction queue (IQ) is used to send control and data packets to
device from host. IQ 0 is used to send device configuration commands
during initialization and later re-allocated as per application
requirement.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/Makefile           |   1 +
 drivers/net/liquidio/base/lio_hw_defs.h |  12 ++
 drivers/net/liquidio/lio_ethdev.c       |   8 ++
 drivers/net/liquidio/lio_rxtx.c         | 206 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         |  57 +++++++++
 drivers/net/liquidio/lio_struct.h       | 112 ++++++++++++++++-
 6 files changed, 395 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/liquidio/lio_rxtx.c
 create mode 100644 drivers/net/liquidio/lio_rxtx.h

diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
index 451f49d..de2ef9b 100644
--- a/drivers/net/liquidio/Makefile
+++ b/drivers/net/liquidio/Makefile
@@ -51,6 +51,7 @@ VPATH += $(RTE_SDK)/drivers/net/liquidio/base
 # all source are stored in SRCS-y
 #
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_ethdev.c
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_rxtx.c
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_23xx_vf.c
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_mbox.c
 
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 9282068..726ce6b 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -76,6 +76,18 @@ enum lio_card_type {
 
 #define LIO_23XX_NAME "23xx"
 
+#define LIO_NUM_DEF_TX_DESCS_CFG(cfg)					\
+		((cfg)->default_config->num_def_tx_descs)
+
+#define LIO_IQ_INSTR_TYPE(cfg)		((cfg)->default_config->iq.instr_type)
+
+/* The following config values are fixed and should not be modified. */
+
+/* Maximum number of Instruction queues */
+#define LIO_MAX_INSTR_QUEUES(lio_dev)		CN23XX_MAX_RINGS_PER_VF
+
+#define LIO_MAX_POSSIBLE_INSTR_QUEUES		CN23XX_MAX_INPUT_QUEUES
+
 #define LIO_DEVICE_NAME_LEN		32
 #define LIO_BASE_MAJOR_VERSION		1
 #define LIO_BASE_MINOR_VERSION		5
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 5aae105..5d7d5a7 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -39,6 +39,7 @@
 #include "lio_logs.h"
 #include "lio_23xx_vf.h"
 #include "lio_ethdev.h"
+#include "lio_rxtx.h"
 
 static void
 lio_check_pf_hs_response(void *lio_dev)
@@ -127,6 +128,11 @@
 		goto error;
 	}
 
+	if (lio_setup_instr_queue0(lio_dev)) {
+		lio_dev_err(lio_dev, "Failed to setup instruction queue 0\n");
+		goto error;
+	}
+
 	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
 
 	lio_dev->max_tx_queues = dpdk_queues;
@@ -137,6 +143,8 @@
 error:
 	if (lio_dev->mbox[0])
 		lio_dev->fn_list.free_mbox(lio_dev);
+	if (lio_dev->instr_queue[0])
+		lio_free_instr_queue0(lio_dev);
 
 	return -1;
 }
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
new file mode 100644
index 0000000..de98fc6
--- /dev/null
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -0,0 +1,206 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+
+#include "lio_logs.h"
+#include "lio_struct.h"
+#include "lio_ethdev.h"
+#include "lio_rxtx.h"
+
+static void
+lio_dma_zone_free(struct lio_device *lio_dev, const struct rte_memzone *mz)
+{
+	const struct rte_memzone *mz_tmp;
+	int ret = 0;
+
+	if (mz == NULL) {
+		lio_dev_err(lio_dev, "Memzone NULL\n");
+		return;
+	}
+
+	mz_tmp = rte_memzone_lookup(mz->name);
+	if (mz_tmp == NULL) {
+		lio_dev_err(lio_dev, "Memzone %s Not Found\n", mz->name);
+		return;
+	}
+
+	ret = rte_memzone_free(mz);
+	if (ret)
+		lio_dev_err(lio_dev, "Memzone free Failed ret %d\n", ret);
+}
+
+/**
+ *  lio_init_instr_queue()
+ *  @param lio_dev	- pointer to the lio device structure.
+ *  @param txpciq	- queue to be initialized.
+ *
+ *  Called at driver init time for each input queue. iq_conf has the
+ *  configuration parameters for the queue.
+ *
+ *  @return  Success: 0	Failure: -1
+ */
+static int
+lio_init_instr_queue(struct lio_device *lio_dev,
+		     union octeon_txpciq txpciq,
+		     uint32_t num_descs, unsigned int socket_id)
+{
+	uint32_t iq_no = (uint32_t)txpciq.s.q_no;
+	struct lio_instr_queue *iq;
+	uint32_t instr_type;
+	uint32_t q_size;
+
+	instr_type = LIO_IQ_INSTR_TYPE(lio_dev);
+
+	q_size = instr_type * num_descs;
+	iq = lio_dev->instr_queue[iq_no];
+	iq->iq_mz = rte_eth_dma_zone_reserve(lio_dev->eth_dev,
+					     "instr_queue", iq_no, q_size,
+					     RTE_CACHE_LINE_SIZE,
+					     socket_id);
+	if (iq->iq_mz == NULL) {
+		lio_dev_err(lio_dev, "Cannot allocate memory for instr queue %d\n",
+			    iq_no);
+		return -1;
+	}
+
+	iq->base_addr_dma = iq->iq_mz->phys_addr;
+	iq->base_addr = (uint8_t *)iq->iq_mz->addr;
+
+	iq->max_count = num_descs;
+
+	/* Initialize a list to holds requests that have been posted to Octeon
+	 * but has yet to be fetched by octeon
+	 */
+	iq->request_list = rte_zmalloc_socket("request_list",
+					      sizeof(*iq->request_list) *
+							num_descs,
+					      RTE_CACHE_LINE_SIZE,
+					      socket_id);
+	if (iq->request_list == NULL) {
+		lio_dev_err(lio_dev, "Alloc failed for IQ[%d] nr free list\n",
+			    iq_no);
+		lio_dma_zone_free(lio_dev, iq->iq_mz);
+		return -1;
+	}
+
+	lio_dev_dbg(lio_dev, "IQ[%d]: base: %p basedma: %lx count: %d\n",
+		    iq_no, iq->base_addr, (unsigned long)iq->base_addr_dma,
+		    iq->max_count);
+
+	iq->lio_dev = lio_dev;
+	iq->txpciq.txpciq64 = txpciq.txpciq64;
+	iq->fill_cnt = 0;
+	iq->host_write_index = 0;
+	iq->lio_read_index = 0;
+	iq->flush_index = 0;
+
+	rte_atomic64_set(&iq->instr_pending, 0);
+
+	/* Initialize the spinlock for this instruction queue */
+	rte_spinlock_init(&iq->lock);
+	rte_spinlock_init(&iq->post_lock);
+
+	rte_atomic64_clear(&iq->iq_flush_running);
+
+	lio_dev->io_qmask.iq |= (1ULL << iq_no);
+
+	/* Set the 32B/64B mode for each input queue */
+	lio_dev->io_qmask.iq64B |= ((instr_type == 64) << iq_no);
+	iq->iqcmd_64B = (instr_type == 64);
+
+	return 0;
+}
+
+int
+lio_setup_instr_queue0(struct lio_device *lio_dev)
+{
+	union octeon_txpciq txpciq;
+	uint32_t num_descs = 0;
+	uint32_t iq_no = 0;
+
+	num_descs = LIO_NUM_DEF_TX_DESCS_CFG(lio_dev);
+
+	lio_dev->num_iqs = 0;
+
+	lio_dev->instr_queue[0] = rte_zmalloc(NULL,
+					sizeof(struct lio_instr_queue), 0);
+	if (lio_dev->instr_queue[0] == NULL)
+		return -ENOMEM;
+
+	lio_dev->instr_queue[0]->q_index = 0;
+	lio_dev->instr_queue[0]->app_ctx = (void *)(size_t)0;
+	txpciq.txpciq64 = 0;
+	txpciq.s.q_no = iq_no;
+	txpciq.s.pkind = lio_dev->pfvf_hsword.pkind;
+	txpciq.s.use_qpg = 0;
+	txpciq.s.qpg = 0;
+	if (lio_init_instr_queue(lio_dev, txpciq, num_descs, SOCKET_ID_ANY)) {
+		rte_free(lio_dev->instr_queue[0]);
+		lio_dev->instr_queue[0] = NULL;
+		return -1;
+	}
+
+	lio_dev->num_iqs++;
+
+	return 0;
+}
+
+/**
+ *  lio_delete_instr_queue()
+ *  @param lio_dev	- pointer to the lio device structure.
+ *  @param iq_no	- queue to be deleted.
+ *
+ *  Called at driver unload time for each input queue. Deletes all
+ *  allocated resources for the input queue.
+ */
+static void
+lio_delete_instr_queue(struct lio_device *lio_dev, uint32_t iq_no)
+{
+	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
+
+	rte_free(iq->request_list);
+	iq->request_list = NULL;
+	lio_dma_zone_free(lio_dev, iq->iq_mz);
+}
+
+void
+lio_free_instr_queue0(struct lio_device *lio_dev)
+{
+	lio_delete_instr_queue(lio_dev, 0);
+	rte_free(lio_dev->instr_queue[0]);
+	lio_dev->instr_queue[0] = NULL;
+	lio_dev->num_iqs--;
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
new file mode 100644
index 0000000..33f178b
--- /dev/null
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -0,0 +1,57 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_RXTX_H_
+#define _LIO_RXTX_H_
+
+#include <stdio.h>
+#include <stdint.h>
+
+#include <rte_spinlock.h>
+#include <rte_memory.h>
+
+#include "lio_struct.h"
+
+struct lio_request_list {
+	uint32_t reqtype;
+	void *buf;
+};
+
+/** Setup instruction queue zero for the device
+ *  @param lio_dev which lio device to setup
+ *
+ *  @return 0 if success. -1 if fails
+ */
+int lio_setup_instr_queue0(struct lio_device *lio_dev);
+void lio_free_instr_queue0(struct lio_device *lio_dev);
+#endif	/* _LIO_RXTX_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index e8b6e1d..29059a5 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -50,7 +50,110 @@ struct lio_version {
 	uint16_t reserved;
 };
 
-struct lio_device;
+/** The txpciq info passed to host from the firmware */
+union octeon_txpciq {
+	uint64_t txpciq64;
+
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t q_no : 8;
+		uint64_t port : 8;
+		uint64_t pkind : 6;
+		uint64_t use_qpg : 1;
+		uint64_t qpg : 11;
+		uint64_t aura_num : 10;
+		uint64_t reserved : 20;
+#else
+		uint64_t reserved : 20;
+		uint64_t aura_num : 10;
+		uint64_t qpg : 11;
+		uint64_t use_qpg : 1;
+		uint64_t pkind : 6;
+		uint64_t port : 8;
+		uint64_t q_no : 8;
+#endif
+	} s;
+};
+
+/** The instruction (input) queue.
+ *  The input queue is used to post raw (instruction) mode data or packet
+ *  data to Octeon device from the host. Each input queue for
+ *  a LIO device has one such structure to represent it.
+ */
+struct lio_instr_queue {
+	/** A spinlock to protect access to the input ring.  */
+	rte_spinlock_t lock;
+
+	rte_spinlock_t post_lock;
+
+	struct lio_device *lio_dev;
+
+	uint32_t pkt_in_done;
+
+	rte_atomic64_t iq_flush_running;
+
+	/** Flag that indicates if the queue uses 64 byte commands. */
+	uint32_t iqcmd_64B:1;
+
+	/** Queue info. */
+	union octeon_txpciq txpciq;
+
+	uint32_t rsvd:17;
+
+	uint32_t status:8;
+
+	/** Maximum no. of instructions in this queue. */
+	uint32_t max_count;
+
+	/** Index in input ring where the driver should write the next packet */
+	uint32_t host_write_index;
+
+	/** Index in input ring where Octeon is expected to read the next
+	 *  packet.
+	 */
+	uint32_t lio_read_index;
+
+	/** This index aids in finding the window in the queue where Octeon
+	 *  has read the commands.
+	 */
+	uint32_t flush_index;
+
+	/** This field keeps track of the instructions pending in this queue. */
+	rte_atomic64_t instr_pending;
+
+	/** Pointer to the Virtual Base addr of the input ring. */
+	uint8_t *base_addr;
+
+	struct lio_request_list *request_list;
+
+	/** Octeon doorbell register for the ring. */
+	void *doorbell_reg;
+
+	/** Octeon instruction count register for this ring. */
+	void *inst_cnt_reg;
+
+	/** Number of instructions pending to be posted to Octeon. */
+	uint32_t fill_cnt;
+
+	/** DMA mapped base address of the input descriptor ring. */
+	uint64_t base_addr_dma;
+
+	/** Application context */
+	void *app_ctx;
+
+	/* network stack queue index */
+	int q_index;
+
+	/* Memory zone */
+	const struct rte_memzone *iq_mz;
+};
+
+struct lio_io_enable {
+	uint64_t iq;
+	uint64_t oq;
+	uint64_t iq64B;
+};
+
 struct lio_fn_list {
 	int (*setup_mbox)(struct lio_device *);
 	void (*free_mbox)(struct lio_device *);
@@ -170,6 +273,13 @@ struct lio_device {
 
 	struct lio_fn_list fn_list;
 
+	uint32_t num_iqs;
+
+	/** The input instruction queues */
+	struct lio_instr_queue *instr_queue[LIO_MAX_POSSIBLE_INSTR_QUEUES];
+
+	struct lio_io_enable io_qmask;
+
 	struct lio_sriov_info sriov_info;
 
 	struct lio_pf_vf_hs_word pfvf_hsword;
-- 
1.8.3.1

[PATCH v2 14/46] net/liquidio: add API to setup IQ

[Shijith Thotton]

Map instruction queue registers and set queue size.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 44 ++++++++++++++++++++++++++++++++-
 drivers/net/liquidio/lio_rxtx.c         |  2 ++
 drivers/net/liquidio/lio_struct.h       |  2 ++
 3 files changed, 47 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index ed5b830..181f830 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -199,6 +199,40 @@
 }
 
 static void
+cn23xx_vf_setup_iq_regs(struct lio_device *lio_dev, uint32_t iq_no)
+{
+	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
+	uint64_t pkt_in_done = 0;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* Write the start of the input queue's ring and its size */
+	lio_write_csr64(lio_dev, CN23XX_SLI_IQ_BASE_ADDR64(iq_no),
+			iq->base_addr_dma);
+	lio_write_csr(lio_dev, CN23XX_SLI_IQ_SIZE(iq_no), iq->max_count);
+
+	/* Remember the doorbell & instruction count register addr
+	 * for this queue
+	 */
+	iq->doorbell_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_SLI_IQ_DOORBELL(iq_no);
+	iq->inst_cnt_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_SLI_IQ_INSTR_COUNT64(iq_no);
+	lio_dev_dbg(lio_dev, "InstQ[%d]:dbell reg @ 0x%p instcnt_reg @ 0x%p\n",
+		    iq_no, iq->doorbell_reg, iq->inst_cnt_reg);
+
+	/* Store the current instruction counter (used in flush_iq
+	 * calculation)
+	 */
+	pkt_in_done = rte_read64(iq->inst_cnt_reg);
+
+	/* Clear the count by writing back what we read, but don't
+	 * enable data traffic here
+	 */
+	rte_write64(pkt_in_done, iq->inst_cnt_reg);
+}
+
+static void
 cn23xx_vf_free_mbox(struct lio_device *lio_dev)
 {
 	PMD_INIT_FUNC_TRACE();
@@ -298,8 +332,8 @@
 {
 	struct lio_mbox_cmd mbox_cmd;
 	struct lio_version *lio_ver = (struct lio_version *)&mbox_cmd.data[0];
+	uint32_t q_no, count = 0;
 	rte_atomic64_t status;
-	uint32_t count = 0;
 	uint32_t pfmajor;
 	uint32_t vfmajor;
 	uint32_t ret;
@@ -341,6 +375,10 @@
 		return -1;
 	}
 
+	for (q_no = 0; q_no < lio_dev->num_iqs; q_no++)
+		lio_dev->instr_queue[q_no]->txpciq.s.pkind =
+						lio_dev->pfvf_hsword.pkind;
+
 	vfmajor = LIO_BASE_MAJOR_VERSION;
 	pfmajor = ret >> 16;
 	if (pfmajor != vfmajor) {
@@ -355,6 +393,9 @@
 		ret = 0;
 	}
 
+	lio_dev_dbg(lio_dev, "got data from PF pkind is %d\n",
+		    lio_dev->pfvf_hsword.pkind);
+
 	return ret;
 }
 
@@ -394,6 +435,7 @@
 	if (lio_dev->default_config == NULL)
 		return -1;
 
+	lio_dev->fn_list.setup_iq_regs		= cn23xx_vf_setup_iq_regs;
 	lio_dev->fn_list.setup_mbox		= cn23xx_vf_setup_mbox;
 	lio_dev->fn_list.free_mbox		= cn23xx_vf_free_mbox;
 
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index de98fc6..4a687d8 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -141,6 +141,8 @@
 	lio_dev->io_qmask.iq64B |= ((instr_type == 64) << iq_no);
 	iq->iqcmd_64B = (instr_type == 64);
 
+	lio_dev->fn_list.setup_iq_regs(lio_dev, iq_no);
+
 	return 0;
 }
 
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 29059a5..2806c37 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -155,6 +155,8 @@ struct lio_io_enable {
 };
 
 struct lio_fn_list {
+	void (*setup_iq_regs)(struct lio_device *, uint32_t);
+
 	int (*setup_mbox)(struct lio_device *);
 	void (*free_mbox)(struct lio_device *);
 
-- 
1.8.3.1

[PATCH v2 15/46] net/liquidio: add APIs to allocate and free SC buffer pool

[Shijith Thotton]

Soft command (SC) holds device control command and related information.
SC buffer pool holds buffers which are used during soft command
allocation.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 12 ++++++++++++
 drivers/net/liquidio/lio_rxtx.c   | 21 +++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  9 +++++++++
 drivers/net/liquidio/lio_struct.h |  3 +++
 4 files changed, 45 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 5d7d5a7..a1dcdf6 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -101,6 +101,12 @@
 		return -1;
 	}
 
+	/* Initialize soft command buffer pool */
+	if (lio_setup_sc_buffer_pool(lio_dev)) {
+		lio_dev_err(lio_dev, "sc buffer pool allocation failed\n");
+		return -1;
+	}
+
 	if (lio_dev->fn_list.setup_mbox(lio_dev)) {
 		lio_dev_err(lio_dev, "Mailbox setup failed\n");
 		goto error;
@@ -141,6 +147,7 @@
 	return 0;
 
 error:
+	lio_free_sc_buffer_pool(lio_dev);
 	if (lio_dev->mbox[0])
 		lio_dev->fn_list.free_mbox(lio_dev);
 	if (lio_dev->instr_queue[0])
@@ -152,11 +159,16 @@
 static int
 lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
 {
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
+	/* lio_free_sc_buffer_pool */
+	lio_free_sc_buffer_pool(lio_dev);
+
 	rte_free(eth_dev->data->mac_addrs);
 	eth_dev->data->mac_addrs = NULL;
 
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 4a687d8..1c6ce59 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -206,3 +206,24 @@
 	lio_dev->instr_queue[0] = NULL;
 	lio_dev->num_iqs--;
 }
+
+int
+lio_setup_sc_buffer_pool(struct lio_device *lio_dev)
+{
+	char sc_pool_name[RTE_MEMPOOL_NAMESIZE];
+	uint16_t buf_size;
+
+	buf_size = LIO_SOFT_COMMAND_BUFFER_SIZE + RTE_PKTMBUF_HEADROOM;
+	snprintf(sc_pool_name, sizeof(sc_pool_name),
+		 "lio_sc_pool_%u", lio_dev->port_id);
+	lio_dev->sc_buf_pool = rte_pktmbuf_pool_create(sc_pool_name,
+						LIO_MAX_SOFT_COMMAND_BUFFERS,
+						0, 0, buf_size, SOCKET_ID_ANY);
+	return 0;
+}
+
+void
+lio_free_sc_buffer_pool(struct lio_device *lio_dev)
+{
+	rte_mempool_free(lio_dev->sc_buf_pool);
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 33f178b..b308211 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -47,6 +47,15 @@ struct lio_request_list {
 	void *buf;
 };
 
+/** The size of each buffer in soft command buffer pool */
+#define LIO_SOFT_COMMAND_BUFFER_SIZE	1536
+
+/** Maximum number of buffers to allocate into soft command buffer pool */
+#define LIO_MAX_SOFT_COMMAND_BUFFERS	255
+
+int lio_setup_sc_buffer_pool(struct lio_device *lio_dev);
+void lio_free_sc_buffer_pool(struct lio_device *lio_dev);
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 2806c37..992ad39 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -277,6 +277,9 @@ struct lio_device {
 
 	uint32_t num_iqs;
 
+	/* The pool containing pre allocated buffers used for soft commands */
+	struct rte_mempool *sc_buf_pool;
+
 	/** The input instruction queues */
 	struct lio_instr_queue *instr_queue[LIO_MAX_POSSIBLE_INSTR_QUEUES];
 
-- 
1.8.3.1

[PATCH v2 16/46] net/liquidio: add APIs to allocate and free soft command

[Shijith Thotton]

Get buffers from SC buffer pool and create soft command. Buffers are
freed to the pool once the command reaches device.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_rxtx.c   | 66 +++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   | 38 ++++++++++++++++++++++
 drivers/net/liquidio/lio_struct.h |  6 ++++
 3 files changed, 110 insertions(+)

diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 1c6ce59..cfec96d 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -227,3 +227,69 @@
 {
 	rte_mempool_free(lio_dev->sc_buf_pool);
 }
+
+struct lio_soft_command *
+lio_alloc_soft_command(struct lio_device *lio_dev, uint32_t datasize,
+		       uint32_t rdatasize, uint32_t ctxsize)
+{
+	uint32_t offset = sizeof(struct lio_soft_command);
+	struct lio_soft_command *sc;
+	struct rte_mbuf *m;
+	uint64_t dma_addr;
+
+	RTE_ASSERT((offset + datasize + rdatasize + ctxsize) <=
+		   LIO_SOFT_COMMAND_BUFFER_SIZE);
+
+	m = rte_pktmbuf_alloc(lio_dev->sc_buf_pool);
+	if (m == NULL) {
+		lio_dev_err(lio_dev, "Cannot allocate mbuf for sc\n");
+		return NULL;
+	}
+
+	/* set rte_mbuf data size and there is only 1 segment */
+	m->pkt_len = LIO_SOFT_COMMAND_BUFFER_SIZE;
+	m->data_len = LIO_SOFT_COMMAND_BUFFER_SIZE;
+
+	/* use rte_mbuf buffer for soft command */
+	sc = rte_pktmbuf_mtod(m, struct lio_soft_command *);
+	memset(sc, 0, LIO_SOFT_COMMAND_BUFFER_SIZE);
+	sc->size = LIO_SOFT_COMMAND_BUFFER_SIZE;
+	sc->dma_addr = rte_mbuf_data_dma_addr(m);
+	sc->mbuf = m;
+
+	dma_addr = sc->dma_addr;
+
+	if (ctxsize) {
+		sc->ctxptr = (uint8_t *)sc + offset;
+		sc->ctxsize = ctxsize;
+	}
+
+	/* Start data at 128 byte boundary */
+	offset = (offset + ctxsize + 127) & 0xffffff80;
+
+	if (datasize) {
+		sc->virtdptr = (uint8_t *)sc + offset;
+		sc->dmadptr = dma_addr + offset;
+		sc->datasize = datasize;
+	}
+
+	/* Start rdata at 128 byte boundary */
+	offset = (offset + datasize + 127) & 0xffffff80;
+
+	if (rdatasize) {
+		RTE_ASSERT(rdatasize >= 16);
+		sc->virtrptr = (uint8_t *)sc + offset;
+		sc->dmarptr = dma_addr + offset;
+		sc->rdatasize = rdatasize;
+		sc->status_word = (uint64_t *)((uint8_t *)(sc->virtrptr) +
+					       rdatasize - 8);
+	}
+
+	return sc;
+}
+
+void
+lio_free_soft_command(struct lio_soft_command *sc)
+{
+	rte_pktmbuf_free(sc->mbuf);
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index b308211..284cb18 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -53,9 +53,47 @@ struct lio_request_list {
 /** Maximum number of buffers to allocate into soft command buffer pool */
 #define LIO_MAX_SOFT_COMMAND_BUFFERS	255
 
+struct lio_soft_command {
+	/** Soft command buffer info. */
+	struct lio_stailq_node node;
+	uint64_t dma_addr;
+	uint32_t size;
+
+#define LIO_COMPLETION_WORD_INIT	0xffffffffffffffffULL
+	uint64_t *status_word;
+
+	/** Data buffer info */
+	void *virtdptr;
+	uint64_t dmadptr;
+	uint32_t datasize;
+
+	/** Return buffer info */
+	void *virtrptr;
+	uint64_t dmarptr;
+	uint32_t rdatasize;
+
+	/** Context buffer info */
+	void *ctxptr;
+	uint32_t ctxsize;
+
+	/** Time out and callback */
+	size_t wait_time;
+	size_t timeout;
+	uint32_t iq_no;
+	void (*callback)(uint32_t, void *);
+	void *callback_arg;
+	struct rte_mbuf *mbuf;
+};
+
 int lio_setup_sc_buffer_pool(struct lio_device *lio_dev);
 void lio_free_sc_buffer_pool(struct lio_device *lio_dev);
 
+struct lio_soft_command *
+lio_alloc_soft_command(struct lio_device *lio_dev,
+		       uint32_t datasize, uint32_t rdatasize,
+		       uint32_t ctxsize);
+void lio_free_soft_command(struct lio_soft_command *sc);
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 992ad39..45f9ac9 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -43,6 +43,12 @@
 
 #include "lio_hw_defs.h"
 
+struct lio_stailq_node {
+	STAILQ_ENTRY(lio_stailq_node) entries;
+};
+
+STAILQ_HEAD(lio_stailq_head, lio_stailq_node);
+
 struct lio_version {
 	uint16_t major;
 	uint16_t minor;
-- 
1.8.3.1

[PATCH v2 17/46] net/liquidio: add APIs for response list

[Shijith Thotton]

Add APIs to setup and process response list. Response list holds soft
commands waiting for response from device. Entries of this list are
processed to check for command response or timeout.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c |   5 ++
 drivers/net/liquidio/lio_rxtx.c   | 106 ++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  73 ++++++++++++++++++++++++++
 drivers/net/liquidio/lio_struct.h |  14 +++++
 4 files changed, 198 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index a1dcdf6..97a7369 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -107,6 +107,11 @@
 		return -1;
 	}
 
+	/* Initialize lists to manage the requests of different types that
+	 * arrive from applications for this lio device.
+	 */
+	lio_setup_response_list(lio_dev);
+
 	if (lio_dev->fn_list.setup_mbox(lio_dev)) {
 		lio_dev_err(lio_dev, "Mailbox setup failed\n");
 		goto error;
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index cfec96d..c61397f 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -293,3 +293,109 @@ struct lio_soft_command *
 {
 	rte_pktmbuf_free(sc->mbuf);
 }
+
+void
+lio_setup_response_list(struct lio_device *lio_dev)
+{
+	STAILQ_INIT(&lio_dev->response_list.head);
+	rte_spinlock_init(&lio_dev->response_list.lock);
+	rte_atomic64_set(&lio_dev->response_list.pending_req_count, 0);
+}
+
+int
+lio_process_ordered_list(struct lio_device *lio_dev)
+{
+	int resp_to_process = LIO_MAX_ORD_REQS_TO_PROCESS;
+	struct lio_response_list *ordered_sc_list;
+	struct lio_soft_command *sc;
+	int request_complete = 0;
+	uint64_t status64;
+	uint32_t status;
+
+	ordered_sc_list = &lio_dev->response_list;
+
+	do {
+		rte_spinlock_lock(&ordered_sc_list->lock);
+
+		if (STAILQ_EMPTY(&ordered_sc_list->head)) {
+			/* ordered_sc_list is empty; there is
+			 * nothing to process
+			 */
+			rte_spinlock_unlock(&ordered_sc_list->lock);
+			return -1;
+		}
+
+		sc = LIO_STQUEUE_FIRST_ENTRY(&ordered_sc_list->head,
+					     struct lio_soft_command, node);
+
+		status = LIO_REQUEST_PENDING;
+
+		/* check if octeon has finished DMA'ing a response
+		 * to where rptr is pointing to
+		 */
+		status64 = *sc->status_word;
+
+		if (status64 != LIO_COMPLETION_WORD_INIT) {
+			/* This logic ensures that all 64b have been written.
+			 * 1. check byte 0 for non-FF
+			 * 2. if non-FF, then swap result from BE to host order
+			 * 3. check byte 7 (swapped to 0) for non-FF
+			 * 4. if non-FF, use the low 32-bit status code
+			 * 5. if either byte 0 or byte 7 is FF, don't use status
+			 */
+			if ((status64 & 0xff) != 0xff) {
+				lio_swap_8B_data(&status64, 1);
+				if (((status64 & 0xff) != 0xff)) {
+					/* retrieve 16-bit firmware status */
+					status = (uint32_t)(status64 &
+							    0xffffULL);
+					if (status) {
+						status =
+						LIO_FIRMWARE_STATUS_CODE(
+									status);
+					} else {
+						/* i.e. no error */
+						status = LIO_REQUEST_DONE;
+					}
+				}
+			}
+		} else if ((sc->timeout && lio_check_timeout(lio_uptime,
+							     sc->timeout))) {
+			lio_dev_err(lio_dev,
+				    "cmd failed, timeout (%ld, %ld)\n",
+				    (long)lio_uptime, (long)sc->timeout);
+			status = LIO_REQUEST_TIMEOUT;
+		}
+
+		if (status != LIO_REQUEST_PENDING) {
+			/* we have received a response or we have timed out.
+			 * remove node from linked list
+			 */
+			STAILQ_REMOVE(&ordered_sc_list->head,
+				      &sc->node, lio_stailq_node, entries);
+			rte_atomic64_dec(
+			    &lio_dev->response_list.pending_req_count);
+			rte_spinlock_unlock(&ordered_sc_list->lock);
+
+			if (sc->callback)
+				sc->callback(status, sc->callback_arg);
+
+			request_complete++;
+		} else {
+			/* no response yet */
+			request_complete = 0;
+			rte_spinlock_unlock(&ordered_sc_list->lock);
+		}
+
+		/* If we hit the Max Ordered requests to process every loop,
+		 * we quit and let this function be invoked the next time
+		 * the poll thread runs to process the remaining requests.
+		 * This function can take up the entire CPU if there is
+		 * no upper limit to the requests processed.
+		 */
+		if (request_complete >= resp_to_process)
+			break;
+	} while (request_complete);
+
+	return 0;
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 284cb18..0e7eb6a 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -42,6 +42,14 @@
 
 #include "lio_struct.h"
 
+#define LIO_STQUEUE_FIRST_ENTRY(ptr, type, elem)	\
+	(type *)((char *)((ptr)->stqh_first) - offsetof(type, elem))
+
+#define lio_check_timeout(cur_time, chk_time) ((cur_time) > (chk_time))
+
+#define lio_uptime		\
+	(size_t)(rte_get_timer_cycles() / rte_get_timer_hz())
+
 struct lio_request_list {
 	uint32_t reqtype;
 	void *buf;
@@ -94,6 +102,71 @@ struct lio_soft_command *
 		       uint32_t ctxsize);
 void lio_free_soft_command(struct lio_soft_command *sc);
 
+/** Maximum ordered requests to process in every invocation of
+ *  lio_process_ordered_list(). The function will continue to process requests
+ *  as long as it can find one that has finished processing. If it keeps
+ *  finding requests that have completed, the function can run for ever. The
+ *  value defined here sets an upper limit on the number of requests it can
+ *  process before it returns control to the poll thread.
+ */
+#define LIO_MAX_ORD_REQS_TO_PROCESS	4096
+
+/** Error codes used in Octeon Host-Core communication.
+ *
+ *   31		16 15		0
+ *   ----------------------------
+ * |		|		|
+ *   ----------------------------
+ *   Error codes are 32-bit wide. The upper 16-bits, called Major Error Number,
+ *   are reserved to identify the group to which the error code belongs. The
+ *   lower 16-bits, called Minor Error Number, carry the actual code.
+ *
+ *   So error codes are (MAJOR NUMBER << 16)| MINOR_NUMBER.
+ */
+/** Status for a request.
+ *  If the request is successfully queued, the driver will return
+ *  a LIO_REQUEST_PENDING status. LIO_REQUEST_TIMEOUT is only returned by
+ *  the driver if the response for request failed to arrive before a
+ *  time-out period or if the request processing * got interrupted due to
+ *  a signal respectively.
+ */
+enum {
+	/** A value of 0x00000000 indicates no error i.e. success */
+	LIO_REQUEST_DONE	= 0x00000000,
+	/** (Major number: 0x0000; Minor Number: 0x0001) */
+	LIO_REQUEST_PENDING	= 0x00000001,
+	LIO_REQUEST_TIMEOUT	= 0x00000003,
+
+};
+
+/*------ Error codes used by firmware (bits 15..0 set by firmware */
+#define LIO_FIRMWARE_MAJOR_ERROR_CODE	 0x0001
+#define LIO_FIRMWARE_STATUS_CODE(status) \
+	((LIO_FIRMWARE_MAJOR_ERROR_CODE << 16) | (status))
+
+/** Initialize the response lists. The number of response lists to create is
+ *  given by count.
+ *  @param lio_dev - the lio device structure.
+ */
+void lio_setup_response_list(struct lio_device *lio_dev);
+
+/** Check the status of first entry in the ordered list. If the instruction at
+ *  that entry finished processing or has timed-out, the entry is cleaned.
+ *  @param lio_dev - the lio device structure.
+ *  @return 1 if the ordered list is empty, 0 otherwise.
+ */
+int lio_process_ordered_list(struct lio_device *lio_dev);
+
+static inline void
+lio_swap_8B_data(uint64_t *data, uint32_t blocks)
+{
+	while (blocks) {
+		*data = rte_cpu_to_be_64(*data);
+		blocks--;
+		data++;
+	}
+}
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 45f9ac9..add0e7d 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -213,6 +213,17 @@ struct lio_sriov_info {
 	uint32_t num_vfs;
 };
 
+/* Head of a response list */
+struct lio_response_list {
+	/** List structure to add delete pending entries to */
+	struct lio_stailq_head head;
+
+	/** A lock for this response list */
+	rte_spinlock_t lock;
+
+	rte_atomic64_t pending_req_count;
+};
+
 /* Structure to define the configuration attributes for each Input queue. */
 struct lio_iq_config {
 	/* Max number of IQs available */
@@ -289,6 +300,9 @@ struct lio_device {
 	/** The input instruction queues */
 	struct lio_instr_queue *instr_queue[LIO_MAX_POSSIBLE_INSTR_QUEUES];
 
+	/** The singly-linked tail queues of instruction response */
+	struct lio_response_list response_list;
+
 	struct lio_io_enable io_qmask;
 
 	struct lio_sriov_info sriov_info;
-- 
1.8.3.1

[PATCH v2 18/46] net/liquidio: add API to send packet to device

[Shijith Thotton]

Add API to send control and data packets to device. Request list keeps
track of host buffers to be freed till it reaches device.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  11 ++
 drivers/net/liquidio/lio_rxtx.c         | 180 ++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         | 291 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_struct.h       |   6 +
 4 files changed, 488 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 726ce6b..dc11406 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -76,6 +76,8 @@ enum lio_card_type {
 
 #define LIO_23XX_NAME "23xx"
 
+#define LIO_DEV_RUNNING		0xc
+
 #define LIO_NUM_DEF_TX_DESCS_CFG(cfg)					\
 		((cfg)->default_config->num_def_tx_descs)
 
@@ -93,6 +95,15 @@ enum lio_card_type {
 #define LIO_BASE_MINOR_VERSION		5
 #define LIO_BASE_MICRO_VERSION		1
 
+/** Tag types used by Octeon cores in its work. */
+enum octeon_tag_type {
+	OCTEON_ORDERED_TAG	= 0,
+	OCTEON_ATOMIC_TAG	= 1,
+};
+
+/* pre-defined host->NIC tag values */
+#define LIO_CONTROL	(0x11111110)
+
 /* Routines for reading and writing CSRs */
 #ifdef RTE_LIBRTE_LIO_DEBUG_REGS
 #define lio_write_csr(lio_dev, reg_off, value)				\
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index c61397f..7e2202c 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -207,6 +207,186 @@
 	lio_dev->num_iqs--;
 }
 
+static inline void
+lio_ring_doorbell(struct lio_device *lio_dev,
+		  struct lio_instr_queue *iq)
+{
+	if (rte_atomic64_read(&lio_dev->status) == LIO_DEV_RUNNING) {
+		rte_write32(iq->fill_cnt, iq->doorbell_reg);
+		/* make sure doorbell write goes through */
+		rte_wmb();
+		iq->fill_cnt = 0;
+	}
+}
+
+static inline void
+copy_cmd_into_iq(struct lio_instr_queue *iq, uint8_t *cmd)
+{
+	uint8_t *iqptr, cmdsize;
+
+	cmdsize = ((iq->iqcmd_64B) ? 64 : 32);
+	iqptr = iq->base_addr + (cmdsize * iq->host_write_index);
+
+	rte_memcpy(iqptr, cmd, cmdsize);
+}
+
+static inline struct lio_iq_post_status
+post_command2(struct lio_instr_queue *iq, uint8_t *cmd)
+{
+	struct lio_iq_post_status st;
+
+	st.status = LIO_IQ_SEND_OK;
+
+	/* This ensures that the read index does not wrap around to the same
+	 * position if queue gets full before Octeon could fetch any instr.
+	 */
+	if (rte_atomic64_read(&iq->instr_pending) >=
+			(int32_t)(iq->max_count - 1)) {
+		st.status = LIO_IQ_SEND_FAILED;
+		st.index = -1;
+		return st;
+	}
+
+	if (rte_atomic64_read(&iq->instr_pending) >=
+			(int32_t)(iq->max_count - 2))
+		st.status = LIO_IQ_SEND_STOP;
+
+	copy_cmd_into_iq(iq, cmd);
+
+	/* "index" is returned, host_write_index is modified. */
+	st.index = iq->host_write_index;
+	iq->host_write_index = lio_incr_index(iq->host_write_index, 1,
+					      iq->max_count);
+	iq->fill_cnt++;
+
+	/* Flush the command into memory. We need to be sure the data is in
+	 * memory before indicating that the instruction is pending.
+	 */
+	rte_wmb();
+
+	rte_atomic64_inc(&iq->instr_pending);
+
+	return st;
+}
+
+static inline void
+lio_add_to_request_list(struct lio_instr_queue *iq,
+			int idx, void *buf, int reqtype)
+{
+	iq->request_list[idx].buf = buf;
+	iq->request_list[idx].reqtype = reqtype;
+}
+
+static int
+lio_send_command(struct lio_device *lio_dev, uint32_t iq_no, void *cmd,
+		 void *buf, uint32_t datasize __rte_unused, uint32_t reqtype)
+{
+	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
+	struct lio_iq_post_status st;
+
+	rte_spinlock_lock(&iq->post_lock);
+
+	st = post_command2(iq, cmd);
+
+	if (st.status != LIO_IQ_SEND_FAILED) {
+		lio_add_to_request_list(iq, st.index, buf, reqtype);
+		lio_ring_doorbell(lio_dev, iq);
+	}
+
+	rte_spinlock_unlock(&iq->post_lock);
+
+	return st.status;
+}
+
+void
+lio_prepare_soft_command(struct lio_device *lio_dev,
+			 struct lio_soft_command *sc, uint8_t opcode,
+			 uint8_t subcode, uint32_t irh_ossp, uint64_t ossp0,
+			 uint64_t ossp1)
+{
+	struct octeon_instr_pki_ih3 *pki_ih3;
+	struct octeon_instr_ih3 *ih3;
+	struct octeon_instr_irh *irh;
+	struct octeon_instr_rdp *rdp;
+
+	RTE_ASSERT(opcode <= 15);
+	RTE_ASSERT(subcode <= 127);
+
+	ih3	  = (struct octeon_instr_ih3 *)&sc->cmd.cmd3.ih3;
+
+	ih3->pkind = lio_dev->instr_queue[sc->iq_no]->txpciq.s.pkind;
+
+	pki_ih3 = (struct octeon_instr_pki_ih3 *)&sc->cmd.cmd3.pki_ih3;
+
+	pki_ih3->w	= 1;
+	pki_ih3->raw	= 1;
+	pki_ih3->utag	= 1;
+	pki_ih3->uqpg	= lio_dev->instr_queue[sc->iq_no]->txpciq.s.use_qpg;
+	pki_ih3->utt	= 1;
+
+	pki_ih3->tag	= LIO_CONTROL;
+	pki_ih3->tagtype = OCTEON_ATOMIC_TAG;
+	pki_ih3->qpg	= lio_dev->instr_queue[sc->iq_no]->txpciq.s.qpg;
+	pki_ih3->pm	= 0x7;
+	pki_ih3->sl	= 8;
+
+	if (sc->datasize)
+		ih3->dlengsz = sc->datasize;
+
+	irh		= (struct octeon_instr_irh *)&sc->cmd.cmd3.irh;
+	irh->opcode	= opcode;
+	irh->subcode	= subcode;
+
+	/* opcode/subcode specific parameters (ossp) */
+	irh->ossp = irh_ossp;
+	sc->cmd.cmd3.ossp[0] = ossp0;
+	sc->cmd.cmd3.ossp[1] = ossp1;
+
+	if (sc->rdatasize) {
+		rdp = (struct octeon_instr_rdp *)&sc->cmd.cmd3.rdp;
+		rdp->pcie_port = lio_dev->pcie_port;
+		rdp->rlen      = sc->rdatasize;
+		irh->rflag = 1;
+		/* PKI IH3 */
+		ih3->fsz    = OCTEON_SOFT_CMD_RESP_IH3;
+	} else {
+		irh->rflag = 0;
+		/* PKI IH3 */
+		ih3->fsz    = OCTEON_PCI_CMD_O3;
+	}
+}
+
+int
+lio_send_soft_command(struct lio_device *lio_dev,
+		      struct lio_soft_command *sc)
+{
+	struct octeon_instr_ih3 *ih3;
+	struct octeon_instr_irh *irh;
+	uint32_t len = 0;
+
+	ih3 = (struct octeon_instr_ih3 *)&sc->cmd.cmd3.ih3;
+	if (ih3->dlengsz) {
+		RTE_ASSERT(sc->dmadptr);
+		sc->cmd.cmd3.dptr = sc->dmadptr;
+	}
+
+	irh = (struct octeon_instr_irh *)&sc->cmd.cmd3.irh;
+	if (irh->rflag) {
+		RTE_ASSERT(sc->dmarptr);
+		RTE_ASSERT(sc->status_word != NULL);
+		*sc->status_word = LIO_COMPLETION_WORD_INIT;
+		sc->cmd.cmd3.rptr = sc->dmarptr;
+	}
+
+	len = (uint32_t)ih3->dlengsz;
+
+	if (sc->wait_time)
+		sc->timeout = lio_uptime + sc->wait_time;
+
+	return lio_send_command(lio_dev, sc->iq_no, &sc->cmd, sc, len,
+				LIO_REQTYPE_SOFT_COMMAND);
+}
+
 int
 lio_setup_sc_buffer_pool(struct lio_device *lio_dev)
 {
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 0e7eb6a..9e2d967 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -50,11 +50,53 @@
 #define lio_uptime		\
 	(size_t)(rte_get_timer_cycles() / rte_get_timer_hz())
 
+#define LIO_IQ_SEND_OK		0
+#define LIO_IQ_SEND_STOP	1
+#define LIO_IQ_SEND_FAILED	-1
+
+/* conditions */
+#define LIO_REQTYPE_NONE		0
+#define LIO_REQTYPE_NORESP_NET		1
+#define LIO_REQTYPE_NORESP_NET_SG	2
+#define LIO_REQTYPE_SOFT_COMMAND	3
+
 struct lio_request_list {
 	uint32_t reqtype;
 	void *buf;
 };
 
+/*----------------------  INSTRUCTION FORMAT ----------------------------*/
+
+struct lio_instr3_64B {
+	/** Pointer where the input data is available. */
+	uint64_t dptr;
+
+	/** Instruction Header. */
+	uint64_t ih3;
+
+	/** Instruction Header. */
+	uint64_t pki_ih3;
+
+	/** Input Request Header. */
+	uint64_t irh;
+
+	/** opcode/subcode specific parameters */
+	uint64_t ossp[2];
+
+	/** Return Data Parameters */
+	uint64_t rdp;
+
+	/** Pointer where the response for a RAW mode packet will be written
+	 *  by Octeon.
+	 */
+	uint64_t rptr;
+
+};
+
+union lio_instr_64B {
+	struct lio_instr3_64B cmd3;
+};
+
 /** The size of each buffer in soft command buffer pool */
 #define LIO_SOFT_COMMAND_BUFFER_SIZE	1536
 
@@ -67,6 +109,9 @@ struct lio_soft_command {
 	uint64_t dma_addr;
 	uint32_t size;
 
+	/** Command and return status */
+	union lio_instr_64B cmd;
+
 #define LIO_COMPLETION_WORD_INIT	0xffffffffffffffffULL
 	uint64_t *status_word;
 
@@ -93,6 +138,230 @@ struct lio_soft_command {
 	struct rte_mbuf *mbuf;
 };
 
+struct lio_iq_post_status {
+	int status;
+	int index;
+};
+
+/*   wqe
+ *  ---------------  0
+ * |  wqe  word0-3 |
+ *  ---------------  32
+ * |    PCI IH     |
+ *  ---------------  40
+ * |     RPTR      |
+ *  ---------------  48
+ * |    PCI IRH    |
+ *  ---------------  56
+ * |    OCTEON_CMD |
+ *  ---------------  64
+ * | Addtl 8-BData |
+ * |               |
+ *  ---------------
+ */
+
+union octeon_cmd {
+	uint64_t cmd64;
+
+	struct	{
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t cmd : 5;
+
+		uint64_t more : 6; /* How many udd words follow the command */
+
+		uint64_t reserved : 29;
+
+		uint64_t param1 : 16;
+
+		uint64_t param2 : 8;
+
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+
+		uint64_t param2 : 8;
+
+		uint64_t param1 : 16;
+
+		uint64_t reserved : 29;
+
+		uint64_t more : 6;
+
+		uint64_t cmd : 5;
+
+#endif
+	} s;
+};
+
+#define OCTEON_CMD_SIZE (sizeof(union octeon_cmd))
+
+/* Instruction Header */
+struct octeon_instr_ih3 {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+
+	/** Reserved3 */
+	uint64_t reserved3 : 1;
+
+	/** Gather indicator 1=gather*/
+	uint64_t gather : 1;
+
+	/** Data length OR no. of entries in gather list */
+	uint64_t dlengsz : 14;
+
+	/** Front Data size */
+	uint64_t fsz : 6;
+
+	/** Reserved2 */
+	uint64_t reserved2 : 4;
+
+	/** PKI port kind - PKIND */
+	uint64_t pkind : 6;
+
+	/** Reserved1 */
+	uint64_t reserved1 : 32;
+
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	/** Reserved1 */
+	uint64_t reserved1 : 32;
+
+	/** PKI port kind - PKIND */
+	uint64_t pkind : 6;
+
+	/** Reserved2 */
+	uint64_t reserved2 : 4;
+
+	/** Front Data size */
+	uint64_t fsz : 6;
+
+	/** Data length OR no. of entries in gather list */
+	uint64_t dlengsz : 14;
+
+	/** Gather indicator 1=gather*/
+	uint64_t gather : 1;
+
+	/** Reserved3 */
+	uint64_t reserved3 : 1;
+
+#endif
+};
+
+/* PKI Instruction Header(PKI IH) */
+struct octeon_instr_pki_ih3 {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+
+	/** Wider bit */
+	uint64_t w : 1;
+
+	/** Raw mode indicator 1 = RAW */
+	uint64_t raw : 1;
+
+	/** Use Tag */
+	uint64_t utag : 1;
+
+	/** Use QPG */
+	uint64_t uqpg : 1;
+
+	/** Reserved2 */
+	uint64_t reserved2 : 1;
+
+	/** Parse Mode */
+	uint64_t pm : 3;
+
+	/** Skip Length */
+	uint64_t sl : 8;
+
+	/** Use Tag Type */
+	uint64_t utt : 1;
+
+	/** Tag type */
+	uint64_t tagtype : 2;
+
+	/** Reserved1 */
+	uint64_t reserved1 : 2;
+
+	/** QPG Value */
+	uint64_t qpg : 11;
+
+	/** Tag Value */
+	uint64_t tag : 32;
+
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+
+	/** Tag Value */
+	uint64_t tag : 32;
+
+	/** QPG Value */
+	uint64_t qpg : 11;
+
+	/** Reserved1 */
+	uint64_t reserved1 : 2;
+
+	/** Tag type */
+	uint64_t tagtype : 2;
+
+	/** Use Tag Type */
+	uint64_t utt : 1;
+
+	/** Skip Length */
+	uint64_t sl : 8;
+
+	/** Parse Mode */
+	uint64_t pm : 3;
+
+	/** Reserved2 */
+	uint64_t reserved2 : 1;
+
+	/** Use QPG */
+	uint64_t uqpg : 1;
+
+	/** Use Tag */
+	uint64_t utag : 1;
+
+	/** Raw mode indicator 1 = RAW */
+	uint64_t raw : 1;
+
+	/** Wider bit */
+	uint64_t w : 1;
+#endif
+};
+
+/** Input Request Header */
+struct octeon_instr_irh {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	uint64_t opcode : 4;
+	uint64_t rflag : 1;
+	uint64_t subcode : 7;
+	uint64_t vlan : 12;
+	uint64_t priority : 3;
+	uint64_t reserved : 5;
+	uint64_t ossp : 32; /* opcode/subcode specific parameters */
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	uint64_t ossp : 32; /* opcode/subcode specific parameters */
+	uint64_t reserved : 5;
+	uint64_t priority : 3;
+	uint64_t vlan : 12;
+	uint64_t subcode : 7;
+	uint64_t rflag : 1;
+	uint64_t opcode : 4;
+#endif
+};
+
+/* pkiih3 + irh + ossp[0] + ossp[1] + rdp + rptr = 40 bytes */
+#define OCTEON_SOFT_CMD_RESP_IH3	(40 + 8)
+/* pki_h3 + irh + ossp[0] + ossp[1] = 32 bytes */
+#define OCTEON_PCI_CMD_O3		(24 + 8)
+
+/** Return Data Parameters */
+struct octeon_instr_rdp {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	uint64_t reserved : 49;
+	uint64_t pcie_port : 3;
+	uint64_t rlen : 12;
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	uint64_t rlen : 12;
+	uint64_t pcie_port : 3;
+	uint64_t reserved : 49;
+#endif
+};
+
 int lio_setup_sc_buffer_pool(struct lio_device *lio_dev);
 void lio_free_sc_buffer_pool(struct lio_device *lio_dev);
 
@@ -100,6 +369,13 @@ struct lio_soft_command *
 lio_alloc_soft_command(struct lio_device *lio_dev,
 		       uint32_t datasize, uint32_t rdatasize,
 		       uint32_t ctxsize);
+void lio_prepare_soft_command(struct lio_device *lio_dev,
+			      struct lio_soft_command *sc,
+			      uint8_t opcode, uint8_t subcode,
+			      uint32_t irh_ossp, uint64_t ossp0,
+			      uint64_t ossp1);
+int lio_send_soft_command(struct lio_device *lio_dev,
+			  struct lio_soft_command *sc);
 void lio_free_soft_command(struct lio_soft_command *sc);
 
 /** Maximum ordered requests to process in every invocation of
@@ -167,6 +443,21 @@ enum {
 	}
 }
 
+/* Macro to increment index.
+ * Index is incremented by count; if the sum exceeds
+ * max, index is wrapped-around to the start.
+ */
+static inline uint32_t
+lio_incr_index(uint32_t index, uint32_t count, uint32_t max)
+{
+	if ((index + count) >= max)
+		index = index + count - max;
+	else
+		index += count;
+
+	return index;
+}
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index add0e7d..f16571b 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -288,6 +288,12 @@ struct lio_device {
 	uint16_t pf_num;
 	uint16_t vf_num;
 
+	/** This device's PCIe port used for traffic. */
+	uint16_t pcie_port;
+
+	/** The state of this device */
+	rte_atomic64_t status;
+
 	uint8_t *hw_addr;
 
 	struct lio_fn_list fn_list;
-- 
1.8.3.1

[PATCH v2 19/46] net/liquidio: add API to configure device

[Shijith Thotton]

Add API to configure device and initialize ethernet device operations.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  14 +++
 drivers/net/liquidio/lio_ethdev.c       | 169 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  36 +++++++
 drivers/net/liquidio/lio_struct.h       |  72 ++++++++++++++
 4 files changed, 291 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index dc11406..3201dc5 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -95,6 +95,8 @@ enum lio_card_type {
 #define LIO_BASE_MINOR_VERSION		5
 #define LIO_BASE_MICRO_VERSION		1
 
+#define LIO_FW_VERSION_LENGTH		32
+
 /** Tag types used by Octeon cores in its work. */
 enum octeon_tag_type {
 	OCTEON_ORDERED_TAG	= 0,
@@ -104,6 +106,18 @@ enum octeon_tag_type {
 /* pre-defined host->NIC tag values */
 #define LIO_CONTROL	(0x11111110)
 
+/* used for NIC operations */
+#define LIO_OPCODE	1
+
+/** LIO_OPCODE subcodes */
+/* This subcode is sent by core PCI driver to indicate cores are ready. */
+#define LIO_OPCODE_IF_CFG		0x09
+
+/* Interface flags communicated between host driver and core app. */
+enum lio_ifflags {
+	LIO_IFFLAG_UNICAST	= 0x10
+};
+
 /* Routines for reading and writing CSRs */
 #ifdef RTE_LIBRTE_LIO_DEBUG_REGS
 #define lio_write_csr(lio_dev, reg_off, value)				\
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 97a7369..194b096 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -41,6 +41,166 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+static uint64_t
+lio_hweight64(uint64_t w)
+{
+	uint64_t res = w - ((w >> 1) & 0x5555555555555555ul);
+
+	res =
+	    (res & 0x3333333333333333ul) + ((res >> 2) & 0x3333333333333333ul);
+	res = (res + (res >> 4)) & 0x0F0F0F0F0F0F0F0Ful;
+	res = res + (res >> 8);
+	res = res + (res >> 16);
+
+	return (res + (res >> 32)) & 0x00000000000000FFul;
+}
+
+static int lio_dev_configure(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
+	int retval, num_iqueues, num_oqueues;
+	uint8_t mac[ETHER_ADDR_LEN], i;
+	struct lio_if_cfg_resp *resp;
+	struct lio_soft_command *sc;
+	union lio_if_cfg if_cfg;
+	uint32_t resp_size;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* Re-configuring firmware not supported.
+	 * Can't change tx/rx queues per port from initial value.
+	 */
+	if (lio_dev->port_configured) {
+		if ((lio_dev->nb_rx_queues != eth_dev->data->nb_rx_queues) ||
+		    (lio_dev->nb_tx_queues != eth_dev->data->nb_tx_queues)) {
+			lio_dev_err(lio_dev,
+				    "rxq/txq re-conf not supported. Restart application with new value.\n");
+			return -ENOTSUP;
+		}
+		return 0;
+	}
+
+	lio_dev->nb_rx_queues = eth_dev->data->nb_rx_queues;
+	lio_dev->nb_tx_queues = eth_dev->data->nb_tx_queues;
+
+	resp_size = sizeof(struct lio_if_cfg_resp);
+	sc = lio_alloc_soft_command(lio_dev, 0, resp_size, 0);
+	if (sc == NULL)
+		return -ENOMEM;
+
+	resp = (struct lio_if_cfg_resp *)sc->virtrptr;
+
+	/* Firmware doesn't have capability to reconfigure the queues,
+	 * Claim all queues, and use as many required
+	 */
+	if_cfg.if_cfg64 = 0;
+	if_cfg.s.num_iqueues = lio_dev->nb_tx_queues;
+	if_cfg.s.num_oqueues = lio_dev->nb_rx_queues;
+	if_cfg.s.base_queue = 0;
+
+	if_cfg.s.gmx_port_id = lio_dev->pf_num;
+
+	lio_prepare_soft_command(lio_dev, sc, LIO_OPCODE,
+				 LIO_OPCODE_IF_CFG, 0,
+				 if_cfg.if_cfg64, 0);
+
+	/* Setting wait time in seconds */
+	sc->wait_time = LIO_MAX_CMD_TIMEOUT / 1000;
+
+	retval = lio_send_soft_command(lio_dev, sc);
+	if (retval == LIO_IQ_SEND_FAILED) {
+		lio_dev_err(lio_dev, "iq/oq config failed status: %x\n",
+			    retval);
+		/* Soft instr is freed by driver in case of failure. */
+		goto nic_config_fail;
+	}
+
+	/* Sleep on a wait queue till the cond flag indicates that the
+	 * response arrived or timed-out.
+	 */
+	while ((*sc->status_word == LIO_COMPLETION_WORD_INIT) && --timeout) {
+		lio_process_ordered_list(lio_dev);
+		rte_delay_ms(1);
+	}
+
+	retval = resp->status;
+	if (retval) {
+		lio_dev_err(lio_dev, "iq/oq config failed\n");
+		goto nic_config_fail;
+	}
+
+	lio_swap_8B_data((uint64_t *)(&resp->cfg_info),
+			 sizeof(struct octeon_if_cfg_info) >> 3);
+
+	num_iqueues = lio_hweight64(resp->cfg_info.iqmask);
+	num_oqueues = lio_hweight64(resp->cfg_info.oqmask);
+
+	if (!(num_iqueues) || !(num_oqueues)) {
+		lio_dev_err(lio_dev,
+			    "Got bad iqueues (%016lx) or oqueues (%016lx) from firmware.\n",
+			    (unsigned long)resp->cfg_info.iqmask,
+			    (unsigned long)resp->cfg_info.oqmask);
+		goto nic_config_fail;
+	}
+
+	lio_dev_dbg(lio_dev,
+		    "interface %d, iqmask %016lx, oqmask %016lx, numiqueues %d, numoqueues %d\n",
+		    eth_dev->data->port_id,
+		    (unsigned long)resp->cfg_info.iqmask,
+		    (unsigned long)resp->cfg_info.oqmask,
+		    num_iqueues, num_oqueues);
+
+	lio_dev->linfo.num_rxpciq = num_oqueues;
+	lio_dev->linfo.num_txpciq = num_iqueues;
+
+	for (i = 0; i < num_oqueues; i++) {
+		lio_dev->linfo.rxpciq[i].rxpciq64 =
+		    resp->cfg_info.linfo.rxpciq[i].rxpciq64;
+		lio_dev_dbg(lio_dev, "index %d OQ %d\n",
+			    i, lio_dev->linfo.rxpciq[i].s.q_no);
+	}
+
+	for (i = 0; i < num_iqueues; i++) {
+		lio_dev->linfo.txpciq[i].txpciq64 =
+		    resp->cfg_info.linfo.txpciq[i].txpciq64;
+		lio_dev_dbg(lio_dev, "index %d IQ %d\n",
+			    i, lio_dev->linfo.txpciq[i].s.q_no);
+	}
+
+	lio_dev->linfo.hw_addr = resp->cfg_info.linfo.hw_addr;
+	lio_dev->linfo.gmxport = resp->cfg_info.linfo.gmxport;
+	lio_dev->linfo.link.link_status64 =
+			resp->cfg_info.linfo.link.link_status64;
+
+	/* 64-bit swap required on LE machines */
+	lio_swap_8B_data(&lio_dev->linfo.hw_addr, 1);
+	for (i = 0; i < ETHER_ADDR_LEN; i++)
+		mac[i] = *((uint8_t *)(((uint8_t *)&lio_dev->linfo.hw_addr) +
+				       2 + i));
+
+	/* Copy the permanent MAC address */
+	ether_addr_copy((struct ether_addr *)mac, &eth_dev->data->mac_addrs[0]);
+
+	lio_dev->port_configured = 1;
+
+	lio_free_soft_command(sc);
+
+	return 0;
+
+nic_config_fail:
+	lio_dev_err(lio_dev, "Failed retval %d\n", retval);
+	lio_free_soft_command(sc);
+	lio_free_instr_queue0(lio_dev);
+
+	return -ENODEV;
+}
+
+/* Define our ethernet definitions */
+static const struct eth_dev_ops liovf_eth_dev_ops = {
+	.dev_configure		= lio_dev_configure,
+};
+
 static void
 lio_check_pf_hs_response(void *lio_dev)
 {
@@ -215,13 +375,22 @@
 		return -EINVAL;
 	}
 
+	eth_dev->dev_ops = &liovf_eth_dev_ops;
 	eth_dev->data->mac_addrs = rte_zmalloc("lio", ETHER_ADDR_LEN, 0);
 	if (eth_dev->data->mac_addrs == NULL) {
 		lio_dev_err(lio_dev,
 			    "MAC addresses memory allocation failed\n");
+		eth_dev->dev_ops = NULL;
 		return -ENOMEM;
 	}
 
+	rte_atomic64_set(&lio_dev->status, LIO_DEV_RUNNING);
+	rte_wmb();
+
+	lio_dev->port_configured = 0;
+	/* Always allow unicast packets */
+	lio_dev->ifflags |= LIO_IFFLAG_UNICAST;
+
 	return 0;
 }
 
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 76c9072..22e3d83 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -36,5 +36,41 @@
 
 #include <stdint.h>
 
+#include "lio_struct.h"
+
+#define LIO_MAX_CMD_TIMEOUT     10000 /* 10000ms (10s) */
+
 #define LIO_DEV(_eth_dev)		((_eth_dev)->data->dev_private)
+
+struct octeon_if_cfg_info {
+	uint64_t iqmask;	/** mask for IQs enabled for the port */
+	uint64_t oqmask;	/** mask for OQs enabled for the port */
+	struct octeon_link_info linfo; /** initial link information */
+	char lio_firmware_version[LIO_FW_VERSION_LENGTH];
+};
+
+union lio_if_cfg {
+	uint64_t if_cfg64;
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t base_queue : 16;
+		uint64_t num_iqueues : 16;
+		uint64_t num_oqueues : 16;
+		uint64_t gmx_port_id : 8;
+		uint64_t vf_id : 8;
+#else
+		uint64_t vf_id : 8;
+		uint64_t gmx_port_id : 8;
+		uint64_t num_oqueues : 16;
+		uint64_t num_iqueues : 16;
+		uint64_t base_queue : 16;
+#endif
+	} s;
+};
+
+struct lio_if_cfg_resp {
+	uint64_t rh;
+	struct octeon_if_cfg_info cfg_info;
+	uint64_t status;
+};
 #endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index f16571b..48c4cae 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -274,6 +274,75 @@ struct lio_config {
 	int def_rx_buf_size;
 };
 
+/** Status of a RGMII Link on Octeon as seen by core driver. */
+union octeon_link_status {
+	uint64_t link_status64;
+
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t duplex : 8;
+		uint64_t mtu : 16;
+		uint64_t speed : 16;
+		uint64_t link_up : 1;
+		uint64_t autoneg : 1;
+		uint64_t if_mode : 5;
+		uint64_t pause : 1;
+		uint64_t flashing : 1;
+		uint64_t reserved : 15;
+#else
+		uint64_t reserved : 15;
+		uint64_t flashing : 1;
+		uint64_t pause : 1;
+		uint64_t if_mode : 5;
+		uint64_t autoneg : 1;
+		uint64_t link_up : 1;
+		uint64_t speed : 16;
+		uint64_t mtu : 16;
+		uint64_t duplex : 8;
+#endif
+	} s;
+};
+
+/** The rxpciq info passed to host from the firmware */
+union octeon_rxpciq {
+	uint64_t rxpciq64;
+
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t q_no : 8;
+		uint64_t reserved : 56;
+#else
+		uint64_t reserved : 56;
+		uint64_t q_no : 8;
+#endif
+	} s;
+};
+
+/** Information for a OCTEON ethernet interface shared between core & host. */
+struct octeon_link_info {
+	union octeon_link_status link;
+	uint64_t hw_addr;
+
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	uint64_t gmxport : 16;
+	uint64_t macaddr_is_admin_assigned : 1;
+	uint64_t vlan_is_admin_assigned : 1;
+	uint64_t rsvd : 30;
+	uint64_t num_txpciq : 8;
+	uint64_t num_rxpciq : 8;
+#else
+	uint64_t num_rxpciq : 8;
+	uint64_t num_txpciq : 8;
+	uint64_t rsvd : 30;
+	uint64_t vlan_is_admin_assigned : 1;
+	uint64_t macaddr_is_admin_assigned : 1;
+	uint64_t gmxport : 16;
+#endif
+
+	union octeon_txpciq txpciq[LIO_MAX_IOQS_PER_IF];
+	union octeon_rxpciq rxpciq[LIO_MAX_IOQS_PER_IF];
+};
+
 /* -----------------------  THE LIO DEVICE  --------------------------- */
 /** The lio device.
  *  Each lio device has this structure to represent all its
@@ -294,6 +363,8 @@ struct lio_device {
 	/** The state of this device */
 	rte_atomic64_t status;
 
+	struct octeon_link_info linfo;
+
 	uint8_t *hw_addr;
 
 	struct lio_fn_list fn_list;
@@ -324,6 +395,7 @@ struct lio_device {
 
 	struct rte_eth_dev      *eth_dev;
 
+	uint64_t ifflags;
 	uint8_t max_rx_queues;
 	uint8_t max_tx_queues;
 	uint8_t nb_rx_queues;
-- 
1.8.3.1

[PATCH v2 20/46] net/liquidio: add API to setup Rx queue

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |   3 +
 drivers/net/liquidio/lio_ethdev.c       |  67 +++++++++++++
 drivers/net/liquidio/lio_rxtx.c         | 168 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         |  56 +++++++++++
 drivers/net/liquidio/lio_struct.h       | 149 ++++++++++++++++++++++++++++
 5 files changed, 443 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 3201dc5..35d59fd 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -78,6 +78,8 @@ enum lio_card_type {
 
 #define LIO_DEV_RUNNING		0xc
 
+#define LIO_OQ_REFILL_THRESHOLD_CFG(cfg)				\
+		((cfg)->default_config->oq.refill_threshold)
 #define LIO_NUM_DEF_TX_DESCS_CFG(cfg)					\
 		((cfg)->default_config->num_def_tx_descs)
 
@@ -89,6 +91,7 @@ enum lio_card_type {
 #define LIO_MAX_INSTR_QUEUES(lio_dev)		CN23XX_MAX_RINGS_PER_VF
 
 #define LIO_MAX_POSSIBLE_INSTR_QUEUES		CN23XX_MAX_INPUT_QUEUES
+#define LIO_MAX_POSSIBLE_OUTPUT_QUEUES		CN23XX_MAX_OUTPUT_QUEUES
 
 #define LIO_DEVICE_NAME_LEN		32
 #define LIO_BASE_MAJOR_VERSION		1
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 194b096..a93fa4a 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -55,6 +55,72 @@
 	return (res + (res >> 32)) & 0x00000000000000FFul;
 }
 
+/**
+ * Setup our receive queue/ringbuffer. This is the
+ * queue the Octeon uses to send us packets and
+ * responses. We are given a memory pool for our
+ * packet buffers that are used to populate the receive
+ * queue.
+ *
+ * @param eth_dev
+ *    Pointer to the structure rte_eth_dev
+ * @param q_no
+ *    Queue number
+ * @param num_rx_descs
+ *    Number of entries in the queue
+ * @param socket_id
+ *    Where to allocate memory
+ * @param rx_conf
+ *    Pointer to the struction rte_eth_rxconf
+ * @param mp
+ *    Pointer to the packet pool
+ *
+ * @return
+ *    - On success, return 0
+ *    - On failure, return -1
+ */
+static int
+lio_dev_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t q_no,
+		       uint16_t num_rx_descs, unsigned int socket_id,
+		       const struct rte_eth_rxconf *rx_conf __rte_unused,
+		       struct rte_mempool *mp)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct rte_pktmbuf_pool_private *mbp_priv;
+	uint32_t fw_mapped_oq;
+	uint16_t buf_size;
+
+	if (q_no >= lio_dev->nb_rx_queues) {
+		lio_dev_err(lio_dev, "Invalid rx queue number %u\n", q_no);
+		return -EINVAL;
+	}
+
+	lio_dev_dbg(lio_dev, "setting up rx queue %u\n", q_no);
+
+	fw_mapped_oq = lio_dev->linfo.rxpciq[q_no].s.q_no;
+
+	if ((lio_dev->droq[fw_mapped_oq]) &&
+	    (num_rx_descs != lio_dev->droq[fw_mapped_oq]->max_count)) {
+		lio_dev_err(lio_dev,
+			    "Reconfiguring Rx descs not supported. Configure descs to same value %u or restart application\n",
+			    lio_dev->droq[fw_mapped_oq]->max_count);
+		return -ENOTSUP;
+	}
+
+	mbp_priv = rte_mempool_get_priv(mp);
+	buf_size = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM;
+
+	if (lio_setup_droq(lio_dev, fw_mapped_oq, num_rx_descs, buf_size, mp,
+			   socket_id)) {
+		lio_dev_err(lio_dev, "droq allocation failed\n");
+		return -1;
+	}
+
+	eth_dev->data->rx_queues[q_no] = lio_dev->droq[fw_mapped_oq];
+
+	return 0;
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -199,6 +265,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 /* Define our ethernet definitions */
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
+	.rx_queue_setup		= lio_dev_rx_queue_setup,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 7e2202c..942fe9b 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -63,6 +63,174 @@
 }
 
 /**
+ *  Frees the space for descriptor ring for the droq.
+ *
+ *  @param lio_dev	- pointer to the lio device structure
+ *  @param q_no		- droq no.
+ */
+static void
+lio_delete_droq(struct lio_device *lio_dev, uint32_t q_no)
+{
+	struct lio_droq *droq = lio_dev->droq[q_no];
+
+	lio_dev_dbg(lio_dev, "OQ[%d]\n", q_no);
+
+	rte_free(droq->recv_buf_list);
+	droq->recv_buf_list = NULL;
+	lio_dma_zone_free(lio_dev, droq->info_mz);
+	lio_dma_zone_free(lio_dev, droq->desc_ring_mz);
+
+	memset(droq, 0, LIO_DROQ_SIZE);
+}
+
+static void *
+lio_alloc_info_buffer(struct lio_device *lio_dev,
+		      struct lio_droq *droq, unsigned int socket_id)
+{
+	droq->info_mz = rte_eth_dma_zone_reserve(lio_dev->eth_dev,
+						 "info_list", droq->q_no,
+						 (droq->max_count *
+							LIO_DROQ_INFO_SIZE),
+						 RTE_CACHE_LINE_SIZE,
+						 socket_id);
+
+	if (droq->info_mz == NULL)
+		return NULL;
+
+	droq->info_list_dma = droq->info_mz->phys_addr;
+	droq->info_alloc_size = droq->info_mz->len;
+	droq->info_base_addr = (size_t)droq->info_mz->addr;
+
+	return droq->info_mz->addr;
+}
+
+/**
+ *  Allocates space for the descriptor ring for the droq and
+ *  sets the base addr, num desc etc in Octeon registers.
+ *
+ * @param lio_dev	- pointer to the lio device structure
+ * @param q_no		- droq no.
+ * @param app_ctx	- pointer to application context
+ * @return Success: 0	Failure: -1
+ */
+static int
+lio_init_droq(struct lio_device *lio_dev, uint32_t q_no,
+	      uint32_t num_descs, uint32_t desc_size,
+	      struct rte_mempool *mpool, unsigned int socket_id)
+{
+	uint32_t c_refill_threshold;
+	uint32_t desc_ring_size;
+	struct lio_droq *droq;
+
+	lio_dev_dbg(lio_dev, "OQ[%d]\n", q_no);
+
+	droq = lio_dev->droq[q_no];
+	droq->lio_dev = lio_dev;
+	droq->q_no = q_no;
+	droq->mpool = mpool;
+
+	c_refill_threshold = LIO_OQ_REFILL_THRESHOLD_CFG(lio_dev);
+
+	droq->max_count = num_descs;
+	droq->buffer_size = desc_size;
+
+	desc_ring_size = droq->max_count * LIO_DROQ_DESC_SIZE;
+	droq->desc_ring_mz = rte_eth_dma_zone_reserve(lio_dev->eth_dev,
+						      "droq", q_no,
+						      desc_ring_size,
+						      RTE_CACHE_LINE_SIZE,
+						      socket_id);
+
+	if (droq->desc_ring_mz == NULL) {
+		lio_dev_err(lio_dev,
+			    "Output queue %d ring alloc failed\n", q_no);
+		return -1;
+	}
+
+	droq->desc_ring_dma = droq->desc_ring_mz->phys_addr;
+	droq->desc_ring = (struct lio_droq_desc *)droq->desc_ring_mz->addr;
+
+	lio_dev_dbg(lio_dev, "droq[%d]: desc_ring: virt: 0x%p, dma: %lx\n",
+		    q_no, droq->desc_ring, (unsigned long)droq->desc_ring_dma);
+	lio_dev_dbg(lio_dev, "droq[%d]: num_desc: %d\n", q_no,
+		    droq->max_count);
+
+	droq->info_list = lio_alloc_info_buffer(lio_dev, droq, socket_id);
+	if (droq->info_list == NULL) {
+		lio_dev_err(lio_dev, "Cannot allocate memory for info list.\n");
+		goto init_droq_fail;
+	}
+
+	droq->recv_buf_list = rte_zmalloc_socket("recv_buf_list",
+						 (droq->max_count *
+							LIO_DROQ_RECVBUF_SIZE),
+						 RTE_CACHE_LINE_SIZE,
+						 socket_id);
+	if (droq->recv_buf_list == NULL) {
+		lio_dev_err(lio_dev,
+			    "Output queue recv buf list alloc failed\n");
+		goto init_droq_fail;
+	}
+
+	droq->refill_threshold = c_refill_threshold;
+
+	rte_spinlock_init(&droq->lock);
+
+	lio_dev->io_qmask.oq |= (1ULL << q_no);
+
+	return 0;
+
+init_droq_fail:
+	lio_delete_droq(lio_dev, q_no);
+
+	return -1;
+}
+
+int
+lio_setup_droq(struct lio_device *lio_dev, int oq_no, int num_descs,
+	       int desc_size, struct rte_mempool *mpool, unsigned int socket_id)
+{
+	struct lio_droq *droq;
+
+	PMD_INIT_FUNC_TRACE();
+
+	if (lio_dev->droq[oq_no]) {
+		lio_dev_dbg(lio_dev, "Droq %d in use\n", oq_no);
+		return 0;
+	}
+
+	/* Allocate the DS for the new droq. */
+	droq = rte_zmalloc_socket("ethdev RX queue", sizeof(*droq),
+				  RTE_CACHE_LINE_SIZE, socket_id);
+	if (droq == NULL)
+		return -ENOMEM;
+
+	lio_dev->droq[oq_no] = droq;
+
+	/* Initialize the Droq */
+	if (lio_init_droq(lio_dev, oq_no, num_descs, desc_size, mpool,
+			  socket_id)) {
+		lio_dev_err(lio_dev, "Droq[%u] Initialization Failed\n", oq_no);
+		rte_free(lio_dev->droq[oq_no]);
+		lio_dev->droq[oq_no] = NULL;
+		return -ENOMEM;
+	}
+
+	lio_dev->num_oqs++;
+
+	lio_dev_dbg(lio_dev, "Total number of OQ: %d\n", lio_dev->num_oqs);
+
+	/* Send credit for octeon output queues. credits are always
+	 * sent after the output queue is enabled.
+	 */
+	rte_write32(lio_dev->droq[oq_no]->max_count,
+		    lio_dev->droq[oq_no]->pkts_credit_reg);
+	rte_wmb();
+
+	return 0;
+}
+
+/**
  *  lio_init_instr_queue()
  *  @param lio_dev	- pointer to the lio device structure.
  *  @param txpciq	- queue to be initialized.
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 9e2d967..05f4704 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -50,6 +50,58 @@
 #define lio_uptime		\
 	(size_t)(rte_get_timer_cycles() / rte_get_timer_hz())
 
+/** Descriptor format.
+ *  The descriptor ring is made of descriptors which have 2 64-bit values:
+ *  -# Physical (bus) address of the data buffer.
+ *  -# Physical (bus) address of a lio_droq_info structure.
+ *  The device DMA's incoming packets and its information at the address
+ *  given by these descriptor fields.
+ */
+struct lio_droq_desc {
+	/** The buffer pointer */
+	uint64_t buffer_ptr;
+
+	/** The Info pointer */
+	uint64_t info_ptr;
+};
+
+#define LIO_DROQ_DESC_SIZE	(sizeof(struct lio_droq_desc))
+
+/** Information about packet DMA'ed by Octeon.
+ *  The format of the information available at Info Pointer after Octeon
+ *  has posted a packet. Not all descriptors have valid information. Only
+ *  the Info field of the first descriptor for a packet has information
+ *  about the packet.
+ */
+struct lio_droq_info {
+	/** The Output Receive Header. */
+	union octeon_rh rh;
+
+	/** The Length of the packet. */
+	uint64_t length;
+};
+
+#define LIO_DROQ_INFO_SIZE	(sizeof(struct lio_droq_info))
+
+/** Pointer to data buffer.
+ *  Driver keeps a pointer to the data buffer that it made available to
+ *  the Octeon device. Since the descriptor ring keeps physical (bus)
+ *  addresses, this field is required for the driver to keep track of
+ *  the virtual address pointers.
+ */
+struct lio_recv_buffer {
+	/** Packet buffer, including meta data. */
+	void *buffer;
+
+	/** Data in the packet buffer. */
+	uint8_t *data;
+
+};
+
+#define LIO_DROQ_RECVBUF_SIZE	(sizeof(struct lio_recv_buffer))
+
+#define LIO_DROQ_SIZE		(sizeof(struct lio_droq))
+
 #define LIO_IQ_SEND_OK		0
 #define LIO_IQ_SEND_STOP	1
 #define LIO_IQ_SEND_FAILED	-1
@@ -458,6 +510,10 @@ enum {
 	return index;
 }
 
+int lio_setup_droq(struct lio_device *lio_dev, int q_no, int num_descs,
+		   int desc_size, struct rte_mempool *mpool,
+		   unsigned int socket_id);
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 48c4cae..3df79e1 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -56,6 +56,150 @@ struct lio_version {
 	uint16_t reserved;
 };
 
+/** The Descriptor Ring Output Queue structure.
+ *  This structure has all the information required to implement a
+ *  DROQ.
+ */
+struct lio_droq {
+	/** A spinlock to protect access to this ring. */
+	rte_spinlock_t lock;
+
+	uint32_t q_no;
+
+	uint32_t pkt_count;
+
+	struct lio_device *lio_dev;
+
+	/** The 8B aligned descriptor ring starts at this address. */
+	struct lio_droq_desc *desc_ring;
+
+	/** Index in the ring where the driver should read the next packet */
+	uint32_t read_idx;
+
+	/** Index in the ring where Octeon will write the next packet */
+	uint32_t write_idx;
+
+	/** Index in the ring where the driver will refill the descriptor's
+	 * buffer
+	 */
+	uint32_t refill_idx;
+
+	/** Packets pending to be processed */
+	rte_atomic64_t pkts_pending;
+
+	/** Number of  descriptors in this ring. */
+	uint32_t max_count;
+
+	/** The number of descriptors pending refill. */
+	uint32_t refill_count;
+
+	uint32_t refill_threshold;
+
+	/** The 8B aligned info ptrs begin from this address. */
+	struct lio_droq_info *info_list;
+
+	/** The receive buffer list. This list has the virtual addresses of the
+	 *  buffers.
+	 */
+	struct lio_recv_buffer *recv_buf_list;
+
+	/** The size of each buffer pointed by the buffer pointer. */
+	uint32_t buffer_size;
+
+	/** Pointer to the mapped packet credit register.
+	 *  Host writes number of info/buffer ptrs available to this register
+	 */
+	void *pkts_credit_reg;
+
+	/** Pointer to the mapped packet sent register.
+	 *  Octeon writes the number of packets DMA'ed to host memory
+	 *  in this register.
+	 */
+	void *pkts_sent_reg;
+
+	/** DMA mapped address of the DROQ descriptor ring. */
+	size_t desc_ring_dma;
+
+	/** Info ptr list are allocated at this virtual address. */
+	size_t info_base_addr;
+
+	/** DMA mapped address of the info list */
+	size_t info_list_dma;
+
+	/** Allocated size of info list. */
+	uint32_t info_alloc_size;
+
+	/** Memory zone **/
+	const struct rte_memzone *desc_ring_mz;
+	const struct rte_memzone *info_mz;
+	struct rte_mempool *mpool;
+};
+
+/** Receive Header */
+union octeon_rh {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	uint64_t rh64;
+	struct	{
+		uint64_t opcode : 4;
+		uint64_t subcode : 8;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t reserved : 17;
+		uint64_t ossp : 32; /** opcode/subcode specific parameters */
+	} r;
+	struct	{
+		uint64_t opcode : 4;
+		uint64_t subcode : 8;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t extra : 28;
+		uint64_t vlan : 12;
+		uint64_t priority : 3;
+		uint64_t csum_verified : 3; /** checksum verified. */
+		uint64_t has_hwtstamp : 1; /** Has hardware timestamp.1 = yes.*/
+		uint64_t encap_on : 1;
+		uint64_t has_hash : 1; /** Has hash (rth or rss). 1 = yes. */
+	} r_dh;
+	struct {
+		uint64_t opcode : 4;
+		uint64_t subcode : 8;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t reserved : 8;
+		uint64_t extra : 25;
+		uint64_t gmxport : 16;
+	} r_nic_info;
+#else
+	uint64_t rh64;
+	struct {
+		uint64_t ossp : 32; /** opcode/subcode specific parameters */
+		uint64_t reserved : 17;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t subcode : 8;
+		uint64_t opcode : 4;
+	} r;
+	struct {
+		uint64_t has_hash : 1; /** Has hash (rth or rss). 1 = yes. */
+		uint64_t encap_on : 1;
+		uint64_t has_hwtstamp : 1;  /** 1 = has hwtstamp */
+		uint64_t csum_verified : 3; /** checksum verified. */
+		uint64_t priority : 3;
+		uint64_t vlan : 12;
+		uint64_t extra : 28;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t subcode : 8;
+		uint64_t opcode : 4;
+	} r_dh;
+	struct {
+		uint64_t gmxport : 16;
+		uint64_t extra : 25;
+		uint64_t reserved : 8;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t subcode : 8;
+		uint64_t opcode : 4;
+	} r_nic_info;
+#endif
+};
+
+#define OCTEON_RH_SIZE (sizeof(union octeon_rh))
+
 /** The txpciq info passed to host from the firmware */
 union octeon_txpciq {
 	uint64_t txpciq64;
@@ -380,6 +524,11 @@ struct lio_device {
 	/** The singly-linked tail queues of instruction response */
 	struct lio_response_list response_list;
 
+	uint32_t num_oqs;
+
+	/** The DROQ output queues  */
+	struct lio_droq *droq[LIO_MAX_POSSIBLE_OUTPUT_QUEUES];
+
 	struct lio_io_enable io_qmask;
 
 	struct lio_sriov_info sriov_info;
-- 
1.8.3.1

[PATCH v2 21/46] net/liquidio: initialize Rx queue

[Shijith Thotton]

Initialize Rx queue registers and allocate packet buffers for Rx queue.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 22 ++++++++
 drivers/net/liquidio/base/lio_hw_defs.h |  2 +
 drivers/net/liquidio/lio_rxtx.c         | 96 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         | 20 +++++++
 drivers/net/liquidio/lio_struct.h       |  1 +
 5 files changed, 141 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index 181f830..44d90c0 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -233,6 +233,27 @@
 }
 
 static void
+cn23xx_vf_setup_oq_regs(struct lio_device *lio_dev, uint32_t oq_no)
+{
+	struct lio_droq *droq = lio_dev->droq[oq_no];
+
+	PMD_INIT_FUNC_TRACE();
+
+	lio_write_csr64(lio_dev, CN23XX_SLI_OQ_BASE_ADDR64(oq_no),
+			droq->desc_ring_dma);
+	lio_write_csr(lio_dev, CN23XX_SLI_OQ_SIZE(oq_no), droq->max_count);
+
+	lio_write_csr(lio_dev, CN23XX_SLI_OQ_BUFF_INFO_SIZE(oq_no),
+		      (droq->buffer_size | (OCTEON_RH_SIZE << 16)));
+
+	/* Get the mapped address of the pkt_sent and pkts_credit regs */
+	droq->pkts_sent_reg = (uint8_t *)lio_dev->hw_addr +
+					CN23XX_SLI_OQ_PKTS_SENT(oq_no);
+	droq->pkts_credit_reg = (uint8_t *)lio_dev->hw_addr +
+					CN23XX_SLI_OQ_PKTS_CREDIT(oq_no);
+}
+
+static void
 cn23xx_vf_free_mbox(struct lio_device *lio_dev)
 {
 	PMD_INIT_FUNC_TRACE();
@@ -436,6 +457,7 @@
 		return -1;
 
 	lio_dev->fn_list.setup_iq_regs		= cn23xx_vf_setup_iq_regs;
+	lio_dev->fn_list.setup_oq_regs		= cn23xx_vf_setup_oq_regs;
 	lio_dev->fn_list.setup_mbox		= cn23xx_vf_setup_mbox;
 	lio_dev->fn_list.free_mbox		= cn23xx_vf_free_mbox;
 
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 35d59fd..4271730 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -116,6 +116,8 @@ enum octeon_tag_type {
 /* This subcode is sent by core PCI driver to indicate cores are ready. */
 #define LIO_OPCODE_IF_CFG		0x09
 
+#define LIO_MAX_RX_PKTLEN		(64 * 1024)
+
 /* Interface flags communicated between host driver and core app. */
 enum lio_ifflags {
 	LIO_IFFLAG_UNICAST	= 0x10
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 942fe9b..9948023 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -41,6 +41,96 @@
 #include "lio_rxtx.h"
 
 static void
+lio_droq_compute_max_packet_bufs(struct lio_droq *droq)
+{
+	uint32_t count = 0;
+
+	do {
+		count += droq->buffer_size;
+	} while (count < LIO_MAX_RX_PKTLEN);
+}
+
+static void
+lio_droq_reset_indices(struct lio_droq *droq)
+{
+	droq->read_idx	= 0;
+	droq->write_idx	= 0;
+	droq->refill_idx = 0;
+	droq->refill_count = 0;
+	rte_atomic64_set(&droq->pkts_pending, 0);
+}
+
+static void
+lio_droq_destroy_ring_buffers(struct lio_droq *droq)
+{
+	uint32_t i;
+
+	for (i = 0; i < droq->max_count; i++) {
+		if (droq->recv_buf_list[i].buffer) {
+			rte_pktmbuf_free((struct rte_mbuf *)
+					 droq->recv_buf_list[i].buffer);
+			droq->recv_buf_list[i].buffer = NULL;
+		}
+	}
+
+	lio_droq_reset_indices(droq);
+}
+
+static void *
+lio_recv_buffer_alloc(struct lio_device *lio_dev, int q_no)
+{
+	struct lio_droq *droq = lio_dev->droq[q_no];
+	struct rte_mempool *mpool = droq->mpool;
+	struct rte_mbuf *m;
+
+	m = rte_pktmbuf_alloc(mpool);
+	if (m == NULL) {
+		lio_dev_err(lio_dev, "Cannot allocate\n");
+		return NULL;
+	}
+
+	rte_mbuf_refcnt_set(m, 1);
+	m->next = NULL;
+	m->data_off = RTE_PKTMBUF_HEADROOM;
+	m->nb_segs = 1;
+	m->pool = mpool;
+
+	return m;
+}
+
+static int
+lio_droq_setup_ring_buffers(struct lio_device *lio_dev,
+			    struct lio_droq *droq)
+{
+	struct lio_droq_desc *desc_ring = droq->desc_ring;
+	uint32_t i;
+	void *buf;
+
+	for (i = 0; i < droq->max_count; i++) {
+		buf = lio_recv_buffer_alloc(lio_dev, droq->q_no);
+		if (buf == NULL) {
+			lio_dev_err(lio_dev, "buffer alloc failed\n");
+			lio_droq_destroy_ring_buffers(droq);
+			return -ENOMEM;
+		}
+
+		droq->recv_buf_list[i].buffer = buf;
+		droq->info_list[i].length = 0;
+
+		/* map ring buffers into memory */
+		desc_ring[i].info_ptr = lio_map_ring_info(droq, i);
+		desc_ring[i].buffer_ptr =
+			lio_map_ring(droq->recv_buf_list[i].buffer);
+	}
+
+	lio_droq_reset_indices(droq);
+
+	lio_droq_compute_max_packet_bufs(droq);
+
+	return 0;
+}
+
+static void
 lio_dma_zone_free(struct lio_device *lio_dev, const struct rte_memzone *mz)
 {
 	const struct rte_memzone *mz_tmp;
@@ -75,6 +165,7 @@
 
 	lio_dev_dbg(lio_dev, "OQ[%d]\n", q_no);
 
+	lio_droq_destroy_ring_buffers(droq);
 	rte_free(droq->recv_buf_list);
 	droq->recv_buf_list = NULL;
 	lio_dma_zone_free(lio_dev, droq->info_mz);
@@ -172,10 +263,15 @@
 		goto init_droq_fail;
 	}
 
+	if (lio_droq_setup_ring_buffers(lio_dev, droq))
+		goto init_droq_fail;
+
 	droq->refill_threshold = c_refill_threshold;
 
 	rte_spinlock_init(&droq->lock);
 
+	lio_dev->fn_list.setup_oq_regs(lio_dev, q_no);
+
 	lio_dev->io_qmask.oq |= (1ULL << q_no);
 
 	return 0;
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 05f4704..fc623ad 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -495,6 +495,26 @@ enum {
 	}
 }
 
+static inline uint64_t
+lio_map_ring(void *buf)
+{
+	phys_addr_t dma_addr;
+
+	dma_addr = rte_mbuf_data_dma_addr_default(((struct rte_mbuf *)buf));
+
+	return (uint64_t)dma_addr;
+}
+
+static inline uint64_t
+lio_map_ring_info(struct lio_droq *droq, uint32_t i)
+{
+	phys_addr_t dma_addr;
+
+	dma_addr = droq->info_list_dma + (i * LIO_DROQ_INFO_SIZE);
+
+	return (uint64_t)dma_addr;
+}
+
 /* Macro to increment index.
  * Index is incremented by count; if the sum exceeds
  * max, index is wrapped-around to the start.
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 3df79e1..7a7a4a6 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -306,6 +306,7 @@ struct lio_io_enable {
 
 struct lio_fn_list {
 	void (*setup_iq_regs)(struct lio_device *, uint32_t);
+	void (*setup_oq_regs)(struct lio_device *, uint32_t);
 
 	int (*setup_mbox)(struct lio_device *);
 	void (*free_mbox)(struct lio_device *);
-- 
1.8.3.1

[PATCH v2 22/46] net/liquidio: add Rx data path

[Shijith Thotton]

Add APIs to receive packets and re-fill ring buffers.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  12 +
 drivers/net/liquidio/lio_ethdev.c       |   5 +
 drivers/net/liquidio/lio_rxtx.c         | 380 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         |  13 ++
 4 files changed, 410 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 4271730..2db7085 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -112,12 +112,24 @@ enum octeon_tag_type {
 /* used for NIC operations */
 #define LIO_OPCODE	1
 
+/* Subcodes are used by host driver/apps to identify the sub-operation
+ * for the core. They only need to by unique for a given subsystem.
+ */
+#define LIO_OPCODE_SUBCODE(op, sub)		\
+		((((op) & 0x0f) << 8) | ((sub) & 0x7f))
+
 /** LIO_OPCODE subcodes */
 /* This subcode is sent by core PCI driver to indicate cores are ready. */
+#define LIO_OPCODE_NW_DATA		0x02 /* network packet data */
 #define LIO_OPCODE_IF_CFG		0x09
 
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
 
+/* RX(packets coming from wire) Checksum verification flags */
+/* TCP/UDP csum */
+#define LIO_L4_CSUM_VERIFIED		0x1
+#define LIO_IP_CSUM_VERIFIED		0x2
+
 /* Interface flags communicated between host driver and core app. */
 enum lio_ifflags {
 	LIO_IFFLAG_UNICAST	= 0x10
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index a93fa4a..ebfdf7a 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -404,6 +404,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	rte_free(eth_dev->data->mac_addrs);
 	eth_dev->data->mac_addrs = NULL;
 
+	eth_dev->rx_pkt_burst = NULL;
+
 	return 0;
 }
 
@@ -415,6 +417,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 
 	PMD_INIT_FUNC_TRACE();
 
+	eth_dev->rx_pkt_burst = &lio_dev_recv_pkts;
+
 	/* Primary does the initialization. */
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
@@ -448,6 +452,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 		lio_dev_err(lio_dev,
 			    "MAC addresses memory allocation failed\n");
 		eth_dev->dev_ops = NULL;
+		eth_dev->rx_pkt_burst = NULL;
 		return -ENOMEM;
 	}
 
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 9948023..9e4da3a 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -326,6 +326,386 @@
 	return 0;
 }
 
+static inline uint32_t
+lio_droq_get_bufcount(uint32_t buf_size, uint32_t total_len)
+{
+	uint32_t buf_cnt = 0;
+
+	while (total_len > (buf_size * buf_cnt))
+		buf_cnt++;
+
+	return buf_cnt;
+}
+
+/* If we were not able to refill all buffers, try to move around
+ * the buffers that were not dispatched.
+ */
+static inline uint32_t
+lio_droq_refill_pullup_descs(struct lio_droq *droq,
+			     struct lio_droq_desc *desc_ring)
+{
+	uint32_t refill_index = droq->refill_idx;
+	uint32_t desc_refilled = 0;
+
+	while (refill_index != droq->read_idx) {
+		if (droq->recv_buf_list[refill_index].buffer) {
+			droq->recv_buf_list[droq->refill_idx].buffer =
+				droq->recv_buf_list[refill_index].buffer;
+			desc_ring[droq->refill_idx].buffer_ptr =
+				desc_ring[refill_index].buffer_ptr;
+			droq->recv_buf_list[refill_index].buffer = NULL;
+			desc_ring[refill_index].buffer_ptr = 0;
+			do {
+				droq->refill_idx = lio_incr_index(
+							droq->refill_idx, 1,
+							droq->max_count);
+				desc_refilled++;
+				droq->refill_count--;
+			} while (droq->recv_buf_list[droq->refill_idx].buffer);
+		}
+		refill_index = lio_incr_index(refill_index, 1,
+					      droq->max_count);
+	}	/* while */
+
+	return desc_refilled;
+}
+
+/* lio_droq_refill
+ *
+ * @param lio_dev	- pointer to the lio device structure
+ * @param droq		- droq in which descriptors require new buffers.
+ *
+ * Description:
+ *  Called during normal DROQ processing in interrupt mode or by the poll
+ *  thread to refill the descriptors from which buffers were dispatched
+ *  to upper layers. Attempts to allocate new buffers. If that fails, moves
+ *  up buffers (that were not dispatched) to form a contiguous ring.
+ *
+ * Returns:
+ *  No of descriptors refilled.
+ *
+ * Locks:
+ * This routine is called with droq->lock held.
+ */
+static uint32_t
+lio_droq_refill(struct lio_device *lio_dev, struct lio_droq *droq)
+{
+	struct lio_droq_desc *desc_ring;
+	uint32_t desc_refilled = 0;
+	void *buf = NULL;
+
+	desc_ring = droq->desc_ring;
+
+	while (droq->refill_count && (desc_refilled < droq->max_count)) {
+		/* If a valid buffer exists (happens if there is no dispatch),
+		 * reuse the buffer, else allocate.
+		 */
+		if (droq->recv_buf_list[droq->refill_idx].buffer == NULL) {
+			buf = lio_recv_buffer_alloc(lio_dev, droq->q_no);
+			/* If a buffer could not be allocated, no point in
+			 * continuing
+			 */
+			if (buf == NULL)
+				break;
+
+			droq->recv_buf_list[droq->refill_idx].buffer = buf;
+		}
+
+		desc_ring[droq->refill_idx].buffer_ptr =
+		    lio_map_ring(droq->recv_buf_list[droq->refill_idx].buffer);
+		/* Reset any previous values in the length field. */
+		droq->info_list[droq->refill_idx].length = 0;
+
+		droq->refill_idx = lio_incr_index(droq->refill_idx, 1,
+						  droq->max_count);
+		desc_refilled++;
+		droq->refill_count--;
+	}
+
+	if (droq->refill_count)
+		desc_refilled += lio_droq_refill_pullup_descs(droq, desc_ring);
+
+	/* if droq->refill_count
+	 * The refill count would not change in pass two. We only moved buffers
+	 * to close the gap in the ring, but we would still have the same no. of
+	 * buffers to refill.
+	 */
+	return desc_refilled;
+}
+
+static int
+lio_droq_fast_process_packet(struct lio_device *lio_dev,
+			     struct lio_droq *droq,
+			     struct rte_mbuf **rx_pkts)
+{
+	struct rte_mbuf *nicbuf = NULL;
+	struct lio_droq_info *info;
+	uint32_t total_len = 0;
+	int data_total_len = 0;
+	uint32_t pkt_len = 0;
+	union octeon_rh *rh;
+	int data_pkts = 0;
+
+	info = &droq->info_list[droq->read_idx];
+	lio_swap_8B_data((uint64_t *)info, 2);
+
+	if (!info->length)
+		return -1;
+
+	/* Len of resp hdr in included in the received data len. */
+	info->length -= OCTEON_RH_SIZE;
+	rh = &info->rh;
+
+	total_len += (uint32_t)info->length;
+
+	if (lio_opcode_slow_path(rh)) {
+		uint32_t buf_cnt;
+
+		buf_cnt = lio_droq_get_bufcount(droq->buffer_size,
+						(uint32_t)info->length);
+		droq->read_idx = lio_incr_index(droq->read_idx, buf_cnt,
+						droq->max_count);
+		droq->refill_count += buf_cnt;
+	} else {
+		if (info->length <= droq->buffer_size) {
+			if (rh->r_dh.has_hash)
+				pkt_len = (uint32_t)(info->length - 8);
+			else
+				pkt_len = (uint32_t)info->length;
+
+			nicbuf = droq->recv_buf_list[droq->read_idx].buffer;
+			droq->recv_buf_list[droq->read_idx].buffer = NULL;
+			droq->read_idx = lio_incr_index(
+						droq->read_idx, 1,
+						droq->max_count);
+			droq->refill_count++;
+
+			if (likely(nicbuf != NULL)) {
+				nicbuf->data_off = RTE_PKTMBUF_HEADROOM;
+				nicbuf->nb_segs = 1;
+				nicbuf->next = NULL;
+				/* We don't have a way to pass flags yet */
+				nicbuf->ol_flags = 0;
+				if (rh->r_dh.has_hash) {
+					uint64_t *hash_ptr;
+
+					nicbuf->ol_flags |= PKT_RX_RSS_HASH;
+					hash_ptr = rte_pktmbuf_mtod(nicbuf,
+								    uint64_t *);
+					lio_swap_8B_data(hash_ptr, 1);
+					nicbuf->hash.rss = (uint32_t)*hash_ptr;
+					nicbuf->data_off += 8;
+				}
+
+				nicbuf->pkt_len = pkt_len;
+				nicbuf->data_len = pkt_len;
+				nicbuf->port = lio_dev->port_id;
+				/* Store the mbuf */
+				rx_pkts[data_pkts++] = nicbuf;
+				data_total_len += pkt_len;
+			}
+
+			/* Prefetch buffer pointers when on a cache line
+			 * boundary
+			 */
+			if ((droq->read_idx & 3) == 0) {
+				rte_prefetch0(
+				    &droq->recv_buf_list[droq->read_idx]);
+				rte_prefetch0(
+				    &droq->info_list[droq->read_idx]);
+			}
+		} else {
+			struct rte_mbuf *first_buf = NULL;
+			struct rte_mbuf *last_buf = NULL;
+
+			while (pkt_len < info->length) {
+				int cpy_len = 0;
+
+				cpy_len = ((pkt_len + droq->buffer_size) >
+						info->length)
+						? ((uint32_t)info->length -
+							pkt_len)
+						: droq->buffer_size;
+
+				nicbuf =
+				    droq->recv_buf_list[droq->read_idx].buffer;
+				droq->recv_buf_list[droq->read_idx].buffer =
+				    NULL;
+
+				if (likely(nicbuf != NULL)) {
+					/* Note the first seg */
+					if (!pkt_len)
+						first_buf = nicbuf;
+
+					nicbuf->data_off = RTE_PKTMBUF_HEADROOM;
+					nicbuf->nb_segs = 1;
+					nicbuf->next = NULL;
+					nicbuf->port = lio_dev->port_id;
+					/* We don't have a way to pass
+					 * flags yet
+					 */
+					nicbuf->ol_flags = 0;
+					if ((!pkt_len) && (rh->r_dh.has_hash)) {
+						uint64_t *hash_ptr;
+
+						nicbuf->ol_flags |=
+						    PKT_RX_RSS_HASH;
+						hash_ptr = rte_pktmbuf_mtod(
+						    nicbuf, uint64_t *);
+						lio_swap_8B_data(hash_ptr, 1);
+						nicbuf->hash.rss =
+						    (uint32_t)*hash_ptr;
+						nicbuf->data_off += 8;
+						nicbuf->pkt_len = cpy_len - 8;
+						nicbuf->data_len = cpy_len - 8;
+					} else {
+						nicbuf->pkt_len = cpy_len;
+						nicbuf->data_len = cpy_len;
+					}
+
+					if (pkt_len)
+						first_buf->nb_segs++;
+
+					if (last_buf)
+						last_buf->next = nicbuf;
+
+					last_buf = nicbuf;
+				} else {
+					PMD_RX_LOG(lio_dev, ERR, "no buf\n");
+				}
+
+				pkt_len += cpy_len;
+				droq->read_idx = lio_incr_index(
+							droq->read_idx,
+							1, droq->max_count);
+				droq->refill_count++;
+
+				/* Prefetch buffer pointers when on a
+				 * cache line boundary
+				 */
+				if ((droq->read_idx & 3) == 0) {
+					rte_prefetch0(&droq->recv_buf_list
+							      [droq->read_idx]);
+
+					rte_prefetch0(
+					    &droq->info_list[droq->read_idx]);
+				}
+			}
+			rx_pkts[data_pkts++] = first_buf;
+			if (rh->r_dh.has_hash)
+				data_total_len += (pkt_len - 8);
+			else
+				data_total_len += pkt_len;
+		}
+
+		/* Inform upper layer about packet checksum verification */
+		struct rte_mbuf *m = rx_pkts[data_pkts - 1];
+
+		if (rh->r_dh.csum_verified & LIO_IP_CSUM_VERIFIED)
+			m->ol_flags |= PKT_RX_IP_CKSUM_GOOD;
+
+		if (rh->r_dh.csum_verified & LIO_L4_CSUM_VERIFIED)
+			m->ol_flags |= PKT_RX_L4_CKSUM_GOOD;
+	}
+
+	if (droq->refill_count >= droq->refill_threshold) {
+		int desc_refilled = lio_droq_refill(lio_dev, droq);
+
+		/* Flush the droq descriptor data to memory to be sure
+		 * that when we update the credits the data in memory is
+		 * accurate.
+		 */
+		rte_wmb();
+		rte_write32(desc_refilled, droq->pkts_credit_reg);
+		/* make sure mmio write completes */
+		rte_wmb();
+	}
+
+	info->length = 0;
+	info->rh.rh64 = 0;
+
+	return data_pkts;
+}
+
+static uint32_t
+lio_droq_fast_process_packets(struct lio_device *lio_dev,
+			      struct lio_droq *droq,
+			      struct rte_mbuf **rx_pkts,
+			      uint32_t pkts_to_process)
+{
+	int ret, data_pkts = 0;
+	uint32_t pkt;
+
+	for (pkt = 0; pkt < pkts_to_process; pkt++) {
+		ret = lio_droq_fast_process_packet(lio_dev, droq,
+						   &rx_pkts[data_pkts]);
+		if (ret < 0) {
+			lio_dev_err(lio_dev, "Port[%d] DROQ[%d] idx: %d len:0, pkt_cnt: %d\n",
+				    lio_dev->port_id, droq->q_no,
+				    droq->read_idx, pkts_to_process);
+			break;
+		}
+		data_pkts += ret;
+	}
+
+	rte_atomic64_sub(&droq->pkts_pending, pkt);
+
+	return data_pkts;
+}
+
+static inline uint32_t
+lio_droq_check_hw_for_pkts(struct lio_droq *droq)
+{
+	uint32_t last_count;
+	uint32_t pkt_count;
+
+	pkt_count = rte_read32(droq->pkts_sent_reg);
+
+	last_count = pkt_count - droq->pkt_count;
+	droq->pkt_count = pkt_count;
+
+	if (last_count)
+		rte_atomic64_add(&droq->pkts_pending, last_count);
+
+	return last_count;
+}
+
+uint16_t
+lio_dev_recv_pkts(void *rx_queue,
+		  struct rte_mbuf **rx_pkts,
+		  uint16_t budget)
+{
+	struct lio_droq *droq = rx_queue;
+	struct lio_device *lio_dev = droq->lio_dev;
+	uint32_t pkts_processed = 0;
+	uint32_t pkt_count = 0;
+
+	lio_droq_check_hw_for_pkts(droq);
+
+	pkt_count = rte_atomic64_read(&droq->pkts_pending);
+	if (!pkt_count)
+		return 0;
+
+	if (pkt_count > budget)
+		pkt_count = budget;
+
+	/* Grab the lock */
+	rte_spinlock_lock(&droq->lock);
+	pkts_processed = lio_droq_fast_process_packets(lio_dev,
+						       droq, rx_pkts,
+						       pkt_count);
+
+	if (droq->pkt_count) {
+		rte_write32(droq->pkt_count, droq->pkts_sent_reg);
+		droq->pkt_count = 0;
+	}
+
+	/* Release the spin lock */
+	rte_spinlock_unlock(&droq->lock);
+
+	return pkts_processed;
+}
+
 /**
  *  lio_init_instr_queue()
  *  @param lio_dev	- pointer to the lio device structure.
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index fc623ad..420b893 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -515,6 +515,17 @@ enum {
 	return (uint64_t)dma_addr;
 }
 
+static inline int
+lio_opcode_slow_path(union octeon_rh *rh)
+{
+	uint16_t subcode1, subcode2;
+
+	subcode1 = LIO_OPCODE_SUBCODE(rh->r.opcode, rh->r.subcode);
+	subcode2 = LIO_OPCODE_SUBCODE(LIO_OPCODE, LIO_OPCODE_NW_DATA);
+
+	return subcode2 != subcode1;
+}
+
 /* Macro to increment index.
  * Index is incremented by count; if the sum exceeds
  * max, index is wrapped-around to the start.
@@ -533,6 +544,8 @@ enum {
 int lio_setup_droq(struct lio_device *lio_dev, int q_no, int num_descs,
 		   int desc_size, struct rte_mempool *mpool,
 		   unsigned int socket_id);
+uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+			   uint16_t budget);
 
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
-- 
1.8.3.1

[PATCH v2 23/46] net/liquidio: add API to release Rx queue

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 28 ++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.c   | 10 ++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  1 +
 3 files changed, 39 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index ebfdf7a..d5f650e 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -121,6 +121,33 @@
 	return 0;
 }
 
+/**
+ * Release the receive queue/ringbuffer. Called by
+ * the upper layers.
+ *
+ * @param rxq
+ *    Opaque pointer to the receive queue to release
+ *
+ * @return
+ *    - nothing
+ */
+static void
+lio_dev_rx_queue_release(void *rxq)
+{
+	struct lio_droq *droq = rxq;
+	struct lio_device *lio_dev = droq->lio_dev;
+	int oq_no;
+
+	/* Run time queue deletion not supported */
+	if (lio_dev->port_configured)
+		return;
+
+	if (droq != NULL) {
+		oq_no = droq->q_no;
+		lio_delete_droq_queue(droq->lio_dev, oq_no);
+	}
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -266,6 +293,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
+	.rx_queue_release	= lio_dev_rx_queue_release,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 9e4da3a..4e63a50 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -706,6 +706,16 @@
 	return pkts_processed;
 }
 
+void
+lio_delete_droq_queue(struct lio_device *lio_dev,
+		      int oq_no)
+{
+	lio_delete_droq(lio_dev, oq_no);
+	lio_dev->num_oqs--;
+	rte_free(lio_dev->droq[oq_no]);
+	lio_dev->droq[oq_no] = NULL;
+}
+
 /**
  *  lio_init_instr_queue()
  *  @param lio_dev	- pointer to the lio device structure.
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 420b893..76d067e 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -546,6 +546,7 @@ int lio_setup_droq(struct lio_device *lio_dev, int q_no, int num_descs,
 		   unsigned int socket_id);
 uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			   uint16_t budget);
+void lio_delete_droq_queue(struct lio_device *lio_dev, int oq_no);
 
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
-- 
1.8.3.1

[PATCH v2 24/46] net/liquidio: add API to setup Tx queue

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 60 +++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.c   | 39 +++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  3 ++
 3 files changed, 102 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index d5f650e..9e2d3f8 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -148,6 +148,65 @@
 	}
 }
 
+/**
+ * Allocate and initialize SW ring. Initialize associated HW registers.
+ *
+ * @param eth_dev
+ *   Pointer to structure rte_eth_dev
+ *
+ * @param q_no
+ *   Queue number
+ *
+ * @param num_tx_descs
+ *   Number of ringbuffer descriptors
+ *
+ * @param socket_id
+ *   NUMA socket id, used for memory allocations
+ *
+ * @param tx_conf
+ *   Pointer to the structure rte_eth_txconf
+ *
+ * @return
+ *   - On success, return 0
+ *   - On failure, return -errno value
+ */
+static int
+lio_dev_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t q_no,
+		       uint16_t num_tx_descs, unsigned int socket_id,
+		       const struct rte_eth_txconf *tx_conf __rte_unused)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	int fw_mapped_iq = lio_dev->linfo.txpciq[q_no].s.q_no;
+	int retval;
+
+	if (q_no >= lio_dev->nb_tx_queues) {
+		lio_dev_err(lio_dev, "Invalid tx queue number %u\n", q_no);
+		return -EINVAL;
+	}
+
+	lio_dev_dbg(lio_dev, "setting up tx queue %u\n", q_no);
+
+	if ((lio_dev->instr_queue[fw_mapped_iq] != NULL) &&
+	    (num_tx_descs != lio_dev->instr_queue[fw_mapped_iq]->max_count)) {
+		lio_dev_err(lio_dev,
+			    "Reconfiguring Tx descs not supported. Configure descs to same value %u or restart application\n",
+			    lio_dev->instr_queue[fw_mapped_iq]->max_count);
+		return -ENOTSUP;
+	}
+
+	retval = lio_setup_iq(lio_dev, q_no, lio_dev->linfo.txpciq[q_no],
+			      num_tx_descs, lio_dev, socket_id);
+
+	if (retval) {
+		lio_dev_err(lio_dev, "Runtime IQ(TxQ) creation failed.\n");
+		return retval;
+	}
+
+	eth_dev->data->tx_queues[q_no] = lio_dev->instr_queue[fw_mapped_iq];
+
+	return 0;
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -294,6 +353,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.dev_configure		= lio_dev_configure,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
+	.tx_queue_setup		= lio_dev_tx_queue_setup,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 4e63a50..b0dfc9b 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -861,6 +861,45 @@
 	lio_dev->num_iqs--;
 }
 
+/* Return 0 on success, -1 on failure */
+int
+lio_setup_iq(struct lio_device *lio_dev, int q_index,
+	     union octeon_txpciq txpciq, uint32_t num_descs, void *app_ctx,
+	     unsigned int socket_id)
+{
+	uint32_t iq_no = (uint32_t)txpciq.s.q_no;
+
+	if (lio_dev->instr_queue[iq_no]) {
+		lio_dev_dbg(lio_dev, "IQ is in use. Cannot create the IQ: %d again\n",
+			    iq_no);
+		lio_dev->instr_queue[iq_no]->txpciq.txpciq64 = txpciq.txpciq64;
+		lio_dev->instr_queue[iq_no]->app_ctx = app_ctx;
+		return 0;
+	}
+
+	lio_dev->instr_queue[iq_no] = rte_zmalloc_socket("ethdev TX queue",
+						sizeof(struct lio_instr_queue),
+						RTE_CACHE_LINE_SIZE, socket_id);
+	if (lio_dev->instr_queue[iq_no] == NULL)
+		return -1;
+
+	lio_dev->instr_queue[iq_no]->q_index = q_index;
+	lio_dev->instr_queue[iq_no]->app_ctx = app_ctx;
+
+	if (lio_init_instr_queue(lio_dev, txpciq, num_descs, socket_id))
+		goto release_lio_iq;
+
+	lio_dev->num_iqs++;
+
+	return 0;
+
+release_lio_iq:
+	rte_free(lio_dev->instr_queue[iq_no]);
+	lio_dev->instr_queue[iq_no] = NULL;
+
+	return -1;
+}
+
 static inline void
 lio_ring_doorbell(struct lio_device *lio_dev,
 		  struct lio_instr_queue *iq)
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 76d067e..86d5864 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -548,6 +548,9 @@ uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			   uint16_t budget);
 void lio_delete_droq_queue(struct lio_device *lio_dev, int oq_no);
 
+int lio_setup_iq(struct lio_device *lio_dev, int q_index,
+		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
+		 unsigned int socket_id);
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
-- 
1.8.3.1

[PATCH v2 25/46] net/liquidio: add APIs for SG list

[Shijith Thotton]

Add APIs to setup and free Scatter-Gather list. SG list is used while
sending packets with multiple segments.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c |  23 ++++++++
 drivers/net/liquidio/lio_rxtx.c   | 107 ++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |   7 +++
 drivers/net/liquidio/lio_struct.h |  40 ++++++++++++++
 4 files changed, 177 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 9e2d3f8..77106f6 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -202,6 +202,15 @@
 		return retval;
 	}
 
+	retval = lio_setup_sglists(lio_dev, q_no, fw_mapped_iq,
+				lio_dev->instr_queue[fw_mapped_iq]->max_count,
+				socket_id);
+
+	if (retval) {
+		lio_delete_instruction_queue(lio_dev, fw_mapped_iq);
+		return retval;
+	}
+
 	eth_dev->data->tx_queues[q_no] = lio_dev->instr_queue[fw_mapped_iq];
 
 	return 0;
@@ -334,6 +343,20 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	/* Copy the permanent MAC address */
 	ether_addr_copy((struct ether_addr *)mac, &eth_dev->data->mac_addrs[0]);
 
+	lio_dev->glist_lock =
+	    rte_zmalloc(NULL, sizeof(*lio_dev->glist_lock) * num_iqueues, 0);
+	if (lio_dev->glist_lock == NULL)
+		return -ENOMEM;
+
+	lio_dev->glist_head =
+		rte_zmalloc(NULL, sizeof(*lio_dev->glist_head) * num_iqueues,
+			    0);
+	if (lio_dev->glist_head == NULL) {
+		rte_free(lio_dev->glist_lock);
+		lio_dev->glist_lock = NULL;
+		return -ENOMEM;
+	}
+
 	lio_dev->port_configured = 1;
 
 	lio_free_soft_command(sc);
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index b0dfc9b..50deaba 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -40,6 +40,8 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+#define LIO_MAX_SG 12
+
 static void
 lio_droq_compute_max_packet_bufs(struct lio_droq *droq)
 {
@@ -1272,3 +1274,108 @@ struct lio_soft_command *
 
 	return 0;
 }
+
+static inline struct lio_stailq_node *
+list_delete_first_node(struct lio_stailq_head *head)
+{
+	struct lio_stailq_node *node;
+
+	if (STAILQ_EMPTY(head))
+		node = NULL;
+	else
+		node = STAILQ_FIRST(head);
+
+	if (node)
+		STAILQ_REMOVE(head, node, lio_stailq_node, entries);
+
+	return node;
+}
+
+static void
+lio_delete_sglist(struct lio_instr_queue *txq)
+{
+	struct lio_device *lio_dev = txq->lio_dev;
+	int iq_no = txq->q_index;
+	struct lio_gather *g;
+
+	if (lio_dev->glist_head == NULL)
+		return;
+
+	do {
+		g = (struct lio_gather *)list_delete_first_node(
+						&lio_dev->glist_head[iq_no]);
+		if (g) {
+			if (g->sg)
+				rte_free(
+				    (void *)((unsigned long)g->sg - g->adjust));
+			rte_free(g);
+		}
+	} while (g);
+}
+
+/**
+ * \brief Setup gather lists
+ * @param lio per-network private data
+ */
+int
+lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
+		  int fw_mapped_iq, int num_descs, unsigned int socket_id)
+{
+	struct lio_gather *g;
+	int i;
+
+	rte_spinlock_init(&lio_dev->glist_lock[iq_no]);
+
+	STAILQ_INIT(&lio_dev->glist_head[iq_no]);
+
+	for (i = 0; i < num_descs; i++) {
+		g = rte_zmalloc_socket(NULL, sizeof(*g), RTE_CACHE_LINE_SIZE,
+				       socket_id);
+		if (g == NULL) {
+			lio_dev_err(lio_dev,
+				    "lio_gather memory allocation failed for qno %d\n",
+				    iq_no);
+			break;
+		}
+
+		g->sg_size =
+		    ((ROUNDUP4(LIO_MAX_SG) >> 2) * LIO_SG_ENTRY_SIZE);
+
+		g->sg = rte_zmalloc_socket(NULL, g->sg_size + 8,
+					   RTE_CACHE_LINE_SIZE, socket_id);
+		if (g->sg == NULL) {
+			lio_dev_err(lio_dev,
+				    "sg list memory allocation failed for qno %d\n",
+				    iq_no);
+			rte_free(g);
+			break;
+		}
+
+		/* The gather component should be aligned on 64-bit boundary */
+		if (((unsigned long)g->sg) & 7) {
+			g->adjust = 8 - (((unsigned long)g->sg) & 7);
+			g->sg =
+			    (struct lio_sg_entry *)((unsigned long)g->sg +
+						       g->adjust);
+		}
+
+		STAILQ_INSERT_TAIL(&lio_dev->glist_head[iq_no], &g->list,
+				   entries);
+	}
+
+	if (i != num_descs) {
+		lio_delete_sglist(lio_dev->instr_queue[fw_mapped_iq]);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+void
+lio_delete_instruction_queue(struct lio_device *lio_dev, int iq_no)
+{
+	lio_delete_instr_queue(lio_dev, iq_no);
+	rte_free(lio_dev->instr_queue[iq_no]);
+	lio_dev->instr_queue[iq_no] = NULL;
+	lio_dev->num_iqs--;
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 86d5864..10bca4c 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -42,6 +42,10 @@
 
 #include "lio_struct.h"
 
+#ifndef ROUNDUP4
+#define ROUNDUP4(val) (((val) + 3) & 0xfffffffc)
+#endif
+
 #define LIO_STQUEUE_FIRST_ENTRY(ptr, type, elem)	\
 	(type *)((char *)((ptr)->stqh_first) - offsetof(type, elem))
 
@@ -548,9 +552,12 @@ uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			   uint16_t budget);
 void lio_delete_droq_queue(struct lio_device *lio_dev, int oq_no);
 
+int lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
+		      int fw_mapped_iq, int num_descs, unsigned int socket_id);
 int lio_setup_iq(struct lio_device *lio_dev, int q_index,
 		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
 		 unsigned int socket_id);
+void lio_delete_instruction_queue(struct lio_device *lio_dev, int iq_no);
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 7a7a4a6..4d67eb6 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -298,6 +298,41 @@ struct lio_instr_queue {
 	const struct rte_memzone *iq_mz;
 };
 
+/* The Scatter-Gather List Entry. The scatter or gather component used with
+ * input instruction has this format.
+ */
+struct lio_sg_entry {
+	/** The first 64 bit gives the size of data in each dptr. */
+	union {
+		uint16_t size[4];
+		uint64_t size64;
+	} u;
+
+	/** The 4 dptr pointers for this entry. */
+	uint64_t ptr[4];
+};
+
+#define LIO_SG_ENTRY_SIZE	(sizeof(struct lio_sg_entry))
+
+/** Structure of a node in list of gather components maintained by
+ *  driver for each network device.
+ */
+struct lio_gather {
+	/** List manipulation. Next and prev pointers. */
+	struct lio_stailq_node list;
+
+	/** Size of the gather component at sg in bytes. */
+	int sg_size;
+
+	/** Number of bytes that sg was adjusted to make it 8B-aligned. */
+	int adjust;
+
+	/** Gather component that can accommodate max sized fragment list
+	 *  received from the IP layer.
+	 */
+	struct lio_sg_entry *sg;
+};
+
 struct lio_io_enable {
 	uint64_t iq;
 	uint64_t oq;
@@ -516,6 +551,11 @@ struct lio_device {
 
 	uint32_t num_iqs;
 
+	/** Guards each glist */
+	rte_spinlock_t *glist_lock;
+	/** Array of gather component linked lists */
+	struct lio_stailq_head *glist_head;
+
 	/* The pool containing pre allocated buffers used for soft commands */
 	struct rte_mempool *sc_buf_pool;
 
-- 
1.8.3.1

[PATCH v2 26/46] net/liquidio: add APIs to enable and disable IO queues

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 70 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.c       | 13 ++++++
 drivers/net/liquidio/lio_rxtx.c         |  5 +++
 drivers/net/liquidio/lio_struct.h       |  2 +
 4 files changed, 90 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index 44d90c0..6ff5b69 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -310,6 +310,73 @@
 	return 0;
 }
 
+static int
+cn23xx_vf_enable_io_queues(struct lio_device *lio_dev)
+{
+	uint32_t q_no;
+
+	PMD_INIT_FUNC_TRACE();
+
+	for (q_no = 0; q_no < lio_dev->num_iqs; q_no++) {
+		uint64_t reg_val;
+
+		/* set the corresponding IQ IS_64B bit */
+		if (lio_dev->io_qmask.iq64B & (1ULL << q_no)) {
+			reg_val = lio_read_csr64(
+					lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+			reg_val = reg_val | CN23XX_PKT_INPUT_CTL_IS_64B;
+			lio_write_csr64(lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+					reg_val);
+		}
+
+		/* set the corresponding IQ ENB bit */
+		if (lio_dev->io_qmask.iq & (1ULL << q_no)) {
+			reg_val = lio_read_csr64(
+					lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+			reg_val = reg_val | CN23XX_PKT_INPUT_CTL_RING_ENB;
+			lio_write_csr64(lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+					reg_val);
+		}
+	}
+	for (q_no = 0; q_no < lio_dev->num_oqs; q_no++) {
+		uint32_t reg_val;
+
+		/* set the corresponding OQ ENB bit */
+		if (lio_dev->io_qmask.oq & (1ULL << q_no)) {
+			reg_val = lio_read_csr(
+					lio_dev,
+					CN23XX_SLI_OQ_PKT_CONTROL(q_no));
+			reg_val = reg_val | CN23XX_PKT_OUTPUT_CTL_RING_ENB;
+			lio_write_csr(lio_dev,
+				      CN23XX_SLI_OQ_PKT_CONTROL(q_no),
+				      reg_val);
+		}
+	}
+
+	return 0;
+}
+
+static void
+cn23xx_vf_disable_io_queues(struct lio_device *lio_dev)
+{
+	uint32_t num_queues;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* per HRM, rings can only be disabled via reset operation,
+	 * NOT via SLI_PKT()_INPUT/OUTPUT_CONTROL[ENB]
+	 */
+	num_queues = lio_dev->num_iqs;
+	if (num_queues < lio_dev->num_oqs)
+		num_queues = lio_dev->num_oqs;
+
+	cn23xx_vf_reset_io_queues(lio_dev, num_queues);
+}
+
 void
 cn23xx_vf_ask_pf_to_do_flr(struct lio_device *lio_dev)
 {
@@ -463,6 +530,9 @@
 
 	lio_dev->fn_list.setup_device_regs	= cn23xx_vf_setup_device_regs;
 
+	lio_dev->fn_list.enable_io_queues	= cn23xx_vf_enable_io_queues;
+	lio_dev->fn_list.disable_io_queues	= cn23xx_vf_disable_io_queues;
+
 	return 0;
 }
 
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 77106f6..b2e7a29 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -361,6 +361,15 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 
 	lio_free_soft_command(sc);
 
+	/* Disable iq_0 for reconf */
+	lio_dev->fn_list.disable_io_queues(lio_dev);
+
+	/* Reset ioq regs */
+	lio_dev->fn_list.setup_device_regs(lio_dev);
+
+	/* Free iq_0 used during init */
+	lio_free_instr_queue0(lio_dev);
+
 	return 0;
 
 nic_config_fail:
@@ -487,6 +496,10 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	lio_dev->max_tx_queues = dpdk_queues;
 	lio_dev->max_rx_queues = dpdk_queues;
 
+	/* Enable input and output queues for this device */
+	if (lio_dev->fn_list.enable_io_queues(lio_dev))
+		goto error;
+
 	return 0;
 
 error:
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 50deaba..63d8896 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -892,9 +892,14 @@
 		goto release_lio_iq;
 
 	lio_dev->num_iqs++;
+	if (lio_dev->fn_list.enable_io_queues(lio_dev))
+		goto delete_lio_iq;
 
 	return 0;
 
+delete_lio_iq:
+	lio_delete_instr_queue(lio_dev, iq_no);
+	lio_dev->num_iqs--;
 release_lio_iq:
 	rte_free(lio_dev->instr_queue[iq_no]);
 	lio_dev->instr_queue[iq_no] = NULL;
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 4d67eb6..906553c 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -347,6 +347,8 @@ struct lio_fn_list {
 	void (*free_mbox)(struct lio_device *);
 
 	int (*setup_device_regs)(struct lio_device *);
+	int (*enable_io_queues)(struct lio_device *);
+	void (*disable_io_queues)(struct lio_device *);
 };
 
 struct lio_pf_vf_hs_word {
-- 
1.8.3.1

[PATCH v2 27/46] net/liquidio: add Tx data path for single segment

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |   1 +
 drivers/net/liquidio/lio_ethdev.c       |   3 +
 drivers/net/liquidio/lio_rxtx.c         |  89 ++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         | 138 ++++++++++++++++++++++++++++++++
 4 files changed, 231 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 2db7085..1c1ad8e 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -108,6 +108,7 @@ enum octeon_tag_type {
 
 /* pre-defined host->NIC tag values */
 #define LIO_CONTROL	(0x11111110)
+#define LIO_DATA(i)	(0x11111111 + (i))
 
 /* used for NIC operations */
 #define LIO_OPCODE	1
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index b2e7a29..5c5aade 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -529,6 +529,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	eth_dev->data->mac_addrs = NULL;
 
 	eth_dev->rx_pkt_burst = NULL;
+	eth_dev->tx_pkt_burst = NULL;
 
 	return 0;
 }
@@ -542,6 +543,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	PMD_INIT_FUNC_TRACE();
 
 	eth_dev->rx_pkt_burst = &lio_dev_recv_pkts;
+	eth_dev->tx_pkt_burst = &lio_dev_xmit_pkts;
 
 	/* Primary does the initialization. */
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
@@ -577,6 +579,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 			    "MAC addresses memory allocation failed\n");
 		eth_dev->dev_ops = NULL;
 		eth_dev->rx_pkt_burst = NULL;
+		eth_dev->tx_pkt_burst = NULL;
 		return -ENOMEM;
 	}
 
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 63d8896..14113e1 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1384,3 +1384,92 @@ struct lio_soft_command *
 	lio_dev->instr_queue[iq_no] = NULL;
 	lio_dev->num_iqs--;
 }
+
+/** Send data packet to the device
+ *  @param lio_dev - lio device pointer
+ *  @param ndata   - control structure with queueing, and buffer information
+ *
+ *  @returns IQ_FAILED if it failed to add to the input queue. IQ_STOP if it the
+ *  queue should be stopped, and LIO_IQ_SEND_OK if it sent okay.
+ */
+static inline int
+lio_send_data_pkt(struct lio_device *lio_dev, struct lio_data_pkt *ndata)
+{
+	return lio_send_command(lio_dev, ndata->q_no, &ndata->cmd,
+				ndata->buf, ndata->datasize, ndata->reqtype);
+}
+
+uint16_t
+lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts, uint16_t nb_pkts)
+{
+	struct lio_instr_queue *txq = tx_queue;
+	union lio_cmd_setup cmdsetup;
+	struct lio_device *lio_dev;
+	struct lio_data_pkt ndata;
+	int i, processed = 0;
+	struct rte_mbuf *m;
+	uint32_t tag = 0;
+	int status = 0;
+	int iq_no;
+
+	lio_dev = txq->lio_dev;
+	iq_no = txq->txpciq.s.q_no;
+
+	if (!lio_dev->linfo.link.s.link_up) {
+		PMD_TX_LOG(lio_dev, ERR, "Transmit failed link_status : %d\n",
+			   lio_dev->linfo.link.s.link_up);
+		goto xmit_failed;
+	}
+
+	for (i = 0; i < nb_pkts; i++) {
+		uint32_t pkt_len = 0;
+
+		m = pkts[i];
+
+		/* Prepare the attributes for the data to be passed to BASE. */
+		memset(&ndata, 0, sizeof(struct lio_data_pkt));
+
+		ndata.buf = m;
+
+		ndata.q_no = iq_no;
+
+		cmdsetup.cmd_setup64 = 0;
+		cmdsetup.s.iq_no = iq_no;
+
+		/* check checksum offload flags to form cmd */
+		if (m->ol_flags & PKT_TX_IP_CKSUM)
+			cmdsetup.s.ip_csum = 1;
+
+		if ((m->ol_flags & PKT_TX_TCP_CKSUM) ||
+				(m->ol_flags & PKT_TX_UDP_CKSUM))
+			cmdsetup.s.transport_csum = 1;
+
+		if (m->nb_segs == 1) {
+			pkt_len = rte_pktmbuf_data_len(m);
+			cmdsetup.s.u.datasize = pkt_len;
+			lio_prepare_pci_cmd(lio_dev, &ndata.cmd,
+					    &cmdsetup, tag);
+			ndata.cmd.cmd3.dptr = rte_mbuf_data_dma_addr(m);
+			ndata.reqtype = LIO_REQTYPE_NORESP_NET;
+		}
+
+		ndata.datasize = pkt_len;
+
+		status = lio_send_data_pkt(lio_dev, &ndata);
+
+		if (unlikely(status == LIO_IQ_SEND_FAILED)) {
+			PMD_TX_LOG(lio_dev, ERR, "send failed\n");
+			break;
+		}
+
+		if (unlikely(status == LIO_IQ_SEND_STOP))
+			PMD_TX_LOG(lio_dev, DEBUG, "iq full\n");
+
+		processed++;
+	}
+
+xmit_failed:
+
+	return processed;
+}
+
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 10bca4c..6813aea 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -249,6 +249,50 @@ struct lio_iq_post_status {
 
 #define OCTEON_CMD_SIZE (sizeof(union octeon_cmd))
 
+/** Structure of data information passed by driver to the BASE
+ *  layer when forwarding data to Octeon device software.
+ */
+struct lio_data_pkt {
+	/** Pointer to information maintained by NIC module for this packet. The
+	 *  BASE layer passes this as-is to the driver.
+	 */
+	void *buf;
+
+	/** Type of buffer passed in "buf" above. */
+	uint32_t reqtype;
+
+	/** Total data bytes to be transferred in this command. */
+	uint32_t datasize;
+
+	/** Command to be passed to the Octeon device software. */
+	union lio_instr_64B cmd;
+
+	/** Input queue to use to send this command. */
+	uint32_t q_no;
+};
+
+/** Structure passed by driver to BASE layer to prepare a command to send
+ *  network data to Octeon.
+ */
+union lio_cmd_setup {
+	struct {
+		uint32_t iq_no : 8;
+		uint32_t gather : 1;
+		uint32_t timestamp : 1;
+		uint32_t ip_csum : 1;
+		uint32_t transport_csum : 1;
+		uint32_t tnl_csum : 1;
+		uint32_t rsvd : 19;
+
+		union {
+			uint32_t datasize;
+			uint32_t gatherptrs;
+		} u;
+	} s;
+
+	uint64_t cmd_setup64;
+};
+
 /* Instruction Header */
 struct octeon_instr_ih3 {
 #if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
@@ -418,6 +462,98 @@ struct octeon_instr_rdp {
 #endif
 };
 
+union octeon_packet_params {
+	uint32_t pkt_params32;
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint32_t reserved : 24;
+		uint32_t ip_csum : 1; /* Perform IP header checksum(s) */
+		/* Perform Outer transport header checksum */
+		uint32_t transport_csum : 1;
+		/* Find tunnel, and perform transport csum. */
+		uint32_t tnl_csum : 1;
+		uint32_t tsflag : 1;   /* Timestamp this packet */
+		uint32_t ipsec_ops : 4; /* IPsec operation */
+#else
+		uint32_t ipsec_ops : 4;
+		uint32_t tsflag : 1;
+		uint32_t tnl_csum : 1;
+		uint32_t transport_csum : 1;
+		uint32_t ip_csum : 1;
+		uint32_t reserved : 7;
+#endif
+	} s;
+};
+
+/** Utility function to prepare a 64B NIC instruction based on a setup command
+ * @param cmd - pointer to instruction to be filled in.
+ * @param setup - pointer to the setup structure
+ * @param q_no - which queue for back pressure
+ *
+ * Assumes the cmd instruction is pre-allocated, but no fields are filled in.
+ */
+static inline void
+lio_prepare_pci_cmd(struct lio_device *lio_dev,
+		    union lio_instr_64B *cmd,
+		    union lio_cmd_setup *setup,
+		    uint32_t tag)
+{
+	union octeon_packet_params packet_params;
+	struct octeon_instr_pki_ih3 *pki_ih3;
+	struct octeon_instr_irh *irh;
+	struct octeon_instr_ih3 *ih3;
+	int port;
+
+	memset(cmd, 0, sizeof(union lio_instr_64B));
+
+	ih3 = (struct octeon_instr_ih3 *)&cmd->cmd3.ih3;
+	pki_ih3 = (struct octeon_instr_pki_ih3 *)&cmd->cmd3.pki_ih3;
+
+	/* assume that rflag is cleared so therefore front data will only have
+	 * irh and ossp[1] and ossp[2] for a total of 24 bytes
+	 */
+	ih3->pkind = lio_dev->instr_queue[setup->s.iq_no]->txpciq.s.pkind;
+	/* PKI IH */
+	ih3->fsz = OCTEON_PCI_CMD_O3;
+
+	if (!setup->s.gather) {
+		ih3->dlengsz = setup->s.u.datasize;
+	} else {
+		ih3->gather = 1;
+		ih3->dlengsz = setup->s.u.gatherptrs;
+	}
+
+	pki_ih3->w = 1;
+	pki_ih3->raw = 0;
+	pki_ih3->utag = 0;
+	pki_ih3->utt = 1;
+	pki_ih3->uqpg = lio_dev->instr_queue[setup->s.iq_no]->txpciq.s.use_qpg;
+
+	port = (int)lio_dev->instr_queue[setup->s.iq_no]->txpciq.s.port;
+
+	if (tag)
+		pki_ih3->tag = tag;
+	else
+		pki_ih3->tag = LIO_DATA(port);
+
+	pki_ih3->tagtype = OCTEON_ORDERED_TAG;
+	pki_ih3->qpg = lio_dev->instr_queue[setup->s.iq_no]->txpciq.s.qpg;
+	pki_ih3->pm = 0x0; /* parse from L2 */
+	pki_ih3->sl = 32;  /* sl will be sizeof(pki_ih3) + irh + ossp0 + ossp1*/
+
+	irh = (struct octeon_instr_irh *)&cmd->cmd3.irh;
+
+	irh->opcode = LIO_OPCODE;
+	irh->subcode = LIO_OPCODE_NW_DATA;
+
+	packet_params.pkt_params32 = 0;
+	packet_params.s.ip_csum = setup->s.ip_csum;
+	packet_params.s.transport_csum = setup->s.transport_csum;
+	packet_params.s.tsflag = setup->s.timestamp;
+
+	irh->ossp = packet_params.pkt_params32;
+}
+
 int lio_setup_sc_buffer_pool(struct lio_device *lio_dev);
 void lio_free_sc_buffer_pool(struct lio_device *lio_dev);
 
@@ -554,6 +690,8 @@ uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 
 int lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
 		      int fw_mapped_iq, int num_descs, unsigned int socket_id);
+uint16_t lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts,
+			   uint16_t nb_pkts);
 int lio_setup_iq(struct lio_device *lio_dev, int q_index,
 		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
 		 unsigned int socket_id);
-- 
1.8.3.1

[PATCH v2 28/46] net/liquidio: add Tx data path for multiple segments

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_rxtx.c   | 62 +++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   | 11 +++++++
 drivers/net/liquidio/lio_struct.h | 24 +++++++++++++++
 3 files changed, 97 insertions(+)

diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 14113e1..75ecbd5 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1451,6 +1451,68 @@ struct lio_soft_command *
 					    &cmdsetup, tag);
 			ndata.cmd.cmd3.dptr = rte_mbuf_data_dma_addr(m);
 			ndata.reqtype = LIO_REQTYPE_NORESP_NET;
+		} else {
+			struct lio_buf_free_info *finfo;
+			struct lio_gather *g;
+			phys_addr_t phyaddr;
+			int i, frags;
+
+			finfo = (struct lio_buf_free_info *)rte_malloc(NULL,
+							sizeof(*finfo), 0);
+			if (finfo == NULL) {
+				PMD_TX_LOG(lio_dev, ERR,
+					   "free buffer alloc failed\n");
+				goto xmit_failed;
+			}
+
+			rte_spinlock_lock(&lio_dev->glist_lock[iq_no]);
+			g = (struct lio_gather *)list_delete_first_node(
+						&lio_dev->glist_head[iq_no]);
+			rte_spinlock_unlock(&lio_dev->glist_lock[iq_no]);
+			if (g == NULL) {
+				PMD_TX_LOG(lio_dev, ERR,
+					   "Transmit scatter gather: glist null!\n");
+				goto xmit_failed;
+			}
+
+			cmdsetup.s.gather = 1;
+			cmdsetup.s.u.gatherptrs = m->nb_segs;
+			lio_prepare_pci_cmd(lio_dev, &ndata.cmd,
+					    &cmdsetup, tag);
+
+			memset(g->sg, 0, g->sg_size);
+			g->sg[0].ptr[0] = rte_mbuf_data_dma_addr(m);
+			lio_add_sg_size(&g->sg[0], m->data_len, 0);
+			pkt_len = m->data_len;
+			finfo->mbuf = m;
+
+			/* First seg taken care above */
+			frags = m->nb_segs - 1;
+			i = 1;
+			m = m->next;
+			while (frags--) {
+				g->sg[(i >> 2)].ptr[(i & 3)] =
+						rte_mbuf_data_dma_addr(m);
+				lio_add_sg_size(&g->sg[(i >> 2)],
+						m->data_len, (i & 3));
+				pkt_len += m->data_len;
+				i++;
+				m = m->next;
+			}
+
+			phyaddr = rte_mem_virt2phy(g->sg);
+			if (phyaddr == RTE_BAD_PHYS_ADDR) {
+				PMD_TX_LOG(lio_dev, ERR, "bad phys addr\n");
+				goto xmit_failed;
+			}
+
+			ndata.cmd.cmd3.dptr = phyaddr;
+			ndata.reqtype = LIO_REQTYPE_NORESP_NET_SG;
+
+			finfo->g = g;
+			finfo->lio_dev = lio_dev;
+			finfo->iq_no = (uint64_t)iq_no;
+			ndata.buf = finfo;
 		}
 
 		ndata.datasize = pkt_len;
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 6813aea..b555bde 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -666,6 +666,17 @@ enum {
 	return subcode2 != subcode1;
 }
 
+static inline void
+lio_add_sg_size(struct lio_sg_entry *sg_entry,
+		uint16_t size, uint32_t pos)
+{
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	sg_entry->u.size[pos] = size;
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	sg_entry->u.size[3 - pos] = size;
+#endif
+}
+
 /* Macro to increment index.
  * Index is incremented by count; if the sum exceeds
  * max, index is wrapped-around to the start.
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 906553c..478a290 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -298,6 +298,30 @@ struct lio_instr_queue {
 	const struct rte_memzone *iq_mz;
 };
 
+/** This structure is used by driver to store information required
+ *  to free the mbuff when the packet has been fetched by Octeon.
+ *  Bytes offset below assume worst-case of a 64-bit system.
+ */
+struct lio_buf_free_info {
+	/** Bytes 1-8. Pointer to network device private structure. */
+	struct lio_device *lio_dev;
+
+	/** Bytes 9-16. Pointer to mbuff. */
+	struct rte_mbuf *mbuf;
+
+	/** Bytes 17-24. Pointer to gather list. */
+	struct lio_gather *g;
+
+	/** Bytes 25-32. Physical address of mbuf->data or gather list. */
+	uint64_t dptr;
+
+	/** Bytes 33-47. Piggybacked soft command, if any */
+	struct lio_soft_command *sc;
+
+	/** Bytes 48-63. iq no */
+	uint64_t iq_no;
+};
+
 /* The Scatter-Gather List Entry. The scatter or gather component used with
  * input instruction has this format.
  */
-- 
1.8.3.1

[PATCH v2 29/46] net/liquidio: add API to flush IQ

[Shijith Thotton]

API to flush instruction queue checks how many packets reached device
and frees associated host buffers using request list.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c |   1 +
 drivers/net/liquidio/lio_rxtx.c   | 187 +++++++++++++++++++++++++++++++++++++-
 drivers/net/liquidio/lio_rxtx.h   |   1 +
 3 files changed, 188 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 5c5aade..b8baa4f 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -281,6 +281,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	 * response arrived or timed-out.
 	 */
 	while ((*sc->status_word == LIO_COMPLETION_WORD_INIT) && --timeout) {
+		lio_flush_iq(lio_dev, lio_dev->instr_queue[sc->iq_no]);
 		lio_process_ordered_list(lio_dev);
 		rte_delay_ms(1);
 	}
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 75ecbd5..64c0385 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -41,6 +41,9 @@
 #include "lio_rxtx.h"
 
 #define LIO_MAX_SG 12
+/* Flush iq if available tx_desc fall below LIO_FLUSH_WM */
+#define LIO_FLUSH_WM(_iq) ((_iq)->max_count / 2)
+#define LIO_PKT_IN_DONE_CNT_MASK 0x00000000FFFFFFFFULL
 
 static void
 lio_droq_compute_max_packet_bufs(struct lio_droq *droq)
@@ -977,6 +980,146 @@
 	iq->request_list[idx].reqtype = reqtype;
 }
 
+static inline void
+lio_free_netsgbuf(void *buf)
+{
+	struct lio_buf_free_info *finfo = buf;
+	struct lio_device *lio_dev = finfo->lio_dev;
+	struct rte_mbuf *m = finfo->mbuf;
+	struct lio_gather *g = finfo->g;
+	uint8_t iq = finfo->iq_no;
+
+	/* This will take care of multiple segments also */
+	rte_pktmbuf_free(m);
+
+	rte_spinlock_lock(&lio_dev->glist_lock[iq]);
+	STAILQ_INSERT_TAIL(&lio_dev->glist_head[iq], &g->list, entries);
+	rte_spinlock_unlock(&lio_dev->glist_lock[iq]);
+	rte_free(finfo);
+}
+
+/* Can only run in process context */
+static int
+lio_process_iq_request_list(struct lio_device *lio_dev,
+			    struct lio_instr_queue *iq)
+{
+	struct octeon_instr_irh *irh = NULL;
+	uint32_t old = iq->flush_index;
+	struct lio_soft_command *sc;
+	uint32_t inst_count = 0;
+	int reqtype;
+	void *buf;
+
+	while (old != iq->lio_read_index) {
+		reqtype = iq->request_list[old].reqtype;
+		buf     = iq->request_list[old].buf;
+
+		if (reqtype == LIO_REQTYPE_NONE)
+			goto skip_this;
+
+		switch (reqtype) {
+		case LIO_REQTYPE_NORESP_NET:
+			rte_pktmbuf_free((struct rte_mbuf *)buf);
+			break;
+		case LIO_REQTYPE_NORESP_NET_SG:
+			lio_free_netsgbuf(buf);
+			break;
+		case LIO_REQTYPE_SOFT_COMMAND:
+			sc = buf;
+			irh = (struct octeon_instr_irh *)&sc->cmd.cmd3.irh;
+			if (irh->rflag) {
+				/* We're expecting a response from Octeon.
+				 * It's up to lio_process_ordered_list() to
+				 * process sc. Add sc to the ordered soft
+				 * command response list because we expect
+				 * a response from Octeon.
+				 */
+				rte_spinlock_lock(&lio_dev->response_list.lock);
+				rte_atomic64_inc(
+				    &lio_dev->response_list.pending_req_count);
+				STAILQ_INSERT_TAIL(
+					&lio_dev->response_list.head,
+					&sc->node, entries);
+				rte_spinlock_unlock(
+						&lio_dev->response_list.lock);
+			} else {
+				if (sc->callback) {
+					/* This callback must not sleep */
+					sc->callback(LIO_REQUEST_DONE,
+						     sc->callback_arg);
+				}
+			}
+			break;
+		default:
+			lio_dev_err(lio_dev,
+				    "Unknown reqtype: %d buf: %p at idx %d\n",
+				    reqtype, buf, old);
+		}
+
+		iq->request_list[old].buf = NULL;
+		iq->request_list[old].reqtype = 0;
+
+skip_this:
+		inst_count++;
+		old = lio_incr_index(old, 1, iq->max_count);
+	}
+
+	iq->flush_index = old;
+
+	return inst_count;
+}
+
+static void
+lio_update_read_index(struct lio_instr_queue *iq)
+{
+	uint32_t pkt_in_done = rte_read32(iq->inst_cnt_reg);
+	uint32_t last_done;
+
+	last_done = pkt_in_done - iq->pkt_in_done;
+	iq->pkt_in_done = pkt_in_done;
+
+	/* Add last_done and modulo with the IQ size to get new index */
+	iq->lio_read_index = (iq->lio_read_index +
+			(uint32_t)(last_done & LIO_PKT_IN_DONE_CNT_MASK)) %
+			iq->max_count;
+}
+
+int
+lio_flush_iq(struct lio_device *lio_dev, struct lio_instr_queue *iq)
+{
+	uint32_t tot_inst_processed = 0;
+	uint32_t inst_processed = 0;
+	int tx_done = 1;
+
+	if (rte_atomic64_test_and_set(&iq->iq_flush_running) == 0)
+		return tx_done;
+
+	rte_spinlock_lock(&iq->lock);
+
+	lio_update_read_index(iq);
+
+	do {
+		/* Process any outstanding IQ packets. */
+		if (iq->flush_index == iq->lio_read_index)
+			break;
+
+		inst_processed = lio_process_iq_request_list(lio_dev, iq);
+
+		if (inst_processed)
+			rte_atomic64_sub(&iq->instr_pending, inst_processed);
+
+		tot_inst_processed += inst_processed;
+		inst_processed = 0;
+
+	} while (1);
+
+	rte_spinlock_unlock(&iq->lock);
+
+	rte_atomic64_clear(&iq->iq_flush_running);
+
+	return tx_done;
+}
+
 static int
 lio_send_command(struct lio_device *lio_dev, uint32_t iq_no, void *cmd,
 		 void *buf, uint32_t datasize __rte_unused, uint32_t reqtype)
@@ -1385,6 +1528,35 @@ struct lio_soft_command *
 	lio_dev->num_iqs--;
 }
 
+static inline uint32_t
+lio_iq_get_available(struct lio_device *lio_dev, uint32_t q_no)
+{
+	return ((lio_dev->instr_queue[q_no]->max_count - 1) -
+		(uint32_t)rte_atomic64_read(
+				&lio_dev->instr_queue[q_no]->instr_pending));
+}
+
+static inline int
+lio_iq_is_full(struct lio_device *lio_dev, uint32_t q_no)
+{
+	return ((uint32_t)rte_atomic64_read(
+				&lio_dev->instr_queue[q_no]->instr_pending) >=
+				(lio_dev->instr_queue[q_no]->max_count - 2));
+}
+
+static int
+lio_dev_cleanup_iq(struct lio_device *lio_dev, int iq_no)
+{
+	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
+	uint32_t count = 10000;
+
+	while ((lio_iq_get_available(lio_dev, iq_no) < LIO_FLUSH_WM(iq)) &&
+			--count)
+		lio_flush_iq(lio_dev, iq);
+
+	return count ? 0 : 1;
+}
+
 /** Send data packet to the device
  *  @param lio_dev - lio device pointer
  *  @param ndata   - control structure with queueing, and buffer information
@@ -1421,6 +1593,8 @@ struct lio_soft_command *
 		goto xmit_failed;
 	}
 
+	lio_dev_cleanup_iq(lio_dev, iq_no);
+
 	for (i = 0; i < nb_pkts; i++) {
 		uint32_t pkt_len = 0;
 
@@ -1432,6 +1606,14 @@ struct lio_soft_command *
 		ndata.buf = m;
 
 		ndata.q_no = iq_no;
+		if (lio_iq_is_full(lio_dev, ndata.q_no)) {
+			if (lio_dev_cleanup_iq(lio_dev, iq_no)) {
+				PMD_TX_LOG(lio_dev, ERR,
+					   "Transmit failed iq:%d full\n",
+					   ndata.q_no);
+				break;
+			}
+		}
 
 		cmdsetup.cmd_setup64 = 0;
 		cmdsetup.s.iq_no = iq_no;
@@ -1524,8 +1706,11 @@ struct lio_soft_command *
 			break;
 		}
 
-		if (unlikely(status == LIO_IQ_SEND_STOP))
+		if (unlikely(status == LIO_IQ_SEND_STOP)) {
 			PMD_TX_LOG(lio_dev, DEBUG, "iq full\n");
+			/* create space as iq is full */
+			lio_dev_cleanup_iq(lio_dev, iq_no);
+		}
 
 		processed++;
 	}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index b555bde..0a4cc2b 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -706,6 +706,7 @@ uint16_t lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts,
 int lio_setup_iq(struct lio_device *lio_dev, int q_index,
 		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
 		 unsigned int socket_id);
+int lio_flush_iq(struct lio_device *lio_dev, struct lio_instr_queue *iq);
 void lio_delete_instruction_queue(struct lio_device *lio_dev, int iq_no);
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
-- 
1.8.3.1

[PATCH v2 30/46] net/liquidio: add API to release Tx queue

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 31 +++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.c   |  2 +-
 drivers/net/liquidio/lio_rxtx.h   |  1 +
 3 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index b8baa4f..97acfcc 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -216,6 +216,36 @@
 	return 0;
 }
 
+/**
+ * Release the transmit queue/ringbuffer. Called by
+ * the upper layers.
+ *
+ * @param txq
+ *    Opaque pointer to the transmit queue to release
+ *
+ * @return
+ *    - nothing
+ */
+static void
+lio_dev_tx_queue_release(void *txq)
+{
+	struct lio_instr_queue *tq = txq;
+	struct lio_device *lio_dev = tq->lio_dev;
+	uint32_t fw_mapped_iq_no;
+
+	/* Run time queue deletion not supported */
+	if (lio_dev->port_configured)
+		return;
+
+	if (tq != NULL) {
+		/* Free sg_list */
+		lio_delete_sglist(tq);
+
+		fw_mapped_iq_no = tq->txpciq.s.q_no;
+		lio_delete_instruction_queue(tq->lio_dev, fw_mapped_iq_no);
+	}
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -387,6 +417,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
+	.tx_queue_release	= lio_dev_tx_queue_release,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 64c0385..6a1d265 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1439,7 +1439,7 @@ struct lio_soft_command *
 	return node;
 }
 
-static void
+void
 lio_delete_sglist(struct lio_instr_queue *txq)
 {
 	struct lio_device *lio_dev = txq->lio_dev;
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 0a4cc2b..964a884 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -699,6 +699,7 @@ uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			   uint16_t budget);
 void lio_delete_droq_queue(struct lio_device *lio_dev, int oq_no);
 
+void lio_delete_sglist(struct lio_instr_queue *txq);
 int lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
 		      int fw_mapped_iq, int num_descs, unsigned int socket_id);
 uint16_t lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts,
-- 
1.8.3.1

[PATCH v2 31/46] net/liquidio: add APIs to start device and update link

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |   1 +
 drivers/net/liquidio/lio_ethdev.c       | 182 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  13 +++
 drivers/net/liquidio/lio_rxtx.c         |   2 +-
 drivers/net/liquidio/lio_struct.h       |   2 +
 5 files changed, 199 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 1c1ad8e..e3f18e3 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -122,6 +122,7 @@ enum octeon_tag_type {
 /** LIO_OPCODE subcodes */
 /* This subcode is sent by core PCI driver to indicate cores are ready. */
 #define LIO_OPCODE_NW_DATA		0x02 /* network packet data */
+#define LIO_OPCODE_INFO			0x04
 #define LIO_OPCODE_IF_CFG		0x09
 
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 97acfcc..4962cad 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -41,6 +41,32 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+/**
+ * Atomically writes the link status information into global
+ * structure rte_eth_dev.
+ *
+ * @param eth_dev
+ *   - Pointer to the structure rte_eth_dev to read from.
+ *   - Pointer to the buffer to be saved with the link status.
+ *
+ * @return
+ *   - On success, zero.
+ *   - On failure, negative value.
+ */
+static inline int
+lio_dev_atomic_write_link_status(struct rte_eth_dev *eth_dev,
+				 struct rte_eth_link *link)
+{
+	struct rte_eth_link *dst = &eth_dev->data->dev_link;
+	struct rte_eth_link *src = link;
+
+	if (rte_atomic64_cmpset((uint64_t *)dst, *(uint64_t *)dst,
+				*(uint64_t *)src) == 0)
+		return -1;
+
+	return 0;
+}
+
 static uint64_t
 lio_hweight64(uint64_t w)
 {
@@ -55,6 +81,49 @@
 	return (res + (res >> 32)) & 0x00000000000000FFul;
 }
 
+static int
+lio_dev_link_update(struct rte_eth_dev *eth_dev,
+		    int wait_to_complete __rte_unused)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct rte_eth_link link, old;
+
+	/* Initialize */
+	link.link_status = ETH_LINK_DOWN;
+	link.link_speed = ETH_SPEED_NUM_NONE;
+	link.link_duplex = ETH_LINK_HALF_DUPLEX;
+	memset(&old, 0, sizeof(old));
+
+	/* Return what we found */
+	if (lio_dev->linfo.link.s.link_up == 0) {
+		/* Interface is down */
+		if (lio_dev_atomic_write_link_status(eth_dev, &link))
+			return -1;
+		if (link.link_status == old.link_status)
+			return -1;
+		return 0;
+	}
+
+	link.link_status = ETH_LINK_UP; /* Interface is up */
+	link.link_duplex = ETH_LINK_FULL_DUPLEX;
+	switch (lio_dev->linfo.link.s.speed) {
+	case LIO_LINK_SPEED_10000:
+		link.link_speed = ETH_SPEED_NUM_10G;
+		break;
+	default:
+		link.link_speed = ETH_SPEED_NUM_NONE;
+		link.link_duplex = ETH_LINK_HALF_DUPLEX;
+	}
+
+	if (lio_dev_atomic_write_link_status(eth_dev, &link))
+		return -1;
+
+	if (link.link_status == old.link_status)
+		return -1;
+
+	return 0;
+}
+
 /**
  * Setup our receive queue/ringbuffer. This is the
  * queue the Octeon uses to send us packets and
@@ -246,6 +315,115 @@
 	}
 }
 
+/**
+ * Api to check link state.
+ */
+static void
+lio_dev_get_link_status(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
+	struct lio_link_status_resp *resp;
+	union octeon_link_status *ls;
+	struct lio_soft_command *sc;
+	uint32_t resp_size;
+
+	if (!lio_dev->intf_open)
+		return;
+
+	resp_size = sizeof(struct lio_link_status_resp);
+	sc = lio_alloc_soft_command(lio_dev, 0, resp_size, 0);
+	if (sc == NULL)
+		return;
+
+	resp = (struct lio_link_status_resp *)sc->virtrptr;
+	lio_prepare_soft_command(lio_dev, sc, LIO_OPCODE,
+				 LIO_OPCODE_INFO, 0, 0, 0);
+
+	/* Setting wait time in seconds */
+	sc->wait_time = LIO_MAX_CMD_TIMEOUT / 1000;
+
+	if (lio_send_soft_command(lio_dev, sc) == LIO_IQ_SEND_FAILED)
+		goto get_status_fail;
+
+	while ((*sc->status_word == LIO_COMPLETION_WORD_INIT) && --timeout) {
+		lio_flush_iq(lio_dev, lio_dev->instr_queue[sc->iq_no]);
+		rte_delay_ms(1);
+	}
+
+	if (resp->status)
+		goto get_status_fail;
+
+	ls = &resp->link_info.link;
+
+	lio_swap_8B_data((uint64_t *)ls, sizeof(union octeon_link_status) >> 3);
+
+	if (lio_dev->linfo.link.link_status64 != ls->link_status64) {
+		lio_dev->linfo.link.link_status64 = ls->link_status64;
+		lio_dev_link_update(eth_dev, 0);
+	}
+
+	lio_free_soft_command(sc);
+
+	return;
+
+get_status_fail:
+	lio_free_soft_command(sc);
+}
+
+/* This function will be invoked every LSC_TIMEOUT ns (100ms)
+ * and will update link state if it changes.
+ */
+static void
+lio_sync_link_state_check(void *eth_dev)
+{
+	struct lio_device *lio_dev =
+		(((struct rte_eth_dev *)eth_dev)->data->dev_private);
+
+	if (lio_dev->port_configured)
+		lio_dev_get_link_status(eth_dev);
+
+	/* Schedule periodic link status check.
+	 * Stop check if interface is close and start again while opening.
+	 */
+	if (lio_dev->intf_open)
+		rte_eal_alarm_set(LIO_LSC_TIMEOUT, lio_sync_link_state_check,
+				  eth_dev);
+}
+
+static int
+lio_dev_start(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	int ret = 0;
+
+	lio_dev_info(lio_dev, "Starting port %d\n", eth_dev->data->port_id);
+
+	if (lio_dev->fn_list.enable_io_queues(lio_dev))
+		return -1;
+
+	/* Ready for link status updates */
+	lio_dev->intf_open = 1;
+	rte_mb();
+
+	/* start polling for lsc */
+	ret = rte_eal_alarm_set(LIO_LSC_TIMEOUT,
+				lio_sync_link_state_check,
+				eth_dev);
+	if (ret) {
+		lio_dev_err(lio_dev,
+			    "link state check handler creation failed\n");
+		goto dev_lsc_handle_error;
+	}
+
+	return 0;
+
+dev_lsc_handle_error:
+	lio_dev->intf_open = 0;
+
+	return ret;
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -388,6 +566,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
+	lio_dev_link_update(eth_dev, 0);
+
 	lio_dev->port_configured = 1;
 
 	lio_free_soft_command(sc);
@@ -414,6 +594,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 /* Define our ethernet definitions */
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
+	.dev_start		= lio_dev_start,
+	.link_update		= lio_dev_link_update,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 22e3d83..98ff493 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -38,10 +38,17 @@
 
 #include "lio_struct.h"
 
+/* timeout to check link state updates from firmware in us */
+#define LIO_LSC_TIMEOUT		100000 /* 100000us (100ms) */
 #define LIO_MAX_CMD_TIMEOUT     10000 /* 10000ms (10s) */
 
 #define LIO_DEV(_eth_dev)		((_eth_dev)->data->dev_private)
 
+enum lio_bus_speed {
+	LIO_LINK_SPEED_UNKNOWN  = 0,
+	LIO_LINK_SPEED_10000    = 10000
+};
+
 struct octeon_if_cfg_info {
 	uint64_t iqmask;	/** mask for IQs enabled for the port */
 	uint64_t oqmask;	/** mask for OQs enabled for the port */
@@ -73,4 +80,10 @@ struct lio_if_cfg_resp {
 	struct octeon_if_cfg_info cfg_info;
 	uint64_t status;
 };
+
+struct lio_link_status_resp {
+	uint64_t rh;
+	struct octeon_link_info link_info;
+	uint64_t status;
+};
 #endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 6a1d265..d7e17bf 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1587,7 +1587,7 @@ struct lio_soft_command *
 	lio_dev = txq->lio_dev;
 	iq_no = txq->txpciq.s.q_no;
 
-	if (!lio_dev->linfo.link.s.link_up) {
+	if (!lio_dev->intf_open || !lio_dev->linfo.link.s.link_up) {
 		PMD_TX_LOG(lio_dev, ERR, "Transmit failed link_status : %d\n",
 			   lio_dev->linfo.link.s.link_up);
 		goto xmit_failed;
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 478a290..da08fe4 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -569,6 +569,8 @@ struct lio_device {
 	/** The state of this device */
 	rte_atomic64_t status;
 
+	uint8_t intf_open;
+
 	struct octeon_link_info linfo;
 
 	uint8_t *hw_addr;
-- 
1.8.3.1

[PATCH v2 32/46] net/liquidio: add APIs to alloc and send control command

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  1 +
 drivers/net/liquidio/lio_ethdev.h       |  6 +++
 drivers/net/liquidio/lio_rxtx.c         | 82 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         | 44 ++++++++++++++++++
 4 files changed, 133 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index e3f18e3..d38c835 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -122,6 +122,7 @@ enum octeon_tag_type {
 /** LIO_OPCODE subcodes */
 /* This subcode is sent by core PCI driver to indicate cores are ready. */
 #define LIO_OPCODE_NW_DATA		0x02 /* network packet data */
+#define LIO_OPCODE_CMD			0x03
 #define LIO_OPCODE_INFO			0x04
 #define LIO_OPCODE_IF_CFG		0x09
 
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 98ff493..7b5343a 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -44,6 +44,12 @@
 
 #define LIO_DEV(_eth_dev)		((_eth_dev)->data->dev_private)
 
+/* LIO Response condition variable */
+struct lio_dev_ctrl_cmd {
+	struct rte_eth_dev *eth_dev;
+	uint64_t cond;
+};
+
 enum lio_bus_speed {
 	LIO_LINK_SPEED_UNKNOWN  = 0,
 	LIO_LINK_SPEED_10000    = 10000
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index d7e17bf..c12960c 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1557,6 +1557,88 @@ struct lio_soft_command *
 	return count ? 0 : 1;
 }
 
+static void
+lio_ctrl_cmd_callback(uint32_t status __rte_unused, void *sc_ptr)
+{
+	struct lio_soft_command *sc = sc_ptr;
+	struct lio_dev_ctrl_cmd *ctrl_cmd;
+	struct lio_ctrl_pkt *ctrl_pkt;
+
+	ctrl_pkt = (struct lio_ctrl_pkt *)sc->ctxptr;
+	ctrl_cmd = ctrl_pkt->ctrl_cmd;
+	ctrl_cmd->cond = 1;
+
+	lio_free_soft_command(sc);
+}
+
+static inline struct lio_soft_command *
+lio_alloc_ctrl_pkt_sc(struct lio_device *lio_dev,
+		      struct lio_ctrl_pkt *ctrl_pkt)
+{
+	struct lio_soft_command *sc = NULL;
+	uint32_t uddsize, datasize;
+	uint32_t rdatasize;
+	uint8_t *data;
+
+	uddsize = (uint32_t)(ctrl_pkt->ncmd.s.more * 8);
+
+	datasize = OCTEON_CMD_SIZE + uddsize;
+	rdatasize = (ctrl_pkt->wait_time) ? 16 : 0;
+
+	sc = lio_alloc_soft_command(lio_dev, datasize,
+				    rdatasize, sizeof(struct lio_ctrl_pkt));
+	if (sc == NULL)
+		return NULL;
+
+	rte_memcpy(sc->ctxptr, ctrl_pkt, sizeof(struct lio_ctrl_pkt));
+
+	data = (uint8_t *)sc->virtdptr;
+
+	rte_memcpy(data, &ctrl_pkt->ncmd, OCTEON_CMD_SIZE);
+
+	lio_swap_8B_data((uint64_t *)data, OCTEON_CMD_SIZE >> 3);
+
+	if (uddsize) {
+		/* Endian-Swap for UDD should have been done by caller. */
+		rte_memcpy(data + OCTEON_CMD_SIZE, ctrl_pkt->udd, uddsize);
+	}
+
+	sc->iq_no = (uint32_t)ctrl_pkt->iq_no;
+
+	lio_prepare_soft_command(lio_dev, sc,
+				 LIO_OPCODE, LIO_OPCODE_CMD,
+				 0, 0, 0);
+
+	sc->callback = lio_ctrl_cmd_callback;
+	sc->callback_arg = sc;
+	sc->wait_time = ctrl_pkt->wait_time;
+
+	return sc;
+}
+
+int
+lio_send_ctrl_pkt(struct lio_device *lio_dev, struct lio_ctrl_pkt *ctrl_pkt)
+{
+	struct lio_soft_command *sc = NULL;
+	int retval;
+
+	sc = lio_alloc_ctrl_pkt_sc(lio_dev, ctrl_pkt);
+	if (sc == NULL) {
+		lio_dev_err(lio_dev, "soft command allocation failed\n");
+		return -1;
+	}
+
+	retval = lio_send_soft_command(lio_dev, sc);
+	if (retval == LIO_IQ_SEND_FAILED) {
+		lio_free_soft_command(sc);
+		lio_dev_err(lio_dev, "Port: %d soft command: %d send failed status: %x\n",
+			    lio_dev->port_id, ctrl_pkt->ncmd.s.cmd, retval);
+		return -1;
+	}
+
+	return retval;
+}
+
 /** Send data packet to the device
  *  @param lio_dev - lio device pointer
  *  @param ndata   - control structure with queueing, and buffer information
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 964a884..95d0007 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -249,6 +249,40 @@ struct lio_iq_post_status {
 
 #define OCTEON_CMD_SIZE (sizeof(union octeon_cmd))
 
+/* Maximum number of 8-byte words can be
+ * sent in a NIC control message.
+ */
+#define LIO_MAX_NCTRL_UDD	32
+
+/* Structure of control information passed by driver to the BASE
+ * layer when sending control commands to Octeon device software.
+ */
+struct lio_ctrl_pkt {
+	/** Command to be passed to the Octeon device software. */
+	union octeon_cmd ncmd;
+
+	/** Send buffer */
+	void *data;
+	uint64_t dmadata;
+
+	/** Response buffer */
+	void *rdata;
+	uint64_t dmardata;
+
+	/** Additional data that may be needed by some commands. */
+	uint64_t udd[LIO_MAX_NCTRL_UDD];
+
+	/** Input queue to use to send this command. */
+	uint64_t iq_no;
+
+	/** Time to wait for Octeon software to respond to this control command.
+	 *  If wait_time is 0, BASE assumes no response is expected.
+	 */
+	size_t wait_time;
+
+	struct lio_dev_ctrl_cmd *ctrl_cmd;
+};
+
 /** Structure of data information passed by driver to the BASE
  *  layer when forwarding data to Octeon device software.
  */
@@ -570,6 +604,16 @@ int lio_send_soft_command(struct lio_device *lio_dev,
 			  struct lio_soft_command *sc);
 void lio_free_soft_command(struct lio_soft_command *sc);
 
+/** Send control packet to the device
+ *  @param lio_dev - lio device pointer
+ *  @param nctrl   - control structure with command, timeout, and callback info
+ *
+ *  @returns IQ_FAILED if it failed to add to the input queue. IQ_STOP if it the
+ *  queue should be stopped, and LIO_IQ_SEND_OK if it sent okay.
+ */
+int lio_send_ctrl_pkt(struct lio_device *lio_dev,
+		      struct lio_ctrl_pkt *ctrl_pkt);
+
 /** Maximum ordered requests to process in every invocation of
  *  lio_process_ordered_list(). The function will continue to process requests
  *  as long as it can find one that has finished processing. If it keeps
-- 
1.8.3.1

[PATCH v2 33/46] net/liquidio: add API to control Rx

[Shijith Thotton]

Enable or disable packet reception.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  3 ++
 drivers/net/liquidio/lio_ethdev.c       | 59 +++++++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index d38c835..59668c0 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -128,6 +128,9 @@ enum octeon_tag_type {
 
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
 
+/* NIC Command types */
+#define LIO_CMD_RX_CTL			0x4
+
 /* RX(packets coming from wire) Checksum verification flags */
 /* TCP/UDP csum */
 #define LIO_L4_CSUM_VERIFIED		0x1
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 4962cad..c698c70 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -41,6 +41,61 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+/* Wait for control command to reach nic. */
+static uint16_t
+lio_wait_for_ctrl_cmd(struct lio_device *lio_dev,
+		      struct lio_dev_ctrl_cmd *ctrl_cmd)
+{
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
+
+	while ((ctrl_cmd->cond == 0) && --timeout) {
+		lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+		rte_delay_ms(1);
+	}
+
+	return !timeout;
+}
+
+/**
+ * \brief Send Rx control command
+ * @param eth_dev Pointer to the structure rte_eth_dev
+ * @param start_stop whether to start or stop
+ */
+static int
+lio_send_rx_ctrl_cmd(struct rte_eth_dev *eth_dev, int start_stop)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_RX_CTL;
+	ctrl_pkt.ncmd.s.param1 = start_stop;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send RX Control message\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "RX Control command timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Atomically writes the link status information into global
  * structure rte_eth_dev.
@@ -402,6 +457,9 @@
 	if (lio_dev->fn_list.enable_io_queues(lio_dev))
 		return -1;
 
+	if (lio_send_rx_ctrl_cmd(eth_dev, 1))
+		return -1;
+
 	/* Ready for link status updates */
 	lio_dev->intf_open = 1;
 	rte_mb();
@@ -420,6 +478,7 @@
 
 dev_lsc_handle_error:
 	lio_dev->intf_open = 0;
+	lio_send_rx_ctrl_cmd(eth_dev, 0);
 
 	return ret;
 }
-- 
1.8.3.1

[PATCH v2 34/46] net/liquidio: add RSS support

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  26 +++
 drivers/net/liquidio/lio_ethdev.c       | 336 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  21 ++
 drivers/net/liquidio/lio_struct.h       |  16 ++
 4 files changed, 399 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 59668c0..8272162 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -130,12 +130,38 @@ enum octeon_tag_type {
 
 /* NIC Command types */
 #define LIO_CMD_RX_CTL			0x4
+#define LIO_CMD_SET_RSS			0xD
 
 /* RX(packets coming from wire) Checksum verification flags */
 /* TCP/UDP csum */
 #define LIO_L4_CSUM_VERIFIED		0x1
 #define LIO_IP_CSUM_VERIFIED		0x2
 
+/* RSS */
+#define LIO_RSS_PARAM_DISABLE_RSS		0x10
+#define LIO_RSS_PARAM_HASH_KEY_UNCHANGED	0x08
+#define LIO_RSS_PARAM_ITABLE_UNCHANGED		0x04
+#define LIO_RSS_PARAM_HASH_INFO_UNCHANGED	0x02
+
+#define LIO_RSS_HASH_IPV4			0x100
+#define LIO_RSS_HASH_TCP_IPV4			0x200
+#define LIO_RSS_HASH_IPV6			0x400
+#define LIO_RSS_HASH_TCP_IPV6			0x1000
+#define LIO_RSS_HASH_IPV6_EX			0x800
+#define LIO_RSS_HASH_TCP_IPV6_EX		0x2000
+
+#define LIO_RSS_OFFLOAD_ALL (		\
+		LIO_RSS_HASH_IPV4 |	\
+		LIO_RSS_HASH_TCP_IPV4 |	\
+		LIO_RSS_HASH_IPV6 |	\
+		LIO_RSS_HASH_TCP_IPV6 |	\
+		LIO_RSS_HASH_IPV6_EX |	\
+		LIO_RSS_HASH_TCP_IPV6_EX)
+
+#define LIO_RSS_MAX_TABLE_SZ		128
+#define LIO_RSS_MAX_KEY_SZ		40
+#define LIO_RSS_PARAM_SIZE		16
+
 /* Interface flags communicated between host driver and core app. */
 enum lio_ifflags {
 	LIO_IFFLAG_UNICAST	= 0x10
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index c698c70..58a932e 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -41,6 +41,15 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+/* Default RSS key in use */
+static uint8_t lio_rss_key[40] = {
+	0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2,
+	0x41, 0x67, 0x25, 0x3D, 0x43, 0xA3, 0x8F, 0xB0,
+	0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+	0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C,
+	0x6A, 0x42, 0xB7, 0x3B, 0xBE, 0xAC, 0x01, 0xFA,
+};
+
 /* Wait for control command to reach nic. */
 static uint16_t
 lio_wait_for_ctrl_cmd(struct lio_device *lio_dev,
@@ -96,6 +105,267 @@
 	return 0;
 }
 
+static int
+lio_dev_rss_reta_update(struct rte_eth_dev *eth_dev,
+			struct rte_eth_rss_reta_entry64 *reta_conf,
+			uint16_t reta_size)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	struct lio_rss_set *rss_param;
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+	int i, j, index;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't update reta\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	if (reta_size != LIO_RSS_MAX_TABLE_SZ) {
+		lio_dev_err(lio_dev,
+			    "The size of hash lookup table configured (%d) doesn't match the number hardware can supported (%d)\n",
+			    reta_size, LIO_RSS_MAX_TABLE_SZ);
+		return -EINVAL;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	rss_param = (struct lio_rss_set *)&ctrl_pkt.udd[0];
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_SET_RSS;
+	ctrl_pkt.ncmd.s.more = sizeof(struct lio_rss_set) >> 3;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	rss_param->param.flags = 0xF;
+	rss_param->param.flags &= ~LIO_RSS_PARAM_ITABLE_UNCHANGED;
+	rss_param->param.itablesize = LIO_RSS_MAX_TABLE_SZ;
+
+	for (i = 0; i < (reta_size / RTE_RETA_GROUP_SIZE); i++) {
+		for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) {
+			if ((reta_conf[i].mask) & ((uint64_t)1 << j)) {
+				index = (i * RTE_RETA_GROUP_SIZE) + j;
+				rss_state->itable[index] = reta_conf[i].reta[j];
+			}
+		}
+	}
+
+	rss_state->itable_size = LIO_RSS_MAX_TABLE_SZ;
+	memcpy(rss_param->itable, rss_state->itable, rss_state->itable_size);
+
+	lio_swap_8B_data((uint64_t *)rss_param, LIO_RSS_PARAM_SIZE >> 3);
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to set rss hash\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "Set rss hash timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+static int
+lio_dev_rss_reta_query(struct rte_eth_dev *eth_dev,
+		       struct rte_eth_rss_reta_entry64 *reta_conf,
+		       uint16_t reta_size)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	int i, num;
+
+	if (reta_size != LIO_RSS_MAX_TABLE_SZ) {
+		lio_dev_err(lio_dev,
+			    "The size of hash lookup table configured (%d) doesn't match the number hardware can supported (%d)\n",
+			    reta_size, LIO_RSS_MAX_TABLE_SZ);
+		return -EINVAL;
+	}
+
+	num = reta_size / RTE_RETA_GROUP_SIZE;
+
+	for (i = 0; i < num; i++) {
+		memcpy(reta_conf->reta,
+		       &rss_state->itable[i * RTE_RETA_GROUP_SIZE],
+		       RTE_RETA_GROUP_SIZE);
+		reta_conf++;
+	}
+
+	return 0;
+}
+
+static int
+lio_dev_rss_hash_conf_get(struct rte_eth_dev *eth_dev,
+			  struct rte_eth_rss_conf *rss_conf)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	uint8_t *hash_key = NULL;
+	uint64_t rss_hf = 0;
+
+	if (rss_state->hash_disable) {
+		lio_dev_info(lio_dev, "RSS disabled in nic\n");
+		rss_conf->rss_hf = 0;
+		return 0;
+	}
+
+	/* Get key value */
+	hash_key = rss_conf->rss_key;
+	if (hash_key != NULL)
+		memcpy(hash_key, rss_state->hash_key, rss_state->hash_key_size);
+
+	if (rss_state->ip)
+		rss_hf |= ETH_RSS_IPV4;
+	if (rss_state->tcp_hash)
+		rss_hf |= ETH_RSS_NONFRAG_IPV4_TCP;
+	if (rss_state->ipv6)
+		rss_hf |= ETH_RSS_IPV6;
+	if (rss_state->ipv6_tcp_hash)
+		rss_hf |= ETH_RSS_NONFRAG_IPV6_TCP;
+	if (rss_state->ipv6_ex)
+		rss_hf |= ETH_RSS_IPV6_EX;
+	if (rss_state->ipv6_tcp_ex_hash)
+		rss_hf |= ETH_RSS_IPV6_TCP_EX;
+
+	rss_conf->rss_hf = rss_hf;
+
+	return 0;
+}
+
+static int
+lio_dev_rss_hash_update(struct rte_eth_dev *eth_dev,
+			struct rte_eth_rss_conf *rss_conf)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	struct lio_rss_set *rss_param;
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't update hash\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	rss_param = (struct lio_rss_set *)&ctrl_pkt.udd[0];
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_SET_RSS;
+	ctrl_pkt.ncmd.s.more = sizeof(struct lio_rss_set) >> 3;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	rss_param->param.flags = 0xF;
+
+	if (rss_conf->rss_key) {
+		rss_param->param.flags &= ~LIO_RSS_PARAM_HASH_KEY_UNCHANGED;
+		rss_state->hash_key_size = LIO_RSS_MAX_KEY_SZ;
+		rss_param->param.hashkeysize = LIO_RSS_MAX_KEY_SZ;
+		memcpy(rss_state->hash_key, rss_conf->rss_key,
+		       rss_state->hash_key_size);
+		memcpy(rss_param->key, rss_state->hash_key,
+		       rss_state->hash_key_size);
+	}
+
+	if ((rss_conf->rss_hf & LIO_RSS_OFFLOAD_ALL) == 0) {
+		/* Can't disable rss through hash flags,
+		 * if it is enabled by default during init
+		 */
+		if (!rss_state->hash_disable)
+			return -EINVAL;
+
+		/* This is for --disable-rss during testpmd launch */
+		rss_param->param.flags |= LIO_RSS_PARAM_DISABLE_RSS;
+	} else {
+		uint32_t hashinfo = 0;
+
+		/* Can't enable rss if disabled by default during init */
+		if (rss_state->hash_disable)
+			return -EINVAL;
+
+		if (rss_conf->rss_hf & ETH_RSS_IPV4) {
+			hashinfo |= LIO_RSS_HASH_IPV4;
+			rss_state->ip = 1;
+		} else {
+			rss_state->ip = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_NONFRAG_IPV4_TCP) {
+			hashinfo |= LIO_RSS_HASH_TCP_IPV4;
+			rss_state->tcp_hash = 1;
+		} else {
+			rss_state->tcp_hash = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_IPV6) {
+			hashinfo |= LIO_RSS_HASH_IPV6;
+			rss_state->ipv6 = 1;
+		} else {
+			rss_state->ipv6 = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_NONFRAG_IPV6_TCP) {
+			hashinfo |= LIO_RSS_HASH_TCP_IPV6;
+			rss_state->ipv6_tcp_hash = 1;
+		} else {
+			rss_state->ipv6_tcp_hash = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_IPV6_EX) {
+			hashinfo |= LIO_RSS_HASH_IPV6_EX;
+			rss_state->ipv6_ex = 1;
+		} else {
+			rss_state->ipv6_ex = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_IPV6_TCP_EX) {
+			hashinfo |= LIO_RSS_HASH_TCP_IPV6_EX;
+			rss_state->ipv6_tcp_ex_hash = 1;
+		} else {
+			rss_state->ipv6_tcp_ex_hash = 0;
+		}
+
+		rss_param->param.flags &= ~LIO_RSS_PARAM_HASH_INFO_UNCHANGED;
+		rss_param->param.hashinfo = hashinfo;
+	}
+
+	lio_swap_8B_data((uint64_t *)rss_param, LIO_RSS_PARAM_SIZE >> 3);
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to set rss hash\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "Set rss hash timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Atomically writes the link status information into global
  * structure rte_eth_dev.
@@ -179,6 +449,65 @@
 	return 0;
 }
 
+static void
+lio_dev_rss_configure(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	struct rte_eth_rss_reta_entry64 reta_conf[8];
+	struct rte_eth_rss_conf rss_conf;
+	uint16_t i;
+
+	/* Configure the RSS key and the RSS protocols used to compute
+	 * the RSS hash of input packets.
+	 */
+	rss_conf = eth_dev->data->dev_conf.rx_adv_conf.rss_conf;
+	if ((rss_conf.rss_hf & LIO_RSS_OFFLOAD_ALL) == 0) {
+		rss_state->hash_disable = 1;
+		lio_dev_rss_hash_update(eth_dev, &rss_conf);
+		return;
+	}
+
+	if (rss_conf.rss_key == NULL)
+		rss_conf.rss_key = lio_rss_key; /* Default hash key */
+
+	lio_dev_rss_hash_update(eth_dev, &rss_conf);
+
+	memset(reta_conf, 0, sizeof(reta_conf));
+	for (i = 0; i < LIO_RSS_MAX_TABLE_SZ; i++) {
+		uint8_t q_idx, conf_idx, reta_idx;
+
+		q_idx = (uint8_t)((eth_dev->data->nb_rx_queues > 1) ?
+				  i % eth_dev->data->nb_rx_queues : 0);
+		conf_idx = i / RTE_RETA_GROUP_SIZE;
+		reta_idx = i % RTE_RETA_GROUP_SIZE;
+		reta_conf[conf_idx].reta[reta_idx] = q_idx;
+		reta_conf[conf_idx].mask |= ((uint64_t)1 << reta_idx);
+	}
+
+	lio_dev_rss_reta_update(eth_dev, reta_conf, LIO_RSS_MAX_TABLE_SZ);
+}
+
+static void
+lio_dev_mq_rx_configure(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	struct rte_eth_rss_conf rss_conf;
+
+	switch (eth_dev->data->dev_conf.rxmode.mq_mode) {
+	case ETH_MQ_RX_RSS:
+		lio_dev_rss_configure(eth_dev);
+		break;
+	case ETH_MQ_RX_NONE:
+	/* if mq_mode is none, disable rss mode. */
+	default:
+		memset(&rss_conf, 0, sizeof(rss_conf));
+		rss_state->hash_disable = 1;
+		lio_dev_rss_hash_update(eth_dev, &rss_conf);
+	}
+}
+
 /**
  * Setup our receive queue/ringbuffer. This is the
  * queue the Octeon uses to send us packets and
@@ -464,6 +793,9 @@
 	lio_dev->intf_open = 1;
 	rte_mb();
 
+	/* Configure RSS if device configured with multiple RX queues. */
+	lio_dev_mq_rx_configure(eth_dev);
+
 	/* start polling for lsc */
 	ret = rte_eal_alarm_set(LIO_LSC_TIMEOUT,
 				lio_sync_link_state_check,
@@ -659,6 +991,10 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
 	.tx_queue_release	= lio_dev_tx_queue_release,
+	.reta_update		= lio_dev_rss_reta_update,
+	.reta_query		= lio_dev_rss_reta_query,
+	.rss_hash_conf_get	= lio_dev_rss_hash_conf_get,
+	.rss_hash_update	= lio_dev_rss_hash_update,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 7b5343a..6543061 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -92,4 +92,25 @@ struct lio_link_status_resp {
 	struct octeon_link_info link_info;
 	uint64_t status;
 };
+
+struct lio_rss_set {
+	struct param {
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+		uint64_t flags : 16;
+		uint64_t hashinfo : 32;
+		uint64_t itablesize : 16;
+		uint64_t hashkeysize : 16;
+		uint64_t reserved : 48;
+#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t itablesize : 16;
+		uint64_t hashinfo : 32;
+		uint64_t flags : 16;
+		uint64_t reserved : 48;
+		uint64_t hashkeysize : 16;
+#endif
+	} param;
+
+	uint8_t itable[LIO_RSS_MAX_TABLE_SZ];
+	uint8_t key[LIO_RSS_MAX_KEY_SZ];
+};
 #endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index da08fe4..c2293f7 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -357,6 +357,21 @@ struct lio_gather {
 	struct lio_sg_entry *sg;
 };
 
+struct lio_rss_ctx {
+	uint16_t hash_key_size;
+	uint8_t  hash_key[LIO_RSS_MAX_KEY_SZ];
+	/* Ideally a factor of number of queues */
+	uint8_t  itable[LIO_RSS_MAX_TABLE_SZ];
+	uint8_t  itable_size;
+	uint8_t  ip;
+	uint8_t  tcp_hash;
+	uint8_t  ipv6;
+	uint8_t  ipv6_tcp_hash;
+	uint8_t  ipv6_ex;
+	uint8_t  ipv6_tcp_ex_hash;
+	uint8_t  hash_disable;
+};
+
 struct lio_io_enable {
 	uint64_t iq;
 	uint64_t oq;
@@ -619,6 +634,7 @@ struct lio_device {
 	uint8_t nb_rx_queues;
 	uint8_t nb_tx_queues;
 	uint8_t port_configured;
+	struct lio_rss_ctx rss_state;
 	uint8_t port_id;
 };
 #endif /* _LIO_STRUCT_H_ */
-- 
1.8.3.1

[PATCH v2 35/46] net/liquidio: add API to get device info

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  1 +
 drivers/net/liquidio/lio_ethdev.c       | 47 +++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 8272162..3ea2e0f 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -126,6 +126,7 @@ enum octeon_tag_type {
 #define LIO_OPCODE_INFO			0x04
 #define LIO_OPCODE_IF_CFG		0x09
 
+#define LIO_MIN_RX_BUF_SIZE		64
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
 
 /* NIC Command types */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 58a932e..e2040b9 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -50,6 +50,18 @@
 	0x6A, 0x42, 0xB7, 0x3B, 0xBE, 0xAC, 0x01, 0xFA,
 };
 
+static const struct rte_eth_desc_lim lio_rx_desc_lim = {
+	.nb_max		= CN23XX_MAX_OQ_DESCRIPTORS,
+	.nb_min		= CN23XX_MIN_OQ_DESCRIPTORS,
+	.nb_align	= 1,
+};
+
+static const struct rte_eth_desc_lim lio_tx_desc_lim = {
+	.nb_max		= CN23XX_MAX_IQ_DESCRIPTORS,
+	.nb_min		= CN23XX_MIN_IQ_DESCRIPTORS,
+	.nb_align	= 1,
+};
+
 /* Wait for control command to reach nic. */
 static uint16_t
 lio_wait_for_ctrl_cmd(struct lio_device *lio_dev,
@@ -105,6 +117,40 @@
 	return 0;
 }
 
+static void
+lio_dev_info_get(struct rte_eth_dev *eth_dev,
+		 struct rte_eth_dev_info *devinfo)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	devinfo->max_rx_queues = lio_dev->max_rx_queues;
+	devinfo->max_tx_queues = lio_dev->max_tx_queues;
+
+	devinfo->min_rx_bufsize = LIO_MIN_RX_BUF_SIZE;
+	devinfo->max_rx_pktlen = LIO_MAX_RX_PKTLEN;
+
+	devinfo->max_mac_addrs = 1;
+
+	devinfo->rx_offload_capa = (DEV_RX_OFFLOAD_IPV4_CKSUM		|
+				    DEV_RX_OFFLOAD_UDP_CKSUM		|
+				    DEV_RX_OFFLOAD_TCP_CKSUM);
+	devinfo->tx_offload_capa = (DEV_TX_OFFLOAD_IPV4_CKSUM		|
+				    DEV_TX_OFFLOAD_UDP_CKSUM		|
+				    DEV_TX_OFFLOAD_TCP_CKSUM);
+
+	devinfo->rx_desc_lim = lio_rx_desc_lim;
+	devinfo->tx_desc_lim = lio_tx_desc_lim;
+
+	devinfo->reta_size = LIO_RSS_MAX_TABLE_SZ;
+	devinfo->hash_key_size = LIO_RSS_MAX_KEY_SZ;
+	devinfo->flow_type_rss_offloads = (ETH_RSS_IPV4			|
+					   ETH_RSS_NONFRAG_IPV4_TCP	|
+					   ETH_RSS_IPV6			|
+					   ETH_RSS_NONFRAG_IPV6_TCP	|
+					   ETH_RSS_IPV6_EX		|
+					   ETH_RSS_IPV6_TCP_EX);
+}
+
 static int
 lio_dev_rss_reta_update(struct rte_eth_dev *eth_dev,
 			struct rte_eth_rss_reta_entry64 *reta_conf,
@@ -987,6 +1033,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.dev_configure		= lio_dev_configure,
 	.dev_start		= lio_dev_start,
 	.link_update		= lio_dev_link_update,
+	.dev_infos_get		= lio_dev_info_get,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
-- 
1.8.3.1

[PATCH v2 36/46] net/liquidio: add API to set MTU

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 47 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index e2040b9..167d41e 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -152,6 +152,33 @@
 }
 
 static int
+lio_dev_change_vf_mtu(struct rte_eth_dev *eth_dev, uint16_t new_mtu)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	PMD_INIT_FUNC_TRACE();
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't change MTU\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	/* Limit the MTU to make sure the ethernet packets are between
+	 * ETHER_MIN_MTU bytes and PF's MTU
+	 */
+	if ((new_mtu < ETHER_MIN_MTU) ||
+			(new_mtu > lio_dev->linfo.link.s.mtu)) {
+		lio_dev_err(lio_dev, "Invalid MTU: %d\n", new_mtu);
+		lio_dev_err(lio_dev, "Valid range %d and %d\n",
+			    ETHER_MIN_MTU, lio_dev->linfo.link.s.mtu);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int
 lio_dev_rss_reta_update(struct rte_eth_dev *eth_dev,
 			struct rte_eth_rss_reta_entry64 *reta_conf,
 			uint16_t reta_size)
@@ -824,7 +851,9 @@
 static int
 lio_dev_start(struct rte_eth_dev *eth_dev)
 {
+	uint16_t mtu = eth_dev->data->dev_conf.rxmode.max_rx_pkt_len;
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
 	int ret = 0;
 
 	lio_dev_info(lio_dev, "Starting port %d\n", eth_dev->data->port_id);
@@ -852,8 +881,25 @@
 		goto dev_lsc_handle_error;
 	}
 
+	while ((lio_dev->linfo.link.link_status64 == 0) && (--timeout))
+		rte_delay_ms(1);
+
+	if (lio_dev->linfo.link.link_status64 == 0) {
+		ret = -1;
+		goto dev_mtu_updation_error;
+	}
+
+	if (lio_dev->linfo.link.s.mtu != mtu) {
+		ret = lio_dev_change_vf_mtu(eth_dev, mtu);
+		if (ret)
+			goto dev_mtu_updation_error;
+	}
+
 	return 0;
 
+dev_mtu_updation_error:
+	rte_eal_alarm_cancel(lio_sync_link_state_check, eth_dev);
+
 dev_lsc_handle_error:
 	lio_dev->intf_open = 0;
 	lio_send_rx_ctrl_cmd(eth_dev, 0);
@@ -1034,6 +1080,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.dev_start		= lio_dev_start,
 	.link_update		= lio_dev_link_update,
 	.dev_infos_get		= lio_dev_info_get,
+	.mtu_set		= lio_dev_change_vf_mtu,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
-- 
1.8.3.1

Re: [PATCH v2 36/46] net/liquidio: add API to set MTU

[Ferruh Yigit]

On 3/2/2017 11:32 AM, Shijith Thotton wrote:
> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>

<...>

>  
>  static int
> +lio_dev_change_vf_mtu(struct rte_eth_dev *eth_dev, uint16_t new_mtu)
> +{
> +	struct lio_device *lio_dev = LIO_DEV(eth_dev);
> +
> +	PMD_INIT_FUNC_TRACE();
> +
> +	if (!lio_dev->intf_open) {
> +		lio_dev_err(lio_dev, "Port %d down, can't change MTU\n",
> +			    lio_dev->port_id);
> +		return -EINVAL;
> +	}
> +
> +	/* Limit the MTU to make sure the ethernet packets are between
> +	 * ETHER_MIN_MTU bytes and PF's MTU
> +	 */
> +	if ((new_mtu < ETHER_MIN_MTU) ||
> +			(new_mtu > lio_dev->linfo.link.s.mtu)) {
> +		lio_dev_err(lio_dev, "Invalid MTU: %d\n", new_mtu);
> +		lio_dev_err(lio_dev, "Valid range %d and %d\n",
> +			    ETHER_MIN_MTU, lio_dev->linfo.link.s.mtu);
> +		return -EINVAL;
> +	}
> +
> +	return 0;
> +}

Is this really sets the MTU?
"new_mtu" seems not used, except limit check, an lio_send_ctrl_pkt()
required perhaps?

Re: [PATCH v2 36/46] net/liquidio: add API to set MTU

[Shijith Thotton]

On Tue, Mar 21, 2017 at 12:24:49PM +0000, Ferruh Yigit wrote:
> On 3/2/2017 11:32 AM, Shijith Thotton wrote:
> > Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> > Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> > Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> > Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
> > Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
> 
> <...>
> 
> >  
> >  static int
> > +lio_dev_change_vf_mtu(struct rte_eth_dev *eth_dev, uint16_t new_mtu)
> > +{
> > +	struct lio_device *lio_dev = LIO_DEV(eth_dev);
> > +
> > +	PMD_INIT_FUNC_TRACE();
> > +
> > +	if (!lio_dev->intf_open) {
> > +		lio_dev_err(lio_dev, "Port %d down, can't change MTU\n",
> > +			    lio_dev->port_id);
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* Limit the MTU to make sure the ethernet packets are between
> > +	 * ETHER_MIN_MTU bytes and PF's MTU
> > +	 */
> > +	if ((new_mtu < ETHER_MIN_MTU) ||
> > +			(new_mtu > lio_dev->linfo.link.s.mtu)) {
> > +		lio_dev_err(lio_dev, "Invalid MTU: %d\n", new_mtu);
> > +		lio_dev_err(lio_dev, "Valid range %d and %d\n",
> > +			    ETHER_MIN_MTU, lio_dev->linfo.link.s.mtu);
> > +		return -EINVAL;
> > +	}
> > +
> > +	return 0;
> > +}
> 
> Is this really sets the MTU?
> "new_mtu" seems not used, except limit check, an lio_send_ctrl_pkt()
> required perhaps?

It won't set MTU for hardware and is possible only by PF. So
lio_send_ctrl_pkt is not required. VF MTU is limited by PF MTU and is
mentioned under limitations in driver documentation. Here we are
allowing upper layer to set MTU up to the value configured by PF.

Shijith

Re: [PATCH v2 36/46] net/liquidio: add API to set MTU

[Ferruh Yigit]

On 3/21/2017 12:53 PM, Shijith Thotton wrote:
> On Tue, Mar 21, 2017 at 12:24:49PM +0000, Ferruh Yigit wrote:
>> On 3/2/2017 11:32 AM, Shijith Thotton wrote:
>>> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
>>> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
>>> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
>>> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
>>> Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
>>> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
>>
>> <...>
>>
>>>  
>>>  static int
>>> +lio_dev_change_vf_mtu(struct rte_eth_dev *eth_dev, uint16_t new_mtu)
>>> +{
>>> +	struct lio_device *lio_dev = LIO_DEV(eth_dev);
>>> +
>>> +	PMD_INIT_FUNC_TRACE();
>>> +
>>> +	if (!lio_dev->intf_open) {
>>> +		lio_dev_err(lio_dev, "Port %d down, can't change MTU\n",
>>> +			    lio_dev->port_id);
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	/* Limit the MTU to make sure the ethernet packets are between
>>> +	 * ETHER_MIN_MTU bytes and PF's MTU
>>> +	 */
>>> +	if ((new_mtu < ETHER_MIN_MTU) ||
>>> +			(new_mtu > lio_dev->linfo.link.s.mtu)) {
>>> +		lio_dev_err(lio_dev, "Invalid MTU: %d\n", new_mtu);
>>> +		lio_dev_err(lio_dev, "Valid range %d and %d\n",
>>> +			    ETHER_MIN_MTU, lio_dev->linfo.link.s.mtu);
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	return 0;
>>> +}
>>
>> Is this really sets the MTU?
>> "new_mtu" seems not used, except limit check, an lio_send_ctrl_pkt()
>> required perhaps?
> 
> It won't set MTU for hardware and is possible only by PF. So
> lio_send_ctrl_pkt is not required. VF MTU is limited by PF MTU and is
> mentioned under limitations in driver documentation. Here we are
> allowing upper layer to set MTU up to the value configured by PF.

I see, but lio_dev_change_vf_mtu() does not set anything at all. If it
is not modifying anything at all, why you claim "MTU update" supported?

And following logic seems wrong for this case:

	...
	if (lio_dev->linfo.link.s.mtu != mtu) {
		ret = lio_dev_change_vf_mtu(eth_dev, mtu);
	...

Should this functions set lio_dev->linfo.link.s.mtu at least, perhaps?

> 
> Shijith
> 

Re: [PATCH v2 36/46] net/liquidio: add API to set MTU

[Shijith Thotton]

On Tue, Mar 21, 2017 at 01:01:58PM +0000, Ferruh Yigit wrote:
> On 3/21/2017 12:53 PM, Shijith Thotton wrote:
> > On Tue, Mar 21, 2017 at 12:24:49PM +0000, Ferruh Yigit wrote:
> >> On 3/2/2017 11:32 AM, Shijith Thotton wrote:
> >>> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> >>> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> >>> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> >>> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> >>> Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
> >>> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
> >>
> >> <...>
> >>
> >>>  
> >>>  static int
> >>> +lio_dev_change_vf_mtu(struct rte_eth_dev *eth_dev, uint16_t new_mtu)
> >>> +{
> >>> +	struct lio_device *lio_dev = LIO_DEV(eth_dev);
> >>> +
> >>> +	PMD_INIT_FUNC_TRACE();
> >>> +
> >>> +	if (!lio_dev->intf_open) {
> >>> +		lio_dev_err(lio_dev, "Port %d down, can't change MTU\n",
> >>> +			    lio_dev->port_id);
> >>> +		return -EINVAL;
> >>> +	}
> >>> +
> >>> +	/* Limit the MTU to make sure the ethernet packets are between
> >>> +	 * ETHER_MIN_MTU bytes and PF's MTU
> >>> +	 */
> >>> +	if ((new_mtu < ETHER_MIN_MTU) ||
> >>> +			(new_mtu > lio_dev->linfo.link.s.mtu)) {
> >>> +		lio_dev_err(lio_dev, "Invalid MTU: %d\n", new_mtu);
> >>> +		lio_dev_err(lio_dev, "Valid range %d and %d\n",
> >>> +			    ETHER_MIN_MTU, lio_dev->linfo.link.s.mtu);
> >>> +		return -EINVAL;
> >>> +	}
> >>> +
> >>> +	return 0;
> >>> +}
> >>
> >> Is this really sets the MTU?
> >> "new_mtu" seems not used, except limit check, an lio_send_ctrl_pkt()
> >> required perhaps?
> > 
> > It won't set MTU for hardware and is possible only by PF. So
> > lio_send_ctrl_pkt is not required. VF MTU is limited by PF MTU and is
> > mentioned under limitations in driver documentation. Here we are
> > allowing upper layer to set MTU up to the value configured by PF.
> 
> I see, but lio_dev_change_vf_mtu() does not set anything at all. If it
> is not modifying anything at all, why you claim "MTU update" supported?

We allow update for the upper layer till the value supported by PF even
though there are no real MTU update happening at hardware level.
Thought it is ok to have it mentioned under limitation.
Two options are:
1. mark support as partial.
2. remove this patch and support.

Please suggest which one is better.

> 
> And following logic seems wrong for this case:
> 
> 	...
> 	if (lio_dev->linfo.link.s.mtu != mtu) {
> 		ret = lio_dev_change_vf_mtu(eth_dev, mtu);
> 	...
> 
> Should this functions set lio_dev->linfo.link.s.mtu at least, perhaps?

lio_dev->linfo.link.s.mtu represents the MTU supported by the device and
it gets updated if there is a change by PF.

> 
<...>

Re: [PATCH v2 36/46] net/liquidio: add API to set MTU

[Ferruh Yigit]

On 3/21/2017 1:56 PM, Shijith Thotton wrote:
> On Tue, Mar 21, 2017 at 01:01:58PM +0000, Ferruh Yigit wrote:
>> On 3/21/2017 12:53 PM, Shijith Thotton wrote:
>>> On Tue, Mar 21, 2017 at 12:24:49PM +0000, Ferruh Yigit wrote:
>>>> On 3/2/2017 11:32 AM, Shijith Thotton wrote:
>>>>> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
>>>>> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
>>>>> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
>>>>> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
>>>>> Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
>>>>> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
>>>>
>>>> <...>
>>>>
>>>>>  
>>>>>  static int
>>>>> +lio_dev_change_vf_mtu(struct rte_eth_dev *eth_dev, uint16_t new_mtu)
>>>>> +{
>>>>> +	struct lio_device *lio_dev = LIO_DEV(eth_dev);
>>>>> +
>>>>> +	PMD_INIT_FUNC_TRACE();
>>>>> +
>>>>> +	if (!lio_dev->intf_open) {
>>>>> +		lio_dev_err(lio_dev, "Port %d down, can't change MTU\n",
>>>>> +			    lio_dev->port_id);
>>>>> +		return -EINVAL;
>>>>> +	}
>>>>> +
>>>>> +	/* Limit the MTU to make sure the ethernet packets are between
>>>>> +	 * ETHER_MIN_MTU bytes and PF's MTU
>>>>> +	 */
>>>>> +	if ((new_mtu < ETHER_MIN_MTU) ||
>>>>> +			(new_mtu > lio_dev->linfo.link.s.mtu)) {
>>>>> +		lio_dev_err(lio_dev, "Invalid MTU: %d\n", new_mtu);
>>>>> +		lio_dev_err(lio_dev, "Valid range %d and %d\n",
>>>>> +			    ETHER_MIN_MTU, lio_dev->linfo.link.s.mtu);
>>>>> +		return -EINVAL;
>>>>> +	}
>>>>> +
>>>>> +	return 0;
>>>>> +}
>>>>
>>>> Is this really sets the MTU?
>>>> "new_mtu" seems not used, except limit check, an lio_send_ctrl_pkt()
>>>> required perhaps?
>>>
>>> It won't set MTU for hardware and is possible only by PF. So
>>> lio_send_ctrl_pkt is not required. VF MTU is limited by PF MTU and is
>>> mentioned under limitations in driver documentation. Here we are
>>> allowing upper layer to set MTU up to the value configured by PF.
>>
>> I see, but lio_dev_change_vf_mtu() does not set anything at all. If it
>> is not modifying anything at all, why you claim "MTU update" supported?
> 
> We allow update for the upper layer till the value supported by PF even
> though there are no real MTU update happening at hardware level.
> Thought it is ok to have it mentioned under limitation.
> Two options are:
> 1. mark support as partial.
> 2. remove this patch and support.
> 
> Please suggest which one is better.

If I get it right, it is not possible to set VF MTU, so I would suggest
removing MTU update support.

But you may want to keep the patch for MTU validation, and change
function name according.

> 
>>
>> And following logic seems wrong for this case:
>>
>> 	...
>> 	if (lio_dev->linfo.link.s.mtu != mtu) {
>> 		ret = lio_dev_change_vf_mtu(eth_dev, mtu);
>> 	...
>>
>> Should this functions set lio_dev->linfo.link.s.mtu at least, perhaps?
> 
> lio_dev->linfo.link.s.mtu represents the MTU supported by the device and
> it gets updated if there is a change by PF.

So, "lio_dev->linfo.link.s.mtu" is device MTU set by PF, "mtu" is VF
eth_dev configured value.
If they are not same, lio_dev_change_vf_mtu() does check if "mtu" is in
valid range and return success or failure, right?
So, this is just configuration validation, nothing changed/set here.

> 
>>
> <...>
> 

Re: [PATCH v2 36/46] net/liquidio: add API to set MTU

[Shijith Thotton]

On Tue, Mar 21, 2017 at 02:09:44PM +0000, Ferruh Yigit wrote:
> On 3/21/2017 1:56 PM, Shijith Thotton wrote:
> > On Tue, Mar 21, 2017 at 01:01:58PM +0000, Ferruh Yigit wrote:
> >> On 3/21/2017 12:53 PM, Shijith Thotton wrote:
> >>> On Tue, Mar 21, 2017 at 12:24:49PM +0000, Ferruh Yigit wrote:
> >>>> On 3/2/2017 11:32 AM, Shijith Thotton wrote:
> >>>>> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> >>>>> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> >>>>> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> >>>>> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> >>>>> Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
> >>>>> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
> >>>>
> >>>> <...>
> >>>>
> >>>>>  
> >>>>>  static int
> >>>>> +lio_dev_change_vf_mtu(struct rte_eth_dev *eth_dev, uint16_t new_mtu)
> >>>>> +{
> >>>>> +	struct lio_device *lio_dev = LIO_DEV(eth_dev);
> >>>>> +
> >>>>> +	PMD_INIT_FUNC_TRACE();
> >>>>> +
> >>>>> +	if (!lio_dev->intf_open) {
> >>>>> +		lio_dev_err(lio_dev, "Port %d down, can't change MTU\n",
> >>>>> +			    lio_dev->port_id);
> >>>>> +		return -EINVAL;
> >>>>> +	}
> >>>>> +
> >>>>> +	/* Limit the MTU to make sure the ethernet packets are between
> >>>>> +	 * ETHER_MIN_MTU bytes and PF's MTU
> >>>>> +	 */
> >>>>> +	if ((new_mtu < ETHER_MIN_MTU) ||
> >>>>> +			(new_mtu > lio_dev->linfo.link.s.mtu)) {
> >>>>> +		lio_dev_err(lio_dev, "Invalid MTU: %d\n", new_mtu);
> >>>>> +		lio_dev_err(lio_dev, "Valid range %d and %d\n",
> >>>>> +			    ETHER_MIN_MTU, lio_dev->linfo.link.s.mtu);
> >>>>> +		return -EINVAL;
> >>>>> +	}
> >>>>> +
> >>>>> +	return 0;
> >>>>> +}
> >>>>
> >>>> Is this really sets the MTU?
> >>>> "new_mtu" seems not used, except limit check, an lio_send_ctrl_pkt()
> >>>> required perhaps?
> >>>
> >>> It won't set MTU for hardware and is possible only by PF. So
> >>> lio_send_ctrl_pkt is not required. VF MTU is limited by PF MTU and is
> >>> mentioned under limitations in driver documentation. Here we are
> >>> allowing upper layer to set MTU up to the value configured by PF.
> >>
> >> I see, but lio_dev_change_vf_mtu() does not set anything at all. If it
> >> is not modifying anything at all, why you claim "MTU update" supported?
> > 
> > We allow update for the upper layer till the value supported by PF even
> > though there are no real MTU update happening at hardware level.
> > Thought it is ok to have it mentioned under limitation.
> > Two options are:
> > 1. mark support as partial.
> > 2. remove this patch and support.
> > 
> > Please suggest which one is better.
> 
> If I get it right, it is not possible to set VF MTU, so I would suggest
> removing MTU update support.

OK.

> 
> But you may want to keep the patch for MTU validation, and change
> function name according.

Will change set to validate.

> 
> > 
> >>
> >> And following logic seems wrong for this case:
> >>
> >> 	...
> >> 	if (lio_dev->linfo.link.s.mtu != mtu) {
> >> 		ret = lio_dev_change_vf_mtu(eth_dev, mtu);
> >> 	...
> >>
> >> Should this functions set lio_dev->linfo.link.s.mtu at least, perhaps?
> > 
> > lio_dev->linfo.link.s.mtu represents the MTU supported by the device and
> > it gets updated if there is a change by PF.
> 
> So, "lio_dev->linfo.link.s.mtu" is device MTU set by PF, "mtu" is VF
> eth_dev configured value.
> If they are not same, lio_dev_change_vf_mtu() does check if "mtu" is in
> valid range and return success or failure, right?
> So, this is just configuration validation, nothing changed/set here.
> 
> > 
> >>
> > <...>
> > 
> 

[PATCH v2 37/46] net/liquidio: add APIs to enable and disable multicast

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  2 +
 drivers/net/liquidio/lio_ethdev.c       | 68 +++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 3ea2e0f..781e929 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -130,6 +130,7 @@ enum octeon_tag_type {
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
 
 /* NIC Command types */
+#define LIO_CMD_CHANGE_DEVFLAGS		0x3
 #define LIO_CMD_RX_CTL			0x4
 #define LIO_CMD_SET_RSS			0xD
 
@@ -165,6 +166,7 @@ enum octeon_tag_type {
 
 /* Interface flags communicated between host driver and core app. */
 enum lio_ifflags {
+	LIO_IFFLAG_ALLMULTI	= 0x02,
 	LIO_IFFLAG_UNICAST	= 0x10
 };
 
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 167d41e..194d219 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -522,6 +522,72 @@
 	return 0;
 }
 
+/**
+ * \brief Net device enable, disable allmulticast
+ * @param eth_dev Pointer to the structure rte_eth_dev
+ */
+static void
+lio_change_dev_flag(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	/* Create a ctrl pkt command to be sent to core app. */
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_CHANGE_DEVFLAGS;
+	ctrl_pkt.ncmd.s.param1 = lio_dev->ifflags;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send change flag message\n");
+		return;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd))
+		lio_dev_err(lio_dev, "Change dev flag command timed out\n");
+}
+
+static void
+lio_dev_allmulticast_enable(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't enable multicast\n",
+			    lio_dev->port_id);
+		return;
+	}
+
+	lio_dev->ifflags |= LIO_IFFLAG_ALLMULTI;
+	lio_change_dev_flag(eth_dev);
+}
+
+static void
+lio_dev_allmulticast_disable(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't disable multicast\n",
+			    lio_dev->port_id);
+		return;
+	}
+
+	lio_dev->ifflags &= ~LIO_IFFLAG_ALLMULTI;
+	lio_change_dev_flag(eth_dev);
+}
+
 static void
 lio_dev_rss_configure(struct rte_eth_dev *eth_dev)
 {
@@ -1078,6 +1144,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
 	.dev_start		= lio_dev_start,
+	.allmulticast_enable	= lio_dev_allmulticast_enable,
+	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
 	.dev_infos_get		= lio_dev_info_get,
 	.mtu_set		= lio_dev_change_vf_mtu,
-- 
1.8.3.1

[PATCH v2 38/46] net/liquidio: add APIs to set link up and down

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 56 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 194d219..27727e8 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -973,6 +973,60 @@
 	return ret;
 }
 
+static int
+lio_dev_set_link_up(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	if (!lio_dev->intf_open) {
+		lio_dev_info(lio_dev, "Port is stopped, Start the port first\n");
+		return 0;
+	}
+
+	if (lio_dev->linfo.link.s.link_up) {
+		lio_dev_info(lio_dev, "Link is already UP\n");
+		return 0;
+	}
+
+	if (lio_send_rx_ctrl_cmd(eth_dev, 1)) {
+		lio_dev_err(lio_dev, "Unable to set Link UP\n");
+		return -1;
+	}
+
+	lio_dev->linfo.link.s.link_up = 1;
+	eth_dev->data->dev_link.link_status = ETH_LINK_UP;
+
+	return 0;
+}
+
+static int
+lio_dev_set_link_down(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	if (!lio_dev->intf_open) {
+		lio_dev_info(lio_dev, "Port is stopped, Start the port first\n");
+		return 0;
+	}
+
+	if (!lio_dev->linfo.link.s.link_up) {
+		lio_dev_info(lio_dev, "Link is already DOWN\n");
+		return 0;
+	}
+
+	lio_dev->linfo.link.s.link_up = 0;
+	eth_dev->data->dev_link.link_status = ETH_LINK_DOWN;
+
+	if (lio_send_rx_ctrl_cmd(eth_dev, 0)) {
+		lio_dev->linfo.link.s.link_up = 1;
+		eth_dev->data->dev_link.link_status = ETH_LINK_UP;
+		lio_dev_err(lio_dev, "Unable to set Link Down\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -1144,6 +1198,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
 	.dev_start		= lio_dev_start,
+	.dev_set_link_up	= lio_dev_set_link_up,
+	.dev_set_link_down	= lio_dev_set_link_down,
 	.allmulticast_enable	= lio_dev_allmulticast_enable,
 	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
-- 
1.8.3.1

[PATCH v2 39/46] net/liquidio: add API to configure UDP tunnel port

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |   8 ++
 drivers/net/liquidio/lio_ethdev.c       | 191 +++++++++++++++++++++++++++++++-
 drivers/net/liquidio/lio_rxtx.c         |   4 +-
 drivers/net/liquidio/lio_rxtx.h         |   1 +
 4 files changed, 202 insertions(+), 2 deletions(-)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 781e929..cc189ad 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -133,6 +133,14 @@ enum octeon_tag_type {
 #define LIO_CMD_CHANGE_DEVFLAGS		0x3
 #define LIO_CMD_RX_CTL			0x4
 #define LIO_CMD_SET_RSS			0xD
+#define LIO_CMD_TNL_RX_CSUM_CTL		0x10
+#define LIO_CMD_TNL_TX_CSUM_CTL		0x11
+#define LIO_CMD_VXLAN_PORT_CONFIG	0x19
+
+#define LIO_CMD_VXLAN_PORT_ADD		0x0
+#define LIO_CMD_VXLAN_PORT_DEL		0x1
+#define LIO_CMD_RXCSUM_ENABLE		0x0
+#define LIO_CMD_TXCSUM_ENABLE		0x0
 
 /* RX(packets coming from wire) Checksum verification flags */
 /* TCP/UDP csum */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 27727e8..eb8e921 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -136,7 +136,8 @@
 				    DEV_RX_OFFLOAD_TCP_CKSUM);
 	devinfo->tx_offload_capa = (DEV_TX_OFFLOAD_IPV4_CKSUM		|
 				    DEV_TX_OFFLOAD_UDP_CKSUM		|
-				    DEV_TX_OFFLOAD_TCP_CKSUM);
+				    DEV_TX_OFFLOAD_TCP_CKSUM		|
+				    DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM);
 
 	devinfo->rx_desc_lim = lio_rx_desc_lim;
 	devinfo->tx_desc_lim = lio_tx_desc_lim;
@@ -440,6 +441,120 @@
 }
 
 /**
+ * Add vxlan dest udp port for an interface.
+ *
+ * @param eth_dev
+ *  Pointer to the structure rte_eth_dev
+ * @param udp_tnl
+ *  udp tunnel conf
+ *
+ * @return
+ *  On success return 0
+ *  On failure return -1
+ */
+static int
+lio_dev_udp_tunnel_add(struct rte_eth_dev *eth_dev,
+		       struct rte_eth_udp_tunnel *udp_tnl)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (udp_tnl == NULL)
+		return -EINVAL;
+
+	if (udp_tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN) {
+		lio_dev_err(lio_dev, "Unsupported tunnel type\n");
+		return -1;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_VXLAN_PORT_CONFIG;
+	ctrl_pkt.ncmd.s.param1 = udp_tnl->udp_port;
+	ctrl_pkt.ncmd.s.more = LIO_CMD_VXLAN_PORT_ADD;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send VXLAN_PORT_ADD command\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "VXLAN_PORT_ADD command timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+/**
+ * Remove vxlan dest udp port for an interface.
+ *
+ * @param eth_dev
+ *  Pointer to the structure rte_eth_dev
+ * @param udp_tnl
+ *  udp tunnel conf
+ *
+ * @return
+ *  On success return 0
+ *  On failure return -1
+ */
+static int
+lio_dev_udp_tunnel_del(struct rte_eth_dev *eth_dev,
+		       struct rte_eth_udp_tunnel *udp_tnl)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (udp_tnl == NULL)
+		return -EINVAL;
+
+	if (udp_tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN) {
+		lio_dev_err(lio_dev, "Unsupported tunnel type\n");
+		return -1;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_VXLAN_PORT_CONFIG;
+	ctrl_pkt.ncmd.s.param1 = udp_tnl->udp_port;
+	ctrl_pkt.ncmd.s.more = LIO_CMD_VXLAN_PORT_DEL;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send VXLAN_PORT_DEL command\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "VXLAN_PORT_DEL command timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+/**
  * Atomically writes the link status information into global
  * structure rte_eth_dev.
  *
@@ -1027,6 +1142,74 @@
 	return 0;
 }
 
+/**
+ * Enable tunnel rx checksum verification from firmware.
+ */
+static void
+lio_enable_hw_tunnel_rx_checksum(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_TNL_RX_CSUM_CTL;
+	ctrl_pkt.ncmd.s.param1 = LIO_CMD_RXCSUM_ENABLE;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send TNL_RX_CSUM command\n");
+		return;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd))
+		lio_dev_err(lio_dev, "TNL_RX_CSUM command timed out\n");
+}
+
+/**
+ * Enable checksum calculation for inner packet in a tunnel.
+ */
+static void
+lio_enable_hw_tunnel_tx_checksum(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_TNL_TX_CSUM_CTL;
+	ctrl_pkt.ncmd.s.param1 = LIO_CMD_TXCSUM_ENABLE;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send TNL_TX_CSUM command\n");
+		return;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd))
+		lio_dev_err(lio_dev, "TNL_TX_CSUM command timed out\n");
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -1155,6 +1338,10 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	/* Copy the permanent MAC address */
 	ether_addr_copy((struct ether_addr *)mac, &eth_dev->data->mac_addrs[0]);
 
+	/* enable firmware checksum support for tunnel packets */
+	lio_enable_hw_tunnel_rx_checksum(eth_dev);
+	lio_enable_hw_tunnel_tx_checksum(eth_dev);
+
 	lio_dev->glist_lock =
 	    rte_zmalloc(NULL, sizeof(*lio_dev->glist_lock) * num_iqueues, 0);
 	if (lio_dev->glist_lock == NULL)
@@ -1213,6 +1400,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.reta_query		= lio_dev_rss_reta_query,
 	.rss_hash_conf_get	= lio_dev_rss_hash_conf_get,
 	.rss_hash_update	= lio_dev_rss_hash_update,
+	.udp_tunnel_port_add	= lio_dev_udp_tunnel_add,
+	.udp_tunnel_port_del	= lio_dev_udp_tunnel_del,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index c12960c..f105457 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1704,7 +1704,9 @@ struct lio_soft_command *
 		if (m->ol_flags & PKT_TX_IP_CKSUM)
 			cmdsetup.s.ip_csum = 1;
 
-		if ((m->ol_flags & PKT_TX_TCP_CKSUM) ||
+		if (m->ol_flags & PKT_TX_OUTER_IP_CKSUM)
+			cmdsetup.s.tnl_csum = 1;
+		else if ((m->ol_flags & PKT_TX_TCP_CKSUM) ||
 				(m->ol_flags & PKT_TX_UDP_CKSUM))
 			cmdsetup.s.transport_csum = 1;
 
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 95d0007..6835430 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -583,6 +583,7 @@ struct octeon_instr_rdp {
 	packet_params.pkt_params32 = 0;
 	packet_params.s.ip_csum = setup->s.ip_csum;
 	packet_params.s.transport_csum = setup->s.transport_csum;
+	packet_params.s.tnl_csum = setup->s.tnl_csum;
 	packet_params.s.tsflag = setup->s.timestamp;
 
 	irh->ossp = packet_params.pkt_params32;
-- 
1.8.3.1

[PATCH v2 40/46] net/liquidio: add support for Rx stats

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 50 +++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.c   | 10 +++++++-
 drivers/net/liquidio/lio_struct.h | 34 ++++++++++++++++++++++++++
 3 files changed, 93 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index eb8e921..794ec78 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -117,6 +117,54 @@
 	return 0;
 }
 
+/* Retrieve the device statistics (# packets in/out, # bytes in/out, etc */
+static void
+lio_dev_stats_get(struct rte_eth_dev *eth_dev,
+		  struct rte_eth_stats *stats)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_droq_stats *oq_stats;
+	struct lio_droq *droq;
+	uint64_t bytes = 0;
+	uint64_t pkts = 0;
+	uint64_t drop = 0;
+	int i, oq_no;
+
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
+		oq_no = lio_dev->linfo.rxpciq[i].s.q_no;
+		droq = lio_dev->droq[oq_no];
+		if (droq != NULL) {
+			oq_stats = &droq->stats;
+			pkts += oq_stats->rx_pkts_received;
+			drop += (oq_stats->rx_dropped +
+					oq_stats->dropped_toomany +
+					oq_stats->dropped_nomem);
+			bytes += oq_stats->rx_bytes_received;
+		}
+	}
+	stats->ibytes = bytes;
+	stats->ipackets = pkts;
+	stats->ierrors = drop;
+}
+
+static void
+lio_dev_stats_reset(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_droq_stats *oq_stats;
+	struct lio_droq *droq;
+	int i, oq_no;
+
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
+		oq_no = lio_dev->linfo.rxpciq[i].s.q_no;
+		droq = lio_dev->droq[oq_no];
+		if (droq != NULL) {
+			oq_stats = &droq->stats;
+			memset(oq_stats, 0, sizeof(struct lio_droq_stats));
+		}
+	}
+}
+
 static void
 lio_dev_info_get(struct rte_eth_dev *eth_dev,
 		 struct rte_eth_dev_info *devinfo)
@@ -1390,6 +1438,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.allmulticast_enable	= lio_dev_allmulticast_enable,
 	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
+	.stats_get		= lio_dev_stats_get,
+	.stats_reset		= lio_dev_stats_reset,
 	.dev_infos_get		= lio_dev_info_get,
 	.mtu_set		= lio_dev_change_vf_mtu,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index f105457..adbd990 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -115,6 +115,7 @@
 		buf = lio_recv_buffer_alloc(lio_dev, droq->q_no);
 		if (buf == NULL) {
 			lio_dev_err(lio_dev, "buffer alloc failed\n");
+			droq->stats.rx_alloc_failure++;
 			lio_droq_destroy_ring_buffers(droq);
 			return -ENOMEM;
 		}
@@ -410,8 +411,10 @@
 			/* If a buffer could not be allocated, no point in
 			 * continuing
 			 */
-			if (buf == NULL)
+			if (buf == NULL) {
+				droq->stats.rx_alloc_failure++;
 				break;
+			}
 
 			droq->recv_buf_list[droq->refill_idx].buffer = buf;
 		}
@@ -629,6 +632,11 @@
 	info->length = 0;
 	info->rh.rh64 = 0;
 
+	droq->stats.pkts_received++;
+	droq->stats.rx_pkts_received += data_pkts;
+	droq->stats.rx_bytes_received += data_total_len;
+	droq->stats.bytes_received += total_len;
+
 	return data_pkts;
 }
 
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index c2293f7..50d5e86 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -56,6 +56,37 @@ struct lio_version {
 	uint16_t reserved;
 };
 
+/** Output Queue statistics. Each output queue has four stats fields. */
+struct lio_droq_stats {
+	/** Number of packets received in this queue. */
+	uint64_t pkts_received;
+
+	/** Bytes received by this queue. */
+	uint64_t bytes_received;
+
+	/** Packets dropped due to no memory available. */
+	uint64_t dropped_nomem;
+
+	/** Packets dropped due to large number of pkts to process. */
+	uint64_t dropped_toomany;
+
+	/** Number of packets  sent to stack from this queue. */
+	uint64_t rx_pkts_received;
+
+	/** Number of Bytes sent to stack from this queue. */
+	uint64_t rx_bytes_received;
+
+	/** Num of Packets dropped due to receive path failures. */
+	uint64_t rx_dropped;
+
+	/** Num of vxlan packets received; */
+	uint64_t rx_vxlan;
+
+	/** Num of failures of lio_recv_buffer_alloc() */
+	uint64_t rx_alloc_failure;
+
+};
+
 /** The Descriptor Ring Output Queue structure.
  *  This structure has all the information required to implement a
  *  DROQ.
@@ -117,6 +148,9 @@ struct lio_droq {
 	 */
 	void *pkts_sent_reg;
 
+	/** Statistics for this DROQ. */
+	struct lio_droq_stats stats;
+
 	/** DMA mapped address of the DROQ descriptor ring. */
 	size_t desc_ring_dma;
 
-- 
1.8.3.1

[PATCH v2 41/46] net/liquidio: add support for Tx stats

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 36 ++++++++++++++++++++++++++++++++++--
 drivers/net/liquidio/lio_rxtx.c   | 18 ++++++++++++++++--
 drivers/net/liquidio/lio_rxtx.h   |  3 +++
 drivers/net/liquidio/lio_struct.h | 15 +++++++++++++++
 4 files changed, 68 insertions(+), 4 deletions(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 794ec78..7f0e9f4 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -124,11 +124,32 @@
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
 	struct lio_droq_stats *oq_stats;
+	struct lio_iq_stats *iq_stats;
+	struct lio_instr_queue *txq;
 	struct lio_droq *droq;
+	int i, iq_no, oq_no;
 	uint64_t bytes = 0;
 	uint64_t pkts = 0;
 	uint64_t drop = 0;
-	int i, oq_no;
+
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
+		iq_no = lio_dev->linfo.txpciq[i].s.q_no;
+		txq = lio_dev->instr_queue[iq_no];
+		if (txq != NULL) {
+			iq_stats = &txq->stats;
+			pkts += iq_stats->tx_done;
+			drop += iq_stats->tx_dropped;
+			bytes += iq_stats->tx_tot_bytes;
+		}
+	}
+
+	stats->opackets = pkts;
+	stats->obytes = bytes;
+	stats->oerrors = drop;
+
+	pkts = 0;
+	drop = 0;
+	bytes = 0;
 
 	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
 		oq_no = lio_dev->linfo.rxpciq[i].s.q_no;
@@ -152,8 +173,19 @@
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
 	struct lio_droq_stats *oq_stats;
+	struct lio_iq_stats *iq_stats;
+	struct lio_instr_queue *txq;
 	struct lio_droq *droq;
-	int i, oq_no;
+	int i, iq_no, oq_no;
+
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
+		iq_no = lio_dev->linfo.txpciq[i].s.q_no;
+		txq = lio_dev->instr_queue[iq_no];
+		if (txq != NULL) {
+			iq_stats = &txq->stats;
+			memset(iq_stats, 0, sizeof(struct lio_iq_stats));
+		}
+	}
 
 	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
 		oq_no = lio_dev->linfo.rxpciq[i].s.q_no;
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index adbd990..470131c 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1113,8 +1113,10 @@
 
 		inst_processed = lio_process_iq_request_list(lio_dev, iq);
 
-		if (inst_processed)
+		if (inst_processed) {
 			rte_atomic64_sub(&iq->instr_pending, inst_processed);
+			iq->stats.instr_processed += inst_processed;
+		}
 
 		tot_inst_processed += inst_processed;
 		inst_processed = 0;
@@ -1130,7 +1132,7 @@
 
 static int
 lio_send_command(struct lio_device *lio_dev, uint32_t iq_no, void *cmd,
-		 void *buf, uint32_t datasize __rte_unused, uint32_t reqtype)
+		 void *buf, uint32_t datasize, uint32_t reqtype)
 {
 	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
 	struct lio_iq_post_status st;
@@ -1141,7 +1143,13 @@
 
 	if (st.status != LIO_IQ_SEND_FAILED) {
 		lio_add_to_request_list(iq, st.index, buf, reqtype);
+		LIO_INCR_INSTRQUEUE_PKT_COUNT(lio_dev, iq_no, bytes_sent,
+					      datasize);
+		LIO_INCR_INSTRQUEUE_PKT_COUNT(lio_dev, iq_no, instr_posted, 1);
+
 		lio_ring_doorbell(lio_dev, iq);
+	} else {
+		LIO_INCR_INSTRQUEUE_PKT_COUNT(lio_dev, iq_no, instr_dropped, 1);
 	}
 
 	rte_spinlock_unlock(&iq->post_lock);
@@ -1667,6 +1675,7 @@ struct lio_soft_command *
 	struct lio_instr_queue *txq = tx_queue;
 	union lio_cmd_setup cmdsetup;
 	struct lio_device *lio_dev;
+	struct lio_iq_stats *stats;
 	struct lio_data_pkt ndata;
 	int i, processed = 0;
 	struct rte_mbuf *m;
@@ -1676,6 +1685,7 @@ struct lio_soft_command *
 
 	lio_dev = txq->lio_dev;
 	iq_no = txq->txpciq.s.q_no;
+	stats = &lio_dev->instr_queue[iq_no]->stats;
 
 	if (!lio_dev->intf_open || !lio_dev->linfo.link.s.link_up) {
 		PMD_TX_LOG(lio_dev, ERR, "Transmit failed link_status : %d\n",
@@ -1697,6 +1707,7 @@ struct lio_soft_command *
 
 		ndata.q_no = iq_no;
 		if (lio_iq_is_full(lio_dev, ndata.q_no)) {
+			stats->tx_iq_busy++;
 			if (lio_dev_cleanup_iq(lio_dev, iq_no)) {
 				PMD_TX_LOG(lio_dev, ERR,
 					   "Transmit failed iq:%d full\n",
@@ -1804,10 +1815,13 @@ struct lio_soft_command *
 			lio_dev_cleanup_iq(lio_dev, iq_no);
 		}
 
+		stats->tx_done++;
+		stats->tx_tot_bytes += pkt_len;
 		processed++;
 	}
 
 xmit_failed:
+	stats->tx_dropped += (nb_pkts - processed);
 
 	return processed;
 }
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 6835430..d5aed6a 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -670,6 +670,9 @@ enum {
  */
 int lio_process_ordered_list(struct lio_device *lio_dev);
 
+#define LIO_INCR_INSTRQUEUE_PKT_COUNT(lio_dev, iq_no, field, count)	\
+	(((lio_dev)->instr_queue[iq_no]->stats.field) += count)
+
 static inline void
 lio_swap_8B_data(uint64_t *data, uint32_t blocks)
 {
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 50d5e86..26f803f 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -56,6 +56,18 @@ struct lio_version {
 	uint16_t reserved;
 };
 
+/** Input Queue statistics. Each input queue has four stats fields. */
+struct lio_iq_stats {
+	uint64_t instr_posted; /**< Instructions posted to this queue. */
+	uint64_t instr_processed; /**< Instructions processed in this queue. */
+	uint64_t instr_dropped; /**< Instructions that could not be processed */
+	uint64_t bytes_sent; /**< Bytes sent through this queue. */
+	uint64_t tx_done; /**< Num of packets sent to network. */
+	uint64_t tx_iq_busy; /**< Num of times this iq was found to be full. */
+	uint64_t tx_dropped; /**< Num of pkts dropped due to xmitpath errors. */
+	uint64_t tx_tot_bytes; /**< Total count of bytes sent to network. */
+};
+
 /** Output Queue statistics. Each output queue has four stats fields. */
 struct lio_droq_stats {
 	/** Number of packets received in this queue. */
@@ -319,6 +331,9 @@ struct lio_instr_queue {
 	/** Number of instructions pending to be posted to Octeon. */
 	uint32_t fill_cnt;
 
+	/** Statistics for this input queue. */
+	struct lio_iq_stats stats;
+
 	/** DMA mapped base address of the input descriptor ring. */
 	uint64_t base_addr_dma;
 
-- 
1.8.3.1

[PATCH v2 42/46] net/liquidio: add APIs for hardware stats

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |   2 +
 drivers/net/liquidio/lio_ethdev.c       | 194 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  83 ++++++++++++++
 3 files changed, 279 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index cc189ad..8a22d10 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -124,6 +124,7 @@ enum octeon_tag_type {
 #define LIO_OPCODE_NW_DATA		0x02 /* network packet data */
 #define LIO_OPCODE_CMD			0x03
 #define LIO_OPCODE_INFO			0x04
+#define LIO_OPCODE_PORT_STATS		0x05
 #define LIO_OPCODE_IF_CFG		0x09
 
 #define LIO_MIN_RX_BUF_SIZE		64
@@ -132,6 +133,7 @@ enum octeon_tag_type {
 /* NIC Command types */
 #define LIO_CMD_CHANGE_DEVFLAGS		0x3
 #define LIO_CMD_RX_CTL			0x4
+#define LIO_CMD_CLEAR_STATS		0x6
 #define LIO_CMD_SET_RSS			0xD
 #define LIO_CMD_TNL_RX_CSUM_CTL		0x10
 #define LIO_CMD_TNL_TX_CSUM_CTL		0x11
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 7f0e9f4..8459986 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -117,6 +117,197 @@
 	return 0;
 }
 
+/* store statistics names and its offset in stats structure */
+struct rte_lio_xstats_name_off {
+	char name[RTE_ETH_XSTATS_NAME_SIZE];
+	unsigned int offset;
+};
+
+static const struct rte_lio_xstats_name_off rte_lio_stats_strings[] = {
+	{"rx_pkts", offsetof(struct octeon_rx_stats, total_rcvd)},
+	{"rx_bytes", offsetof(struct octeon_rx_stats, bytes_rcvd)},
+	{"rx_broadcast_pkts", offsetof(struct octeon_rx_stats, total_bcst)},
+	{"rx_multicast_pkts", offsetof(struct octeon_rx_stats, total_mcst)},
+	{"rx_flow_ctrl_pkts", offsetof(struct octeon_rx_stats, ctl_rcvd)},
+	{"rx_fifo_err", offsetof(struct octeon_rx_stats, fifo_err)},
+	{"rx_dmac_drop", offsetof(struct octeon_rx_stats, dmac_drop)},
+	{"rx_fcs_err", offsetof(struct octeon_rx_stats, fcs_err)},
+	{"rx_jabber_err", offsetof(struct octeon_rx_stats, jabber_err)},
+	{"rx_l2_err", offsetof(struct octeon_rx_stats, l2_err)},
+	{"rx_vxlan_pkts", offsetof(struct octeon_rx_stats, fw_rx_vxlan)},
+	{"rx_vxlan_err", offsetof(struct octeon_rx_stats, fw_rx_vxlan_err)},
+	{"rx_lro_pkts", offsetof(struct octeon_rx_stats, fw_lro_pkts)},
+	{"tx_pkts", (offsetof(struct octeon_tx_stats, total_pkts_sent)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_bytes", (offsetof(struct octeon_tx_stats, total_bytes_sent)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_broadcast_pkts",
+		(offsetof(struct octeon_tx_stats, bcast_pkts_sent)) +
+			sizeof(struct octeon_rx_stats)},
+	{"tx_multicast_pkts",
+		(offsetof(struct octeon_tx_stats, mcast_pkts_sent)) +
+			sizeof(struct octeon_rx_stats)},
+	{"tx_flow_ctrl_pkts", (offsetof(struct octeon_tx_stats, ctl_sent)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_fifo_err", (offsetof(struct octeon_tx_stats, fifo_err)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_total_collisions", (offsetof(struct octeon_tx_stats,
+					  total_collisions)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_tso", (offsetof(struct octeon_tx_stats, fw_tso)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_vxlan_pkts", (offsetof(struct octeon_tx_stats, fw_tx_vxlan)) +
+						sizeof(struct octeon_rx_stats)},
+};
+
+#define LIO_NB_XSTATS	RTE_DIM(rte_lio_stats_strings)
+
+/* Get hw stats of the port */
+static int
+lio_dev_xstats_get(struct rte_eth_dev *eth_dev, struct rte_eth_xstat *xstats,
+		   unsigned int n)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
+	struct octeon_link_stats *hw_stats;
+	struct lio_link_stats_resp *resp;
+	struct lio_soft_command *sc;
+	uint32_t resp_size;
+	unsigned int i;
+	int retval;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	if (n < LIO_NB_XSTATS)
+		return LIO_NB_XSTATS;
+
+	resp_size = sizeof(struct lio_link_stats_resp);
+	sc = lio_alloc_soft_command(lio_dev, 0, resp_size, 0);
+	if (sc == NULL)
+		return -ENOMEM;
+
+	resp = (struct lio_link_stats_resp *)sc->virtrptr;
+	lio_prepare_soft_command(lio_dev, sc, LIO_OPCODE,
+				 LIO_OPCODE_PORT_STATS, 0, 0, 0);
+
+	/* Setting wait time in seconds */
+	sc->wait_time = LIO_MAX_CMD_TIMEOUT / 1000;
+
+	retval = lio_send_soft_command(lio_dev, sc);
+	if (retval == LIO_IQ_SEND_FAILED) {
+		lio_dev_err(lio_dev, "failed to get port stats from firmware. status: %x\n",
+			    retval);
+		goto get_stats_fail;
+	}
+
+	while ((*sc->status_word == LIO_COMPLETION_WORD_INIT) && --timeout) {
+		lio_flush_iq(lio_dev, lio_dev->instr_queue[sc->iq_no]);
+		lio_process_ordered_list(lio_dev);
+		rte_delay_ms(1);
+	}
+
+	retval = resp->status;
+	if (retval) {
+		lio_dev_err(lio_dev, "failed to get port stats from firmware\n");
+		goto get_stats_fail;
+	}
+
+	lio_swap_8B_data((uint64_t *)(&resp->link_stats),
+			 sizeof(struct octeon_link_stats) >> 3);
+
+	hw_stats = &resp->link_stats;
+
+	for (i = 0; i < LIO_NB_XSTATS; i++) {
+		xstats[i].id = i;
+		xstats[i].value =
+		    *(uint64_t *)(((char *)hw_stats) +
+					rte_lio_stats_strings[i].offset);
+	}
+
+	lio_free_soft_command(sc);
+
+	return LIO_NB_XSTATS;
+
+get_stats_fail:
+	lio_free_soft_command(sc);
+
+	return -1;
+}
+
+static int
+lio_dev_xstats_get_names(struct rte_eth_dev *eth_dev,
+			 struct rte_eth_xstat_name *xstats_names,
+			 unsigned limit __rte_unused)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	unsigned int i;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	if (xstats_names == NULL)
+		return LIO_NB_XSTATS;
+
+	/* Note: limit checked in rte_eth_xstats_names() */
+
+	for (i = 0; i < LIO_NB_XSTATS; i++) {
+		snprintf(xstats_names[i].name, sizeof(xstats_names[i].name),
+			 "%s", rte_lio_stats_strings[i].name);
+	}
+
+	return LIO_NB_XSTATS;
+}
+
+/* Reset hw stats for the port */
+static void
+lio_dev_xstats_reset(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down\n",
+			    lio_dev->port_id);
+		return;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_CLEAR_STATS;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send clear stats command\n");
+		return;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "Clear stats command timed out\n");
+		return;
+	}
+
+	/* clear stored per queue stats */
+	RTE_FUNC_PTR_OR_RET(*eth_dev->dev_ops->stats_reset);
+	(*eth_dev->dev_ops->stats_reset)(eth_dev);
+}
+
 /* Retrieve the device statistics (# packets in/out, # bytes in/out, etc */
 static void
 lio_dev_stats_get(struct rte_eth_dev *eth_dev,
@@ -1471,7 +1662,10 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
 	.stats_get		= lio_dev_stats_get,
+	.xstats_get		= lio_dev_xstats_get,
+	.xstats_get_names	= lio_dev_xstats_get_names,
 	.stats_reset		= lio_dev_stats_reset,
+	.xstats_reset		= lio_dev_xstats_reset,
 	.dev_infos_get		= lio_dev_info_get,
 	.mtu_set		= lio_dev_change_vf_mtu,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 6543061..150e9c9 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -62,6 +62,83 @@ struct octeon_if_cfg_info {
 	char lio_firmware_version[LIO_FW_VERSION_LENGTH];
 };
 
+/** Stats for each NIC port in RX direction. */
+struct octeon_rx_stats {
+	/* link-level stats */
+	uint64_t total_rcvd;
+	uint64_t bytes_rcvd;
+	uint64_t total_bcst;
+	uint64_t total_mcst;
+	uint64_t runts;
+	uint64_t ctl_rcvd;
+	uint64_t fifo_err; /* Accounts for over/under-run of buffers */
+	uint64_t dmac_drop;
+	uint64_t fcs_err;
+	uint64_t jabber_err;
+	uint64_t l2_err;
+	uint64_t frame_err;
+
+	/* firmware stats */
+	uint64_t fw_total_rcvd;
+	uint64_t fw_total_fwd;
+	uint64_t fw_total_fwd_bytes;
+	uint64_t fw_err_pko;
+	uint64_t fw_err_link;
+	uint64_t fw_err_drop;
+	uint64_t fw_rx_vxlan;
+	uint64_t fw_rx_vxlan_err;
+
+	/* LRO */
+	uint64_t fw_lro_pkts;   /* Number of packets that are LROed */
+	uint64_t fw_lro_octs;   /* Number of octets that are LROed */
+	uint64_t fw_total_lro;  /* Number of LRO packets formed */
+	uint64_t fw_lro_aborts; /* Number of times lRO of packet aborted */
+	uint64_t fw_lro_aborts_port;
+	uint64_t fw_lro_aborts_seq;
+	uint64_t fw_lro_aborts_tsval;
+	uint64_t fw_lro_aborts_timer;
+	/* intrmod: packet forward rate */
+	uint64_t fwd_rate;
+};
+
+/** Stats for each NIC port in RX direction. */
+struct octeon_tx_stats {
+	/* link-level stats */
+	uint64_t total_pkts_sent;
+	uint64_t total_bytes_sent;
+	uint64_t mcast_pkts_sent;
+	uint64_t bcast_pkts_sent;
+	uint64_t ctl_sent;
+	uint64_t one_collision_sent;	/* Packets sent after one collision */
+	/* Packets sent after multiple collision */
+	uint64_t multi_collision_sent;
+	/* Packets not sent due to max collisions */
+	uint64_t max_collision_fail;
+	/* Packets not sent due to max deferrals */
+	uint64_t max_deferral_fail;
+	/* Accounts for over/under-run of buffers */
+	uint64_t fifo_err;
+	uint64_t runts;
+	uint64_t total_collisions; /* Total number of collisions detected */
+
+	/* firmware stats */
+	uint64_t fw_total_sent;
+	uint64_t fw_total_fwd;
+	uint64_t fw_total_fwd_bytes;
+	uint64_t fw_err_pko;
+	uint64_t fw_err_link;
+	uint64_t fw_err_drop;
+	uint64_t fw_err_tso;
+	uint64_t fw_tso;     /* number of tso requests */
+	uint64_t fw_tso_fwd; /* number of packets segmented in tso */
+	uint64_t fw_tx_vxlan;
+};
+
+struct octeon_link_stats {
+	struct octeon_rx_stats fromwire;
+	struct octeon_tx_stats fromhost;
+};
+
 union lio_if_cfg {
 	uint64_t if_cfg64;
 	struct {
@@ -87,6 +164,12 @@ struct lio_if_cfg_resp {
 	uint64_t status;
 };
 
+struct lio_link_stats_resp {
+	uint64_t rh;
+	struct octeon_link_stats link_stats;
+	uint64_t status;
+};
+
 struct lio_link_status_resp {
 	uint64_t rh;
 	struct octeon_link_info link_info;
-- 
1.8.3.1

[PATCH v2 43/46] net/liquidio: add API to stop device

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 8459986..b0db8ea 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1359,6 +1359,25 @@ struct rte_lio_xstats_name_off {
 	return ret;
 }
 
+/* Stop device and disable input/output functions */
+static void
+lio_dev_stop(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	lio_dev_info(lio_dev, "Stopping port %d\n", eth_dev->data->port_id);
+	lio_dev->intf_open = 0;
+	rte_mb();
+
+	/* Cancel callback if still running. */
+	rte_eal_alarm_cancel(lio_sync_link_state_check, eth_dev);
+
+	lio_send_rx_ctrl_cmd(eth_dev, 0);
+
+	/* Clear recorded link status */
+	lio_dev->linfo.link.link_status64 = 0;
+}
+
 static int
 lio_dev_set_link_up(struct rte_eth_dev *eth_dev)
 {
@@ -1656,6 +1675,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
 	.dev_start		= lio_dev_start,
+	.dev_stop		= lio_dev_stop,
 	.dev_set_link_up	= lio_dev_set_link_up,
 	.dev_set_link_down	= lio_dev_set_link_down,
 	.allmulticast_enable	= lio_dev_allmulticast_enable,
-- 
1.8.3.1

[PATCH v2 44/46] net/liquidio: add API to close device

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 68 +++++++++++++++++++++++++++++++++++++--
 drivers/net/liquidio/lio_ethdev.h |  5 +++
 drivers/net/liquidio/lio_rxtx.c   | 57 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  2 ++
 4 files changed, 130 insertions(+), 2 deletions(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index b0db8ea..9a936bf 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1109,7 +1109,7 @@ struct rte_lio_xstats_name_off {
  * @return
  *    - nothing
  */
-static void
+void
 lio_dev_rx_queue_release(void *rxq)
 {
 	struct lio_droq *droq = rxq;
@@ -1204,7 +1204,7 @@ struct rte_lio_xstats_name_off {
  * @return
  *    - nothing
  */
-static void
+void
 lio_dev_tx_queue_release(void *txq)
 {
 	struct lio_instr_queue *tq = txq;
@@ -1433,6 +1433,68 @@ struct rte_lio_xstats_name_off {
 }
 
 /**
+ * Reset and stop the device. This occurs on the first
+ * call to this routine. Subsequent calls will simply
+ * return. NB: This will require the NIC to be rebooted.
+ *
+ * @param eth_dev
+ *    Pointer to the structure rte_eth_dev
+ *
+ * @return
+ *    - nothing
+ */
+static void
+lio_dev_close(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint32_t i;
+
+	lio_dev_info(lio_dev, "closing port %d\n", eth_dev->data->port_id);
+
+	if (lio_dev->intf_open)
+		lio_dev_stop(eth_dev);
+
+	lio_wait_for_instr_fetch(lio_dev);
+
+	lio_dev->fn_list.disable_io_queues(lio_dev);
+
+	cn23xx_vf_set_io_queues_off(lio_dev);
+
+	/* Reset iq regs (IQ_DBELL).
+	 * Clear sli_pktx_cnts (OQ_PKTS_SENT).
+	 */
+	for (i = 0; i < lio_dev->nb_rx_queues; i++) {
+		struct lio_droq *droq = lio_dev->droq[i];
+
+		if (droq == NULL)
+			break;
+
+		uint32_t pkt_count = rte_read32(droq->pkts_sent_reg);
+
+		lio_dev_dbg(lio_dev,
+			    "pending oq count %u\n", pkt_count);
+		rte_write32(pkt_count, droq->pkts_sent_reg);
+	}
+
+	/* Do FLR for the VF */
+	cn23xx_vf_ask_pf_to_do_flr(lio_dev);
+
+	/* lio_free_mbox */
+	lio_dev->fn_list.free_mbox(lio_dev);
+
+	/* Free glist resources */
+	rte_free(lio_dev->glist_head);
+	rte_free(lio_dev->glist_lock);
+	lio_dev->glist_head = NULL;
+	lio_dev->glist_lock = NULL;
+
+	lio_dev->port_configured = 0;
+
+	 /* Delete all queues */
+	lio_dev_clear_queues(eth_dev);
+}
+
+/**
  * Enable tunnel rx checksum verification from firmware.
  */
 static void
@@ -1678,6 +1740,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.dev_stop		= lio_dev_stop,
 	.dev_set_link_up	= lio_dev_set_link_up,
 	.dev_set_link_down	= lio_dev_set_link_down,
+	.dev_close		= lio_dev_close,
 	.allmulticast_enable	= lio_dev_allmulticast_enable,
 	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
@@ -1840,6 +1903,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	rte_free(eth_dev->data->mac_addrs);
 	eth_dev->data->mac_addrs = NULL;
 
+	eth_dev->dev_ops = NULL;
 	eth_dev->rx_pkt_burst = NULL;
 	eth_dev->tx_pkt_burst = NULL;
 
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 150e9c9..ee30615 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -196,4 +196,9 @@ struct lio_rss_set {
 	uint8_t itable[LIO_RSS_MAX_TABLE_SZ];
 	uint8_t key[LIO_RSS_MAX_KEY_SZ];
 };
+
+void lio_dev_rx_queue_release(void *rxq);
+
+void lio_dev_tx_queue_release(void *txq);
+
 #endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 470131c..9533015 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -918,6 +918,40 @@
 	return -1;
 }
 
+int
+lio_wait_for_instr_fetch(struct lio_device *lio_dev)
+{
+	int pending, instr_cnt;
+	int i, retry = 1000;
+
+	do {
+		instr_cnt = 0;
+
+		for (i = 0; i < LIO_MAX_INSTR_QUEUES(lio_dev); i++) {
+			if (!(lio_dev->io_qmask.iq & (1ULL << i)))
+				continue;
+
+			if (lio_dev->instr_queue[i] == NULL)
+				break;
+
+			pending = rte_atomic64_read(
+			    &lio_dev->instr_queue[i]->instr_pending);
+			if (pending)
+				lio_flush_iq(lio_dev, lio_dev->instr_queue[i]);
+
+			instr_cnt += pending;
+		}
+
+		if (instr_cnt == 0)
+			break;
+
+		rte_delay_ms(1);
+
+	} while (retry-- && instr_cnt);
+
+	return instr_cnt;
+}
+
 static inline void
 lio_ring_doorbell(struct lio_device *lio_dev,
 		  struct lio_instr_queue *iq)
@@ -1826,3 +1860,26 @@ struct lio_soft_command *
 	return processed;
 }
 
+void
+lio_dev_clear_queues(struct rte_eth_dev *eth_dev)
+{
+	struct lio_instr_queue *txq;
+	struct lio_droq *rxq;
+	uint16_t i;
+
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
+		txq = eth_dev->data->tx_queues[i];
+		if (txq != NULL) {
+			lio_dev_tx_queue_release(txq);
+			eth_dev->data->tx_queues[i] = NULL;
+		}
+	}
+
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
+		rxq = eth_dev->data->rx_queues[i];
+		if (rxq != NULL) {
+			lio_dev_rx_queue_release(rxq);
+			eth_dev->data->rx_queues[i] = NULL;
+		}
+	}
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index d5aed6a..85685dc 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -752,6 +752,7 @@ int lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
 		      int fw_mapped_iq, int num_descs, unsigned int socket_id);
 uint16_t lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts,
 			   uint16_t nb_pkts);
+int lio_wait_for_instr_fetch(struct lio_device *lio_dev);
 int lio_setup_iq(struct lio_device *lio_dev, int q_index,
 		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
 		 unsigned int socket_id);
@@ -764,4 +765,5 @@ int lio_setup_iq(struct lio_device *lio_dev, int q_index,
  */
 int lio_setup_instr_queue0(struct lio_device *lio_dev);
 void lio_free_instr_queue0(struct lio_device *lio_dev);
+void lio_dev_clear_queues(struct rte_eth_dev *eth_dev);
 #endif	/* _LIO_RXTX_H_ */
-- 
1.8.3.1

[PATCH v2 45/46] net/liquidio: add API to add and remove VLAN port

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  2 ++
 drivers/net/liquidio/lio_ethdev.c       | 45 ++++++++++++++++++++++++++++++++-
 2 files changed, 46 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 8a22d10..67eaa45 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -137,6 +137,8 @@ enum octeon_tag_type {
 #define LIO_CMD_SET_RSS			0xD
 #define LIO_CMD_TNL_RX_CSUM_CTL		0x10
 #define LIO_CMD_TNL_TX_CSUM_CTL		0x11
+#define LIO_CMD_ADD_VLAN_FILTER		0x17
+#define LIO_CMD_DEL_VLAN_FILTER		0x18
 #define LIO_CMD_VXLAN_PORT_CONFIG	0x19
 
 #define LIO_CMD_VXLAN_PORT_ADD		0x0
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 9a936bf..920a7f8 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -404,7 +404,8 @@ struct rte_lio_xstats_name_off {
 
 	devinfo->rx_offload_capa = (DEV_RX_OFFLOAD_IPV4_CKSUM		|
 				    DEV_RX_OFFLOAD_UDP_CKSUM		|
-				    DEV_RX_OFFLOAD_TCP_CKSUM);
+				    DEV_RX_OFFLOAD_TCP_CKSUM		|
+				    DEV_RX_OFFLOAD_VLAN_STRIP);
 	devinfo->tx_offload_capa = (DEV_TX_OFFLOAD_IPV4_CKSUM		|
 				    DEV_TX_OFFLOAD_UDP_CKSUM		|
 				    DEV_TX_OFFLOAD_TCP_CKSUM		|
@@ -825,6 +826,47 @@ struct rte_lio_xstats_name_off {
 	return 0;
 }
 
+static int
+lio_dev_vlan_filter_set(struct rte_eth_dev *eth_dev, uint16_t vlan_id, int on)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (lio_dev->linfo.vlan_is_admin_assigned)
+		return -EPERM;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = on ?
+			LIO_CMD_ADD_VLAN_FILTER : LIO_CMD_DEL_VLAN_FILTER;
+	ctrl_pkt.ncmd.s.param1 = vlan_id;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to %s VLAN port\n",
+			    on ? "add" : "remove");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "Command to %s VLAN port timed out\n",
+			    on ? "add" : "remove");
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Atomically writes the link status information into global
  * structure rte_eth_dev.
@@ -1751,6 +1793,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.xstats_reset		= lio_dev_xstats_reset,
 	.dev_infos_get		= lio_dev_info_get,
 	.mtu_set		= lio_dev_change_vf_mtu,
+	.vlan_filter_set	= lio_dev_vlan_filter_set,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
-- 
1.8.3.1

[PATCH v2 46/46] doc: add doc for liquidio

[Shijith Thotton]

Added doc/guides/nics/liquidio.rst and
doc/guides/nics/features/liquidio.ini. Updated release notes.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 MAINTAINERS                            |   2 +
 doc/guides/nics/features/liquidio.ini  |  29 ++++
 doc/guides/nics/index.rst              |   1 +
 doc/guides/nics/liquidio.rst           | 280 +++++++++++++++++++++++++++++++++
 doc/guides/rel_notes/release_17_05.rst |   3 +
 5 files changed, 315 insertions(+)
 create mode 100644 doc/guides/nics/features/liquidio.ini
 create mode 100644 doc/guides/nics/liquidio.rst

diff --git a/MAINTAINERS b/MAINTAINERS
index 2e16c5d..3529e0a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -303,6 +303,8 @@ Cavium LiquidIO
 M: Shijith Thotton <shijith.thotton@cavium.com>
 M: Srisivasubramanian Srinivasan <ssrinivasan@cavium.com>
 F: drivers/net/liquidio/
+F: doc/guides/nics/liquidio.rst
+F: doc/guides/nics/features/liquidio.ini
 
 Chelsio cxgbe
 M: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
new file mode 100644
index 0000000..eac32ba
--- /dev/null
+++ b/doc/guides/nics/features/liquidio.ini
@@ -0,0 +1,29 @@
+;
+; Supported features of the 'LiquidIO' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+Link status          = Y
+Link status event    = Y
+MTU update           = Y
+Jumbo frame          = Y
+Scattered Rx         = Y
+Allmulticast mode    = Y
+RSS hash             = Y
+RSS key update       = Y
+RSS reta update      = Y
+SR-IOV               = Y
+VLAN filter          = Y
+CRC offload          = Y
+VLAN offload         = P
+L3 checksum offload  = Y
+L4 checksum offload  = Y
+Inner L3 checksum    = Y
+Inner L4 checksum    = Y
+Basic stats          = Y
+Extended stats       = Y
+Linux UIO            = Y
+Linux VFIO           = Y
+x86-64               = Y
+Usage doc            = Y
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 5248625..37e6416 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -47,6 +47,7 @@ Network Interface Controller Drivers
     ixgbe
     intel_vf
     kni
+    liquidio
     mlx4
     mlx5
     nfp
diff --git a/doc/guides/nics/liquidio.rst b/doc/guides/nics/liquidio.rst
new file mode 100644
index 0000000..9ffdc35
--- /dev/null
+++ b/doc/guides/nics/liquidio.rst
@@ -0,0 +1,280 @@
+..  BSD LICENSE
+    Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+    All rights reserved.
+
+    Redistribution and use in source and binary forms, with or without
+    modification, are permitted provided that the following conditions
+    are met:
+
+    * Redistributions of source code must retain the above copyright
+    notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright
+    notice, this list of conditions and the following disclaimer in
+    the documentation and/or other materials provided with the
+    distribution.
+    * Neither the name of Cavium, Inc. nor the names of its
+    contributors may be used to endorse or promote products derived
+    from this software without specific prior written permission.
+
+    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+    OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+LiquidIO VF Poll Mode Driver
+============================
+
+The LiquidIO VF PMD library (librte_pmd_lio) provides poll mode driver support for
+Cavium LiquidIO® II server adapter VFs. PF management and VF creation can be
+done using kernel driver.
+
+More information can be found at `Cavium Official Website
+<http://cavium.com/LiquidIO_Adapters.html>`_.
+
+Supported LiquidIO Adapters
+-----------------------------
+
+- LiquidIO II CN2350 210SV
+- LiquidIO II CN2360 210SV
+
+
+Pre-Installation Configuration
+------------------------------
+
+The following options can be modified in the ``config`` file.
+Please note that enabling debugging options may affect system performance.
+
+- ``CONFIG_RTE_LIBRTE_LIO_PMD`` (default ``y``)
+
+  Toggle compilation of LiquidIO PMD.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_DRIVER`` (default ``n``)
+
+  Toggle display of generic debugging messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_INIT`` (default ``n``)
+
+  Toggle display of initialization related messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_RX`` (default ``n``)
+
+  Toggle display of receive fast path run-time messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_TX`` (default ``n``)
+
+  Toggle display of transmit fast path run-time messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_MBOX`` (default ``n``)
+
+  Toggle display of mailbox messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_REGS`` (default ``n``)
+
+  Toggle display of register reads and writes.
+
+
+.. _lio_driver-compilation:
+
+Driver Compilation
+------------------
+
+To compile LiquidIO PMD for Linux x86_64 gcc target, run the following "make"
+command:
+
+.. code-block:: console
+
+   cd <DPDK-source-directory>
+   make install T=x86_64-native-linuxapp-gcc
+
+
+Sample Application Notes
+------------------------
+
+This section demonstrates how to launch ``testpmd`` with LiquidIO® CN23XX
+device managed by ``librte_pmd_lio`` in Linux operating system.
+
+#. Mount huge pages:
+
+   .. code-block:: console
+
+      mkdir /mnt/huge
+      mount -t hugetlbfs nodev /mnt/huge
+
+#. Request huge pages:
+
+   .. code-block:: console
+
+      echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages/nr_hugepages
+
+#. Load ``vfio-pci`` driver:
+
+   .. code-block:: console
+
+      modprobe vfio-pci
+
+#. Bind the LiquidIO VFs to ``vfio-pci`` loaded in previous step:
+
+   Setup VFIO permissions for regular users and then bind to ``vfio-pci``:
+
+   .. code-block:: console
+
+      sudo chmod a+x /dev/vfio
+
+      sudo chmod 0666 /dev/vfio/*
+
+      ./usertools/dpdk-devbind.py --bind vfio-pci 0000:03:00.3 0000:03:08.3
+
+#. Start ``testpmd`` with basic parameters:
+
+   .. code-block:: console
+
+      ./build/app/testpmd -c 0xf -n 4 -- -i
+
+   Example output:
+
+   .. code-block:: console
+
+      [...]
+      EAL: PCI device 0000:03:00.3 on NUMA socket 0
+      EAL:   probe driver: 177d:9712 net_liovf
+      EAL:   using IOMMU type 1 (Type 1)
+      PMD: net_liovf[03:00.3]INFO: DEVICE : CN23XX VF
+      EAL: PCI device 0000:03:08.3 on NUMA socket 0
+      EAL:   probe driver: 177d:9712 net_liovf
+      PMD: net_liovf[03:08.3]INFO: DEVICE : CN23XX VF
+      Interactive-mode selected
+      USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=171456, size=2176, socket=0
+      Configuring Port 0 (socket 0)
+      PMD: net_liovf[03:00.3]INFO: Starting port 0
+      Port 0: F2:A8:1B:5E:B4:66
+      Configuring Port 1 (socket 0)
+      PMD: net_liovf[03:08.3]INFO: Starting port 1
+      Port 1: 32:76:CC:EE:56:D7
+      Checking link statuses...
+      Port 0 Link Up - speed 10000 Mbps - full-duplex
+      Port 1 Link Up - speed 10000 Mbps - full-duplex
+      Done
+      testpmd>
+
+
+SR-IOV: Prerequisites and Sample Application Notes
+--------------------------------------------------
+
+This section provides instructions to configure SR-IOV with Linux OS.
+
+#. Verify SR-IOV and ARI capabilities are enabled on the adapter using ``lspci``:
+
+   .. code-block:: console
+
+      lspci -s <slot> -vvv
+
+   Example output:
+
+   .. code-block:: console
+
+      [...]
+      Capabilities: [148 v1] Alternative Routing-ID Interpretation (ARI)
+      [...]
+      Capabilities: [178 v1] Single Root I/O Virtualization (SR-IOV)
+      [...]
+      Kernel driver in use: LiquidIO
+
+#. Load the kernel module:
+
+   .. code-block:: console
+
+      modprobe liquidio
+
+#. Bring up the PF ports:
+
+   .. code-block:: console
+
+      ifconfig p4p1 up
+      ifconfig p4p2 up
+
+#. Change PF MTU if required:
+
+   .. code-block:: console
+
+      ifconfig p4p1 mtu 9000
+      ifconfig p4p2 mtu 9000
+
+#. Create VF device(s):
+
+   Echo number of VFs to be created into ``"sriov_numvfs"`` sysfs entry
+   of the parent PF.
+
+   .. code-block:: console
+
+      echo 1 > /sys/bus/pci/devices/0000:03:00.0/sriov_numvfs
+      echo 1 > /sys/bus/pci/devices/0000:03:00.1/sriov_numvfs
+
+
+#. Assign VF MAC address:
+
+   Assign MAC address to the VF using iproute2 utility. The syntax is::
+
+      ip link set <PF iface> vf <VF id> mac <macaddr>
+
+   Example output:
+
+   .. code-block:: console
+
+      ip link set p4p1 vf 0 mac F2:A8:1B:5E:B4:66
+
+
+#. Assign VF(s) to VM.
+
+   The VF devices may be passed through to the guest VM using qemu or
+   virt-manager or virsh etc.
+
+   Example qemu guest launch command:
+
+   .. code-block:: console
+
+      ./qemu-system-x86_64 -name lio-vm -machine accel=kvm \
+      -cpu host -m 4096 -smp 4 \
+      -drive file=<disk_file>,if=none,id=disk1,format=<type> \
+      -device virtio-blk-pci,scsi=off,drive=disk1,id=virtio-disk1,bootindex=1 \
+      -device vfio-pci,host=03:00.3 -device vfio-pci,host=03:08.3
+
+
+#. Running testpmd
+
+   Refer :ref:`notes above <lio_driver-compilation>`
+   to compile and run ``testpmd`` application.
+   Use ``igb_uio`` instead of ``vfio-pci`` in VM.
+
+
+Limitations
+-----------
+
+VF MTU
+~~~~~~
+
+VF MTU is limited by PF MTU. Raise PF value before configuring VF for larger packet size.
+
+VLAN offload
+~~~~~~~~~~~~
+
+Tx VLAN insertion is not supported and consequently VLAN offload feature is
+marked partial.
+
+Ring size
+~~~~~~~~~
+
+Number of descriptors for Rx/Tx ring should be in the range 128 to 512.
+
+CRC striping
+~~~~~~~~~~~~
+
+LiquidIO adapters strip ethernet FCS of every packet coming to the host
+interface. So, CRC will be stripped even when the ``rxmode.hw_strip_crc``
+member is set to 0 in ``struct rte_eth_conf``.
diff --git a/doc/guides/rel_notes/release_17_05.rst b/doc/guides/rel_notes/release_17_05.rst
index e25ea9f..7ba6e10 100644
--- a/doc/guides/rel_notes/release_17_05.rst
+++ b/doc/guides/rel_notes/release_17_05.rst
@@ -41,6 +41,9 @@ New Features
      Also, make sure to start the actual text at the margin.
      =========================================================
 
+* **Added LiquidIO network PMD.**
+
+  Added poll mode driver support for Cavium LiquidIO II server adapter VFs.
 
 Resolved Issues
 ---------------
-- 
1.8.3.1

Re: [PATCH v2 46/46] doc: add doc for liquidio

[Ferruh Yigit]

On 3/2/2017 11:32 AM, Shijith Thotton wrote:
> Added doc/guides/nics/liquidio.rst and
> doc/guides/nics/features/liquidio.ini. Updated release notes.
> 
> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>

<...>

> --- /dev/null
> +++ b/doc/guides/nics/features/liquidio.ini
> @@ -0,0 +1,29 @@
> +;
> +; Supported features of the 'LiquidIO' network poll mode driver.
> +;
> +; Refer to default.ini for the full list of available PMD features.
> +;
> +[Features]
> +Link status          = Y
> +Link status event    = Y
> +MTU update           = Y
> +Jumbo frame          = Y
> +Scattered Rx         = Y
> +Allmulticast mode    = Y
> +RSS hash             = Y
> +RSS key update       = Y
> +RSS reta update      = Y
> +SR-IOV               = Y

This has been discussed before, but I am not still clear what does
setting this feature in VF driver means. What is the intention here?

> +VLAN filter          = Y
> +CRC offload          = Y
> +VLAN offload         = P
> +L3 checksum offload  = Y
> +L4 checksum offload  = Y
> +Inner L3 checksum    = Y
> +Inner L4 checksum    = Y
> +Basic stats          = Y
> +Extended stats       = Y
> +Linux UIO            = Y
> +Linux VFIO           = Y
> +x86-64               = Y

Is other platforms not supported?
If that is the case, PMD should not be enabled by default.

> +Usage doc            = Y

Instead of features file in one patch, can you please split some into
other patches?
Each patch that implements the feature can update the relevant item in
the doc?

<...>

> +.. _lio_driver-compilation:

Thank you for the documentation but below part seems generic to all
PMDs, and already documented in other PMDs.
I am not sure about this part, what do you think only keeping LiquidIO
related part? And perhaps we can prepare a common document that you can
reference here.

> +
> +Driver Compilation
> +------------------
> +
> +To compile LiquidIO PMD for Linux x86_64 gcc target, run the following "make"
> +command:
> +
> +.. code-block:: console
> +
> +   cd <DPDK-source-directory>
> +   make install T=x86_64-native-linuxapp-gcc
> +
> +
> +Sample Application Notes
> +------------------------
> +
> +This section demonstrates how to launch ``testpmd`` with LiquidIO® CN23XX
> +device managed by ``librte_pmd_lio`` in Linux operating system.
> +
> +#. Mount huge pages:
> +
> +   .. code-block:: console
> +
> +      mkdir /mnt/huge
> +      mount -t hugetlbfs nodev /mnt/huge
> +
> +#. Request huge pages:
> +
> +   .. code-block:: console
> +
> +      echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages/nr_hugepages
> +
> +#. Load ``vfio-pci`` driver:
> +
> +   .. code-block:: console
> +
> +      modprobe vfio-pci
> +
> +#. Bind the LiquidIO VFs to ``vfio-pci`` loaded in previous step:
> +
> +   Setup VFIO permissions for regular users and then bind to ``vfio-pci``:
> +
> +   .. code-block:: console
> +
> +      sudo chmod a+x /dev/vfio
> +
> +      sudo chmod 0666 /dev/vfio/*
> +
> +      ./usertools/dpdk-devbind.py --bind vfio-pci 0000:03:00.3 0000:03:08.3
> +
> +#. Start ``testpmd`` with basic parameters:
> +
> +   .. code-block:: console
> +
> +      ./build/app/testpmd -c 0xf -n 4 -- -i
> +
> +   Example output:
> +
> +   .. code-block:: console
> +
> +      [...]
> +      EAL: PCI device 0000:03:00.3 on NUMA socket 0
> +      EAL:   probe driver: 177d:9712 net_liovf
> +      EAL:   using IOMMU type 1 (Type 1)
> +      PMD: net_liovf[03:00.3]INFO: DEVICE : CN23XX VF
> +      EAL: PCI device 0000:03:08.3 on NUMA socket 0
> +      EAL:   probe driver: 177d:9712 net_liovf
> +      PMD: net_liovf[03:08.3]INFO: DEVICE : CN23XX VF
> +      Interactive-mode selected
> +      USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=171456, size=2176, socket=0
> +      Configuring Port 0 (socket 0)
> +      PMD: net_liovf[03:00.3]INFO: Starting port 0
> +      Port 0: F2:A8:1B:5E:B4:66
> +      Configuring Port 1 (socket 0)
> +      PMD: net_liovf[03:08.3]INFO: Starting port 1
> +      Port 1: 32:76:CC:EE:56:D7
> +      Checking link statuses...
> +      Port 0 Link Up - speed 10000 Mbps - full-duplex
> +      Port 1 Link Up - speed 10000 Mbps - full-duplex
> +      Done
> +      testpmd>
> +
> +
> +SR-IOV: Prerequisites and Sample Application Notes
> +--------------------------------------------------
> +
> +This section provides instructions to configure SR-IOV with Linux OS.
> +
> +#. Verify SR-IOV and ARI capabilities are enabled on the adapter using ``lspci``:
> +
> +   .. code-block:: console
> +
> +      lspci -s <slot> -vvv
> +
> +   Example output:
> +
> +   .. code-block:: console
> +
> +      [...]
> +      Capabilities: [148 v1] Alternative Routing-ID Interpretation (ARI)
> +      [...]
> +      Capabilities: [178 v1] Single Root I/O Virtualization (SR-IOV)
> +      [...]
> +      Kernel driver in use: LiquidIO
> +
> +#. Load the kernel module:
> +
> +   .. code-block:: console
> +
> +      modprobe liquidio
> +
> +#. Bring up the PF ports:
> +
> +   .. code-block:: console
> +
> +      ifconfig p4p1 up
> +      ifconfig p4p2 up
> +
> +#. Change PF MTU if required:
> +
> +   .. code-block:: console
> +
> +      ifconfig p4p1 mtu 9000
> +      ifconfig p4p2 mtu 9000
> +
> +#. Create VF device(s):
> +
> +   Echo number of VFs to be created into ``"sriov_numvfs"`` sysfs entry
> +   of the parent PF.
> +
> +   .. code-block:: console
> +
> +      echo 1 > /sys/bus/pci/devices/0000:03:00.0/sriov_numvfs
> +      echo 1 > /sys/bus/pci/devices/0000:03:00.1/sriov_numvfs
> +
> +
> +#. Assign VF MAC address:
> +
> +   Assign MAC address to the VF using iproute2 utility. The syntax is::
> +
> +      ip link set <PF iface> vf <VF id> mac <macaddr>
> +
> +   Example output:
> +
> +   .. code-block:: console
> +
> +      ip link set p4p1 vf 0 mac F2:A8:1B:5E:B4:66
> +
> +
> +#. Assign VF(s) to VM.
> +
> +   The VF devices may be passed through to the guest VM using qemu or
> +   virt-manager or virsh etc.
> +
> +   Example qemu guest launch command:
> +
> +   .. code-block:: console
> +
> +      ./qemu-system-x86_64 -name lio-vm -machine accel=kvm \
> +      -cpu host -m 4096 -smp 4 \
> +      -drive file=<disk_file>,if=none,id=disk1,format=<type> \
> +      -device virtio-blk-pci,scsi=off,drive=disk1,id=virtio-disk1,bootindex=1 \
> +      -device vfio-pci,host=03:00.3 -device vfio-pci,host=03:08.3
> +
> +
> +#. Running testpmd
> +
> +   Refer :ref:`notes above <lio_driver-compilation>`
> +   to compile and run ``testpmd`` application.
> +   Use ``igb_uio`` instead of ``vfio-pci`` in VM.
> +
> +
> +Limitations
> +-----------
> +
> +VF MTU
> +~~~~~~
> +
> +VF MTU is limited by PF MTU. Raise PF value before configuring VF for larger packet size.
> +
> +VLAN offload
> +~~~~~~~~~~~~
> +
> +Tx VLAN insertion is not supported and consequently VLAN offload feature is
> +marked partial.
> +
> +Ring size
> +~~~~~~~~~
> +
> +Number of descriptors for Rx/Tx ring should be in the range 128 to 512.
> +
> +CRC striping
> +~~~~~~~~~~~~
> +

<...>

Re: [PATCH v2 46/46] doc: add doc for liquidio

[Shijith Thotton]

On Tue, Mar 21, 2017 at 12:33:30PM +0000, Ferruh Yigit wrote:
> On 3/2/2017 11:32 AM, Shijith Thotton wrote:
> > Added doc/guides/nics/liquidio.rst and
> > doc/guides/nics/features/liquidio.ini. Updated release notes.
> > 
> > Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> > Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> > Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> > Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
> > Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
> 
> <...>
> 
> > --- /dev/null
> > +++ b/doc/guides/nics/features/liquidio.ini
> > @@ -0,0 +1,29 @@
> > +;
> > +; Supported features of the 'LiquidIO' network poll mode driver.
> > +;
> > +; Refer to default.ini for the full list of available PMD features.
> > +;
> > +[Features]
> > +Link status          = Y
> > +Link status event    = Y
> > +MTU update           = Y
> > +Jumbo frame          = Y
> > +Scattered Rx         = Y
> > +Allmulticast mode    = Y
> > +RSS hash             = Y
> > +RSS key update       = Y
> > +RSS reta update      = Y
> > +SR-IOV               = Y
> 
> This has been discussed before, but I am not still clear what does
> setting this feature in VF driver means. What is the intention here?
> 

Will remove.

> > +VLAN filter          = Y
> > +CRC offload          = Y
> > +VLAN offload         = P
> > +L3 checksum offload  = Y
> > +L4 checksum offload  = Y
> > +Inner L3 checksum    = Y
> > +Inner L4 checksum    = Y
> > +Basic stats          = Y
> > +Extended stats       = Y
> > +Linux UIO            = Y
> > +Linux VFIO           = Y
> > +x86-64               = Y
> 
> Is other platforms not supported?
> If that is the case, PMD should not be enabled by default.
> 

Verified build on x86-32, x86-64, ARM and for BSD targets. Hope that
qualifies to be enabled by default.

> > +Usage doc            = Y
> 
> Instead of features file in one patch, can you please split some into
> other patches?
> Each patch that implements the feature can update the relevant item in
> the doc?

Will do.

> 
> <...>
> 
> > +.. _lio_driver-compilation:
> 
> Thank you for the documentation but below part seems generic to all
> PMDs, and already documented in other PMDs.
> I am not sure about this part, what do you think only keeping LiquidIO
> related part? And perhaps we can prepare a common document that you can
> reference here.
> 

I can submit a patchset aimed at this later, as this need to touch
documentation of all PMDs with common part.

> > +
> > +Driver Compilation
> > +------------------
> > +
> > +To compile LiquidIO PMD for Linux x86_64 gcc target, run the following "make"
> > +command:
> > +
> > +.. code-block:: console
> > +
> > +   cd <DPDK-source-directory>
> > +   make install T=x86_64-native-linuxapp-gcc
> > +
> > +
> > +Sample Application Notes
> > +------------------------
> > +
> > +This section demonstrates how to launch ``testpmd`` with LiquidIO® CN23XX
> > +device managed by ``librte_pmd_lio`` in Linux operating system.
> > +
> > +#. Mount huge pages:
> > +
> > +   .. code-block:: console
> > +
> > +      mkdir /mnt/huge
> > +      mount -t hugetlbfs nodev /mnt/huge
> > +
> > +#. Request huge pages:
> > +
> > +   .. code-block:: console
> > +
> > +      echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages/nr_hugepages
> > +
> > +#. Load ``vfio-pci`` driver:
> > +
> > +   .. code-block:: console
> > +
> > +      modprobe vfio-pci
> > +
> > +#. Bind the LiquidIO VFs to ``vfio-pci`` loaded in previous step:
> > +
> > +   Setup VFIO permissions for regular users and then bind to ``vfio-pci``:
> > +
> > +   .. code-block:: console
> > +
> > +      sudo chmod a+x /dev/vfio
> > +
> > +      sudo chmod 0666 /dev/vfio/*
> > +
> > +      ./usertools/dpdk-devbind.py --bind vfio-pci 0000:03:00.3 0000:03:08.3
> > +
> > +#. Start ``testpmd`` with basic parameters:
> > +
> > +   .. code-block:: console
> > +
> > +      ./build/app/testpmd -c 0xf -n 4 -- -i
> > +
> > +   Example output:
> > +
> > +   .. code-block:: console
> > +
> > +      [...]
> > +      EAL: PCI device 0000:03:00.3 on NUMA socket 0
> > +      EAL:   probe driver: 177d:9712 net_liovf
> > +      EAL:   using IOMMU type 1 (Type 1)
> > +      PMD: net_liovf[03:00.3]INFO: DEVICE : CN23XX VF
> > +      EAL: PCI device 0000:03:08.3 on NUMA socket 0
> > +      EAL:   probe driver: 177d:9712 net_liovf
> > +      PMD: net_liovf[03:08.3]INFO: DEVICE : CN23XX VF
> > +      Interactive-mode selected
> > +      USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=171456, size=2176, socket=0
> > +      Configuring Port 0 (socket 0)
> > +      PMD: net_liovf[03:00.3]INFO: Starting port 0
> > +      Port 0: F2:A8:1B:5E:B4:66
> > +      Configuring Port 1 (socket 0)
> > +      PMD: net_liovf[03:08.3]INFO: Starting port 1
> > +      Port 1: 32:76:CC:EE:56:D7
> > +      Checking link statuses...
> > +      Port 0 Link Up - speed 10000 Mbps - full-duplex
> > +      Port 1 Link Up - speed 10000 Mbps - full-duplex
> > +      Done
> > +      testpmd>
> > +
> > +
> > +SR-IOV: Prerequisites and Sample Application Notes
> > +--------------------------------------------------
> > +
> > +This section provides instructions to configure SR-IOV with Linux OS.
> > +
> > +#. Verify SR-IOV and ARI capabilities are enabled on the adapter using ``lspci``:
> > +
> > +   .. code-block:: console
> > +
> > +      lspci -s <slot> -vvv
> > +
> > +   Example output:
> > +
> > +   .. code-block:: console
> > +
> > +      [...]
> > +      Capabilities: [148 v1] Alternative Routing-ID Interpretation (ARI)
> > +      [...]
> > +      Capabilities: [178 v1] Single Root I/O Virtualization (SR-IOV)
> > +      [...]
> > +      Kernel driver in use: LiquidIO
> > +
> > +#. Load the kernel module:
> > +
> > +   .. code-block:: console
> > +
> > +      modprobe liquidio
> > +
> > +#. Bring up the PF ports:
> > +
> > +   .. code-block:: console
> > +
> > +      ifconfig p4p1 up
> > +      ifconfig p4p2 up
> > +
> > +#. Change PF MTU if required:
> > +
> > +   .. code-block:: console
> > +
> > +      ifconfig p4p1 mtu 9000
> > +      ifconfig p4p2 mtu 9000
> > +
> > +#. Create VF device(s):
> > +
> > +   Echo number of VFs to be created into ``"sriov_numvfs"`` sysfs entry
> > +   of the parent PF.
> > +
> > +   .. code-block:: console
> > +
> > +      echo 1 > /sys/bus/pci/devices/0000:03:00.0/sriov_numvfs
> > +      echo 1 > /sys/bus/pci/devices/0000:03:00.1/sriov_numvfs
> > +
> > +
> > +#. Assign VF MAC address:
> > +
> > +   Assign MAC address to the VF using iproute2 utility. The syntax is::
> > +
> > +      ip link set <PF iface> vf <VF id> mac <macaddr>
> > +
> > +   Example output:
> > +
> > +   .. code-block:: console
> > +
> > +      ip link set p4p1 vf 0 mac F2:A8:1B:5E:B4:66
> > +
> > +
> > +#. Assign VF(s) to VM.
> > +
> > +   The VF devices may be passed through to the guest VM using qemu or
> > +   virt-manager or virsh etc.
> > +
> > +   Example qemu guest launch command:
> > +
> > +   .. code-block:: console
> > +
> > +      ./qemu-system-x86_64 -name lio-vm -machine accel=kvm \
> > +      -cpu host -m 4096 -smp 4 \
> > +      -drive file=<disk_file>,if=none,id=disk1,format=<type> \
> > +      -device virtio-blk-pci,scsi=off,drive=disk1,id=virtio-disk1,bootindex=1 \
> > +      -device vfio-pci,host=03:00.3 -device vfio-pci,host=03:08.3
> > +
> > +
> > +#. Running testpmd
> > +
> > +   Refer :ref:`notes above <lio_driver-compilation>`
> > +   to compile and run ``testpmd`` application.
> > +   Use ``igb_uio`` instead of ``vfio-pci`` in VM.
> > +
> > +
> > +Limitations
> > +-----------
> > +
> > +VF MTU
> > +~~~~~~
> > +
> > +VF MTU is limited by PF MTU. Raise PF value before configuring VF for larger packet size.
> > +
> > +VLAN offload
> > +~~~~~~~~~~~~
> > +
> > +Tx VLAN insertion is not supported and consequently VLAN offload feature is
> > +marked partial.
> > +
> > +Ring size
> > +~~~~~~~~~
> > +
> > +Number of descriptors for Rx/Tx ring should be in the range 128 to 512.
> > +
> > +CRC striping
> > +~~~~~~~~~~~~
> > +
> 
> <...>

Re: [PATCH v2 46/46] doc: add doc for liquidio

[Ferruh Yigit]

On 3/23/2017 5:44 AM, Shijith Thotton wrote:
> On Tue, Mar 21, 2017 at 12:33:30PM +0000, Ferruh Yigit wrote:
>> On 3/2/2017 11:32 AM, Shijith Thotton wrote:
>>> Added doc/guides/nics/liquidio.rst and
>>> doc/guides/nics/features/liquidio.ini. Updated release notes.
>>>
>>> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
>>> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
>>> Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
>>> Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
>>> Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
>>> Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
>>

<...>

> 
>>> +VLAN filter          = Y
>>> +CRC offload          = Y
>>> +VLAN offload         = P
>>> +L3 checksum offload  = Y
>>> +L4 checksum offload  = Y
>>> +Inner L3 checksum    = Y
>>> +Inner L4 checksum    = Y
>>> +Basic stats          = Y
>>> +Extended stats       = Y
>>> +Linux UIO            = Y
>>> +Linux VFIO           = Y
>>> +x86-64               = Y
>>
>> Is other platforms not supported?
>> If that is the case, PMD should not be enabled by default.
>>
> 
> Verified build on x86-32, x86-64, ARM and for BSD targets. Hope that
> qualifies to be enabled by default.

That should be OK, thanks.

<...>

>>> +.. _lio_driver-compilation:
>>
>> Thank you for the documentation but below part seems generic to all
>> PMDs, and already documented in other PMDs.
>> I am not sure about this part, what do you think only keeping LiquidIO
>> related part? And perhaps we can prepare a common document that you can
>> reference here.
>>
> 
> I can submit a patchset aimed at this later, as this need to touch
> documentation of all PMDs with common part.

OK, thanks.
Please leave this part in patch as it is as this needs to be resolves in
a higher level.

Re: [PATCH v2 00/46] LiquidIO PMD

[Ferruh Yigit]

On 3/2/2017 11:32 AM, Shijith Thotton wrote:
> The patch series provides initial version of virtual function poll mode
> driver for Cavium LiquidIO II server adapters. This version adds support
> for LiquidIO II CN23XX 210SV adapters.
> 
> Patch series includes driver documentation doc/guides/nics/liquidio.rst
> and list of supported features doc/guides/nics/features/liquidio.ini.
> Updated release notes to notify the addition of new PMD.
> 
> v2 changes:
> * Restructured patches as suggested by Ferruh.
>  - http://dpdk.org/ml/archives/dev/2017-February/058186.html
> * Addressed review comments on driver from Ferruh.
>  - http://dpdk.org/ml/archives/dev/2017-February/058188.html
>  - http://dpdk.org/ml/archives/dev/2017-February/058194.html
> * Modified commit logs as suggested by Ferruh.
>  - http://dpdk.org/ml/archives/dev/2017-February/058189.html
>  - http://dpdk.org/ml/archives/dev/2017-February/058190.html
>  - http://dpdk.org/ml/archives/dev/2017-February/058193.html
> * Made changes to documentation as per review comments from John.
>  - http://dpdk.org/ml/archives/dev/2017-February/058206.html
> * Fixed FreeBSD build failure.
>  - http://dpdk.org/ml/archives/test-report/2017-February/011272.html
> * Updated driver documentation:
>  - Added CN2360 under supported LiquidIO adapters.
>  - Added CRC strip under limitations.
> 
> Shijith Thotton (46):
>   config: add liquidio PMD skeleton
>   net/liquidio/base: hardware register definitions
>   net/liquidio: definitions for log
>   net/liquidio: liquidio VF PMD driver registration
>   net/liquidio/base: macros to read and write register
>   net/liquidio: liquidio device init
>   net/liquidio: add API to disable IO queues
>   net/liquidio: add API to setup IO queue registers
>   net/liquidio: add mbox APIs for PF VF communication
>   net/liquidio: add API to setup mbox registers
>   net/liquidio: add API for PF VF handshake
>   net/liquidio: add API for VF FLR
>   net/liquidio: add APIs to allocate and free IQ
>   net/liquidio: add API to setup IQ
>   net/liquidio: add APIs to allocate and free SC buffer pool
>   net/liquidio: add APIs to allocate and free soft command
>   net/liquidio: add APIs for response list
>   net/liquidio: add API to send packet to device
>   net/liquidio: add API to configure device
>   net/liquidio: add API to setup Rx queue
>   net/liquidio: initialize Rx queue
>   net/liquidio: add Rx data path
>   net/liquidio: add API to release Rx queue
>   net/liquidio: add API to setup Tx queue
>   net/liquidio: add APIs for SG list
>   net/liquidio: add APIs to enable and disable IO queues
>   net/liquidio: add Tx data path for single segment
>   net/liquidio: add Tx data path for multiple segments
>   net/liquidio: add API to flush IQ
>   net/liquidio: add API to release Tx queue
>   net/liquidio: add APIs to start device and update link
>   net/liquidio: add APIs to alloc and send control command
>   net/liquidio: add API to control Rx
>   net/liquidio: add RSS support
>   net/liquidio: add API to get device info
>   net/liquidio: add API to set MTU
>   net/liquidio: add APIs to enable and disable multicast
>   net/liquidio: add APIs to set link up and down
>   net/liquidio: add API to configure UDP tunnel port
>   net/liquidio: add support for Rx stats
>   net/liquidio: add support for Tx stats
>   net/liquidio: add APIs for hardware stats
>   net/liquidio: add API to stop device
>   net/liquidio: add API to close device
>   net/liquidio: add API to add and remove VLAN port
>   doc: add doc for liquidio

Hi Shijith,

Overall patch looks good to me, I comment on a few issues.
I think PMD is ready to merge after mentioned issues addressed.

And can you also send a web page patch to list PMD in DPDK supported
NICs web page:
http://dpdk.org/doc/nics
(web repo: http://dpdk.org/browse/tools/dpdk-web/)

Thanks,
ferruh

Re: [PATCH v2 00/46] LiquidIO PMD

[Shijith Thotton]

On Tue, Mar 21, 2017 at 12:38:18PM +0000, Ferruh Yigit wrote:
> On 3/2/2017 11:32 AM, Shijith Thotton wrote:
> > The patch series provides initial version of virtual function poll mode
> > driver for Cavium LiquidIO II server adapters. This version adds support
> > for LiquidIO II CN23XX 210SV adapters.
> > 
> > Patch series includes driver documentation doc/guides/nics/liquidio.rst
> > and list of supported features doc/guides/nics/features/liquidio.ini.
> > Updated release notes to notify the addition of new PMD.
> > 
> > v2 changes:
> > * Restructured patches as suggested by Ferruh.
> >  - http://dpdk.org/ml/archives/dev/2017-February/058186.html
> > * Addressed review comments on driver from Ferruh.
> >  - http://dpdk.org/ml/archives/dev/2017-February/058188.html
> >  - http://dpdk.org/ml/archives/dev/2017-February/058194.html
> > * Modified commit logs as suggested by Ferruh.
> >  - http://dpdk.org/ml/archives/dev/2017-February/058189.html
> >  - http://dpdk.org/ml/archives/dev/2017-February/058190.html
> >  - http://dpdk.org/ml/archives/dev/2017-February/058193.html
> > * Made changes to documentation as per review comments from John.
> >  - http://dpdk.org/ml/archives/dev/2017-February/058206.html
> > * Fixed FreeBSD build failure.
> >  - http://dpdk.org/ml/archives/test-report/2017-February/011272.html
> > * Updated driver documentation:
> >  - Added CN2360 under supported LiquidIO adapters.
> >  - Added CRC strip under limitations.
> > 
> > Shijith Thotton (46):
> >   config: add liquidio PMD skeleton
> >   net/liquidio/base: hardware register definitions
> >   net/liquidio: definitions for log
> >   net/liquidio: liquidio VF PMD driver registration
> >   net/liquidio/base: macros to read and write register
> >   net/liquidio: liquidio device init
> >   net/liquidio: add API to disable IO queues
> >   net/liquidio: add API to setup IO queue registers
> >   net/liquidio: add mbox APIs for PF VF communication
> >   net/liquidio: add API to setup mbox registers
> >   net/liquidio: add API for PF VF handshake
> >   net/liquidio: add API for VF FLR
> >   net/liquidio: add APIs to allocate and free IQ
> >   net/liquidio: add API to setup IQ
> >   net/liquidio: add APIs to allocate and free SC buffer pool
> >   net/liquidio: add APIs to allocate and free soft command
> >   net/liquidio: add APIs for response list
> >   net/liquidio: add API to send packet to device
> >   net/liquidio: add API to configure device
> >   net/liquidio: add API to setup Rx queue
> >   net/liquidio: initialize Rx queue
> >   net/liquidio: add Rx data path
> >   net/liquidio: add API to release Rx queue
> >   net/liquidio: add API to setup Tx queue
> >   net/liquidio: add APIs for SG list
> >   net/liquidio: add APIs to enable and disable IO queues
> >   net/liquidio: add Tx data path for single segment
> >   net/liquidio: add Tx data path for multiple segments
> >   net/liquidio: add API to flush IQ
> >   net/liquidio: add API to release Tx queue
> >   net/liquidio: add APIs to start device and update link
> >   net/liquidio: add APIs to alloc and send control command
> >   net/liquidio: add API to control Rx
> >   net/liquidio: add RSS support
> >   net/liquidio: add API to get device info
> >   net/liquidio: add API to set MTU
> >   net/liquidio: add APIs to enable and disable multicast
> >   net/liquidio: add APIs to set link up and down
> >   net/liquidio: add API to configure UDP tunnel port
> >   net/liquidio: add support for Rx stats
> >   net/liquidio: add support for Tx stats
> >   net/liquidio: add APIs for hardware stats
> >   net/liquidio: add API to stop device
> >   net/liquidio: add API to close device
> >   net/liquidio: add API to add and remove VLAN port
> >   doc: add doc for liquidio
> 
> Hi Shijith,
> 

Hi Ferruh,

> Overall patch looks good to me, I comment on a few issues.
> I think PMD is ready to merge after mentioned issues addressed.
> 

Thanks for the review. Will send v3 with the changes.

> And can you also send a web page patch to list PMD in DPDK supported
> NICs web page:
> http://dpdk.org/doc/nics
> (web repo: http://dpdk.org/browse/tools/dpdk-web/)
> 

We will send the patch to update web page.

> Thanks,
> ferruh

Thanks,
Shijith

[PATCH v3 00/46] LiquidIO PMD

[Shijith Thotton]

The patch series provides initial version of virtual function poll mode
driver for Cavium LiquidIO II server adapters. This version adds support
for LiquidIO II CN23XX 210SV adapters.

Patch series includes driver documentation doc/guides/nics/liquidio.rst
and list of supported features doc/guides/nics/features/liquidio.ini.
Updated release notes to notify the addition of new PMD.

v3 changes:
* Addressed review comments from Ferruh.
 - http://dpdk.org/ml/archives/dev/2017-March/060778.html
 - http://dpdk.org/ml/archives/dev/2017-March/060772.html

v2 changes:
* Restructured patches as suggested by Ferruh.
 - http://dpdk.org/ml/archives/dev/2017-February/058186.html
* Addressed review comments on driver from Ferruh.
 - http://dpdk.org/ml/archives/dev/2017-February/058188.html
 - http://dpdk.org/ml/archives/dev/2017-February/058194.html
* Modified commit logs as suggested by Ferruh.
 - http://dpdk.org/ml/archives/dev/2017-February/058189.html
 - http://dpdk.org/ml/archives/dev/2017-February/058190.html
 - http://dpdk.org/ml/archives/dev/2017-February/058193.html
* Made changes to documentation as per review comments from John.
 - http://dpdk.org/ml/archives/dev/2017-February/058206.html
* Fixed FreeBSD build failure.
 - http://dpdk.org/ml/archives/test-report/2017-February/011272.html
* Updated driver documentation:
 - Added CN2360 under supported LiquidIO adapters.
 - Added CRC strip under limitations.

Shijith Thotton (46):
  net/liquidio: add liquidio PMD skeleton
  net/liquidio/base: hardware register definitions
  net/liquidio: definitions for log
  net/liquidio: liquidio VF PMD driver registration
  net/liquidio/base: macros to read and write register
  net/liquidio: liquidio device init
  net/liquidio: add API to disable IO queues
  net/liquidio: add API to setup IO queue registers
  net/liquidio: add mbox APIs for PF VF communication
  net/liquidio: add API to setup mbox registers
  net/liquidio: add API for PF VF handshake
  net/liquidio: add API for VF FLR
  net/liquidio: add APIs to allocate and free IQ
  net/liquidio: add API to setup IQ
  net/liquidio: add APIs to allocate and free SC buffer pool
  net/liquidio: add APIs to allocate and free soft command
  net/liquidio: add APIs for response list
  net/liquidio: add API to send packet to device
  net/liquidio: add API to configure device
  net/liquidio: add API to setup Rx queue
  net/liquidio: initialize Rx queue
  net/liquidio: add Rx data path
  net/liquidio: add API to release Rx queue
  net/liquidio: add API to setup Tx queue
  net/liquidio: add APIs for SG list
  net/liquidio: add APIs to enable and disable IO queues
  net/liquidio: add Tx data path for single segment
  net/liquidio: add Tx data path for multiple segments
  net/liquidio: add API to flush IQ
  net/liquidio: add API to release Tx queue
  net/liquidio: add APIs to start device and update link
  net/liquidio: add APIs to alloc and send control command
  net/liquidio: add API to control Rx
  net/liquidio: add RSS support
  net/liquidio: add API to get device info
  net/liquidio: add API to validate VF MTU
  net/liquidio: add APIs to enable and disable multicast
  net/liquidio: add APIs to set link up and down
  net/liquidio: add APIs to configure UDP tunnel port
  net/liquidio: add support for Rx stats
  net/liquidio: add support for Tx stats
  net/liquidio: add APIs for hardware stats
  net/liquidio: add API to stop device
  net/liquidio: add API to close device
  net/liquidio: add API to add and remove VLAN port
  doc: add doc for liquidio and update release notes

 MAINTAINERS                                  |    7 +
 config/common_base                           |   11 +
 doc/guides/nics/features/liquidio.ini        |   28 +
 doc/guides/nics/index.rst                    |    1 +
 doc/guides/nics/liquidio.rst                 |  280 ++++
 doc/guides/rel_notes/release_17_05.rst       |    4 +
 drivers/net/Makefile                         |    1 +
 drivers/net/liquidio/Makefile                |   62 +
 drivers/net/liquidio/base/lio_23xx_reg.h     |  194 +++
 drivers/net/liquidio/base/lio_23xx_vf.c      |  586 ++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h      |   97 ++
 drivers/net/liquidio/base/lio_hw_defs.h      |  249 ++++
 drivers/net/liquidio/base/lio_mbox.c         |  275 ++++
 drivers/net/liquidio/base/lio_mbox.h         |  131 ++
 drivers/net/liquidio/lio_ethdev.c            | 2034 ++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h            |  204 +++
 drivers/net/liquidio/lio_logs.h              |   91 ++
 drivers/net/liquidio/lio_rxtx.c              | 1885 ++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h              |  769 ++++++++++
 drivers/net/liquidio/lio_struct.h            |  689 +++++++++
 drivers/net/liquidio/rte_pmd_lio_version.map |    4 +
 mk/rte.app.mk                                |    1 +
 22 files changed, 7603 insertions(+)
 create mode 100644 doc/guides/nics/features/liquidio.ini
 create mode 100644 doc/guides/nics/liquidio.rst
 create mode 100644 drivers/net/liquidio/Makefile
 create mode 100644 drivers/net/liquidio/base/lio_23xx_reg.h
 create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.c
 create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.h
 create mode 100644 drivers/net/liquidio/base/lio_hw_defs.h
 create mode 100644 drivers/net/liquidio/base/lio_mbox.c
 create mode 100644 drivers/net/liquidio/base/lio_mbox.h
 create mode 100644 drivers/net/liquidio/lio_ethdev.c
 create mode 100644 drivers/net/liquidio/lio_ethdev.h
 create mode 100644 drivers/net/liquidio/lio_logs.h
 create mode 100644 drivers/net/liquidio/lio_rxtx.c
 create mode 100644 drivers/net/liquidio/lio_rxtx.h
 create mode 100644 drivers/net/liquidio/lio_struct.h
 create mode 100644 drivers/net/liquidio/rte_pmd_lio_version.map

-- 
1.8.3.1

[PATCH v3 01/46] net/liquidio: add liquidio PMD skeleton

[Shijith Thotton]

Add makefile and config file options to compile PMD. Add feature and
version map file. Update maintainers file to claim responsibility.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 MAINTAINERS                                  |  6 +++
 config/common_base                           |  5 +++
 doc/guides/nics/features/liquidio.ini        |  7 ++++
 drivers/net/Makefile                         |  1 +
 drivers/net/liquidio/Makefile                | 59 ++++++++++++++++++++++++++++
 drivers/net/liquidio/rte_pmd_lio_version.map |  4 ++
 mk/rte.app.mk                                |  1 +
 7 files changed, 83 insertions(+)
 create mode 100644 doc/guides/nics/features/liquidio.ini
 create mode 100644 drivers/net/liquidio/Makefile
 create mode 100644 drivers/net/liquidio/rte_pmd_lio_version.map

diff --git a/MAINTAINERS b/MAINTAINERS
index 4c2fcac..1421c85 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -291,6 +291,12 @@ M: Maciej Czekaj <maciej.czekaj@caviumnetworks.com>
 F: drivers/net/thunderx/
 F: doc/guides/nics/thunderx.rst
 
+Cavium LiquidIO
+M: Shijith Thotton <shijith.thotton@cavium.com>
+M: Srisivasubramanian Srinivasan <ssrinivasan@cavium.com>
+F: drivers/net/liquidio/
+F: doc/guides/nics/features/liquidio.ini
+
 Chelsio cxgbe
 M: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
 F: drivers/net/cxgbe/
diff --git a/config/common_base b/config/common_base
index 12143ae..a5463a3 100644
--- a/config/common_base
+++ b/config/common_base
@@ -292,6 +292,11 @@ CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_DRIVER=n
 CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_MBOX=n
 
 #
+# Compile burst-oriented Cavium LiquidIO PMD driver
+#
+CONFIG_RTE_LIBRTE_LIO_PMD=y
+
+#
 # Compile Support Libraries for NXP DPAA2
 #
 CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL=n
diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
new file mode 100644
index 0000000..3422661
--- /dev/null
+++ b/doc/guides/nics/features/liquidio.ini
@@ -0,0 +1,7 @@
+;
+; Supported features of the 'LiquidIO' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+x86-64               = Y
diff --git a/drivers/net/Makefile b/drivers/net/Makefile
index 7b8b6a2..0d1ff82 100644
--- a/drivers/net/Makefile
+++ b/drivers/net/Makefile
@@ -42,6 +42,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_ENIC_PMD) += enic
 DIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k
 DIRS-$(CONFIG_RTE_LIBRTE_I40E_PMD) += i40e
 DIRS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) += ixgbe
+DIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += liquidio
 DIRS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += mlx4
 DIRS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5
 DIRS-$(CONFIG_RTE_LIBRTE_NFP_PMD) += nfp
diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
new file mode 100644
index 0000000..ce2b7fc
--- /dev/null
+++ b/drivers/net/liquidio/Makefile
@@ -0,0 +1,59 @@
+#
+#   BSD LICENSE
+#
+#   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Cavium, Inc. nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+LIB = librte_pmd_lio.a
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)/base -I$(SRCDIR)
+
+EXPORT_MAP := rte_pmd_lio_version.map
+
+LIBABIVER := 1
+
+VPATH += $(RTE_SDK)/drivers/net/liquidio/base
+
+#
+# all source are stored in SRCS-y
+#
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) +=
+
+# this lib depends upon:
+DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_eal lib/librte_ether
+DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_mempool lib/librte_mbuf
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/liquidio/rte_pmd_lio_version.map b/drivers/net/liquidio/rte_pmd_lio_version.map
new file mode 100644
index 0000000..8591cc0
--- /dev/null
+++ b/drivers/net/liquidio/rte_pmd_lio_version.map
@@ -0,0 +1,4 @@
+DPDK_17.05 {
+
+	local: *;
+};
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 48c7a1c..a63464f 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -117,6 +117,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD)      += -lrte_pmd_ixgbe
 ifeq ($(CONFIG_RTE_LIBRTE_KNI),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KNI)        += -lrte_pmd_kni
 endif
+_LDLIBS-$(CONFIG_RTE_LIBRTE_LIO_PMD)        += -lrte_pmd_lio
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -lrte_pmd_mlx4 -libverbs
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -lrte_pmd_mlx5 -libverbs
 _LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lrte_pmd_nfp -lm
-- 
1.8.3.1

[PATCH v3 02/46] net/liquidio/base: hardware register definitions

[Shijith Thotton]

Add HW register definitions for LiquidIO II CN23XX adapter.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_reg.h | 194 +++++++++++++++++++++++++++++++
 1 file changed, 194 insertions(+)
 create mode 100644 drivers/net/liquidio/base/lio_23xx_reg.h

diff --git a/drivers/net/liquidio/base/lio_23xx_reg.h b/drivers/net/liquidio/base/lio_23xx_reg.h
new file mode 100644
index 0000000..794bc2c
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_23xx_reg.h
@@ -0,0 +1,194 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_23XX_REG_H_
+#define _LIO_23XX_REG_H_
+
+/* ###################### REQUEST QUEUE ######################### */
+
+/* 64 registers for Input Queues Start Addr - SLI_PKT(0..63)_INSTR_BADDR */
+#define CN23XX_SLI_PKT_INSTR_BADDR_START64	0x10010
+
+/* 64 registers for Input Doorbell - SLI_PKT(0..63)_INSTR_BAOFF_DBELL */
+#define CN23XX_SLI_PKT_INSTR_BADDR_DBELL_START	0x10020
+
+/* 64 registers for Input Queue size - SLI_PKT(0..63)_INSTR_FIFO_RSIZE */
+#define CN23XX_SLI_PKT_INSTR_FIFO_RSIZE_START	0x10030
+
+/* 64 registers for Input Queue Instr Count - SLI_PKT_IN_DONE(0..63)_CNTS */
+#define CN23XX_SLI_PKT_IN_DONE_CNTS_START64	0x10040
+
+/* 64 registers (64-bit) - ES, RO, NS, Arbitration for Input Queue Data &
+ * gather list fetches. SLI_PKT(0..63)_INPUT_CONTROL.
+ */
+#define CN23XX_SLI_PKT_INPUT_CONTROL_START64	0x10000
+
+/* ------- Request Queue Macros --------- */
+
+/* Each Input Queue register is at a 16-byte Offset in BAR0 */
+#define CN23XX_IQ_OFFSET			0x20000
+
+#define CN23XX_SLI_IQ_PKT_CONTROL64(iq)					\
+	(CN23XX_SLI_PKT_INPUT_CONTROL_START64 + ((iq) * CN23XX_IQ_OFFSET))
+
+#define CN23XX_SLI_IQ_BASE_ADDR64(iq)					\
+	(CN23XX_SLI_PKT_INSTR_BADDR_START64 + ((iq) * CN23XX_IQ_OFFSET))
+
+#define CN23XX_SLI_IQ_SIZE(iq)						\
+	(CN23XX_SLI_PKT_INSTR_FIFO_RSIZE_START + ((iq) * CN23XX_IQ_OFFSET))
+
+#define CN23XX_SLI_IQ_DOORBELL(iq)					\
+	(CN23XX_SLI_PKT_INSTR_BADDR_DBELL_START + ((iq) * CN23XX_IQ_OFFSET))
+
+#define CN23XX_SLI_IQ_INSTR_COUNT64(iq)					\
+	(CN23XX_SLI_PKT_IN_DONE_CNTS_START64 + ((iq) * CN23XX_IQ_OFFSET))
+
+/* Number of instructions to be read in one MAC read request.
+ * setting to Max value(4)
+ */
+#define CN23XX_PKT_INPUT_CTL_RDSIZE			(3 << 25)
+#define CN23XX_PKT_INPUT_CTL_IS_64B			(1 << 24)
+#define CN23XX_PKT_INPUT_CTL_RST			(1 << 23)
+#define CN23XX_PKT_INPUT_CTL_QUIET			(1 << 28)
+#define CN23XX_PKT_INPUT_CTL_RING_ENB			(1 << 22)
+#define CN23XX_PKT_INPUT_CTL_DATA_ES_64B_SWAP		(1 << 6)
+#define CN23XX_PKT_INPUT_CTL_USE_CSR			(1 << 4)
+#define CN23XX_PKT_INPUT_CTL_GATHER_ES_64B_SWAP		(2)
+
+/* These bits[47:44] select the Physical function number within the MAC */
+#define CN23XX_PKT_INPUT_CTL_PF_NUM_POS		45
+/* These bits[43:32] select the function number within the PF */
+#define CN23XX_PKT_INPUT_CTL_VF_NUM_POS		32
+
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+#define CN23XX_PKT_INPUT_CTL_MASK			\
+	(CN23XX_PKT_INPUT_CTL_RDSIZE |			\
+	 CN23XX_PKT_INPUT_CTL_DATA_ES_64B_SWAP |	\
+	 CN23XX_PKT_INPUT_CTL_USE_CSR)
+#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+#define CN23XX_PKT_INPUT_CTL_MASK			\
+	(CN23XX_PKT_INPUT_CTL_RDSIZE |			\
+	 CN23XX_PKT_INPUT_CTL_DATA_ES_64B_SWAP |	\
+	 CN23XX_PKT_INPUT_CTL_USE_CSR |			\
+	 CN23XX_PKT_INPUT_CTL_GATHER_ES_64B_SWAP)
+#endif
+
+/* ############################ OUTPUT QUEUE ######################### */
+
+/* 64 registers for Output queue control - SLI_PKT(0..63)_OUTPUT_CONTROL */
+#define CN23XX_SLI_PKT_OUTPUT_CONTROL_START	0x10050
+
+/* 64 registers for Output queue buffer and info size
+ * SLI_PKT(0..63)_OUT_SIZE
+ */
+#define CN23XX_SLI_PKT_OUT_SIZE			0x10060
+
+/* 64 registers for Output Queue Start Addr - SLI_PKT(0..63)_SLIST_BADDR */
+#define CN23XX_SLI_SLIST_BADDR_START64		0x10070
+
+/* 64 registers for Output Queue Packet Credits
+ * SLI_PKT(0..63)_SLIST_BAOFF_DBELL
+ */
+#define CN23XX_SLI_PKT_SLIST_BAOFF_DBELL_START	0x10080
+
+/* 64 registers for Output Queue size - SLI_PKT(0..63)_SLIST_FIFO_RSIZE */
+#define CN23XX_SLI_PKT_SLIST_FIFO_RSIZE_START	0x10090
+
+/* 64 registers for Output Queue Packet Count - SLI_PKT(0..63)_CNTS */
+#define CN23XX_SLI_PKT_CNTS_START		0x100B0
+
+/* Each Output Queue register is at a 16-byte Offset in BAR0 */
+#define CN23XX_OQ_OFFSET			0x20000
+
+/* ------- Output Queue Macros --------- */
+
+#define CN23XX_SLI_OQ_PKT_CONTROL(oq)					\
+	(CN23XX_SLI_PKT_OUTPUT_CONTROL_START + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_BASE_ADDR64(oq)					\
+	(CN23XX_SLI_SLIST_BADDR_START64 + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_SIZE(oq)						\
+	(CN23XX_SLI_PKT_SLIST_FIFO_RSIZE_START + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_BUFF_INFO_SIZE(oq)				\
+	(CN23XX_SLI_PKT_OUT_SIZE + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_PKTS_SENT(oq)					\
+	(CN23XX_SLI_PKT_CNTS_START + ((oq) * CN23XX_OQ_OFFSET))
+
+#define CN23XX_SLI_OQ_PKTS_CREDIT(oq)					\
+	(CN23XX_SLI_PKT_SLIST_BAOFF_DBELL_START + ((oq) * CN23XX_OQ_OFFSET))
+
+/* ------------------ Masks ---------------- */
+#define CN23XX_PKT_OUTPUT_CTL_IPTR		(1 << 11)
+#define CN23XX_PKT_OUTPUT_CTL_ES		(1 << 9)
+#define CN23XX_PKT_OUTPUT_CTL_NSR		(1 << 8)
+#define CN23XX_PKT_OUTPUT_CTL_ROR		(1 << 7)
+#define CN23XX_PKT_OUTPUT_CTL_DPTR		(1 << 6)
+#define CN23XX_PKT_OUTPUT_CTL_BMODE		(1 << 5)
+#define CN23XX_PKT_OUTPUT_CTL_ES_P		(1 << 3)
+#define CN23XX_PKT_OUTPUT_CTL_NSR_P		(1 << 2)
+#define CN23XX_PKT_OUTPUT_CTL_ROR_P		(1 << 1)
+#define CN23XX_PKT_OUTPUT_CTL_RING_ENB		(1 << 0)
+
+/* Rings per Virtual Function [RO] */
+#define CN23XX_PKT_INPUT_CTL_RPVF_MASK		0x3F
+#define CN23XX_PKT_INPUT_CTL_RPVF_POS		48
+
+/* These bits[47:44][RO] give the Physical function
+ * number info within the MAC
+ */
+#define CN23XX_PKT_INPUT_CTL_PF_NUM_MASK	0x7
+
+/* These bits[43:32][RO] give the virtual function
+ * number info within the PF
+ */
+#define CN23XX_PKT_INPUT_CTL_VF_NUM_MASK	0x1FFF
+
+/* ######################### Mailbox Reg Macros ######################## */
+#define CN23XX_SLI_PKT_PF_VF_MBOX_SIG_START	0x10200
+#define CN23XX_VF_SLI_PKT_MBOX_INT_START	0x10210
+
+#define CN23XX_SLI_MBOX_OFFSET			0x20000
+#define CN23XX_SLI_MBOX_SIG_IDX_OFFSET		0x8
+
+#define CN23XX_SLI_PKT_PF_VF_MBOX_SIG(q, idx)				\
+	(CN23XX_SLI_PKT_PF_VF_MBOX_SIG_START +				\
+	 ((q) * CN23XX_SLI_MBOX_OFFSET +				\
+	  (idx) * CN23XX_SLI_MBOX_SIG_IDX_OFFSET))
+
+#define CN23XX_VF_SLI_PKT_MBOX_INT(q)					\
+	(CN23XX_VF_SLI_PKT_MBOX_INT_START + ((q) * CN23XX_SLI_MBOX_OFFSET))
+
+#endif /* _LIO_23XX_REG_H_ */
-- 
1.8.3.1

[PATCH v3 03/46] net/liquidio: definitions for log

[Shijith Thotton]

Add debug options to config file. Define macros used for log and make
use of config file options to enable them.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 config/common_base              |  6 +++
 drivers/net/liquidio/lio_logs.h | 91 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 97 insertions(+)
 create mode 100644 drivers/net/liquidio/lio_logs.h

diff --git a/config/common_base b/config/common_base
index a5463a3..44353e4 100644
--- a/config/common_base
+++ b/config/common_base
@@ -295,6 +295,12 @@ CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_MBOX=n
 # Compile burst-oriented Cavium LiquidIO PMD driver
 #
 CONFIG_RTE_LIBRTE_LIO_PMD=y
+CONFIG_RTE_LIBRTE_LIO_DEBUG_DRIVER=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_INIT=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_RX=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_TX=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_MBOX=n
+CONFIG_RTE_LIBRTE_LIO_DEBUG_REGS=n
 
 #
 # Compile Support Libraries for NXP DPAA2
diff --git a/drivers/net/liquidio/lio_logs.h b/drivers/net/liquidio/lio_logs.h
new file mode 100644
index 0000000..a4c9ca4
--- /dev/null
+++ b/drivers/net/liquidio/lio_logs.h
@@ -0,0 +1,91 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_LOGS_H_
+#define _LIO_LOGS_H_
+
+#define lio_dev_printf(lio_dev, level, fmt, args...)			\
+	RTE_LOG(level, PMD, "%s" fmt, (lio_dev)->dev_string, ##args)
+
+#define lio_dev_info(lio_dev, fmt, args...)				\
+	lio_dev_printf(lio_dev, INFO, "INFO: " fmt, ##args)
+
+#define lio_dev_err(lio_dev, fmt, args...)				\
+	lio_dev_printf(lio_dev, ERR, "ERROR: %s() " fmt, __func__, ##args)
+
+#define PMD_INIT_LOG(level, fmt, args...) RTE_LOG(level, PMD, fmt, ## args)
+
+/* Enable these through config options */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_INIT
+#define PMD_INIT_FUNC_TRACE() PMD_INIT_LOG(DEBUG, "%s() >>\n", __func__)
+#else /* !RTE_LIBRTE_LIO_DEBUG_INIT */
+#define PMD_INIT_FUNC_TRACE() do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_INIT */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_DRIVER
+#define lio_dev_dbg(lio_dev, fmt, args...)				\
+	lio_dev_printf(lio_dev, DEBUG, "DEBUG: %s() " fmt, __func__, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_DRIVER */
+#define lio_dev_dbg(lio_dev, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_DRIVER */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_RX
+#define PMD_RX_LOG(lio_dev, level, fmt, args...)			\
+	lio_dev_printf(lio_dev, level, "RX: %s() " fmt, __func__, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_RX */
+#define PMD_RX_LOG(lio_dev, level, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_RX */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_TX
+#define PMD_TX_LOG(lio_dev, level, fmt, args...)			\
+	lio_dev_printf(lio_dev, level, "TX: %s() " fmt, __func__, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_TX */
+#define PMD_TX_LOG(lio_dev, level, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_TX */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_MBOX
+#define PMD_MBOX_LOG(lio_dev, level, fmt, args...)			\
+	lio_dev_printf(lio_dev, level, "MBOX: %s() " fmt, __func__, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_MBOX */
+#define PMD_MBOX_LOG(level, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_MBOX */
+
+#ifdef RTE_LIBRTE_LIO_DEBUG_REGS
+#define PMD_REGS_LOG(lio_dev, fmt, args...)				\
+	lio_dev_printf(lio_dev, DEBUG, "REGS: " fmt, ##args)
+#else /* !RTE_LIBRTE_LIO_DEBUG_REGS */
+#define PMD_REGS_LOG(level, fmt, args...) do { } while (0)
+#endif /* RTE_LIBRTE_LIO_DEBUG_REGS */
+
+#endif  /* _LIO_LOGS_H_ */
-- 
1.8.3.1

[PATCH v3 04/46] net/liquidio: liquidio VF PMD driver registration

[Shijith Thotton]

Register LiquidIO PMD (net_liovf) and define APIs to init and uninit.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 doc/guides/nics/features/liquidio.ini   |   3 +
 drivers/net/liquidio/Makefile           |   2 +-
 drivers/net/liquidio/base/lio_hw_defs.h |  44 ++++++++++++
 drivers/net/liquidio/lio_ethdev.c       | 117 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  40 +++++++++++
 drivers/net/liquidio/lio_struct.h       |  65 ++++++++++++++++++
 6 files changed, 270 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/liquidio/base/lio_hw_defs.h
 create mode 100644 drivers/net/liquidio/lio_ethdev.c
 create mode 100644 drivers/net/liquidio/lio_ethdev.h
 create mode 100644 drivers/net/liquidio/lio_struct.h

diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
index 3422661..6c5d8d1 100644
--- a/doc/guides/nics/features/liquidio.ini
+++ b/doc/guides/nics/features/liquidio.ini
@@ -4,4 +4,7 @@
 ; Refer to default.ini for the full list of available PMD features.
 ;
 [Features]
+Multiprocess aware   = Y
+Linux UIO            = Y
+Linux VFIO           = Y
 x86-64               = Y
diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
index ce2b7fc..25685a7 100644
--- a/drivers/net/liquidio/Makefile
+++ b/drivers/net/liquidio/Makefile
@@ -50,7 +50,7 @@ VPATH += $(RTE_SDK)/drivers/net/liquidio/base
 #
 # all source are stored in SRCS-y
 #
-SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) +=
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_ethdev.c
 
 # this lib depends upon:
 DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_eal lib/librte_ether
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
new file mode 100644
index 0000000..db42f3e
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -0,0 +1,44 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_HW_DEFS_H_
+#define _LIO_HW_DEFS_H_
+
+#ifndef PCI_VENDOR_ID_CAVIUM
+#define PCI_VENDOR_ID_CAVIUM	0x177D
+#endif
+
+#define LIO_CN23XX_VF_VID	0x9712
+
+#define LIO_DEVICE_NAME_LEN		32
+#endif /* _LIO_HW_DEFS_H_ */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
new file mode 100644
index 0000000..49efede
--- /dev/null
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -0,0 +1,117 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+#include <rte_alarm.h>
+
+#include "lio_logs.h"
+#include "lio_struct.h"
+#include "lio_ethdev.h"
+
+static int
+lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
+{
+	PMD_INIT_FUNC_TRACE();
+
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return -EPERM;
+
+	rte_free(eth_dev->data->mac_addrs);
+	eth_dev->data->mac_addrs = NULL;
+
+	return 0;
+}
+
+static int
+lio_eth_dev_init(struct rte_eth_dev *eth_dev)
+{
+	struct rte_pci_device *pdev = RTE_DEV_TO_PCI(eth_dev->device);
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* Primary does the initialization. */
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	rte_eth_copy_pci_info(eth_dev, pdev);
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_DETACHABLE;
+
+	if (pdev->mem_resource[0].addr) {
+		lio_dev->hw_addr = pdev->mem_resource[0].addr;
+	} else {
+		PMD_INIT_LOG(ERR, "ERROR: Failed to map BAR0\n");
+		return -ENODEV;
+	}
+
+	lio_dev->eth_dev = eth_dev;
+	/* set lio device print string */
+	snprintf(lio_dev->dev_string, sizeof(lio_dev->dev_string),
+		 "%s[%02x:%02x.%x]", pdev->driver->driver.name,
+		 pdev->addr.bus, pdev->addr.devid, pdev->addr.function);
+
+	lio_dev->port_id = eth_dev->data->port_id;
+
+	eth_dev->data->mac_addrs = rte_zmalloc("lio", ETHER_ADDR_LEN, 0);
+	if (eth_dev->data->mac_addrs == NULL) {
+		lio_dev_err(lio_dev,
+			    "MAC addresses memory allocation failed\n");
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+/* Set of PCI devices this driver supports */
+static const struct rte_pci_id pci_id_liovf_map[] = {
+	{ RTE_PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, LIO_CN23XX_VF_VID) },
+	{ .vendor_id = 0, /* sentinel */ }
+};
+
+static struct eth_driver rte_liovf_pmd = {
+	.pci_drv = {
+		.id_table	= pci_id_liovf_map,
+		.drv_flags      = RTE_PCI_DRV_NEED_MAPPING,
+		.probe		= rte_eth_dev_pci_probe,
+		.remove		= rte_eth_dev_pci_remove,
+	},
+	.eth_dev_init		= lio_eth_dev_init,
+	.eth_dev_uninit		= lio_eth_dev_uninit,
+	.dev_private_size	= sizeof(struct lio_device),
+};
+
+RTE_PMD_REGISTER_PCI(net_liovf, rte_liovf_pmd.pci_drv);
+RTE_PMD_REGISTER_PCI_TABLE(net_liovf, pci_id_liovf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_liovf, "* igb_uio | vfio");
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
new file mode 100644
index 0000000..76c9072
--- /dev/null
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -0,0 +1,40 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_ETHDEV_H_
+#define _LIO_ETHDEV_H_
+
+#include <stdint.h>
+
+#define LIO_DEV(_eth_dev)		((_eth_dev)->data->dev_private)
+#endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
new file mode 100644
index 0000000..dcf99ce
--- /dev/null
+++ b/drivers/net/liquidio/lio_struct.h
@@ -0,0 +1,65 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_STRUCT_H_
+#define _LIO_STRUCT_H_
+
+#include <stdio.h>
+#include <stdint.h>
+#include <sys/queue.h>
+
+#include <rte_spinlock.h>
+#include <rte_atomic.h>
+
+#include "lio_hw_defs.h"
+
+/* -----------------------  THE LIO DEVICE  --------------------------- */
+/** The lio device.
+ *  Each lio device has this structure to represent all its
+ *  components.
+ */
+struct lio_device {
+	uint8_t *hw_addr;
+
+	char dev_string[LIO_DEVICE_NAME_LEN]; /* Device print string */
+
+	struct rte_eth_dev      *eth_dev;
+
+	uint8_t max_rx_queues;
+	uint8_t max_tx_queues;
+	uint8_t nb_rx_queues;
+	uint8_t nb_tx_queues;
+	uint8_t port_configured;
+	uint8_t port_id;
+};
+#endif /* _LIO_STRUCT_H_ */
-- 
1.8.3.1

[PATCH v3 05/46] net/liquidio/base: macros to read and write register

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h | 67 +++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index db42f3e..9a7a894 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -34,6 +34,8 @@
 #ifndef _LIO_HW_DEFS_H_
 #define _LIO_HW_DEFS_H_
 
+#include <rte_io.h>
+
 #ifndef PCI_VENDOR_ID_CAVIUM
 #define PCI_VENDOR_ID_CAVIUM	0x177D
 #endif
@@ -41,4 +43,69 @@
 #define LIO_CN23XX_VF_VID	0x9712
 
 #define LIO_DEVICE_NAME_LEN		32
+
+/* Routines for reading and writing CSRs */
+#ifdef RTE_LIBRTE_LIO_DEBUG_REGS
+#define lio_write_csr(lio_dev, reg_off, value)				\
+	do {								\
+		typeof(lio_dev) _dev = lio_dev;				\
+		typeof(reg_off) _reg_off = reg_off;			\
+		typeof(value) _value = value;				\
+		PMD_REGS_LOG(_dev,					\
+			     "Write32: Reg: 0x%08lx Val: 0x%08lx\n",	\
+			     (unsigned long)_reg_off,			\
+			     (unsigned long)_value);			\
+		rte_write32(_value, _dev->hw_addr + _reg_off);		\
+	} while (0)
+
+#define lio_write_csr64(lio_dev, reg_off, val64)			\
+	do {								\
+		typeof(lio_dev) _dev = lio_dev;				\
+		typeof(reg_off) _reg_off = reg_off;			\
+		typeof(val64) _val64 = val64;				\
+		PMD_REGS_LOG(						\
+		    _dev,						\
+		    "Write64: Reg: 0x%08lx Val: 0x%016llx\n",		\
+		    (unsigned long)_reg_off,				\
+		    (unsigned long long)_val64);			\
+		rte_write64(_val64, _dev->hw_addr + _reg_off);		\
+	} while (0)
+
+#define lio_read_csr(lio_dev, reg_off)					\
+	({								\
+		typeof(lio_dev) _dev = lio_dev;				\
+		typeof(reg_off) _reg_off = reg_off;			\
+		uint32_t val = rte_read32(_dev->hw_addr + _reg_off);	\
+		PMD_REGS_LOG(_dev,					\
+			     "Read32: Reg: 0x%08lx Val: 0x%08lx\n",	\
+			     (unsigned long)_reg_off,			\
+			     (unsigned long)val);			\
+		val;							\
+	})
+
+#define lio_read_csr64(lio_dev, reg_off)				\
+	({								\
+		typeof(lio_dev) _dev = lio_dev;				\
+		typeof(reg_off) _reg_off = reg_off;			\
+		uint64_t val64 = rte_read64(_dev->hw_addr + _reg_off);	\
+		PMD_REGS_LOG(						\
+		    _dev,						\
+		    "Read64: Reg: 0x%08lx Val: 0x%016llx\n",		\
+		    (unsigned long)_reg_off,				\
+		    (unsigned long long)val64);				\
+		val64;							\
+	})
+#else
+#define lio_write_csr(lio_dev, reg_off, value)				\
+	rte_write32(value, (lio_dev)->hw_addr + (reg_off))
+
+#define lio_write_csr64(lio_dev, reg_off, val64)			\
+	rte_write64(val64, (lio_dev)->hw_addr + (reg_off))
+
+#define lio_read_csr(lio_dev, reg_off)					\
+	rte_read32((lio_dev)->hw_addr + (reg_off))
+
+#define lio_read_csr64(lio_dev, reg_off)				\
+	rte_read64((lio_dev)->hw_addr + (reg_off))
+#endif
 #endif /* _LIO_HW_DEFS_H_ */
-- 
1.8.3.1

[PATCH v3 06/46] net/liquidio: liquidio device init

[Shijith Thotton]

Default device configuration and initialization code.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/Makefile           |  1 +
 drivers/net/liquidio/base/lio_23xx_vf.c | 67 ++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h | 84 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_hw_defs.h | 34 +++++++++++++
 drivers/net/liquidio/lio_ethdev.c       | 61 +++++++++++++++++++++++-
 drivers/net/liquidio/lio_struct.h       | 70 +++++++++++++++++++++++++++
 6 files changed, 316 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.c
 create mode 100644 drivers/net/liquidio/base/lio_23xx_vf.h

diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
index 25685a7..8880a10 100644
--- a/drivers/net/liquidio/Makefile
+++ b/drivers/net/liquidio/Makefile
@@ -51,6 +51,7 @@ VPATH += $(RTE_SDK)/drivers/net/liquidio/base
 # all source are stored in SRCS-y
 #
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_ethdev.c
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_23xx_vf.c
 
 # this lib depends upon:
 DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_eal lib/librte_ether
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
new file mode 100644
index 0000000..dd5e3a6
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -0,0 +1,67 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+
+#include "lio_logs.h"
+#include "lio_23xx_vf.h"
+#include "lio_23xx_reg.h"
+
+int
+cn23xx_vf_setup_device(struct lio_device *lio_dev)
+{
+	uint64_t reg_val;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* INPUT_CONTROL[RPVF] gives the VF IOq count */
+	reg_val = lio_read_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(0));
+
+	lio_dev->pf_num = (reg_val >> CN23XX_PKT_INPUT_CTL_PF_NUM_POS) &
+				CN23XX_PKT_INPUT_CTL_PF_NUM_MASK;
+	lio_dev->vf_num = (reg_val >> CN23XX_PKT_INPUT_CTL_VF_NUM_POS) &
+				CN23XX_PKT_INPUT_CTL_VF_NUM_MASK;
+
+	reg_val = reg_val >> CN23XX_PKT_INPUT_CTL_RPVF_POS;
+
+	lio_dev->sriov_info.rings_per_vf =
+				reg_val & CN23XX_PKT_INPUT_CTL_RPVF_MASK;
+
+	lio_dev->default_config = lio_get_conf(lio_dev);
+	if (lio_dev->default_config == NULL)
+		return -1;
+
+	return 0;
+}
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.h b/drivers/net/liquidio/base/lio_23xx_vf.h
new file mode 100644
index 0000000..1c234bf
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_23xx_vf.h
@@ -0,0 +1,84 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_23XX_VF_H_
+#define _LIO_23XX_VF_H_
+
+#include <stdio.h>
+
+#include "lio_struct.h"
+
+static const struct lio_config default_cn23xx_conf	= {
+	.card_type				= LIO_23XX,
+	.card_name				= LIO_23XX_NAME,
+	/** IQ attributes */
+	.iq					= {
+		.max_iqs			= CN23XX_CFG_IO_QUEUES,
+		.pending_list_size		=
+			(CN23XX_MAX_IQ_DESCRIPTORS * CN23XX_CFG_IO_QUEUES),
+		.instr_type			= OCTEON_64BYTE_INSTR,
+	},
+
+	/** OQ attributes */
+	.oq					= {
+		.max_oqs			= CN23XX_CFG_IO_QUEUES,
+		.info_ptr			= OCTEON_OQ_INFOPTR_MODE,
+		.refill_threshold		= CN23XX_OQ_REFIL_THRESHOLD,
+	},
+
+	.num_nic_ports				= CN23XX_DEFAULT_NUM_PORTS,
+	.num_def_rx_descs			= CN23XX_MAX_OQ_DESCRIPTORS,
+	.num_def_tx_descs			= CN23XX_MAX_IQ_DESCRIPTORS,
+	.def_rx_buf_size			= CN23XX_OQ_BUF_SIZE,
+};
+
+static inline const struct lio_config *
+lio_get_conf(struct lio_device *lio_dev)
+{
+	const struct lio_config *default_lio_conf = NULL;
+
+	/* check the LIO Device model & return the corresponding lio
+	 * configuration
+	 */
+	default_lio_conf = &default_cn23xx_conf;
+
+	if (default_lio_conf == NULL) {
+		lio_dev_err(lio_dev, "Configuration verification failed\n");
+		return NULL;
+	}
+
+	return default_lio_conf;
+}
+
+int cn23xx_vf_setup_device(struct lio_device  *lio_dev);
+#endif /* _LIO_23XX_VF_H_  */
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 9a7a894..a2654cd 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -42,6 +42,40 @@
 
 #define LIO_CN23XX_VF_VID	0x9712
 
+/* --------------------------CONFIG VALUES------------------------ */
+
+/* CN23xx IQ configuration macros */
+#define CN23XX_MAX_RINGS_PER_PF			64
+#define CN23XX_MAX_RINGS_PER_VF			8
+
+#define CN23XX_MAX_INPUT_QUEUES			CN23XX_MAX_RINGS_PER_PF
+#define CN23XX_MAX_IQ_DESCRIPTORS		512
+#define CN23XX_MIN_IQ_DESCRIPTORS		128
+
+#define CN23XX_MAX_OUTPUT_QUEUES		CN23XX_MAX_RINGS_PER_PF
+#define CN23XX_MAX_OQ_DESCRIPTORS		512
+#define CN23XX_MIN_OQ_DESCRIPTORS		128
+#define CN23XX_OQ_BUF_SIZE			1536
+
+#define CN23XX_OQ_REFIL_THRESHOLD		16
+
+#define CN23XX_DEFAULT_NUM_PORTS		1
+
+#define CN23XX_CFG_IO_QUEUES			CN23XX_MAX_RINGS_PER_PF
+
+/* common OCTEON configuration macros */
+#define OCTEON_64BYTE_INSTR			64
+#define OCTEON_OQ_INFOPTR_MODE			1
+
+/* Max IOQs per LIO Link */
+#define LIO_MAX_IOQS_PER_IF			64
+
+enum lio_card_type {
+	LIO_23XX /* 23xx */
+};
+
+#define LIO_23XX_NAME "23xx"
+
 #define LIO_DEVICE_NAME_LEN		32
 
 /* Routines for reading and writing CSRs */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 49efede..734f6c4 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -37,9 +37,63 @@
 #include <rte_alarm.h>
 
 #include "lio_logs.h"
-#include "lio_struct.h"
+#include "lio_23xx_vf.h"
 #include "lio_ethdev.h"
 
+/**
+ * \brief Identify the LIO device and to map the BAR address space
+ * @param lio_dev lio device
+ */
+static int
+lio_chip_specific_setup(struct lio_device *lio_dev)
+{
+	struct rte_pci_device *pdev = lio_dev->pci_dev;
+	uint32_t dev_id = pdev->id.device_id;
+	const char *s;
+	int ret = 1;
+
+	switch (dev_id) {
+	case LIO_CN23XX_VF_VID:
+		lio_dev->chip_id = LIO_CN23XX_VF_VID;
+		ret = cn23xx_vf_setup_device(lio_dev);
+		s = "CN23XX VF";
+		break;
+	default:
+		s = "?";
+		lio_dev_err(lio_dev, "Unsupported Chip\n");
+	}
+
+	if (!ret)
+		lio_dev_info(lio_dev, "DEVICE : %s\n", s);
+
+	return ret;
+}
+
+static int
+lio_first_time_init(struct lio_device *lio_dev,
+		    struct rte_pci_device *pdev)
+{
+	int dpdk_queues;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* set dpdk specific pci device pointer */
+	lio_dev->pci_dev = pdev;
+
+	/* Identify the LIO type and set device ops */
+	if (lio_chip_specific_setup(lio_dev)) {
+		lio_dev_err(lio_dev, "Chip specific setup failed\n");
+		return -1;
+	}
+
+	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
+
+	lio_dev->max_tx_queues = dpdk_queues;
+	lio_dev->max_rx_queues = dpdk_queues;
+
+	return 0;
+}
+
 static int
 lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
 {
@@ -84,6 +138,11 @@
 
 	lio_dev->port_id = eth_dev->data->port_id;
 
+	if (lio_first_time_init(lio_dev, pdev)) {
+		lio_dev_err(lio_dev, "Device init failed\n");
+		return -EINVAL;
+	}
+
 	eth_dev->data->mac_addrs = rte_zmalloc("lio", ETHER_ADDR_LEN, 0);
 	if (eth_dev->data->mac_addrs == NULL) {
 		lio_dev_err(lio_dev,
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index dcf99ce..a1203e4 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -43,16 +43,86 @@
 
 #include "lio_hw_defs.h"
 
+struct lio_sriov_info {
+	/** Number of rings assigned to VF */
+	uint32_t rings_per_vf;
+
+	/** Number of VF devices enabled */
+	uint32_t num_vfs;
+};
+
+/* Structure to define the configuration attributes for each Input queue. */
+struct lio_iq_config {
+	/* Max number of IQs available */
+	uint8_t max_iqs;
+
+	/** Pending list size (usually set to the sum of the size of all Input
+	 *  queues)
+	 */
+	uint32_t pending_list_size;
+
+	/** Command size - 32 or 64 bytes */
+	uint32_t instr_type;
+};
+
+/* Structure to define the configuration attributes for each Output queue. */
+struct lio_oq_config {
+	/* Max number of OQs available */
+	uint8_t max_oqs;
+
+	/** If set, the Output queue uses info-pointer mode. (Default: 1 ) */
+	uint32_t info_ptr;
+
+	/** The number of buffers that were consumed during packet processing by
+	 *  the driver on this Output queue before the driver attempts to
+	 *  replenish the descriptor ring with new buffers.
+	 */
+	uint32_t refill_threshold;
+};
+
+/* Structure to define the configuration. */
+struct lio_config {
+	uint16_t card_type;
+	const char *card_name;
+
+	/** Input Queue attributes. */
+	struct lio_iq_config iq;
+
+	/** Output Queue attributes. */
+	struct lio_oq_config oq;
+
+	int num_nic_ports;
+
+	int num_def_tx_descs;
+
+	/* Num of desc for rx rings */
+	int num_def_rx_descs;
+
+	int def_rx_buf_size;
+};
+
 /* -----------------------  THE LIO DEVICE  --------------------------- */
 /** The lio device.
  *  Each lio device has this structure to represent all its
  *  components.
  */
 struct lio_device {
+	/** PCI device pointer */
+	struct rte_pci_device *pci_dev;
+
+	/** Octeon Chip type */
+	uint16_t chip_id;
+	uint16_t pf_num;
+	uint16_t vf_num;
+
 	uint8_t *hw_addr;
 
+	struct lio_sriov_info sriov_info;
+
 	char dev_string[LIO_DEVICE_NAME_LEN]; /* Device print string */
 
+	const struct lio_config *default_config;
+
 	struct rte_eth_dev      *eth_dev;
 
 	uint8_t max_rx_queues;
-- 
1.8.3.1

[PATCH v3 07/46] net/liquidio: add API to disable IO queues

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 49 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h |  7 +++++
 drivers/net/liquidio/lio_ethdev.c       |  5 ++++
 3 files changed, 61 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index dd5e3a6..d9b9e2a 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -65,3 +65,52 @@
 
 	return 0;
 }
+
+int
+cn23xx_vf_set_io_queues_off(struct lio_device *lio_dev)
+{
+	uint32_t loop = CN23XX_VF_BUSY_READING_REG_LOOP_COUNT;
+	uint64_t q_no;
+
+	/* Disable the i/p and o/p queues for this Octeon.
+	 * IOQs will already be in reset.
+	 * If RST bit is set, wait for Quiet bit to be set
+	 * Once Quiet bit is set, clear the RST bit
+	 */
+	PMD_INIT_FUNC_TRACE();
+
+	for (q_no = 0; q_no < lio_dev->sriov_info.rings_per_vf; q_no++) {
+		volatile uint64_t reg_val;
+
+		reg_val = lio_read_csr64(lio_dev,
+					 CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		while ((reg_val & CN23XX_PKT_INPUT_CTL_RST) && !(reg_val &
+					 CN23XX_PKT_INPUT_CTL_QUIET) && loop) {
+			reg_val = lio_read_csr64(
+					lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+			loop = loop - 1;
+		}
+
+		if (loop == 0) {
+			lio_dev_err(lio_dev,
+				    "clearing the reset reg failed or setting the quiet reg failed for qno %lu\n",
+				    (unsigned long)q_no);
+			return -1;
+		}
+
+		reg_val = reg_val & ~CN23XX_PKT_INPUT_CTL_RST;
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+				reg_val);
+
+		reg_val = lio_read_csr64(lio_dev,
+					 CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		if (reg_val & CN23XX_PKT_INPUT_CTL_RST) {
+			lio_dev_err(lio_dev, "unable to reset qno %lu\n",
+				    (unsigned long)q_no);
+			return -1;
+		}
+	}
+
+	return 0;
+}
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.h b/drivers/net/liquidio/base/lio_23xx_vf.h
index 1c234bf..1af09d0 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.h
+++ b/drivers/net/liquidio/base/lio_23xx_vf.h
@@ -80,5 +80,12 @@
 	return default_lio_conf;
 }
 
+/** Turns off the input and output queues for the device
+ *  @param lio_dev which device io queues to disable
+ */
+int cn23xx_vf_set_io_queues_off(struct lio_device *lio_dev);
+
+#define CN23XX_VF_BUSY_READING_REG_LOOP_COUNT	100000
+
 int cn23xx_vf_setup_device(struct lio_device  *lio_dev);
 #endif /* _LIO_23XX_VF_H_  */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 734f6c4..0487f3d 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -86,6 +86,11 @@
 		return -1;
 	}
 
+	if (cn23xx_vf_set_io_queues_off(lio_dev)) {
+		lio_dev_err(lio_dev, "Setting io queues off failed\n");
+		return -1;
+	}
+
 	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
 
 	lio_dev->max_tx_queues = dpdk_queues;
-- 
1.8.3.1

[PATCH v3 08/46] net/liquidio: add API to setup IO queue registers

[Shijith Thotton]

Set default configuration values for input and output queue registers.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 160 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.c       |   5 +
 drivers/net/liquidio/lio_struct.h       |   7 ++
 3 files changed, 172 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index d9b9e2a..f61f185 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -39,6 +39,164 @@
 #include "lio_23xx_vf.h"
 #include "lio_23xx_reg.h"
 
+static int
+cn23xx_vf_reset_io_queues(struct lio_device *lio_dev, uint32_t num_queues)
+{
+	uint32_t loop = CN23XX_VF_BUSY_READING_REG_LOOP_COUNT;
+	uint64_t d64, q_no;
+	int ret_val = 0;
+
+	PMD_INIT_FUNC_TRACE();
+
+	for (q_no = 0; q_no < num_queues; q_no++) {
+		/* set RST bit to 1. This bit applies to both IQ and OQ */
+		d64 = lio_read_csr64(lio_dev,
+				     CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		d64 = d64 | CN23XX_PKT_INPUT_CTL_RST;
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+				d64);
+	}
+
+	/* wait until the RST bit is clear or the RST and QUIET bits are set */
+	for (q_no = 0; q_no < num_queues; q_no++) {
+		volatile uint64_t reg_val;
+
+		reg_val	= lio_read_csr64(lio_dev,
+					 CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		while ((reg_val & CN23XX_PKT_INPUT_CTL_RST) &&
+				!(reg_val & CN23XX_PKT_INPUT_CTL_QUIET) &&
+				loop) {
+			reg_val = lio_read_csr64(
+					lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+			loop = loop - 1;
+		}
+
+		if (loop == 0) {
+			lio_dev_err(lio_dev,
+				    "clearing the reset reg failed or setting the quiet reg failed for qno: %lu\n",
+				    (unsigned long)q_no);
+			return -1;
+		}
+
+		reg_val = reg_val & ~CN23XX_PKT_INPUT_CTL_RST;
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+				reg_val);
+
+		reg_val = lio_read_csr64(
+		    lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+		if (reg_val & CN23XX_PKT_INPUT_CTL_RST) {
+			lio_dev_err(lio_dev,
+				    "clearing the reset failed for qno: %lu\n",
+				    (unsigned long)q_no);
+			ret_val = -1;
+		}
+	}
+
+	return ret_val;
+}
+
+static int
+cn23xx_vf_setup_global_input_regs(struct lio_device *lio_dev)
+{
+	uint64_t q_no;
+	uint64_t d64;
+
+	PMD_INIT_FUNC_TRACE();
+
+	if (cn23xx_vf_reset_io_queues(lio_dev,
+				      lio_dev->sriov_info.rings_per_vf))
+		return -1;
+
+	for (q_no = 0; q_no < (lio_dev->sriov_info.rings_per_vf); q_no++) {
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_DOORBELL(q_no),
+				0xFFFFFFFF);
+
+		d64 = lio_read_csr64(lio_dev,
+				     CN23XX_SLI_IQ_INSTR_COUNT64(q_no));
+
+		d64 &= 0xEFFFFFFFFFFFFFFFL;
+
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_INSTR_COUNT64(q_no),
+				d64);
+
+		/* Select ES, RO, NS, RDSIZE,DPTR Fomat#0 for
+		 * the Input Queues
+		 */
+		lio_write_csr64(lio_dev, CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+				CN23XX_PKT_INPUT_CTL_MASK);
+	}
+
+	return 0;
+}
+
+static void
+cn23xx_vf_setup_global_output_regs(struct lio_device *lio_dev)
+{
+	uint32_t reg_val;
+	uint32_t q_no;
+
+	PMD_INIT_FUNC_TRACE();
+
+	for (q_no = 0; q_no < lio_dev->sriov_info.rings_per_vf; q_no++) {
+		lio_write_csr(lio_dev, CN23XX_SLI_OQ_PKTS_CREDIT(q_no),
+			      0xFFFFFFFF);
+
+		reg_val =
+		    lio_read_csr(lio_dev, CN23XX_SLI_OQ_PKTS_SENT(q_no));
+
+		reg_val &= 0xEFFFFFFFFFFFFFFFL;
+
+		reg_val =
+		    lio_read_csr(lio_dev, CN23XX_SLI_OQ_PKT_CONTROL(q_no));
+
+		/* set IPTR & DPTR */
+		reg_val |=
+		    (CN23XX_PKT_OUTPUT_CTL_IPTR | CN23XX_PKT_OUTPUT_CTL_DPTR);
+
+		/* reset BMODE */
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_BMODE);
+
+		/* No Relaxed Ordering, No Snoop, 64-bit Byte swap
+		 * for Output Queue Scatter List
+		 * reset ROR_P, NSR_P
+		 */
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_ROR_P);
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_NSR_P);
+
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_ES_P);
+#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		reg_val |= (CN23XX_PKT_OUTPUT_CTL_ES_P);
+#endif
+		/* No Relaxed Ordering, No Snoop, 64-bit Byte swap
+		 * for Output Queue Data
+		 * reset ROR, NSR
+		 */
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_ROR);
+		reg_val &= ~(CN23XX_PKT_OUTPUT_CTL_NSR);
+		/* set the ES bit */
+		reg_val |= (CN23XX_PKT_OUTPUT_CTL_ES);
+
+		/* write all the selected settings */
+		lio_write_csr(lio_dev, CN23XX_SLI_OQ_PKT_CONTROL(q_no),
+			      reg_val);
+	}
+}
+
+static int
+cn23xx_vf_setup_device_regs(struct lio_device *lio_dev)
+{
+	PMD_INIT_FUNC_TRACE();
+
+	if (cn23xx_vf_setup_global_input_regs(lio_dev))
+		return -1;
+
+	cn23xx_vf_setup_global_output_regs(lio_dev);
+
+	return 0;
+}
+
 int
 cn23xx_vf_setup_device(struct lio_device *lio_dev)
 {
@@ -63,6 +221,8 @@
 	if (lio_dev->default_config == NULL)
 		return -1;
 
+	lio_dev->fn_list.setup_device_regs	= cn23xx_vf_setup_device_regs;
+
 	return 0;
 }
 
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 0487f3d..34b7b54 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -91,6 +91,11 @@
 		return -1;
 	}
 
+	if (lio_dev->fn_list.setup_device_regs(lio_dev)) {
+		lio_dev_err(lio_dev, "Failed to configure device registers\n");
+		return -1;
+	}
+
 	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
 
 	lio_dev->max_tx_queues = dpdk_queues;
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index a1203e4..577ea49 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -43,6 +43,11 @@
 
 #include "lio_hw_defs.h"
 
+struct lio_device;
+struct lio_fn_list {
+	int (*setup_device_regs)(struct lio_device *);
+};
+
 struct lio_sriov_info {
 	/** Number of rings assigned to VF */
 	uint32_t rings_per_vf;
@@ -117,6 +122,8 @@ struct lio_device {
 
 	uint8_t *hw_addr;
 
+	struct lio_fn_list fn_list;
+
 	struct lio_sriov_info sriov_info;
 
 	char dev_string[LIO_DEVICE_NAME_LEN]; /* Device print string */
-- 
1.8.3.1

[PATCH v3 09/46] net/liquidio: add mbox APIs for PF VF communication

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/Makefile        |   1 +
 drivers/net/liquidio/base/lio_mbox.c | 275 +++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_mbox.h | 129 ++++++++++++++++
 drivers/net/liquidio/lio_struct.h    |   3 +
 4 files changed, 408 insertions(+)
 create mode 100644 drivers/net/liquidio/base/lio_mbox.c
 create mode 100644 drivers/net/liquidio/base/lio_mbox.h

diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
index 8880a10..451f49d 100644
--- a/drivers/net/liquidio/Makefile
+++ b/drivers/net/liquidio/Makefile
@@ -52,6 +52,7 @@ VPATH += $(RTE_SDK)/drivers/net/liquidio/base
 #
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_ethdev.c
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_23xx_vf.c
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_mbox.c
 
 # this lib depends upon:
 DEPDIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lib/librte_eal lib/librte_ether
diff --git a/drivers/net/liquidio/base/lio_mbox.c b/drivers/net/liquidio/base/lio_mbox.c
new file mode 100644
index 0000000..b4abc62
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_mbox.c
@@ -0,0 +1,275 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+
+#include "lio_logs.h"
+#include "lio_struct.h"
+#include "lio_mbox.h"
+
+/**
+ * lio_mbox_read:
+ * @mbox: Pointer mailbox
+ *
+ * Reads the 8-bytes of data from the mbox register
+ * Writes back the acknowledgment indicating completion of read
+ */
+int
+lio_mbox_read(struct lio_mbox *mbox)
+{
+	union lio_mbox_message msg;
+	int ret = 0;
+
+	msg.mbox_msg64 = rte_read64(mbox->mbox_read_reg);
+
+	if ((msg.mbox_msg64 == LIO_PFVFACK) || (msg.mbox_msg64 == LIO_PFVFSIG))
+		return 0;
+
+	if (mbox->state & LIO_MBOX_STATE_REQ_RECEIVING) {
+		mbox->mbox_req.data[mbox->mbox_req.recv_len - 1] =
+					msg.mbox_msg64;
+		mbox->mbox_req.recv_len++;
+	} else {
+		if (mbox->state & LIO_MBOX_STATE_RES_RECEIVING) {
+			mbox->mbox_resp.data[mbox->mbox_resp.recv_len - 1] =
+					msg.mbox_msg64;
+			mbox->mbox_resp.recv_len++;
+		} else {
+			if ((mbox->state & LIO_MBOX_STATE_IDLE) &&
+					(msg.s.type == LIO_MBOX_REQUEST)) {
+				mbox->state &= ~LIO_MBOX_STATE_IDLE;
+				mbox->state |= LIO_MBOX_STATE_REQ_RECEIVING;
+				mbox->mbox_req.msg.mbox_msg64 = msg.mbox_msg64;
+				mbox->mbox_req.q_no = mbox->q_no;
+				mbox->mbox_req.recv_len = 1;
+			} else {
+				if ((mbox->state &
+				     LIO_MBOX_STATE_RES_PENDING) &&
+				    (msg.s.type == LIO_MBOX_RESPONSE)) {
+					mbox->state &=
+						~LIO_MBOX_STATE_RES_PENDING;
+					mbox->state |=
+						LIO_MBOX_STATE_RES_RECEIVING;
+					mbox->mbox_resp.msg.mbox_msg64 =
+								msg.mbox_msg64;
+					mbox->mbox_resp.q_no = mbox->q_no;
+					mbox->mbox_resp.recv_len = 1;
+				} else {
+					rte_write64(LIO_PFVFERR,
+						    mbox->mbox_read_reg);
+					mbox->state |= LIO_MBOX_STATE_ERROR;
+					return -1;
+				}
+			}
+		}
+	}
+
+	if (mbox->state & LIO_MBOX_STATE_REQ_RECEIVING) {
+		if (mbox->mbox_req.recv_len < msg.s.len) {
+			ret = 0;
+		} else {
+			mbox->state &= ~LIO_MBOX_STATE_REQ_RECEIVING;
+			mbox->state |= LIO_MBOX_STATE_REQ_RECEIVED;
+			ret = 1;
+		}
+	} else {
+		if (mbox->state & LIO_MBOX_STATE_RES_RECEIVING) {
+			if (mbox->mbox_resp.recv_len < msg.s.len) {
+				ret = 0;
+			} else {
+				mbox->state &= ~LIO_MBOX_STATE_RES_RECEIVING;
+				mbox->state |= LIO_MBOX_STATE_RES_RECEIVED;
+				ret = 1;
+			}
+		} else {
+			RTE_ASSERT(0);
+		}
+	}
+
+	rte_write64(LIO_PFVFACK, mbox->mbox_read_reg);
+
+	return ret;
+}
+
+/**
+ * lio_mbox_write:
+ * @lio_dev: Pointer lio device
+ * @mbox_cmd: Cmd to send to mailbox.
+ *
+ * Populates the queue specific mbox structure
+ * with cmd information.
+ * Write the cmd to mbox register
+ */
+int
+lio_mbox_write(struct lio_device *lio_dev,
+	       struct lio_mbox_cmd *mbox_cmd)
+{
+	struct lio_mbox *mbox = lio_dev->mbox[mbox_cmd->q_no];
+	uint32_t count, i, ret = LIO_MBOX_STATUS_SUCCESS;
+
+	if ((mbox_cmd->msg.s.type == LIO_MBOX_RESPONSE) &&
+			!(mbox->state & LIO_MBOX_STATE_REQ_RECEIVED))
+		return LIO_MBOX_STATUS_FAILED;
+
+	if ((mbox_cmd->msg.s.type == LIO_MBOX_REQUEST) &&
+			!(mbox->state & LIO_MBOX_STATE_IDLE))
+		return LIO_MBOX_STATUS_BUSY;
+
+	if (mbox_cmd->msg.s.type == LIO_MBOX_REQUEST) {
+		rte_memcpy(&mbox->mbox_resp, mbox_cmd,
+			   sizeof(struct lio_mbox_cmd));
+		mbox->state = LIO_MBOX_STATE_RES_PENDING;
+	}
+
+	count = 0;
+
+	while (rte_read64(mbox->mbox_write_reg) != LIO_PFVFSIG) {
+		rte_delay_ms(1);
+		if (count++ == 1000) {
+			ret = LIO_MBOX_STATUS_FAILED;
+			break;
+		}
+	}
+
+	if (ret == LIO_MBOX_STATUS_SUCCESS) {
+		rte_write64(mbox_cmd->msg.mbox_msg64, mbox->mbox_write_reg);
+		for (i = 0; i < (uint32_t)(mbox_cmd->msg.s.len - 1); i++) {
+			count = 0;
+			while (rte_read64(mbox->mbox_write_reg) !=
+					LIO_PFVFACK) {
+				rte_delay_ms(1);
+				if (count++ == 1000) {
+					ret = LIO_MBOX_STATUS_FAILED;
+					break;
+				}
+			}
+			rte_write64(mbox_cmd->data[i], mbox->mbox_write_reg);
+		}
+	}
+
+	if (mbox_cmd->msg.s.type == LIO_MBOX_RESPONSE) {
+		mbox->state = LIO_MBOX_STATE_IDLE;
+		rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+	} else {
+		if ((!mbox_cmd->msg.s.resp_needed) ||
+				(ret == LIO_MBOX_STATUS_FAILED)) {
+			mbox->state &= ~LIO_MBOX_STATE_RES_PENDING;
+			if (!(mbox->state & (LIO_MBOX_STATE_REQ_RECEIVING |
+					     LIO_MBOX_STATE_REQ_RECEIVED)))
+				mbox->state = LIO_MBOX_STATE_IDLE;
+		}
+	}
+
+	return ret;
+}
+
+/**
+ * lio_mbox_process_cmd:
+ * @mbox: Pointer mailbox
+ * @mbox_cmd: Pointer to command received
+ *
+ * Process the cmd received in mbox
+ */
+static int
+lio_mbox_process_cmd(struct lio_mbox *mbox,
+		     struct lio_mbox_cmd *mbox_cmd)
+{
+	struct lio_device *lio_dev = mbox->lio_dev;
+
+	if (mbox_cmd->msg.s.cmd == LIO_CORES_CRASHED)
+		lio_dev_err(lio_dev, "Octeon core(s) crashed or got stuck!\n");
+
+	return 0;
+}
+
+/**
+ * Process the received mbox message.
+ */
+int
+lio_mbox_process_message(struct lio_mbox *mbox)
+{
+	struct lio_mbox_cmd mbox_cmd;
+
+	if (mbox->state & LIO_MBOX_STATE_ERROR) {
+		if (mbox->state & (LIO_MBOX_STATE_RES_PENDING |
+				   LIO_MBOX_STATE_RES_RECEIVING)) {
+			rte_memcpy(&mbox_cmd, &mbox->mbox_resp,
+				   sizeof(struct lio_mbox_cmd));
+			mbox->state = LIO_MBOX_STATE_IDLE;
+			rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+			mbox_cmd.recv_status = 1;
+			if (mbox_cmd.fn)
+				mbox_cmd.fn(mbox->lio_dev, &mbox_cmd,
+					    mbox_cmd.fn_arg);
+
+			return 0;
+		}
+
+		mbox->state = LIO_MBOX_STATE_IDLE;
+		rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+
+		return 0;
+	}
+
+	if (mbox->state & LIO_MBOX_STATE_RES_RECEIVED) {
+		rte_memcpy(&mbox_cmd, &mbox->mbox_resp,
+			   sizeof(struct lio_mbox_cmd));
+		mbox->state = LIO_MBOX_STATE_IDLE;
+		rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+		mbox_cmd.recv_status = 0;
+		if (mbox_cmd.fn)
+			mbox_cmd.fn(mbox->lio_dev, &mbox_cmd, mbox_cmd.fn_arg);
+
+		return 0;
+	}
+
+	if (mbox->state & LIO_MBOX_STATE_REQ_RECEIVED) {
+		rte_memcpy(&mbox_cmd, &mbox->mbox_req,
+			   sizeof(struct lio_mbox_cmd));
+		if (!mbox_cmd.msg.s.resp_needed) {
+			mbox->state &= ~LIO_MBOX_STATE_REQ_RECEIVED;
+			if (!(mbox->state & LIO_MBOX_STATE_RES_PENDING))
+				mbox->state = LIO_MBOX_STATE_IDLE;
+			rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+		}
+
+		lio_mbox_process_cmd(mbox, &mbox_cmd);
+
+		return 0;
+	}
+
+	RTE_ASSERT(0);
+
+	return 0;
+}
diff --git a/drivers/net/liquidio/base/lio_mbox.h b/drivers/net/liquidio/base/lio_mbox.h
new file mode 100644
index 0000000..28c9e1a
--- /dev/null
+++ b/drivers/net/liquidio/base/lio_mbox.h
@@ -0,0 +1,129 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_MBOX_H_
+#define _LIO_MBOX_H_
+
+#include <stdint.h>
+
+#include <rte_spinlock.h>
+
+/* Macros for Mail Box Communication */
+
+#define LIO_MBOX_DATA_MAX			32
+
+#define LIO_CORES_CRASHED			0x3
+
+/* Macro for Read acknowledgment */
+#define LIO_PFVFACK				0xffffffffffffffff
+#define LIO_PFVFSIG				0x1122334455667788
+#define LIO_PFVFERR				0xDEADDEADDEADDEAD
+
+enum lio_mbox_cmd_status {
+	LIO_MBOX_STATUS_SUCCESS		= 0,
+	LIO_MBOX_STATUS_FAILED		= 1,
+	LIO_MBOX_STATUS_BUSY		= 2
+};
+
+enum lio_mbox_message_type {
+	LIO_MBOX_REQUEST	= 0,
+	LIO_MBOX_RESPONSE	= 1
+};
+
+union lio_mbox_message {
+	uint64_t mbox_msg64;
+	struct {
+		uint16_t type : 1;
+		uint16_t resp_needed : 1;
+		uint16_t cmd : 6;
+		uint16_t len : 8;
+		uint8_t params[6];
+	} s;
+};
+
+typedef void (*lio_mbox_callback)(void *, void *, void *);
+
+struct lio_mbox_cmd {
+	union lio_mbox_message msg;
+	uint64_t data[LIO_MBOX_DATA_MAX];
+	uint32_t q_no;
+	uint32_t recv_len;
+	uint32_t recv_status;
+	lio_mbox_callback fn;
+	void *fn_arg;
+};
+
+enum lio_mbox_state {
+	LIO_MBOX_STATE_IDLE		= 1,
+	LIO_MBOX_STATE_REQ_RECEIVING	= 2,
+	LIO_MBOX_STATE_REQ_RECEIVED	= 4,
+	LIO_MBOX_STATE_RES_PENDING	= 8,
+	LIO_MBOX_STATE_RES_RECEIVING	= 16,
+	LIO_MBOX_STATE_RES_RECEIVED	= 16,
+	LIO_MBOX_STATE_ERROR		= 32
+};
+
+struct lio_mbox {
+	/* A spinlock to protect access to this q_mbox. */
+	rte_spinlock_t lock;
+
+	struct lio_device *lio_dev;
+
+	uint32_t q_no;
+
+	enum lio_mbox_state state;
+
+	/* SLI_MAC_PF_MBOX_INT for PF, SLI_PKT_MBOX_INT for VF. */
+	void *mbox_int_reg;
+
+	/* SLI_PKT_PF_VF_MBOX_SIG(0) for PF,
+	 * SLI_PKT_PF_VF_MBOX_SIG(1) for VF.
+	 */
+	void *mbox_write_reg;
+
+	/* SLI_PKT_PF_VF_MBOX_SIG(1) for PF,
+	 * SLI_PKT_PF_VF_MBOX_SIG(0) for VF.
+	 */
+	void *mbox_read_reg;
+
+	struct lio_mbox_cmd mbox_req;
+
+	struct lio_mbox_cmd mbox_resp;
+
+};
+
+int lio_mbox_read(struct lio_mbox *mbox);
+int lio_mbox_write(struct lio_device *lio_dev,
+		   struct lio_mbox_cmd *mbox_cmd);
+int lio_mbox_process_message(struct lio_mbox *mbox);
+#endif	/* _LIO_MBOX_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 577ea49..0af4fe3 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -126,6 +126,9 @@ struct lio_device {
 
 	struct lio_sriov_info sriov_info;
 
+	/** Mail Box details of each lio queue. */
+	struct lio_mbox **mbox;
+
 	char dev_string[LIO_DEVICE_NAME_LEN]; /* Device print string */
 
 	const struct lio_config *default_config;
-- 
1.8.3.1

[PATCH v3 10/46] net/liquidio: add API to setup mbox registers

[Shijith Thotton]

Map and initialize mbox registers.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 61 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.c       | 15 ++++++--
 drivers/net/liquidio/lio_struct.h       |  3 ++
 3 files changed, 77 insertions(+), 2 deletions(-)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index f61f185..70faa9b 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -38,6 +38,7 @@
 #include "lio_logs.h"
 #include "lio_23xx_vf.h"
 #include "lio_23xx_reg.h"
+#include "lio_mbox.h"
 
 static int
 cn23xx_vf_reset_io_queues(struct lio_device *lio_dev, uint32_t num_queues)
@@ -197,6 +198,63 @@
 	return 0;
 }
 
+static void
+cn23xx_vf_free_mbox(struct lio_device *lio_dev)
+{
+	PMD_INIT_FUNC_TRACE();
+
+	rte_free(lio_dev->mbox[0]);
+	lio_dev->mbox[0] = NULL;
+
+	rte_free(lio_dev->mbox);
+	lio_dev->mbox = NULL;
+}
+
+static int
+cn23xx_vf_setup_mbox(struct lio_device *lio_dev)
+{
+	struct lio_mbox *mbox;
+
+	PMD_INIT_FUNC_TRACE();
+
+	if (lio_dev->mbox == NULL) {
+		lio_dev->mbox = rte_zmalloc(NULL, sizeof(void *), 0);
+		if (lio_dev->mbox == NULL)
+			return -ENOMEM;
+	}
+
+	mbox = rte_zmalloc(NULL, sizeof(struct lio_mbox), 0);
+	if (mbox == NULL) {
+		rte_free(lio_dev->mbox);
+		lio_dev->mbox = NULL;
+		return -ENOMEM;
+	}
+
+	rte_spinlock_init(&mbox->lock);
+
+	mbox->lio_dev = lio_dev;
+
+	mbox->q_no = 0;
+
+	mbox->state = LIO_MBOX_STATE_IDLE;
+
+	/* VF mbox interrupt reg */
+	mbox->mbox_int_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_VF_SLI_PKT_MBOX_INT(0);
+	/* VF reads from SIG0 reg */
+	mbox->mbox_read_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_SLI_PKT_PF_VF_MBOX_SIG(0, 0);
+	/* VF writes into SIG1 reg */
+	mbox->mbox_write_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_SLI_PKT_PF_VF_MBOX_SIG(0, 1);
+
+	lio_dev->mbox[0] = mbox;
+
+	rte_write64(LIO_PFVFSIG, mbox->mbox_read_reg);
+
+	return 0;
+}
+
 int
 cn23xx_vf_setup_device(struct lio_device *lio_dev)
 {
@@ -221,6 +279,9 @@
 	if (lio_dev->default_config == NULL)
 		return -1;
 
+	lio_dev->fn_list.setup_mbox		= cn23xx_vf_setup_mbox;
+	lio_dev->fn_list.free_mbox		= cn23xx_vf_free_mbox;
+
 	lio_dev->fn_list.setup_device_regs	= cn23xx_vf_setup_device_regs;
 
 	return 0;
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 34b7b54..5ee1bb5 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -86,14 +86,19 @@
 		return -1;
 	}
 
+	if (lio_dev->fn_list.setup_mbox(lio_dev)) {
+		lio_dev_err(lio_dev, "Mailbox setup failed\n");
+		goto error;
+	}
+
 	if (cn23xx_vf_set_io_queues_off(lio_dev)) {
 		lio_dev_err(lio_dev, "Setting io queues off failed\n");
-		return -1;
+		goto error;
 	}
 
 	if (lio_dev->fn_list.setup_device_regs(lio_dev)) {
 		lio_dev_err(lio_dev, "Failed to configure device registers\n");
-		return -1;
+		goto error;
 	}
 
 	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
@@ -102,6 +107,12 @@
 	lio_dev->max_rx_queues = dpdk_queues;
 
 	return 0;
+
+error:
+	if (lio_dev->mbox[0])
+		lio_dev->fn_list.free_mbox(lio_dev);
+
+	return -1;
 }
 
 static int
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 0af4fe3..01b5716 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -45,6 +45,9 @@
 
 struct lio_device;
 struct lio_fn_list {
+	int (*setup_mbox)(struct lio_device *);
+	void (*free_mbox)(struct lio_device *);
+
 	int (*setup_device_regs)(struct lio_device *);
 };
 
-- 
1.8.3.1

[PATCH v3 11/46] net/liquidio: add API for PF VF handshake

[Shijith Thotton]

Handshake with PF kernel driver to check driver version compatibility.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 96 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h |  4 ++
 drivers/net/liquidio/base/lio_hw_defs.h |  3 ++
 drivers/net/liquidio/base/lio_mbox.h    |  1 +
 drivers/net/liquidio/lio_ethdev.c       | 21 ++++++++
 drivers/net/liquidio/lio_struct.h       | 45 ++++++++++++++++
 6 files changed, 170 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index 70faa9b..6270af5 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -255,6 +255,102 @@
 	return 0;
 }
 
+static void
+cn23xx_pfvf_hs_callback(struct lio_device *lio_dev,
+			struct lio_mbox_cmd *cmd, void *arg)
+{
+	uint32_t major = 0;
+
+	PMD_INIT_FUNC_TRACE();
+
+	rte_memcpy((uint8_t *)&lio_dev->pfvf_hsword, cmd->msg.s.params, 6);
+	if (cmd->recv_len > 1) {
+		struct lio_version *lio_ver = (struct lio_version *)cmd->data;
+
+		major = lio_ver->major;
+		major = major << 16;
+	}
+
+	rte_atomic64_set((rte_atomic64_t *)arg, major | 1);
+}
+
+int
+cn23xx_pfvf_handshake(struct lio_device *lio_dev)
+{
+	struct lio_mbox_cmd mbox_cmd;
+	struct lio_version *lio_ver = (struct lio_version *)&mbox_cmd.data[0];
+	rte_atomic64_t status;
+	uint32_t count = 0;
+	uint32_t pfmajor;
+	uint32_t vfmajor;
+	uint32_t ret;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* Sending VF_ACTIVE indication to the PF driver */
+	lio_dev_dbg(lio_dev, "requesting info from PF\n");
+
+	mbox_cmd.msg.mbox_msg64 = 0;
+	mbox_cmd.msg.s.type = LIO_MBOX_REQUEST;
+	mbox_cmd.msg.s.resp_needed = 1;
+	mbox_cmd.msg.s.cmd = LIO_VF_ACTIVE;
+	mbox_cmd.msg.s.len = 2;
+	mbox_cmd.data[0] = 0;
+	lio_ver->major = LIO_BASE_MAJOR_VERSION;
+	lio_ver->minor = LIO_BASE_MINOR_VERSION;
+	lio_ver->micro = LIO_BASE_MICRO_VERSION;
+	mbox_cmd.q_no = 0;
+	mbox_cmd.recv_len = 0;
+	mbox_cmd.recv_status = 0;
+	mbox_cmd.fn = (lio_mbox_callback)cn23xx_pfvf_hs_callback;
+	mbox_cmd.fn_arg = (void *)&status;
+
+	if (lio_mbox_write(lio_dev, &mbox_cmd)) {
+		lio_dev_err(lio_dev, "Write to mailbox failed\n");
+		return -1;
+	}
+
+	rte_atomic64_set(&status, 0);
+
+	do {
+		rte_delay_ms(1);
+	} while ((rte_atomic64_read(&status) == 0) && (count++ < 10000));
+
+	ret = rte_atomic64_read(&status);
+	if (ret == 0) {
+		lio_dev_err(lio_dev, "cn23xx_pfvf_handshake timeout\n");
+		return -1;
+	}
+
+	vfmajor = LIO_BASE_MAJOR_VERSION;
+	pfmajor = ret >> 16;
+	if (pfmajor != vfmajor) {
+		lio_dev_err(lio_dev,
+			    "VF LiquidIO driver (major version %d) is not compatible with LiquidIO PF driver (major version %d)\n",
+			    vfmajor, pfmajor);
+		ret = -EPERM;
+	} else {
+		lio_dev_dbg(lio_dev,
+			    "VF LiquidIO driver (major version %d), LiquidIO PF driver (major version %d)\n",
+			    vfmajor, pfmajor);
+		ret = 0;
+	}
+
+	return ret;
+}
+
+void
+cn23xx_vf_handle_mbox(struct lio_device *lio_dev)
+{
+	uint64_t mbox_int_val;
+
+	/* read and clear by writing 1 */
+	mbox_int_val = rte_read64(lio_dev->mbox[0]->mbox_int_reg);
+	rte_write64(mbox_int_val, lio_dev->mbox[0]->mbox_int_reg);
+	if (lio_mbox_read(lio_dev->mbox[0]))
+		lio_mbox_process_message(lio_dev->mbox[0]);
+}
+
 int
 cn23xx_vf_setup_device(struct lio_device *lio_dev)
 {
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.h b/drivers/net/liquidio/base/lio_23xx_vf.h
index 1af09d0..83dc053 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.h
+++ b/drivers/net/liquidio/base/lio_23xx_vf.h
@@ -87,5 +87,9 @@
 
 #define CN23XX_VF_BUSY_READING_REG_LOOP_COUNT	100000
 
+int cn23xx_pfvf_handshake(struct lio_device *lio_dev);
+
 int cn23xx_vf_setup_device(struct lio_device  *lio_dev);
+
+void cn23xx_vf_handle_mbox(struct lio_device *lio_dev);
 #endif /* _LIO_23XX_VF_H_  */
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index a2654cd..9282068 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -77,6 +77,9 @@ enum lio_card_type {
 #define LIO_23XX_NAME "23xx"
 
 #define LIO_DEVICE_NAME_LEN		32
+#define LIO_BASE_MAJOR_VERSION		1
+#define LIO_BASE_MINOR_VERSION		5
+#define LIO_BASE_MICRO_VERSION		1
 
 /* Routines for reading and writing CSRs */
 #ifdef RTE_LIBRTE_LIO_DEBUG_REGS
diff --git a/drivers/net/liquidio/base/lio_mbox.h b/drivers/net/liquidio/base/lio_mbox.h
index 28c9e1a..f1c5b8e 100644
--- a/drivers/net/liquidio/base/lio_mbox.h
+++ b/drivers/net/liquidio/base/lio_mbox.h
@@ -42,6 +42,7 @@
 
 #define LIO_MBOX_DATA_MAX			32
 
+#define LIO_VF_ACTIVE				0x1
 #define LIO_CORES_CRASHED			0x3
 
 /* Macro for Read acknowledgment */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 5ee1bb5..bebe0e8 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -40,6 +40,20 @@
 #include "lio_23xx_vf.h"
 #include "lio_ethdev.h"
 
+static void
+lio_check_pf_hs_response(void *lio_dev)
+{
+	struct lio_device *dev = lio_dev;
+
+	/* check till response arrives */
+	if (dev->pfvf_hsword.coproc_tics_per_us)
+		return;
+
+	cn23xx_vf_handle_mbox(dev);
+
+	rte_eal_alarm_set(1, lio_check_pf_hs_response, lio_dev);
+}
+
 /**
  * \brief Identify the LIO device and to map the BAR address space
  * @param lio_dev lio device
@@ -91,6 +105,13 @@
 		goto error;
 	}
 
+	/* Check PF response */
+	lio_check_pf_hs_response((void *)lio_dev);
+
+	/* Do handshake and exit if incompatible PF driver */
+	if (cn23xx_pfvf_handshake(lio_dev))
+		goto error;
+
 	if (cn23xx_vf_set_io_queues_off(lio_dev)) {
 		lio_dev_err(lio_dev, "Setting io queues off failed\n");
 		goto error;
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 01b5716..e8b6e1d 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -43,6 +43,13 @@
 
 #include "lio_hw_defs.h"
 
+struct lio_version {
+	uint16_t major;
+	uint16_t minor;
+	uint16_t micro;
+	uint16_t reserved;
+};
+
 struct lio_device;
 struct lio_fn_list {
 	int (*setup_mbox)(struct lio_device *);
@@ -51,6 +58,42 @@ struct lio_fn_list {
 	int (*setup_device_regs)(struct lio_device *);
 };
 
+struct lio_pf_vf_hs_word {
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	/** PKIND value assigned for the DPI interface */
+	uint64_t pkind : 8;
+
+	/** OCTEON core clock multiplier */
+	uint64_t core_tics_per_us : 16;
+
+	/** OCTEON coprocessor clock multiplier */
+	uint64_t coproc_tics_per_us : 16;
+
+	/** app that currently running on OCTEON */
+	uint64_t app_mode : 8;
+
+	/** RESERVED */
+	uint64_t reserved : 16;
+
+#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+
+	/** RESERVED */
+	uint64_t reserved : 16;
+
+	/** app that currently running on OCTEON */
+	uint64_t app_mode : 8;
+
+	/** OCTEON coprocessor clock multiplier */
+	uint64_t coproc_tics_per_us : 16;
+
+	/** OCTEON core clock multiplier */
+	uint64_t core_tics_per_us : 16;
+
+	/** PKIND value assigned for the DPI interface */
+	uint64_t pkind : 8;
+#endif
+};
+
 struct lio_sriov_info {
 	/** Number of rings assigned to VF */
 	uint32_t rings_per_vf;
@@ -129,6 +172,8 @@ struct lio_device {
 
 	struct lio_sriov_info sriov_info;
 
+	struct lio_pf_vf_hs_word pfvf_hsword;
+
 	/** Mail Box details of each lio queue. */
 	struct lio_mbox **mbox;
 
-- 
1.8.3.1

[PATCH v3 12/46] net/liquidio: add API for VF FLR

[Shijith Thotton]

VF sends Function Level Reset request to PF using mbox and PF does the
reset.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 19 +++++++++++++++++++
 drivers/net/liquidio/base/lio_23xx_vf.h |  2 ++
 drivers/net/liquidio/base/lio_mbox.h    |  1 +
 drivers/net/liquidio/lio_ethdev.c       |  5 +++++
 4 files changed, 27 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index 6270af5..ed5b830 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -255,6 +255,25 @@
 	return 0;
 }
 
+void
+cn23xx_vf_ask_pf_to_do_flr(struct lio_device *lio_dev)
+{
+	struct lio_mbox_cmd mbox_cmd;
+
+	mbox_cmd.msg.mbox_msg64 = 0;
+	mbox_cmd.msg.s.type = LIO_MBOX_REQUEST;
+	mbox_cmd.msg.s.resp_needed = 0;
+	mbox_cmd.msg.s.cmd = LIO_VF_FLR_REQUEST;
+	mbox_cmd.msg.s.len = 1;
+	mbox_cmd.q_no = 0;
+	mbox_cmd.recv_len = 0;
+	mbox_cmd.recv_status = 0;
+	mbox_cmd.fn = NULL;
+	mbox_cmd.fn_arg = 0;
+
+	lio_mbox_write(lio_dev, &mbox_cmd);
+}
+
 static void
 cn23xx_pfvf_hs_callback(struct lio_device *lio_dev,
 			struct lio_mbox_cmd *cmd, void *arg)
diff --git a/drivers/net/liquidio/base/lio_23xx_vf.h b/drivers/net/liquidio/base/lio_23xx_vf.h
index 83dc053..ad8db0d 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.h
+++ b/drivers/net/liquidio/base/lio_23xx_vf.h
@@ -87,6 +87,8 @@
 
 #define CN23XX_VF_BUSY_READING_REG_LOOP_COUNT	100000
 
+void cn23xx_vf_ask_pf_to_do_flr(struct lio_device *lio_dev);
+
 int cn23xx_pfvf_handshake(struct lio_device *lio_dev);
 
 int cn23xx_vf_setup_device(struct lio_device  *lio_dev);
diff --git a/drivers/net/liquidio/base/lio_mbox.h b/drivers/net/liquidio/base/lio_mbox.h
index f1c5b8e..b0875d6 100644
--- a/drivers/net/liquidio/base/lio_mbox.h
+++ b/drivers/net/liquidio/base/lio_mbox.h
@@ -43,6 +43,7 @@
 #define LIO_MBOX_DATA_MAX			32
 
 #define LIO_VF_ACTIVE				0x1
+#define LIO_VF_FLR_REQUEST			0x2
 #define LIO_CORES_CRASHED			0x3
 
 /* Macro for Read acknowledgment */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index bebe0e8..5aae105 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -112,6 +112,11 @@
 	if (cn23xx_pfvf_handshake(lio_dev))
 		goto error;
 
+	/* Initial reset */
+	cn23xx_vf_ask_pf_to_do_flr(lio_dev);
+	/* Wait for FLR for 100ms per SRIOV specification */
+	rte_delay_ms(100);
+
 	if (cn23xx_vf_set_io_queues_off(lio_dev)) {
 		lio_dev_err(lio_dev, "Setting io queues off failed\n");
 		goto error;
-- 
1.8.3.1

[PATCH v3 13/46] net/liquidio: add APIs to allocate and free IQ

[Shijith Thotton]

Instruction queue (IQ) is used to send control and data packets to
device from host. IQ 0 is used to send device configuration commands
during initialization and later re-allocated as per application
requirement.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/Makefile           |   1 +
 drivers/net/liquidio/base/lio_hw_defs.h |  12 ++
 drivers/net/liquidio/lio_ethdev.c       |   8 ++
 drivers/net/liquidio/lio_rxtx.c         | 206 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         |  57 +++++++++
 drivers/net/liquidio/lio_struct.h       | 112 ++++++++++++++++-
 6 files changed, 395 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/liquidio/lio_rxtx.c
 create mode 100644 drivers/net/liquidio/lio_rxtx.h

diff --git a/drivers/net/liquidio/Makefile b/drivers/net/liquidio/Makefile
index 451f49d..de2ef9b 100644
--- a/drivers/net/liquidio/Makefile
+++ b/drivers/net/liquidio/Makefile
@@ -51,6 +51,7 @@ VPATH += $(RTE_SDK)/drivers/net/liquidio/base
 # all source are stored in SRCS-y
 #
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_ethdev.c
+SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_rxtx.c
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_23xx_vf.c
 SRCS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += lio_mbox.c
 
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 9282068..726ce6b 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -76,6 +76,18 @@ enum lio_card_type {
 
 #define LIO_23XX_NAME "23xx"
 
+#define LIO_NUM_DEF_TX_DESCS_CFG(cfg)					\
+		((cfg)->default_config->num_def_tx_descs)
+
+#define LIO_IQ_INSTR_TYPE(cfg)		((cfg)->default_config->iq.instr_type)
+
+/* The following config values are fixed and should not be modified. */
+
+/* Maximum number of Instruction queues */
+#define LIO_MAX_INSTR_QUEUES(lio_dev)		CN23XX_MAX_RINGS_PER_VF
+
+#define LIO_MAX_POSSIBLE_INSTR_QUEUES		CN23XX_MAX_INPUT_QUEUES
+
 #define LIO_DEVICE_NAME_LEN		32
 #define LIO_BASE_MAJOR_VERSION		1
 #define LIO_BASE_MINOR_VERSION		5
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 5aae105..5d7d5a7 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -39,6 +39,7 @@
 #include "lio_logs.h"
 #include "lio_23xx_vf.h"
 #include "lio_ethdev.h"
+#include "lio_rxtx.h"
 
 static void
 lio_check_pf_hs_response(void *lio_dev)
@@ -127,6 +128,11 @@
 		goto error;
 	}
 
+	if (lio_setup_instr_queue0(lio_dev)) {
+		lio_dev_err(lio_dev, "Failed to setup instruction queue 0\n");
+		goto error;
+	}
+
 	dpdk_queues = (int)lio_dev->sriov_info.rings_per_vf;
 
 	lio_dev->max_tx_queues = dpdk_queues;
@@ -137,6 +143,8 @@
 error:
 	if (lio_dev->mbox[0])
 		lio_dev->fn_list.free_mbox(lio_dev);
+	if (lio_dev->instr_queue[0])
+		lio_free_instr_queue0(lio_dev);
 
 	return -1;
 }
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
new file mode 100644
index 0000000..de98fc6
--- /dev/null
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -0,0 +1,206 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+
+#include "lio_logs.h"
+#include "lio_struct.h"
+#include "lio_ethdev.h"
+#include "lio_rxtx.h"
+
+static void
+lio_dma_zone_free(struct lio_device *lio_dev, const struct rte_memzone *mz)
+{
+	const struct rte_memzone *mz_tmp;
+	int ret = 0;
+
+	if (mz == NULL) {
+		lio_dev_err(lio_dev, "Memzone NULL\n");
+		return;
+	}
+
+	mz_tmp = rte_memzone_lookup(mz->name);
+	if (mz_tmp == NULL) {
+		lio_dev_err(lio_dev, "Memzone %s Not Found\n", mz->name);
+		return;
+	}
+
+	ret = rte_memzone_free(mz);
+	if (ret)
+		lio_dev_err(lio_dev, "Memzone free Failed ret %d\n", ret);
+}
+
+/**
+ *  lio_init_instr_queue()
+ *  @param lio_dev	- pointer to the lio device structure.
+ *  @param txpciq	- queue to be initialized.
+ *
+ *  Called at driver init time for each input queue. iq_conf has the
+ *  configuration parameters for the queue.
+ *
+ *  @return  Success: 0	Failure: -1
+ */
+static int
+lio_init_instr_queue(struct lio_device *lio_dev,
+		     union octeon_txpciq txpciq,
+		     uint32_t num_descs, unsigned int socket_id)
+{
+	uint32_t iq_no = (uint32_t)txpciq.s.q_no;
+	struct lio_instr_queue *iq;
+	uint32_t instr_type;
+	uint32_t q_size;
+
+	instr_type = LIO_IQ_INSTR_TYPE(lio_dev);
+
+	q_size = instr_type * num_descs;
+	iq = lio_dev->instr_queue[iq_no];
+	iq->iq_mz = rte_eth_dma_zone_reserve(lio_dev->eth_dev,
+					     "instr_queue", iq_no, q_size,
+					     RTE_CACHE_LINE_SIZE,
+					     socket_id);
+	if (iq->iq_mz == NULL) {
+		lio_dev_err(lio_dev, "Cannot allocate memory for instr queue %d\n",
+			    iq_no);
+		return -1;
+	}
+
+	iq->base_addr_dma = iq->iq_mz->phys_addr;
+	iq->base_addr = (uint8_t *)iq->iq_mz->addr;
+
+	iq->max_count = num_descs;
+
+	/* Initialize a list to holds requests that have been posted to Octeon
+	 * but has yet to be fetched by octeon
+	 */
+	iq->request_list = rte_zmalloc_socket("request_list",
+					      sizeof(*iq->request_list) *
+							num_descs,
+					      RTE_CACHE_LINE_SIZE,
+					      socket_id);
+	if (iq->request_list == NULL) {
+		lio_dev_err(lio_dev, "Alloc failed for IQ[%d] nr free list\n",
+			    iq_no);
+		lio_dma_zone_free(lio_dev, iq->iq_mz);
+		return -1;
+	}
+
+	lio_dev_dbg(lio_dev, "IQ[%d]: base: %p basedma: %lx count: %d\n",
+		    iq_no, iq->base_addr, (unsigned long)iq->base_addr_dma,
+		    iq->max_count);
+
+	iq->lio_dev = lio_dev;
+	iq->txpciq.txpciq64 = txpciq.txpciq64;
+	iq->fill_cnt = 0;
+	iq->host_write_index = 0;
+	iq->lio_read_index = 0;
+	iq->flush_index = 0;
+
+	rte_atomic64_set(&iq->instr_pending, 0);
+
+	/* Initialize the spinlock for this instruction queue */
+	rte_spinlock_init(&iq->lock);
+	rte_spinlock_init(&iq->post_lock);
+
+	rte_atomic64_clear(&iq->iq_flush_running);
+
+	lio_dev->io_qmask.iq |= (1ULL << iq_no);
+
+	/* Set the 32B/64B mode for each input queue */
+	lio_dev->io_qmask.iq64B |= ((instr_type == 64) << iq_no);
+	iq->iqcmd_64B = (instr_type == 64);
+
+	return 0;
+}
+
+int
+lio_setup_instr_queue0(struct lio_device *lio_dev)
+{
+	union octeon_txpciq txpciq;
+	uint32_t num_descs = 0;
+	uint32_t iq_no = 0;
+
+	num_descs = LIO_NUM_DEF_TX_DESCS_CFG(lio_dev);
+
+	lio_dev->num_iqs = 0;
+
+	lio_dev->instr_queue[0] = rte_zmalloc(NULL,
+					sizeof(struct lio_instr_queue), 0);
+	if (lio_dev->instr_queue[0] == NULL)
+		return -ENOMEM;
+
+	lio_dev->instr_queue[0]->q_index = 0;
+	lio_dev->instr_queue[0]->app_ctx = (void *)(size_t)0;
+	txpciq.txpciq64 = 0;
+	txpciq.s.q_no = iq_no;
+	txpciq.s.pkind = lio_dev->pfvf_hsword.pkind;
+	txpciq.s.use_qpg = 0;
+	txpciq.s.qpg = 0;
+	if (lio_init_instr_queue(lio_dev, txpciq, num_descs, SOCKET_ID_ANY)) {
+		rte_free(lio_dev->instr_queue[0]);
+		lio_dev->instr_queue[0] = NULL;
+		return -1;
+	}
+
+	lio_dev->num_iqs++;
+
+	return 0;
+}
+
+/**
+ *  lio_delete_instr_queue()
+ *  @param lio_dev	- pointer to the lio device structure.
+ *  @param iq_no	- queue to be deleted.
+ *
+ *  Called at driver unload time for each input queue. Deletes all
+ *  allocated resources for the input queue.
+ */
+static void
+lio_delete_instr_queue(struct lio_device *lio_dev, uint32_t iq_no)
+{
+	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
+
+	rte_free(iq->request_list);
+	iq->request_list = NULL;
+	lio_dma_zone_free(lio_dev, iq->iq_mz);
+}
+
+void
+lio_free_instr_queue0(struct lio_device *lio_dev)
+{
+	lio_delete_instr_queue(lio_dev, 0);
+	rte_free(lio_dev->instr_queue[0]);
+	lio_dev->instr_queue[0] = NULL;
+	lio_dev->num_iqs--;
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
new file mode 100644
index 0000000..33f178b
--- /dev/null
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -0,0 +1,57 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Cavium, Inc. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _LIO_RXTX_H_
+#define _LIO_RXTX_H_
+
+#include <stdio.h>
+#include <stdint.h>
+
+#include <rte_spinlock.h>
+#include <rte_memory.h>
+
+#include "lio_struct.h"
+
+struct lio_request_list {
+	uint32_t reqtype;
+	void *buf;
+};
+
+/** Setup instruction queue zero for the device
+ *  @param lio_dev which lio device to setup
+ *
+ *  @return 0 if success. -1 if fails
+ */
+int lio_setup_instr_queue0(struct lio_device *lio_dev);
+void lio_free_instr_queue0(struct lio_device *lio_dev);
+#endif	/* _LIO_RXTX_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index e8b6e1d..29059a5 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -50,7 +50,110 @@ struct lio_version {
 	uint16_t reserved;
 };
 
-struct lio_device;
+/** The txpciq info passed to host from the firmware */
+union octeon_txpciq {
+	uint64_t txpciq64;
+
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t q_no : 8;
+		uint64_t port : 8;
+		uint64_t pkind : 6;
+		uint64_t use_qpg : 1;
+		uint64_t qpg : 11;
+		uint64_t aura_num : 10;
+		uint64_t reserved : 20;
+#else
+		uint64_t reserved : 20;
+		uint64_t aura_num : 10;
+		uint64_t qpg : 11;
+		uint64_t use_qpg : 1;
+		uint64_t pkind : 6;
+		uint64_t port : 8;
+		uint64_t q_no : 8;
+#endif
+	} s;
+};
+
+/** The instruction (input) queue.
+ *  The input queue is used to post raw (instruction) mode data or packet
+ *  data to Octeon device from the host. Each input queue for
+ *  a LIO device has one such structure to represent it.
+ */
+struct lio_instr_queue {
+	/** A spinlock to protect access to the input ring.  */
+	rte_spinlock_t lock;
+
+	rte_spinlock_t post_lock;
+
+	struct lio_device *lio_dev;
+
+	uint32_t pkt_in_done;
+
+	rte_atomic64_t iq_flush_running;
+
+	/** Flag that indicates if the queue uses 64 byte commands. */
+	uint32_t iqcmd_64B:1;
+
+	/** Queue info. */
+	union octeon_txpciq txpciq;
+
+	uint32_t rsvd:17;
+
+	uint32_t status:8;
+
+	/** Maximum no. of instructions in this queue. */
+	uint32_t max_count;
+
+	/** Index in input ring where the driver should write the next packet */
+	uint32_t host_write_index;
+
+	/** Index in input ring where Octeon is expected to read the next
+	 *  packet.
+	 */
+	uint32_t lio_read_index;
+
+	/** This index aids in finding the window in the queue where Octeon
+	 *  has read the commands.
+	 */
+	uint32_t flush_index;
+
+	/** This field keeps track of the instructions pending in this queue. */
+	rte_atomic64_t instr_pending;
+
+	/** Pointer to the Virtual Base addr of the input ring. */
+	uint8_t *base_addr;
+
+	struct lio_request_list *request_list;
+
+	/** Octeon doorbell register for the ring. */
+	void *doorbell_reg;
+
+	/** Octeon instruction count register for this ring. */
+	void *inst_cnt_reg;
+
+	/** Number of instructions pending to be posted to Octeon. */
+	uint32_t fill_cnt;
+
+	/** DMA mapped base address of the input descriptor ring. */
+	uint64_t base_addr_dma;
+
+	/** Application context */
+	void *app_ctx;
+
+	/* network stack queue index */
+	int q_index;
+
+	/* Memory zone */
+	const struct rte_memzone *iq_mz;
+};
+
+struct lio_io_enable {
+	uint64_t iq;
+	uint64_t oq;
+	uint64_t iq64B;
+};
+
 struct lio_fn_list {
 	int (*setup_mbox)(struct lio_device *);
 	void (*free_mbox)(struct lio_device *);
@@ -170,6 +273,13 @@ struct lio_device {
 
 	struct lio_fn_list fn_list;
 
+	uint32_t num_iqs;
+
+	/** The input instruction queues */
+	struct lio_instr_queue *instr_queue[LIO_MAX_POSSIBLE_INSTR_QUEUES];
+
+	struct lio_io_enable io_qmask;
+
 	struct lio_sriov_info sriov_info;
 
 	struct lio_pf_vf_hs_word pfvf_hsword;
-- 
1.8.3.1

[PATCH v3 14/46] net/liquidio: add API to setup IQ

[Shijith Thotton]

Map instruction queue registers and set queue size.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 44 ++++++++++++++++++++++++++++++++-
 drivers/net/liquidio/lio_rxtx.c         |  2 ++
 drivers/net/liquidio/lio_struct.h       |  2 ++
 3 files changed, 47 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index ed5b830..181f830 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -199,6 +199,40 @@
 }
 
 static void
+cn23xx_vf_setup_iq_regs(struct lio_device *lio_dev, uint32_t iq_no)
+{
+	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
+	uint64_t pkt_in_done = 0;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* Write the start of the input queue's ring and its size */
+	lio_write_csr64(lio_dev, CN23XX_SLI_IQ_BASE_ADDR64(iq_no),
+			iq->base_addr_dma);
+	lio_write_csr(lio_dev, CN23XX_SLI_IQ_SIZE(iq_no), iq->max_count);
+
+	/* Remember the doorbell & instruction count register addr
+	 * for this queue
+	 */
+	iq->doorbell_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_SLI_IQ_DOORBELL(iq_no);
+	iq->inst_cnt_reg = (uint8_t *)lio_dev->hw_addr +
+				CN23XX_SLI_IQ_INSTR_COUNT64(iq_no);
+	lio_dev_dbg(lio_dev, "InstQ[%d]:dbell reg @ 0x%p instcnt_reg @ 0x%p\n",
+		    iq_no, iq->doorbell_reg, iq->inst_cnt_reg);
+
+	/* Store the current instruction counter (used in flush_iq
+	 * calculation)
+	 */
+	pkt_in_done = rte_read64(iq->inst_cnt_reg);
+
+	/* Clear the count by writing back what we read, but don't
+	 * enable data traffic here
+	 */
+	rte_write64(pkt_in_done, iq->inst_cnt_reg);
+}
+
+static void
 cn23xx_vf_free_mbox(struct lio_device *lio_dev)
 {
 	PMD_INIT_FUNC_TRACE();
@@ -298,8 +332,8 @@
 {
 	struct lio_mbox_cmd mbox_cmd;
 	struct lio_version *lio_ver = (struct lio_version *)&mbox_cmd.data[0];
+	uint32_t q_no, count = 0;
 	rte_atomic64_t status;
-	uint32_t count = 0;
 	uint32_t pfmajor;
 	uint32_t vfmajor;
 	uint32_t ret;
@@ -341,6 +375,10 @@
 		return -1;
 	}
 
+	for (q_no = 0; q_no < lio_dev->num_iqs; q_no++)
+		lio_dev->instr_queue[q_no]->txpciq.s.pkind =
+						lio_dev->pfvf_hsword.pkind;
+
 	vfmajor = LIO_BASE_MAJOR_VERSION;
 	pfmajor = ret >> 16;
 	if (pfmajor != vfmajor) {
@@ -355,6 +393,9 @@
 		ret = 0;
 	}
 
+	lio_dev_dbg(lio_dev, "got data from PF pkind is %d\n",
+		    lio_dev->pfvf_hsword.pkind);
+
 	return ret;
 }
 
@@ -394,6 +435,7 @@
 	if (lio_dev->default_config == NULL)
 		return -1;
 
+	lio_dev->fn_list.setup_iq_regs		= cn23xx_vf_setup_iq_regs;
 	lio_dev->fn_list.setup_mbox		= cn23xx_vf_setup_mbox;
 	lio_dev->fn_list.free_mbox		= cn23xx_vf_free_mbox;
 
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index de98fc6..4a687d8 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -141,6 +141,8 @@
 	lio_dev->io_qmask.iq64B |= ((instr_type == 64) << iq_no);
 	iq->iqcmd_64B = (instr_type == 64);
 
+	lio_dev->fn_list.setup_iq_regs(lio_dev, iq_no);
+
 	return 0;
 }
 
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 29059a5..2806c37 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -155,6 +155,8 @@ struct lio_io_enable {
 };
 
 struct lio_fn_list {
+	void (*setup_iq_regs)(struct lio_device *, uint32_t);
+
 	int (*setup_mbox)(struct lio_device *);
 	void (*free_mbox)(struct lio_device *);
 
-- 
1.8.3.1

[PATCH v3 15/46] net/liquidio: add APIs to allocate and free SC buffer pool

[Shijith Thotton]

Soft command (SC) holds device control command and related information.
SC buffer pool holds buffers which are used during soft command
allocation.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 12 ++++++++++++
 drivers/net/liquidio/lio_rxtx.c   | 21 +++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  9 +++++++++
 drivers/net/liquidio/lio_struct.h |  3 +++
 4 files changed, 45 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 5d7d5a7..a1dcdf6 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -101,6 +101,12 @@
 		return -1;
 	}
 
+	/* Initialize soft command buffer pool */
+	if (lio_setup_sc_buffer_pool(lio_dev)) {
+		lio_dev_err(lio_dev, "sc buffer pool allocation failed\n");
+		return -1;
+	}
+
 	if (lio_dev->fn_list.setup_mbox(lio_dev)) {
 		lio_dev_err(lio_dev, "Mailbox setup failed\n");
 		goto error;
@@ -141,6 +147,7 @@
 	return 0;
 
 error:
+	lio_free_sc_buffer_pool(lio_dev);
 	if (lio_dev->mbox[0])
 		lio_dev->fn_list.free_mbox(lio_dev);
 	if (lio_dev->instr_queue[0])
@@ -152,11 +159,16 @@
 static int
 lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
 {
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
+	/* lio_free_sc_buffer_pool */
+	lio_free_sc_buffer_pool(lio_dev);
+
 	rte_free(eth_dev->data->mac_addrs);
 	eth_dev->data->mac_addrs = NULL;
 
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 4a687d8..1c6ce59 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -206,3 +206,24 @@
 	lio_dev->instr_queue[0] = NULL;
 	lio_dev->num_iqs--;
 }
+
+int
+lio_setup_sc_buffer_pool(struct lio_device *lio_dev)
+{
+	char sc_pool_name[RTE_MEMPOOL_NAMESIZE];
+	uint16_t buf_size;
+
+	buf_size = LIO_SOFT_COMMAND_BUFFER_SIZE + RTE_PKTMBUF_HEADROOM;
+	snprintf(sc_pool_name, sizeof(sc_pool_name),
+		 "lio_sc_pool_%u", lio_dev->port_id);
+	lio_dev->sc_buf_pool = rte_pktmbuf_pool_create(sc_pool_name,
+						LIO_MAX_SOFT_COMMAND_BUFFERS,
+						0, 0, buf_size, SOCKET_ID_ANY);
+	return 0;
+}
+
+void
+lio_free_sc_buffer_pool(struct lio_device *lio_dev)
+{
+	rte_mempool_free(lio_dev->sc_buf_pool);
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 33f178b..b308211 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -47,6 +47,15 @@ struct lio_request_list {
 	void *buf;
 };
 
+/** The size of each buffer in soft command buffer pool */
+#define LIO_SOFT_COMMAND_BUFFER_SIZE	1536
+
+/** Maximum number of buffers to allocate into soft command buffer pool */
+#define LIO_MAX_SOFT_COMMAND_BUFFERS	255
+
+int lio_setup_sc_buffer_pool(struct lio_device *lio_dev);
+void lio_free_sc_buffer_pool(struct lio_device *lio_dev);
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 2806c37..992ad39 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -277,6 +277,9 @@ struct lio_device {
 
 	uint32_t num_iqs;
 
+	/* The pool containing pre allocated buffers used for soft commands */
+	struct rte_mempool *sc_buf_pool;
+
 	/** The input instruction queues */
 	struct lio_instr_queue *instr_queue[LIO_MAX_POSSIBLE_INSTR_QUEUES];
 
-- 
1.8.3.1

[PATCH v3 16/46] net/liquidio: add APIs to allocate and free soft command

[Shijith Thotton]

Get buffers from SC buffer pool and create soft command. Buffers are
freed to the pool once the command reaches device.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_rxtx.c   | 66 +++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   | 38 ++++++++++++++++++++++
 drivers/net/liquidio/lio_struct.h |  6 ++++
 3 files changed, 110 insertions(+)

diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 1c6ce59..cfec96d 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -227,3 +227,69 @@
 {
 	rte_mempool_free(lio_dev->sc_buf_pool);
 }
+
+struct lio_soft_command *
+lio_alloc_soft_command(struct lio_device *lio_dev, uint32_t datasize,
+		       uint32_t rdatasize, uint32_t ctxsize)
+{
+	uint32_t offset = sizeof(struct lio_soft_command);
+	struct lio_soft_command *sc;
+	struct rte_mbuf *m;
+	uint64_t dma_addr;
+
+	RTE_ASSERT((offset + datasize + rdatasize + ctxsize) <=
+		   LIO_SOFT_COMMAND_BUFFER_SIZE);
+
+	m = rte_pktmbuf_alloc(lio_dev->sc_buf_pool);
+	if (m == NULL) {
+		lio_dev_err(lio_dev, "Cannot allocate mbuf for sc\n");
+		return NULL;
+	}
+
+	/* set rte_mbuf data size and there is only 1 segment */
+	m->pkt_len = LIO_SOFT_COMMAND_BUFFER_SIZE;
+	m->data_len = LIO_SOFT_COMMAND_BUFFER_SIZE;
+
+	/* use rte_mbuf buffer for soft command */
+	sc = rte_pktmbuf_mtod(m, struct lio_soft_command *);
+	memset(sc, 0, LIO_SOFT_COMMAND_BUFFER_SIZE);
+	sc->size = LIO_SOFT_COMMAND_BUFFER_SIZE;
+	sc->dma_addr = rte_mbuf_data_dma_addr(m);
+	sc->mbuf = m;
+
+	dma_addr = sc->dma_addr;
+
+	if (ctxsize) {
+		sc->ctxptr = (uint8_t *)sc + offset;
+		sc->ctxsize = ctxsize;
+	}
+
+	/* Start data at 128 byte boundary */
+	offset = (offset + ctxsize + 127) & 0xffffff80;
+
+	if (datasize) {
+		sc->virtdptr = (uint8_t *)sc + offset;
+		sc->dmadptr = dma_addr + offset;
+		sc->datasize = datasize;
+	}
+
+	/* Start rdata at 128 byte boundary */
+	offset = (offset + datasize + 127) & 0xffffff80;
+
+	if (rdatasize) {
+		RTE_ASSERT(rdatasize >= 16);
+		sc->virtrptr = (uint8_t *)sc + offset;
+		sc->dmarptr = dma_addr + offset;
+		sc->rdatasize = rdatasize;
+		sc->status_word = (uint64_t *)((uint8_t *)(sc->virtrptr) +
+					       rdatasize - 8);
+	}
+
+	return sc;
+}
+
+void
+lio_free_soft_command(struct lio_soft_command *sc)
+{
+	rte_pktmbuf_free(sc->mbuf);
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index b308211..284cb18 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -53,9 +53,47 @@ struct lio_request_list {
 /** Maximum number of buffers to allocate into soft command buffer pool */
 #define LIO_MAX_SOFT_COMMAND_BUFFERS	255
 
+struct lio_soft_command {
+	/** Soft command buffer info. */
+	struct lio_stailq_node node;
+	uint64_t dma_addr;
+	uint32_t size;
+
+#define LIO_COMPLETION_WORD_INIT	0xffffffffffffffffULL
+	uint64_t *status_word;
+
+	/** Data buffer info */
+	void *virtdptr;
+	uint64_t dmadptr;
+	uint32_t datasize;
+
+	/** Return buffer info */
+	void *virtrptr;
+	uint64_t dmarptr;
+	uint32_t rdatasize;
+
+	/** Context buffer info */
+	void *ctxptr;
+	uint32_t ctxsize;
+
+	/** Time out and callback */
+	size_t wait_time;
+	size_t timeout;
+	uint32_t iq_no;
+	void (*callback)(uint32_t, void *);
+	void *callback_arg;
+	struct rte_mbuf *mbuf;
+};
+
 int lio_setup_sc_buffer_pool(struct lio_device *lio_dev);
 void lio_free_sc_buffer_pool(struct lio_device *lio_dev);
 
+struct lio_soft_command *
+lio_alloc_soft_command(struct lio_device *lio_dev,
+		       uint32_t datasize, uint32_t rdatasize,
+		       uint32_t ctxsize);
+void lio_free_soft_command(struct lio_soft_command *sc);
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 992ad39..45f9ac9 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -43,6 +43,12 @@
 
 #include "lio_hw_defs.h"
 
+struct lio_stailq_node {
+	STAILQ_ENTRY(lio_stailq_node) entries;
+};
+
+STAILQ_HEAD(lio_stailq_head, lio_stailq_node);
+
 struct lio_version {
 	uint16_t major;
 	uint16_t minor;
-- 
1.8.3.1

[PATCH v3 17/46] net/liquidio: add APIs for response list

[Shijith Thotton]

Add APIs to setup and process response list. Response list holds soft
commands waiting for response from device. Entries of this list are
processed to check for command response or timeout.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c |   5 ++
 drivers/net/liquidio/lio_rxtx.c   | 106 ++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  73 ++++++++++++++++++++++++++
 drivers/net/liquidio/lio_struct.h |  14 +++++
 4 files changed, 198 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index a1dcdf6..97a7369 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -107,6 +107,11 @@
 		return -1;
 	}
 
+	/* Initialize lists to manage the requests of different types that
+	 * arrive from applications for this lio device.
+	 */
+	lio_setup_response_list(lio_dev);
+
 	if (lio_dev->fn_list.setup_mbox(lio_dev)) {
 		lio_dev_err(lio_dev, "Mailbox setup failed\n");
 		goto error;
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index cfec96d..c61397f 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -293,3 +293,109 @@ struct lio_soft_command *
 {
 	rte_pktmbuf_free(sc->mbuf);
 }
+
+void
+lio_setup_response_list(struct lio_device *lio_dev)
+{
+	STAILQ_INIT(&lio_dev->response_list.head);
+	rte_spinlock_init(&lio_dev->response_list.lock);
+	rte_atomic64_set(&lio_dev->response_list.pending_req_count, 0);
+}
+
+int
+lio_process_ordered_list(struct lio_device *lio_dev)
+{
+	int resp_to_process = LIO_MAX_ORD_REQS_TO_PROCESS;
+	struct lio_response_list *ordered_sc_list;
+	struct lio_soft_command *sc;
+	int request_complete = 0;
+	uint64_t status64;
+	uint32_t status;
+
+	ordered_sc_list = &lio_dev->response_list;
+
+	do {
+		rte_spinlock_lock(&ordered_sc_list->lock);
+
+		if (STAILQ_EMPTY(&ordered_sc_list->head)) {
+			/* ordered_sc_list is empty; there is
+			 * nothing to process
+			 */
+			rte_spinlock_unlock(&ordered_sc_list->lock);
+			return -1;
+		}
+
+		sc = LIO_STQUEUE_FIRST_ENTRY(&ordered_sc_list->head,
+					     struct lio_soft_command, node);
+
+		status = LIO_REQUEST_PENDING;
+
+		/* check if octeon has finished DMA'ing a response
+		 * to where rptr is pointing to
+		 */
+		status64 = *sc->status_word;
+
+		if (status64 != LIO_COMPLETION_WORD_INIT) {
+			/* This logic ensures that all 64b have been written.
+			 * 1. check byte 0 for non-FF
+			 * 2. if non-FF, then swap result from BE to host order
+			 * 3. check byte 7 (swapped to 0) for non-FF
+			 * 4. if non-FF, use the low 32-bit status code
+			 * 5. if either byte 0 or byte 7 is FF, don't use status
+			 */
+			if ((status64 & 0xff) != 0xff) {
+				lio_swap_8B_data(&status64, 1);
+				if (((status64 & 0xff) != 0xff)) {
+					/* retrieve 16-bit firmware status */
+					status = (uint32_t)(status64 &
+							    0xffffULL);
+					if (status) {
+						status =
+						LIO_FIRMWARE_STATUS_CODE(
+									status);
+					} else {
+						/* i.e. no error */
+						status = LIO_REQUEST_DONE;
+					}
+				}
+			}
+		} else if ((sc->timeout && lio_check_timeout(lio_uptime,
+							     sc->timeout))) {
+			lio_dev_err(lio_dev,
+				    "cmd failed, timeout (%ld, %ld)\n",
+				    (long)lio_uptime, (long)sc->timeout);
+			status = LIO_REQUEST_TIMEOUT;
+		}
+
+		if (status != LIO_REQUEST_PENDING) {
+			/* we have received a response or we have timed out.
+			 * remove node from linked list
+			 */
+			STAILQ_REMOVE(&ordered_sc_list->head,
+				      &sc->node, lio_stailq_node, entries);
+			rte_atomic64_dec(
+			    &lio_dev->response_list.pending_req_count);
+			rte_spinlock_unlock(&ordered_sc_list->lock);
+
+			if (sc->callback)
+				sc->callback(status, sc->callback_arg);
+
+			request_complete++;
+		} else {
+			/* no response yet */
+			request_complete = 0;
+			rte_spinlock_unlock(&ordered_sc_list->lock);
+		}
+
+		/* If we hit the Max Ordered requests to process every loop,
+		 * we quit and let this function be invoked the next time
+		 * the poll thread runs to process the remaining requests.
+		 * This function can take up the entire CPU if there is
+		 * no upper limit to the requests processed.
+		 */
+		if (request_complete >= resp_to_process)
+			break;
+	} while (request_complete);
+
+	return 0;
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 284cb18..0e7eb6a 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -42,6 +42,14 @@
 
 #include "lio_struct.h"
 
+#define LIO_STQUEUE_FIRST_ENTRY(ptr, type, elem)	\
+	(type *)((char *)((ptr)->stqh_first) - offsetof(type, elem))
+
+#define lio_check_timeout(cur_time, chk_time) ((cur_time) > (chk_time))
+
+#define lio_uptime		\
+	(size_t)(rte_get_timer_cycles() / rte_get_timer_hz())
+
 struct lio_request_list {
 	uint32_t reqtype;
 	void *buf;
@@ -94,6 +102,71 @@ struct lio_soft_command *
 		       uint32_t ctxsize);
 void lio_free_soft_command(struct lio_soft_command *sc);
 
+/** Maximum ordered requests to process in every invocation of
+ *  lio_process_ordered_list(). The function will continue to process requests
+ *  as long as it can find one that has finished processing. If it keeps
+ *  finding requests that have completed, the function can run for ever. The
+ *  value defined here sets an upper limit on the number of requests it can
+ *  process before it returns control to the poll thread.
+ */
+#define LIO_MAX_ORD_REQS_TO_PROCESS	4096
+
+/** Error codes used in Octeon Host-Core communication.
+ *
+ *   31		16 15		0
+ *   ----------------------------
+ * |		|		|
+ *   ----------------------------
+ *   Error codes are 32-bit wide. The upper 16-bits, called Major Error Number,
+ *   are reserved to identify the group to which the error code belongs. The
+ *   lower 16-bits, called Minor Error Number, carry the actual code.
+ *
+ *   So error codes are (MAJOR NUMBER << 16)| MINOR_NUMBER.
+ */
+/** Status for a request.
+ *  If the request is successfully queued, the driver will return
+ *  a LIO_REQUEST_PENDING status. LIO_REQUEST_TIMEOUT is only returned by
+ *  the driver if the response for request failed to arrive before a
+ *  time-out period or if the request processing * got interrupted due to
+ *  a signal respectively.
+ */
+enum {
+	/** A value of 0x00000000 indicates no error i.e. success */
+	LIO_REQUEST_DONE	= 0x00000000,
+	/** (Major number: 0x0000; Minor Number: 0x0001) */
+	LIO_REQUEST_PENDING	= 0x00000001,
+	LIO_REQUEST_TIMEOUT	= 0x00000003,
+
+};
+
+/*------ Error codes used by firmware (bits 15..0 set by firmware */
+#define LIO_FIRMWARE_MAJOR_ERROR_CODE	 0x0001
+#define LIO_FIRMWARE_STATUS_CODE(status) \
+	((LIO_FIRMWARE_MAJOR_ERROR_CODE << 16) | (status))
+
+/** Initialize the response lists. The number of response lists to create is
+ *  given by count.
+ *  @param lio_dev - the lio device structure.
+ */
+void lio_setup_response_list(struct lio_device *lio_dev);
+
+/** Check the status of first entry in the ordered list. If the instruction at
+ *  that entry finished processing or has timed-out, the entry is cleaned.
+ *  @param lio_dev - the lio device structure.
+ *  @return 1 if the ordered list is empty, 0 otherwise.
+ */
+int lio_process_ordered_list(struct lio_device *lio_dev);
+
+static inline void
+lio_swap_8B_data(uint64_t *data, uint32_t blocks)
+{
+	while (blocks) {
+		*data = rte_cpu_to_be_64(*data);
+		blocks--;
+		data++;
+	}
+}
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 45f9ac9..add0e7d 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -213,6 +213,17 @@ struct lio_sriov_info {
 	uint32_t num_vfs;
 };
 
+/* Head of a response list */
+struct lio_response_list {
+	/** List structure to add delete pending entries to */
+	struct lio_stailq_head head;
+
+	/** A lock for this response list */
+	rte_spinlock_t lock;
+
+	rte_atomic64_t pending_req_count;
+};
+
 /* Structure to define the configuration attributes for each Input queue. */
 struct lio_iq_config {
 	/* Max number of IQs available */
@@ -289,6 +300,9 @@ struct lio_device {
 	/** The input instruction queues */
 	struct lio_instr_queue *instr_queue[LIO_MAX_POSSIBLE_INSTR_QUEUES];
 
+	/** The singly-linked tail queues of instruction response */
+	struct lio_response_list response_list;
+
 	struct lio_io_enable io_qmask;
 
 	struct lio_sriov_info sriov_info;
-- 
1.8.3.1

[PATCH v3 18/46] net/liquidio: add API to send packet to device

[Shijith Thotton]

Add API to send control and data packets to device. Request list keeps
track of host buffers to be freed till it reaches device.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  11 ++
 drivers/net/liquidio/lio_rxtx.c         | 180 ++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         | 291 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_struct.h       |   6 +
 4 files changed, 488 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 726ce6b..dc11406 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -76,6 +76,8 @@ enum lio_card_type {
 
 #define LIO_23XX_NAME "23xx"
 
+#define LIO_DEV_RUNNING		0xc
+
 #define LIO_NUM_DEF_TX_DESCS_CFG(cfg)					\
 		((cfg)->default_config->num_def_tx_descs)
 
@@ -93,6 +95,15 @@ enum lio_card_type {
 #define LIO_BASE_MINOR_VERSION		5
 #define LIO_BASE_MICRO_VERSION		1
 
+/** Tag types used by Octeon cores in its work. */
+enum octeon_tag_type {
+	OCTEON_ORDERED_TAG	= 0,
+	OCTEON_ATOMIC_TAG	= 1,
+};
+
+/* pre-defined host->NIC tag values */
+#define LIO_CONTROL	(0x11111110)
+
 /* Routines for reading and writing CSRs */
 #ifdef RTE_LIBRTE_LIO_DEBUG_REGS
 #define lio_write_csr(lio_dev, reg_off, value)				\
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index c61397f..7e2202c 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -207,6 +207,186 @@
 	lio_dev->num_iqs--;
 }
 
+static inline void
+lio_ring_doorbell(struct lio_device *lio_dev,
+		  struct lio_instr_queue *iq)
+{
+	if (rte_atomic64_read(&lio_dev->status) == LIO_DEV_RUNNING) {
+		rte_write32(iq->fill_cnt, iq->doorbell_reg);
+		/* make sure doorbell write goes through */
+		rte_wmb();
+		iq->fill_cnt = 0;
+	}
+}
+
+static inline void
+copy_cmd_into_iq(struct lio_instr_queue *iq, uint8_t *cmd)
+{
+	uint8_t *iqptr, cmdsize;
+
+	cmdsize = ((iq->iqcmd_64B) ? 64 : 32);
+	iqptr = iq->base_addr + (cmdsize * iq->host_write_index);
+
+	rte_memcpy(iqptr, cmd, cmdsize);
+}
+
+static inline struct lio_iq_post_status
+post_command2(struct lio_instr_queue *iq, uint8_t *cmd)
+{
+	struct lio_iq_post_status st;
+
+	st.status = LIO_IQ_SEND_OK;
+
+	/* This ensures that the read index does not wrap around to the same
+	 * position if queue gets full before Octeon could fetch any instr.
+	 */
+	if (rte_atomic64_read(&iq->instr_pending) >=
+			(int32_t)(iq->max_count - 1)) {
+		st.status = LIO_IQ_SEND_FAILED;
+		st.index = -1;
+		return st;
+	}
+
+	if (rte_atomic64_read(&iq->instr_pending) >=
+			(int32_t)(iq->max_count - 2))
+		st.status = LIO_IQ_SEND_STOP;
+
+	copy_cmd_into_iq(iq, cmd);
+
+	/* "index" is returned, host_write_index is modified. */
+	st.index = iq->host_write_index;
+	iq->host_write_index = lio_incr_index(iq->host_write_index, 1,
+					      iq->max_count);
+	iq->fill_cnt++;
+
+	/* Flush the command into memory. We need to be sure the data is in
+	 * memory before indicating that the instruction is pending.
+	 */
+	rte_wmb();
+
+	rte_atomic64_inc(&iq->instr_pending);
+
+	return st;
+}
+
+static inline void
+lio_add_to_request_list(struct lio_instr_queue *iq,
+			int idx, void *buf, int reqtype)
+{
+	iq->request_list[idx].buf = buf;
+	iq->request_list[idx].reqtype = reqtype;
+}
+
+static int
+lio_send_command(struct lio_device *lio_dev, uint32_t iq_no, void *cmd,
+		 void *buf, uint32_t datasize __rte_unused, uint32_t reqtype)
+{
+	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
+	struct lio_iq_post_status st;
+
+	rte_spinlock_lock(&iq->post_lock);
+
+	st = post_command2(iq, cmd);
+
+	if (st.status != LIO_IQ_SEND_FAILED) {
+		lio_add_to_request_list(iq, st.index, buf, reqtype);
+		lio_ring_doorbell(lio_dev, iq);
+	}
+
+	rte_spinlock_unlock(&iq->post_lock);
+
+	return st.status;
+}
+
+void
+lio_prepare_soft_command(struct lio_device *lio_dev,
+			 struct lio_soft_command *sc, uint8_t opcode,
+			 uint8_t subcode, uint32_t irh_ossp, uint64_t ossp0,
+			 uint64_t ossp1)
+{
+	struct octeon_instr_pki_ih3 *pki_ih3;
+	struct octeon_instr_ih3 *ih3;
+	struct octeon_instr_irh *irh;
+	struct octeon_instr_rdp *rdp;
+
+	RTE_ASSERT(opcode <= 15);
+	RTE_ASSERT(subcode <= 127);
+
+	ih3	  = (struct octeon_instr_ih3 *)&sc->cmd.cmd3.ih3;
+
+	ih3->pkind = lio_dev->instr_queue[sc->iq_no]->txpciq.s.pkind;
+
+	pki_ih3 = (struct octeon_instr_pki_ih3 *)&sc->cmd.cmd3.pki_ih3;
+
+	pki_ih3->w	= 1;
+	pki_ih3->raw	= 1;
+	pki_ih3->utag	= 1;
+	pki_ih3->uqpg	= lio_dev->instr_queue[sc->iq_no]->txpciq.s.use_qpg;
+	pki_ih3->utt	= 1;
+
+	pki_ih3->tag	= LIO_CONTROL;
+	pki_ih3->tagtype = OCTEON_ATOMIC_TAG;
+	pki_ih3->qpg	= lio_dev->instr_queue[sc->iq_no]->txpciq.s.qpg;
+	pki_ih3->pm	= 0x7;
+	pki_ih3->sl	= 8;
+
+	if (sc->datasize)
+		ih3->dlengsz = sc->datasize;
+
+	irh		= (struct octeon_instr_irh *)&sc->cmd.cmd3.irh;
+	irh->opcode	= opcode;
+	irh->subcode	= subcode;
+
+	/* opcode/subcode specific parameters (ossp) */
+	irh->ossp = irh_ossp;
+	sc->cmd.cmd3.ossp[0] = ossp0;
+	sc->cmd.cmd3.ossp[1] = ossp1;
+
+	if (sc->rdatasize) {
+		rdp = (struct octeon_instr_rdp *)&sc->cmd.cmd3.rdp;
+		rdp->pcie_port = lio_dev->pcie_port;
+		rdp->rlen      = sc->rdatasize;
+		irh->rflag = 1;
+		/* PKI IH3 */
+		ih3->fsz    = OCTEON_SOFT_CMD_RESP_IH3;
+	} else {
+		irh->rflag = 0;
+		/* PKI IH3 */
+		ih3->fsz    = OCTEON_PCI_CMD_O3;
+	}
+}
+
+int
+lio_send_soft_command(struct lio_device *lio_dev,
+		      struct lio_soft_command *sc)
+{
+	struct octeon_instr_ih3 *ih3;
+	struct octeon_instr_irh *irh;
+	uint32_t len = 0;
+
+	ih3 = (struct octeon_instr_ih3 *)&sc->cmd.cmd3.ih3;
+	if (ih3->dlengsz) {
+		RTE_ASSERT(sc->dmadptr);
+		sc->cmd.cmd3.dptr = sc->dmadptr;
+	}
+
+	irh = (struct octeon_instr_irh *)&sc->cmd.cmd3.irh;
+	if (irh->rflag) {
+		RTE_ASSERT(sc->dmarptr);
+		RTE_ASSERT(sc->status_word != NULL);
+		*sc->status_word = LIO_COMPLETION_WORD_INIT;
+		sc->cmd.cmd3.rptr = sc->dmarptr;
+	}
+
+	len = (uint32_t)ih3->dlengsz;
+
+	if (sc->wait_time)
+		sc->timeout = lio_uptime + sc->wait_time;
+
+	return lio_send_command(lio_dev, sc->iq_no, &sc->cmd, sc, len,
+				LIO_REQTYPE_SOFT_COMMAND);
+}
+
 int
 lio_setup_sc_buffer_pool(struct lio_device *lio_dev)
 {
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 0e7eb6a..9e2d967 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -50,11 +50,53 @@
 #define lio_uptime		\
 	(size_t)(rte_get_timer_cycles() / rte_get_timer_hz())
 
+#define LIO_IQ_SEND_OK		0
+#define LIO_IQ_SEND_STOP	1
+#define LIO_IQ_SEND_FAILED	-1
+
+/* conditions */
+#define LIO_REQTYPE_NONE		0
+#define LIO_REQTYPE_NORESP_NET		1
+#define LIO_REQTYPE_NORESP_NET_SG	2
+#define LIO_REQTYPE_SOFT_COMMAND	3
+
 struct lio_request_list {
 	uint32_t reqtype;
 	void *buf;
 };
 
+/*----------------------  INSTRUCTION FORMAT ----------------------------*/
+
+struct lio_instr3_64B {
+	/** Pointer where the input data is available. */
+	uint64_t dptr;
+
+	/** Instruction Header. */
+	uint64_t ih3;
+
+	/** Instruction Header. */
+	uint64_t pki_ih3;
+
+	/** Input Request Header. */
+	uint64_t irh;
+
+	/** opcode/subcode specific parameters */
+	uint64_t ossp[2];
+
+	/** Return Data Parameters */
+	uint64_t rdp;
+
+	/** Pointer where the response for a RAW mode packet will be written
+	 *  by Octeon.
+	 */
+	uint64_t rptr;
+
+};
+
+union lio_instr_64B {
+	struct lio_instr3_64B cmd3;
+};
+
 /** The size of each buffer in soft command buffer pool */
 #define LIO_SOFT_COMMAND_BUFFER_SIZE	1536
 
@@ -67,6 +109,9 @@ struct lio_soft_command {
 	uint64_t dma_addr;
 	uint32_t size;
 
+	/** Command and return status */
+	union lio_instr_64B cmd;
+
 #define LIO_COMPLETION_WORD_INIT	0xffffffffffffffffULL
 	uint64_t *status_word;
 
@@ -93,6 +138,230 @@ struct lio_soft_command {
 	struct rte_mbuf *mbuf;
 };
 
+struct lio_iq_post_status {
+	int status;
+	int index;
+};
+
+/*   wqe
+ *  ---------------  0
+ * |  wqe  word0-3 |
+ *  ---------------  32
+ * |    PCI IH     |
+ *  ---------------  40
+ * |     RPTR      |
+ *  ---------------  48
+ * |    PCI IRH    |
+ *  ---------------  56
+ * |    OCTEON_CMD |
+ *  ---------------  64
+ * | Addtl 8-BData |
+ * |               |
+ *  ---------------
+ */
+
+union octeon_cmd {
+	uint64_t cmd64;
+
+	struct	{
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t cmd : 5;
+
+		uint64_t more : 6; /* How many udd words follow the command */
+
+		uint64_t reserved : 29;
+
+		uint64_t param1 : 16;
+
+		uint64_t param2 : 8;
+
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+
+		uint64_t param2 : 8;
+
+		uint64_t param1 : 16;
+
+		uint64_t reserved : 29;
+
+		uint64_t more : 6;
+
+		uint64_t cmd : 5;
+
+#endif
+	} s;
+};
+
+#define OCTEON_CMD_SIZE (sizeof(union octeon_cmd))
+
+/* Instruction Header */
+struct octeon_instr_ih3 {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+
+	/** Reserved3 */
+	uint64_t reserved3 : 1;
+
+	/** Gather indicator 1=gather*/
+	uint64_t gather : 1;
+
+	/** Data length OR no. of entries in gather list */
+	uint64_t dlengsz : 14;
+
+	/** Front Data size */
+	uint64_t fsz : 6;
+
+	/** Reserved2 */
+	uint64_t reserved2 : 4;
+
+	/** PKI port kind - PKIND */
+	uint64_t pkind : 6;
+
+	/** Reserved1 */
+	uint64_t reserved1 : 32;
+
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	/** Reserved1 */
+	uint64_t reserved1 : 32;
+
+	/** PKI port kind - PKIND */
+	uint64_t pkind : 6;
+
+	/** Reserved2 */
+	uint64_t reserved2 : 4;
+
+	/** Front Data size */
+	uint64_t fsz : 6;
+
+	/** Data length OR no. of entries in gather list */
+	uint64_t dlengsz : 14;
+
+	/** Gather indicator 1=gather*/
+	uint64_t gather : 1;
+
+	/** Reserved3 */
+	uint64_t reserved3 : 1;
+
+#endif
+};
+
+/* PKI Instruction Header(PKI IH) */
+struct octeon_instr_pki_ih3 {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+
+	/** Wider bit */
+	uint64_t w : 1;
+
+	/** Raw mode indicator 1 = RAW */
+	uint64_t raw : 1;
+
+	/** Use Tag */
+	uint64_t utag : 1;
+
+	/** Use QPG */
+	uint64_t uqpg : 1;
+
+	/** Reserved2 */
+	uint64_t reserved2 : 1;
+
+	/** Parse Mode */
+	uint64_t pm : 3;
+
+	/** Skip Length */
+	uint64_t sl : 8;
+
+	/** Use Tag Type */
+	uint64_t utt : 1;
+
+	/** Tag type */
+	uint64_t tagtype : 2;
+
+	/** Reserved1 */
+	uint64_t reserved1 : 2;
+
+	/** QPG Value */
+	uint64_t qpg : 11;
+
+	/** Tag Value */
+	uint64_t tag : 32;
+
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+
+	/** Tag Value */
+	uint64_t tag : 32;
+
+	/** QPG Value */
+	uint64_t qpg : 11;
+
+	/** Reserved1 */
+	uint64_t reserved1 : 2;
+
+	/** Tag type */
+	uint64_t tagtype : 2;
+
+	/** Use Tag Type */
+	uint64_t utt : 1;
+
+	/** Skip Length */
+	uint64_t sl : 8;
+
+	/** Parse Mode */
+	uint64_t pm : 3;
+
+	/** Reserved2 */
+	uint64_t reserved2 : 1;
+
+	/** Use QPG */
+	uint64_t uqpg : 1;
+
+	/** Use Tag */
+	uint64_t utag : 1;
+
+	/** Raw mode indicator 1 = RAW */
+	uint64_t raw : 1;
+
+	/** Wider bit */
+	uint64_t w : 1;
+#endif
+};
+
+/** Input Request Header */
+struct octeon_instr_irh {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	uint64_t opcode : 4;
+	uint64_t rflag : 1;
+	uint64_t subcode : 7;
+	uint64_t vlan : 12;
+	uint64_t priority : 3;
+	uint64_t reserved : 5;
+	uint64_t ossp : 32; /* opcode/subcode specific parameters */
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	uint64_t ossp : 32; /* opcode/subcode specific parameters */
+	uint64_t reserved : 5;
+	uint64_t priority : 3;
+	uint64_t vlan : 12;
+	uint64_t subcode : 7;
+	uint64_t rflag : 1;
+	uint64_t opcode : 4;
+#endif
+};
+
+/* pkiih3 + irh + ossp[0] + ossp[1] + rdp + rptr = 40 bytes */
+#define OCTEON_SOFT_CMD_RESP_IH3	(40 + 8)
+/* pki_h3 + irh + ossp[0] + ossp[1] = 32 bytes */
+#define OCTEON_PCI_CMD_O3		(24 + 8)
+
+/** Return Data Parameters */
+struct octeon_instr_rdp {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	uint64_t reserved : 49;
+	uint64_t pcie_port : 3;
+	uint64_t rlen : 12;
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	uint64_t rlen : 12;
+	uint64_t pcie_port : 3;
+	uint64_t reserved : 49;
+#endif
+};
+
 int lio_setup_sc_buffer_pool(struct lio_device *lio_dev);
 void lio_free_sc_buffer_pool(struct lio_device *lio_dev);
 
@@ -100,6 +369,13 @@ struct lio_soft_command *
 lio_alloc_soft_command(struct lio_device *lio_dev,
 		       uint32_t datasize, uint32_t rdatasize,
 		       uint32_t ctxsize);
+void lio_prepare_soft_command(struct lio_device *lio_dev,
+			      struct lio_soft_command *sc,
+			      uint8_t opcode, uint8_t subcode,
+			      uint32_t irh_ossp, uint64_t ossp0,
+			      uint64_t ossp1);
+int lio_send_soft_command(struct lio_device *lio_dev,
+			  struct lio_soft_command *sc);
 void lio_free_soft_command(struct lio_soft_command *sc);
 
 /** Maximum ordered requests to process in every invocation of
@@ -167,6 +443,21 @@ enum {
 	}
 }
 
+/* Macro to increment index.
+ * Index is incremented by count; if the sum exceeds
+ * max, index is wrapped-around to the start.
+ */
+static inline uint32_t
+lio_incr_index(uint32_t index, uint32_t count, uint32_t max)
+{
+	if ((index + count) >= max)
+		index = index + count - max;
+	else
+		index += count;
+
+	return index;
+}
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index add0e7d..f16571b 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -288,6 +288,12 @@ struct lio_device {
 	uint16_t pf_num;
 	uint16_t vf_num;
 
+	/** This device's PCIe port used for traffic. */
+	uint16_t pcie_port;
+
+	/** The state of this device */
+	rte_atomic64_t status;
+
 	uint8_t *hw_addr;
 
 	struct lio_fn_list fn_list;
-- 
1.8.3.1

[PATCH v3 19/46] net/liquidio: add API to configure device

[Shijith Thotton]

Add API to configure device and initialize ethernet device operations.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  14 +++
 drivers/net/liquidio/lio_ethdev.c       | 169 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  36 +++++++
 drivers/net/liquidio/lio_struct.h       |  72 ++++++++++++++
 4 files changed, 291 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index dc11406..3201dc5 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -95,6 +95,8 @@ enum lio_card_type {
 #define LIO_BASE_MINOR_VERSION		5
 #define LIO_BASE_MICRO_VERSION		1
 
+#define LIO_FW_VERSION_LENGTH		32
+
 /** Tag types used by Octeon cores in its work. */
 enum octeon_tag_type {
 	OCTEON_ORDERED_TAG	= 0,
@@ -104,6 +106,18 @@ enum octeon_tag_type {
 /* pre-defined host->NIC tag values */
 #define LIO_CONTROL	(0x11111110)
 
+/* used for NIC operations */
+#define LIO_OPCODE	1
+
+/** LIO_OPCODE subcodes */
+/* This subcode is sent by core PCI driver to indicate cores are ready. */
+#define LIO_OPCODE_IF_CFG		0x09
+
+/* Interface flags communicated between host driver and core app. */
+enum lio_ifflags {
+	LIO_IFFLAG_UNICAST	= 0x10
+};
+
 /* Routines for reading and writing CSRs */
 #ifdef RTE_LIBRTE_LIO_DEBUG_REGS
 #define lio_write_csr(lio_dev, reg_off, value)				\
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 97a7369..194b096 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -41,6 +41,166 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+static uint64_t
+lio_hweight64(uint64_t w)
+{
+	uint64_t res = w - ((w >> 1) & 0x5555555555555555ul);
+
+	res =
+	    (res & 0x3333333333333333ul) + ((res >> 2) & 0x3333333333333333ul);
+	res = (res + (res >> 4)) & 0x0F0F0F0F0F0F0F0Ful;
+	res = res + (res >> 8);
+	res = res + (res >> 16);
+
+	return (res + (res >> 32)) & 0x00000000000000FFul;
+}
+
+static int lio_dev_configure(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
+	int retval, num_iqueues, num_oqueues;
+	uint8_t mac[ETHER_ADDR_LEN], i;
+	struct lio_if_cfg_resp *resp;
+	struct lio_soft_command *sc;
+	union lio_if_cfg if_cfg;
+	uint32_t resp_size;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* Re-configuring firmware not supported.
+	 * Can't change tx/rx queues per port from initial value.
+	 */
+	if (lio_dev->port_configured) {
+		if ((lio_dev->nb_rx_queues != eth_dev->data->nb_rx_queues) ||
+		    (lio_dev->nb_tx_queues != eth_dev->data->nb_tx_queues)) {
+			lio_dev_err(lio_dev,
+				    "rxq/txq re-conf not supported. Restart application with new value.\n");
+			return -ENOTSUP;
+		}
+		return 0;
+	}
+
+	lio_dev->nb_rx_queues = eth_dev->data->nb_rx_queues;
+	lio_dev->nb_tx_queues = eth_dev->data->nb_tx_queues;
+
+	resp_size = sizeof(struct lio_if_cfg_resp);
+	sc = lio_alloc_soft_command(lio_dev, 0, resp_size, 0);
+	if (sc == NULL)
+		return -ENOMEM;
+
+	resp = (struct lio_if_cfg_resp *)sc->virtrptr;
+
+	/* Firmware doesn't have capability to reconfigure the queues,
+	 * Claim all queues, and use as many required
+	 */
+	if_cfg.if_cfg64 = 0;
+	if_cfg.s.num_iqueues = lio_dev->nb_tx_queues;
+	if_cfg.s.num_oqueues = lio_dev->nb_rx_queues;
+	if_cfg.s.base_queue = 0;
+
+	if_cfg.s.gmx_port_id = lio_dev->pf_num;
+
+	lio_prepare_soft_command(lio_dev, sc, LIO_OPCODE,
+				 LIO_OPCODE_IF_CFG, 0,
+				 if_cfg.if_cfg64, 0);
+
+	/* Setting wait time in seconds */
+	sc->wait_time = LIO_MAX_CMD_TIMEOUT / 1000;
+
+	retval = lio_send_soft_command(lio_dev, sc);
+	if (retval == LIO_IQ_SEND_FAILED) {
+		lio_dev_err(lio_dev, "iq/oq config failed status: %x\n",
+			    retval);
+		/* Soft instr is freed by driver in case of failure. */
+		goto nic_config_fail;
+	}
+
+	/* Sleep on a wait queue till the cond flag indicates that the
+	 * response arrived or timed-out.
+	 */
+	while ((*sc->status_word == LIO_COMPLETION_WORD_INIT) && --timeout) {
+		lio_process_ordered_list(lio_dev);
+		rte_delay_ms(1);
+	}
+
+	retval = resp->status;
+	if (retval) {
+		lio_dev_err(lio_dev, "iq/oq config failed\n");
+		goto nic_config_fail;
+	}
+
+	lio_swap_8B_data((uint64_t *)(&resp->cfg_info),
+			 sizeof(struct octeon_if_cfg_info) >> 3);
+
+	num_iqueues = lio_hweight64(resp->cfg_info.iqmask);
+	num_oqueues = lio_hweight64(resp->cfg_info.oqmask);
+
+	if (!(num_iqueues) || !(num_oqueues)) {
+		lio_dev_err(lio_dev,
+			    "Got bad iqueues (%016lx) or oqueues (%016lx) from firmware.\n",
+			    (unsigned long)resp->cfg_info.iqmask,
+			    (unsigned long)resp->cfg_info.oqmask);
+		goto nic_config_fail;
+	}
+
+	lio_dev_dbg(lio_dev,
+		    "interface %d, iqmask %016lx, oqmask %016lx, numiqueues %d, numoqueues %d\n",
+		    eth_dev->data->port_id,
+		    (unsigned long)resp->cfg_info.iqmask,
+		    (unsigned long)resp->cfg_info.oqmask,
+		    num_iqueues, num_oqueues);
+
+	lio_dev->linfo.num_rxpciq = num_oqueues;
+	lio_dev->linfo.num_txpciq = num_iqueues;
+
+	for (i = 0; i < num_oqueues; i++) {
+		lio_dev->linfo.rxpciq[i].rxpciq64 =
+		    resp->cfg_info.linfo.rxpciq[i].rxpciq64;
+		lio_dev_dbg(lio_dev, "index %d OQ %d\n",
+			    i, lio_dev->linfo.rxpciq[i].s.q_no);
+	}
+
+	for (i = 0; i < num_iqueues; i++) {
+		lio_dev->linfo.txpciq[i].txpciq64 =
+		    resp->cfg_info.linfo.txpciq[i].txpciq64;
+		lio_dev_dbg(lio_dev, "index %d IQ %d\n",
+			    i, lio_dev->linfo.txpciq[i].s.q_no);
+	}
+
+	lio_dev->linfo.hw_addr = resp->cfg_info.linfo.hw_addr;
+	lio_dev->linfo.gmxport = resp->cfg_info.linfo.gmxport;
+	lio_dev->linfo.link.link_status64 =
+			resp->cfg_info.linfo.link.link_status64;
+
+	/* 64-bit swap required on LE machines */
+	lio_swap_8B_data(&lio_dev->linfo.hw_addr, 1);
+	for (i = 0; i < ETHER_ADDR_LEN; i++)
+		mac[i] = *((uint8_t *)(((uint8_t *)&lio_dev->linfo.hw_addr) +
+				       2 + i));
+
+	/* Copy the permanent MAC address */
+	ether_addr_copy((struct ether_addr *)mac, &eth_dev->data->mac_addrs[0]);
+
+	lio_dev->port_configured = 1;
+
+	lio_free_soft_command(sc);
+
+	return 0;
+
+nic_config_fail:
+	lio_dev_err(lio_dev, "Failed retval %d\n", retval);
+	lio_free_soft_command(sc);
+	lio_free_instr_queue0(lio_dev);
+
+	return -ENODEV;
+}
+
+/* Define our ethernet definitions */
+static const struct eth_dev_ops liovf_eth_dev_ops = {
+	.dev_configure		= lio_dev_configure,
+};
+
 static void
 lio_check_pf_hs_response(void *lio_dev)
 {
@@ -215,13 +375,22 @@
 		return -EINVAL;
 	}
 
+	eth_dev->dev_ops = &liovf_eth_dev_ops;
 	eth_dev->data->mac_addrs = rte_zmalloc("lio", ETHER_ADDR_LEN, 0);
 	if (eth_dev->data->mac_addrs == NULL) {
 		lio_dev_err(lio_dev,
 			    "MAC addresses memory allocation failed\n");
+		eth_dev->dev_ops = NULL;
 		return -ENOMEM;
 	}
 
+	rte_atomic64_set(&lio_dev->status, LIO_DEV_RUNNING);
+	rte_wmb();
+
+	lio_dev->port_configured = 0;
+	/* Always allow unicast packets */
+	lio_dev->ifflags |= LIO_IFFLAG_UNICAST;
+
 	return 0;
 }
 
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 76c9072..22e3d83 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -36,5 +36,41 @@
 
 #include <stdint.h>
 
+#include "lio_struct.h"
+
+#define LIO_MAX_CMD_TIMEOUT     10000 /* 10000ms (10s) */
+
 #define LIO_DEV(_eth_dev)		((_eth_dev)->data->dev_private)
+
+struct octeon_if_cfg_info {
+	uint64_t iqmask;	/** mask for IQs enabled for the port */
+	uint64_t oqmask;	/** mask for OQs enabled for the port */
+	struct octeon_link_info linfo; /** initial link information */
+	char lio_firmware_version[LIO_FW_VERSION_LENGTH];
+};
+
+union lio_if_cfg {
+	uint64_t if_cfg64;
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t base_queue : 16;
+		uint64_t num_iqueues : 16;
+		uint64_t num_oqueues : 16;
+		uint64_t gmx_port_id : 8;
+		uint64_t vf_id : 8;
+#else
+		uint64_t vf_id : 8;
+		uint64_t gmx_port_id : 8;
+		uint64_t num_oqueues : 16;
+		uint64_t num_iqueues : 16;
+		uint64_t base_queue : 16;
+#endif
+	} s;
+};
+
+struct lio_if_cfg_resp {
+	uint64_t rh;
+	struct octeon_if_cfg_info cfg_info;
+	uint64_t status;
+};
 #endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index f16571b..48c4cae 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -274,6 +274,75 @@ struct lio_config {
 	int def_rx_buf_size;
 };
 
+/** Status of a RGMII Link on Octeon as seen by core driver. */
+union octeon_link_status {
+	uint64_t link_status64;
+
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t duplex : 8;
+		uint64_t mtu : 16;
+		uint64_t speed : 16;
+		uint64_t link_up : 1;
+		uint64_t autoneg : 1;
+		uint64_t if_mode : 5;
+		uint64_t pause : 1;
+		uint64_t flashing : 1;
+		uint64_t reserved : 15;
+#else
+		uint64_t reserved : 15;
+		uint64_t flashing : 1;
+		uint64_t pause : 1;
+		uint64_t if_mode : 5;
+		uint64_t autoneg : 1;
+		uint64_t link_up : 1;
+		uint64_t speed : 16;
+		uint64_t mtu : 16;
+		uint64_t duplex : 8;
+#endif
+	} s;
+};
+
+/** The rxpciq info passed to host from the firmware */
+union octeon_rxpciq {
+	uint64_t rxpciq64;
+
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t q_no : 8;
+		uint64_t reserved : 56;
+#else
+		uint64_t reserved : 56;
+		uint64_t q_no : 8;
+#endif
+	} s;
+};
+
+/** Information for a OCTEON ethernet interface shared between core & host. */
+struct octeon_link_info {
+	union octeon_link_status link;
+	uint64_t hw_addr;
+
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	uint64_t gmxport : 16;
+	uint64_t macaddr_is_admin_assigned : 1;
+	uint64_t vlan_is_admin_assigned : 1;
+	uint64_t rsvd : 30;
+	uint64_t num_txpciq : 8;
+	uint64_t num_rxpciq : 8;
+#else
+	uint64_t num_rxpciq : 8;
+	uint64_t num_txpciq : 8;
+	uint64_t rsvd : 30;
+	uint64_t vlan_is_admin_assigned : 1;
+	uint64_t macaddr_is_admin_assigned : 1;
+	uint64_t gmxport : 16;
+#endif
+
+	union octeon_txpciq txpciq[LIO_MAX_IOQS_PER_IF];
+	union octeon_rxpciq rxpciq[LIO_MAX_IOQS_PER_IF];
+};
+
 /* -----------------------  THE LIO DEVICE  --------------------------- */
 /** The lio device.
  *  Each lio device has this structure to represent all its
@@ -294,6 +363,8 @@ struct lio_device {
 	/** The state of this device */
 	rte_atomic64_t status;
 
+	struct octeon_link_info linfo;
+
 	uint8_t *hw_addr;
 
 	struct lio_fn_list fn_list;
@@ -324,6 +395,7 @@ struct lio_device {
 
 	struct rte_eth_dev      *eth_dev;
 
+	uint64_t ifflags;
 	uint8_t max_rx_queues;
 	uint8_t max_tx_queues;
 	uint8_t nb_rx_queues;
-- 
1.8.3.1

[PATCH v3 20/46] net/liquidio: add API to setup Rx queue

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |   3 +
 drivers/net/liquidio/lio_ethdev.c       |  67 +++++++++++++
 drivers/net/liquidio/lio_rxtx.c         | 168 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         |  56 +++++++++++
 drivers/net/liquidio/lio_struct.h       | 149 ++++++++++++++++++++++++++++
 5 files changed, 443 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 3201dc5..35d59fd 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -78,6 +78,8 @@ enum lio_card_type {
 
 #define LIO_DEV_RUNNING		0xc
 
+#define LIO_OQ_REFILL_THRESHOLD_CFG(cfg)				\
+		((cfg)->default_config->oq.refill_threshold)
 #define LIO_NUM_DEF_TX_DESCS_CFG(cfg)					\
 		((cfg)->default_config->num_def_tx_descs)
 
@@ -89,6 +91,7 @@ enum lio_card_type {
 #define LIO_MAX_INSTR_QUEUES(lio_dev)		CN23XX_MAX_RINGS_PER_VF
 
 #define LIO_MAX_POSSIBLE_INSTR_QUEUES		CN23XX_MAX_INPUT_QUEUES
+#define LIO_MAX_POSSIBLE_OUTPUT_QUEUES		CN23XX_MAX_OUTPUT_QUEUES
 
 #define LIO_DEVICE_NAME_LEN		32
 #define LIO_BASE_MAJOR_VERSION		1
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 194b096..a93fa4a 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -55,6 +55,72 @@
 	return (res + (res >> 32)) & 0x00000000000000FFul;
 }
 
+/**
+ * Setup our receive queue/ringbuffer. This is the
+ * queue the Octeon uses to send us packets and
+ * responses. We are given a memory pool for our
+ * packet buffers that are used to populate the receive
+ * queue.
+ *
+ * @param eth_dev
+ *    Pointer to the structure rte_eth_dev
+ * @param q_no
+ *    Queue number
+ * @param num_rx_descs
+ *    Number of entries in the queue
+ * @param socket_id
+ *    Where to allocate memory
+ * @param rx_conf
+ *    Pointer to the struction rte_eth_rxconf
+ * @param mp
+ *    Pointer to the packet pool
+ *
+ * @return
+ *    - On success, return 0
+ *    - On failure, return -1
+ */
+static int
+lio_dev_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t q_no,
+		       uint16_t num_rx_descs, unsigned int socket_id,
+		       const struct rte_eth_rxconf *rx_conf __rte_unused,
+		       struct rte_mempool *mp)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct rte_pktmbuf_pool_private *mbp_priv;
+	uint32_t fw_mapped_oq;
+	uint16_t buf_size;
+
+	if (q_no >= lio_dev->nb_rx_queues) {
+		lio_dev_err(lio_dev, "Invalid rx queue number %u\n", q_no);
+		return -EINVAL;
+	}
+
+	lio_dev_dbg(lio_dev, "setting up rx queue %u\n", q_no);
+
+	fw_mapped_oq = lio_dev->linfo.rxpciq[q_no].s.q_no;
+
+	if ((lio_dev->droq[fw_mapped_oq]) &&
+	    (num_rx_descs != lio_dev->droq[fw_mapped_oq]->max_count)) {
+		lio_dev_err(lio_dev,
+			    "Reconfiguring Rx descs not supported. Configure descs to same value %u or restart application\n",
+			    lio_dev->droq[fw_mapped_oq]->max_count);
+		return -ENOTSUP;
+	}
+
+	mbp_priv = rte_mempool_get_priv(mp);
+	buf_size = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM;
+
+	if (lio_setup_droq(lio_dev, fw_mapped_oq, num_rx_descs, buf_size, mp,
+			   socket_id)) {
+		lio_dev_err(lio_dev, "droq allocation failed\n");
+		return -1;
+	}
+
+	eth_dev->data->rx_queues[q_no] = lio_dev->droq[fw_mapped_oq];
+
+	return 0;
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -199,6 +265,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 /* Define our ethernet definitions */
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
+	.rx_queue_setup		= lio_dev_rx_queue_setup,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 7e2202c..942fe9b 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -63,6 +63,174 @@
 }
 
 /**
+ *  Frees the space for descriptor ring for the droq.
+ *
+ *  @param lio_dev	- pointer to the lio device structure
+ *  @param q_no		- droq no.
+ */
+static void
+lio_delete_droq(struct lio_device *lio_dev, uint32_t q_no)
+{
+	struct lio_droq *droq = lio_dev->droq[q_no];
+
+	lio_dev_dbg(lio_dev, "OQ[%d]\n", q_no);
+
+	rte_free(droq->recv_buf_list);
+	droq->recv_buf_list = NULL;
+	lio_dma_zone_free(lio_dev, droq->info_mz);
+	lio_dma_zone_free(lio_dev, droq->desc_ring_mz);
+
+	memset(droq, 0, LIO_DROQ_SIZE);
+}
+
+static void *
+lio_alloc_info_buffer(struct lio_device *lio_dev,
+		      struct lio_droq *droq, unsigned int socket_id)
+{
+	droq->info_mz = rte_eth_dma_zone_reserve(lio_dev->eth_dev,
+						 "info_list", droq->q_no,
+						 (droq->max_count *
+							LIO_DROQ_INFO_SIZE),
+						 RTE_CACHE_LINE_SIZE,
+						 socket_id);
+
+	if (droq->info_mz == NULL)
+		return NULL;
+
+	droq->info_list_dma = droq->info_mz->phys_addr;
+	droq->info_alloc_size = droq->info_mz->len;
+	droq->info_base_addr = (size_t)droq->info_mz->addr;
+
+	return droq->info_mz->addr;
+}
+
+/**
+ *  Allocates space for the descriptor ring for the droq and
+ *  sets the base addr, num desc etc in Octeon registers.
+ *
+ * @param lio_dev	- pointer to the lio device structure
+ * @param q_no		- droq no.
+ * @param app_ctx	- pointer to application context
+ * @return Success: 0	Failure: -1
+ */
+static int
+lio_init_droq(struct lio_device *lio_dev, uint32_t q_no,
+	      uint32_t num_descs, uint32_t desc_size,
+	      struct rte_mempool *mpool, unsigned int socket_id)
+{
+	uint32_t c_refill_threshold;
+	uint32_t desc_ring_size;
+	struct lio_droq *droq;
+
+	lio_dev_dbg(lio_dev, "OQ[%d]\n", q_no);
+
+	droq = lio_dev->droq[q_no];
+	droq->lio_dev = lio_dev;
+	droq->q_no = q_no;
+	droq->mpool = mpool;
+
+	c_refill_threshold = LIO_OQ_REFILL_THRESHOLD_CFG(lio_dev);
+
+	droq->max_count = num_descs;
+	droq->buffer_size = desc_size;
+
+	desc_ring_size = droq->max_count * LIO_DROQ_DESC_SIZE;
+	droq->desc_ring_mz = rte_eth_dma_zone_reserve(lio_dev->eth_dev,
+						      "droq", q_no,
+						      desc_ring_size,
+						      RTE_CACHE_LINE_SIZE,
+						      socket_id);
+
+	if (droq->desc_ring_mz == NULL) {
+		lio_dev_err(lio_dev,
+			    "Output queue %d ring alloc failed\n", q_no);
+		return -1;
+	}
+
+	droq->desc_ring_dma = droq->desc_ring_mz->phys_addr;
+	droq->desc_ring = (struct lio_droq_desc *)droq->desc_ring_mz->addr;
+
+	lio_dev_dbg(lio_dev, "droq[%d]: desc_ring: virt: 0x%p, dma: %lx\n",
+		    q_no, droq->desc_ring, (unsigned long)droq->desc_ring_dma);
+	lio_dev_dbg(lio_dev, "droq[%d]: num_desc: %d\n", q_no,
+		    droq->max_count);
+
+	droq->info_list = lio_alloc_info_buffer(lio_dev, droq, socket_id);
+	if (droq->info_list == NULL) {
+		lio_dev_err(lio_dev, "Cannot allocate memory for info list.\n");
+		goto init_droq_fail;
+	}
+
+	droq->recv_buf_list = rte_zmalloc_socket("recv_buf_list",
+						 (droq->max_count *
+							LIO_DROQ_RECVBUF_SIZE),
+						 RTE_CACHE_LINE_SIZE,
+						 socket_id);
+	if (droq->recv_buf_list == NULL) {
+		lio_dev_err(lio_dev,
+			    "Output queue recv buf list alloc failed\n");
+		goto init_droq_fail;
+	}
+
+	droq->refill_threshold = c_refill_threshold;
+
+	rte_spinlock_init(&droq->lock);
+
+	lio_dev->io_qmask.oq |= (1ULL << q_no);
+
+	return 0;
+
+init_droq_fail:
+	lio_delete_droq(lio_dev, q_no);
+
+	return -1;
+}
+
+int
+lio_setup_droq(struct lio_device *lio_dev, int oq_no, int num_descs,
+	       int desc_size, struct rte_mempool *mpool, unsigned int socket_id)
+{
+	struct lio_droq *droq;
+
+	PMD_INIT_FUNC_TRACE();
+
+	if (lio_dev->droq[oq_no]) {
+		lio_dev_dbg(lio_dev, "Droq %d in use\n", oq_no);
+		return 0;
+	}
+
+	/* Allocate the DS for the new droq. */
+	droq = rte_zmalloc_socket("ethdev RX queue", sizeof(*droq),
+				  RTE_CACHE_LINE_SIZE, socket_id);
+	if (droq == NULL)
+		return -ENOMEM;
+
+	lio_dev->droq[oq_no] = droq;
+
+	/* Initialize the Droq */
+	if (lio_init_droq(lio_dev, oq_no, num_descs, desc_size, mpool,
+			  socket_id)) {
+		lio_dev_err(lio_dev, "Droq[%u] Initialization Failed\n", oq_no);
+		rte_free(lio_dev->droq[oq_no]);
+		lio_dev->droq[oq_no] = NULL;
+		return -ENOMEM;
+	}
+
+	lio_dev->num_oqs++;
+
+	lio_dev_dbg(lio_dev, "Total number of OQ: %d\n", lio_dev->num_oqs);
+
+	/* Send credit for octeon output queues. credits are always
+	 * sent after the output queue is enabled.
+	 */
+	rte_write32(lio_dev->droq[oq_no]->max_count,
+		    lio_dev->droq[oq_no]->pkts_credit_reg);
+	rte_wmb();
+
+	return 0;
+}
+
+/**
  *  lio_init_instr_queue()
  *  @param lio_dev	- pointer to the lio device structure.
  *  @param txpciq	- queue to be initialized.
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 9e2d967..05f4704 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -50,6 +50,58 @@
 #define lio_uptime		\
 	(size_t)(rte_get_timer_cycles() / rte_get_timer_hz())
 
+/** Descriptor format.
+ *  The descriptor ring is made of descriptors which have 2 64-bit values:
+ *  -# Physical (bus) address of the data buffer.
+ *  -# Physical (bus) address of a lio_droq_info structure.
+ *  The device DMA's incoming packets and its information at the address
+ *  given by these descriptor fields.
+ */
+struct lio_droq_desc {
+	/** The buffer pointer */
+	uint64_t buffer_ptr;
+
+	/** The Info pointer */
+	uint64_t info_ptr;
+};
+
+#define LIO_DROQ_DESC_SIZE	(sizeof(struct lio_droq_desc))
+
+/** Information about packet DMA'ed by Octeon.
+ *  The format of the information available at Info Pointer after Octeon
+ *  has posted a packet. Not all descriptors have valid information. Only
+ *  the Info field of the first descriptor for a packet has information
+ *  about the packet.
+ */
+struct lio_droq_info {
+	/** The Output Receive Header. */
+	union octeon_rh rh;
+
+	/** The Length of the packet. */
+	uint64_t length;
+};
+
+#define LIO_DROQ_INFO_SIZE	(sizeof(struct lio_droq_info))
+
+/** Pointer to data buffer.
+ *  Driver keeps a pointer to the data buffer that it made available to
+ *  the Octeon device. Since the descriptor ring keeps physical (bus)
+ *  addresses, this field is required for the driver to keep track of
+ *  the virtual address pointers.
+ */
+struct lio_recv_buffer {
+	/** Packet buffer, including meta data. */
+	void *buffer;
+
+	/** Data in the packet buffer. */
+	uint8_t *data;
+
+};
+
+#define LIO_DROQ_RECVBUF_SIZE	(sizeof(struct lio_recv_buffer))
+
+#define LIO_DROQ_SIZE		(sizeof(struct lio_droq))
+
 #define LIO_IQ_SEND_OK		0
 #define LIO_IQ_SEND_STOP	1
 #define LIO_IQ_SEND_FAILED	-1
@@ -458,6 +510,10 @@ enum {
 	return index;
 }
 
+int lio_setup_droq(struct lio_device *lio_dev, int q_no, int num_descs,
+		   int desc_size, struct rte_mempool *mpool,
+		   unsigned int socket_id);
+
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 48c4cae..3df79e1 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -56,6 +56,150 @@ struct lio_version {
 	uint16_t reserved;
 };
 
+/** The Descriptor Ring Output Queue structure.
+ *  This structure has all the information required to implement a
+ *  DROQ.
+ */
+struct lio_droq {
+	/** A spinlock to protect access to this ring. */
+	rte_spinlock_t lock;
+
+	uint32_t q_no;
+
+	uint32_t pkt_count;
+
+	struct lio_device *lio_dev;
+
+	/** The 8B aligned descriptor ring starts at this address. */
+	struct lio_droq_desc *desc_ring;
+
+	/** Index in the ring where the driver should read the next packet */
+	uint32_t read_idx;
+
+	/** Index in the ring where Octeon will write the next packet */
+	uint32_t write_idx;
+
+	/** Index in the ring where the driver will refill the descriptor's
+	 * buffer
+	 */
+	uint32_t refill_idx;
+
+	/** Packets pending to be processed */
+	rte_atomic64_t pkts_pending;
+
+	/** Number of  descriptors in this ring. */
+	uint32_t max_count;
+
+	/** The number of descriptors pending refill. */
+	uint32_t refill_count;
+
+	uint32_t refill_threshold;
+
+	/** The 8B aligned info ptrs begin from this address. */
+	struct lio_droq_info *info_list;
+
+	/** The receive buffer list. This list has the virtual addresses of the
+	 *  buffers.
+	 */
+	struct lio_recv_buffer *recv_buf_list;
+
+	/** The size of each buffer pointed by the buffer pointer. */
+	uint32_t buffer_size;
+
+	/** Pointer to the mapped packet credit register.
+	 *  Host writes number of info/buffer ptrs available to this register
+	 */
+	void *pkts_credit_reg;
+
+	/** Pointer to the mapped packet sent register.
+	 *  Octeon writes the number of packets DMA'ed to host memory
+	 *  in this register.
+	 */
+	void *pkts_sent_reg;
+
+	/** DMA mapped address of the DROQ descriptor ring. */
+	size_t desc_ring_dma;
+
+	/** Info ptr list are allocated at this virtual address. */
+	size_t info_base_addr;
+
+	/** DMA mapped address of the info list */
+	size_t info_list_dma;
+
+	/** Allocated size of info list. */
+	uint32_t info_alloc_size;
+
+	/** Memory zone **/
+	const struct rte_memzone *desc_ring_mz;
+	const struct rte_memzone *info_mz;
+	struct rte_mempool *mpool;
+};
+
+/** Receive Header */
+union octeon_rh {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	uint64_t rh64;
+	struct	{
+		uint64_t opcode : 4;
+		uint64_t subcode : 8;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t reserved : 17;
+		uint64_t ossp : 32; /** opcode/subcode specific parameters */
+	} r;
+	struct	{
+		uint64_t opcode : 4;
+		uint64_t subcode : 8;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t extra : 28;
+		uint64_t vlan : 12;
+		uint64_t priority : 3;
+		uint64_t csum_verified : 3; /** checksum verified. */
+		uint64_t has_hwtstamp : 1; /** Has hardware timestamp.1 = yes.*/
+		uint64_t encap_on : 1;
+		uint64_t has_hash : 1; /** Has hash (rth or rss). 1 = yes. */
+	} r_dh;
+	struct {
+		uint64_t opcode : 4;
+		uint64_t subcode : 8;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t reserved : 8;
+		uint64_t extra : 25;
+		uint64_t gmxport : 16;
+	} r_nic_info;
+#else
+	uint64_t rh64;
+	struct {
+		uint64_t ossp : 32; /** opcode/subcode specific parameters */
+		uint64_t reserved : 17;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t subcode : 8;
+		uint64_t opcode : 4;
+	} r;
+	struct {
+		uint64_t has_hash : 1; /** Has hash (rth or rss). 1 = yes. */
+		uint64_t encap_on : 1;
+		uint64_t has_hwtstamp : 1;  /** 1 = has hwtstamp */
+		uint64_t csum_verified : 3; /** checksum verified. */
+		uint64_t priority : 3;
+		uint64_t vlan : 12;
+		uint64_t extra : 28;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t subcode : 8;
+		uint64_t opcode : 4;
+	} r_dh;
+	struct {
+		uint64_t gmxport : 16;
+		uint64_t extra : 25;
+		uint64_t reserved : 8;
+		uint64_t len : 3; /** additional 64-bit words */
+		uint64_t subcode : 8;
+		uint64_t opcode : 4;
+	} r_nic_info;
+#endif
+};
+
+#define OCTEON_RH_SIZE (sizeof(union octeon_rh))
+
 /** The txpciq info passed to host from the firmware */
 union octeon_txpciq {
 	uint64_t txpciq64;
@@ -380,6 +524,11 @@ struct lio_device {
 	/** The singly-linked tail queues of instruction response */
 	struct lio_response_list response_list;
 
+	uint32_t num_oqs;
+
+	/** The DROQ output queues  */
+	struct lio_droq *droq[LIO_MAX_POSSIBLE_OUTPUT_QUEUES];
+
 	struct lio_io_enable io_qmask;
 
 	struct lio_sriov_info sriov_info;
-- 
1.8.3.1

[PATCH v3 21/46] net/liquidio: initialize Rx queue

[Shijith Thotton]

Initialize Rx queue registers and allocate packet buffers for Rx queue.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 22 ++++++++
 drivers/net/liquidio/base/lio_hw_defs.h |  2 +
 drivers/net/liquidio/lio_rxtx.c         | 96 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         | 20 +++++++
 drivers/net/liquidio/lio_struct.h       |  1 +
 5 files changed, 141 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index 181f830..44d90c0 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -233,6 +233,27 @@
 }
 
 static void
+cn23xx_vf_setup_oq_regs(struct lio_device *lio_dev, uint32_t oq_no)
+{
+	struct lio_droq *droq = lio_dev->droq[oq_no];
+
+	PMD_INIT_FUNC_TRACE();
+
+	lio_write_csr64(lio_dev, CN23XX_SLI_OQ_BASE_ADDR64(oq_no),
+			droq->desc_ring_dma);
+	lio_write_csr(lio_dev, CN23XX_SLI_OQ_SIZE(oq_no), droq->max_count);
+
+	lio_write_csr(lio_dev, CN23XX_SLI_OQ_BUFF_INFO_SIZE(oq_no),
+		      (droq->buffer_size | (OCTEON_RH_SIZE << 16)));
+
+	/* Get the mapped address of the pkt_sent and pkts_credit regs */
+	droq->pkts_sent_reg = (uint8_t *)lio_dev->hw_addr +
+					CN23XX_SLI_OQ_PKTS_SENT(oq_no);
+	droq->pkts_credit_reg = (uint8_t *)lio_dev->hw_addr +
+					CN23XX_SLI_OQ_PKTS_CREDIT(oq_no);
+}
+
+static void
 cn23xx_vf_free_mbox(struct lio_device *lio_dev)
 {
 	PMD_INIT_FUNC_TRACE();
@@ -436,6 +457,7 @@
 		return -1;
 
 	lio_dev->fn_list.setup_iq_regs		= cn23xx_vf_setup_iq_regs;
+	lio_dev->fn_list.setup_oq_regs		= cn23xx_vf_setup_oq_regs;
 	lio_dev->fn_list.setup_mbox		= cn23xx_vf_setup_mbox;
 	lio_dev->fn_list.free_mbox		= cn23xx_vf_free_mbox;
 
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 35d59fd..4271730 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -116,6 +116,8 @@ enum octeon_tag_type {
 /* This subcode is sent by core PCI driver to indicate cores are ready. */
 #define LIO_OPCODE_IF_CFG		0x09
 
+#define LIO_MAX_RX_PKTLEN		(64 * 1024)
+
 /* Interface flags communicated between host driver and core app. */
 enum lio_ifflags {
 	LIO_IFFLAG_UNICAST	= 0x10
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 942fe9b..9948023 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -41,6 +41,96 @@
 #include "lio_rxtx.h"
 
 static void
+lio_droq_compute_max_packet_bufs(struct lio_droq *droq)
+{
+	uint32_t count = 0;
+
+	do {
+		count += droq->buffer_size;
+	} while (count < LIO_MAX_RX_PKTLEN);
+}
+
+static void
+lio_droq_reset_indices(struct lio_droq *droq)
+{
+	droq->read_idx	= 0;
+	droq->write_idx	= 0;
+	droq->refill_idx = 0;
+	droq->refill_count = 0;
+	rte_atomic64_set(&droq->pkts_pending, 0);
+}
+
+static void
+lio_droq_destroy_ring_buffers(struct lio_droq *droq)
+{
+	uint32_t i;
+
+	for (i = 0; i < droq->max_count; i++) {
+		if (droq->recv_buf_list[i].buffer) {
+			rte_pktmbuf_free((struct rte_mbuf *)
+					 droq->recv_buf_list[i].buffer);
+			droq->recv_buf_list[i].buffer = NULL;
+		}
+	}
+
+	lio_droq_reset_indices(droq);
+}
+
+static void *
+lio_recv_buffer_alloc(struct lio_device *lio_dev, int q_no)
+{
+	struct lio_droq *droq = lio_dev->droq[q_no];
+	struct rte_mempool *mpool = droq->mpool;
+	struct rte_mbuf *m;
+
+	m = rte_pktmbuf_alloc(mpool);
+	if (m == NULL) {
+		lio_dev_err(lio_dev, "Cannot allocate\n");
+		return NULL;
+	}
+
+	rte_mbuf_refcnt_set(m, 1);
+	m->next = NULL;
+	m->data_off = RTE_PKTMBUF_HEADROOM;
+	m->nb_segs = 1;
+	m->pool = mpool;
+
+	return m;
+}
+
+static int
+lio_droq_setup_ring_buffers(struct lio_device *lio_dev,
+			    struct lio_droq *droq)
+{
+	struct lio_droq_desc *desc_ring = droq->desc_ring;
+	uint32_t i;
+	void *buf;
+
+	for (i = 0; i < droq->max_count; i++) {
+		buf = lio_recv_buffer_alloc(lio_dev, droq->q_no);
+		if (buf == NULL) {
+			lio_dev_err(lio_dev, "buffer alloc failed\n");
+			lio_droq_destroy_ring_buffers(droq);
+			return -ENOMEM;
+		}
+
+		droq->recv_buf_list[i].buffer = buf;
+		droq->info_list[i].length = 0;
+
+		/* map ring buffers into memory */
+		desc_ring[i].info_ptr = lio_map_ring_info(droq, i);
+		desc_ring[i].buffer_ptr =
+			lio_map_ring(droq->recv_buf_list[i].buffer);
+	}
+
+	lio_droq_reset_indices(droq);
+
+	lio_droq_compute_max_packet_bufs(droq);
+
+	return 0;
+}
+
+static void
 lio_dma_zone_free(struct lio_device *lio_dev, const struct rte_memzone *mz)
 {
 	const struct rte_memzone *mz_tmp;
@@ -75,6 +165,7 @@
 
 	lio_dev_dbg(lio_dev, "OQ[%d]\n", q_no);
 
+	lio_droq_destroy_ring_buffers(droq);
 	rte_free(droq->recv_buf_list);
 	droq->recv_buf_list = NULL;
 	lio_dma_zone_free(lio_dev, droq->info_mz);
@@ -172,10 +263,15 @@
 		goto init_droq_fail;
 	}
 
+	if (lio_droq_setup_ring_buffers(lio_dev, droq))
+		goto init_droq_fail;
+
 	droq->refill_threshold = c_refill_threshold;
 
 	rte_spinlock_init(&droq->lock);
 
+	lio_dev->fn_list.setup_oq_regs(lio_dev, q_no);
+
 	lio_dev->io_qmask.oq |= (1ULL << q_no);
 
 	return 0;
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 05f4704..fc623ad 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -495,6 +495,26 @@ enum {
 	}
 }
 
+static inline uint64_t
+lio_map_ring(void *buf)
+{
+	phys_addr_t dma_addr;
+
+	dma_addr = rte_mbuf_data_dma_addr_default(((struct rte_mbuf *)buf));
+
+	return (uint64_t)dma_addr;
+}
+
+static inline uint64_t
+lio_map_ring_info(struct lio_droq *droq, uint32_t i)
+{
+	phys_addr_t dma_addr;
+
+	dma_addr = droq->info_list_dma + (i * LIO_DROQ_INFO_SIZE);
+
+	return (uint64_t)dma_addr;
+}
+
 /* Macro to increment index.
  * Index is incremented by count; if the sum exceeds
  * max, index is wrapped-around to the start.
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 3df79e1..7a7a4a6 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -306,6 +306,7 @@ struct lio_io_enable {
 
 struct lio_fn_list {
 	void (*setup_iq_regs)(struct lio_device *, uint32_t);
+	void (*setup_oq_regs)(struct lio_device *, uint32_t);
 
 	int (*setup_mbox)(struct lio_device *);
 	void (*free_mbox)(struct lio_device *);
-- 
1.8.3.1

[PATCH v3 22/46] net/liquidio: add Rx data path

[Shijith Thotton]

Add APIs to receive packets and re-fill ring buffers.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 doc/guides/nics/features/liquidio.ini   |   4 +
 drivers/net/liquidio/base/lio_hw_defs.h |  12 +
 drivers/net/liquidio/lio_ethdev.c       |   5 +
 drivers/net/liquidio/lio_rxtx.c         | 380 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         |  13 ++
 5 files changed, 414 insertions(+)

diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
index 6c5d8d1..554d921 100644
--- a/doc/guides/nics/features/liquidio.ini
+++ b/doc/guides/nics/features/liquidio.ini
@@ -4,6 +4,10 @@
 ; Refer to default.ini for the full list of available PMD features.
 ;
 [Features]
+Scattered Rx         = Y
+CRC offload          = Y
+L3 checksum offload  = Y
+L4 checksum offload  = Y
 Multiprocess aware   = Y
 Linux UIO            = Y
 Linux VFIO           = Y
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 4271730..2db7085 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -112,12 +112,24 @@ enum octeon_tag_type {
 /* used for NIC operations */
 #define LIO_OPCODE	1
 
+/* Subcodes are used by host driver/apps to identify the sub-operation
+ * for the core. They only need to by unique for a given subsystem.
+ */
+#define LIO_OPCODE_SUBCODE(op, sub)		\
+		((((op) & 0x0f) << 8) | ((sub) & 0x7f))
+
 /** LIO_OPCODE subcodes */
 /* This subcode is sent by core PCI driver to indicate cores are ready. */
+#define LIO_OPCODE_NW_DATA		0x02 /* network packet data */
 #define LIO_OPCODE_IF_CFG		0x09
 
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
 
+/* RX(packets coming from wire) Checksum verification flags */
+/* TCP/UDP csum */
+#define LIO_L4_CSUM_VERIFIED		0x1
+#define LIO_IP_CSUM_VERIFIED		0x2
+
 /* Interface flags communicated between host driver and core app. */
 enum lio_ifflags {
 	LIO_IFFLAG_UNICAST	= 0x10
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index a93fa4a..ebfdf7a 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -404,6 +404,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	rte_free(eth_dev->data->mac_addrs);
 	eth_dev->data->mac_addrs = NULL;
 
+	eth_dev->rx_pkt_burst = NULL;
+
 	return 0;
 }
 
@@ -415,6 +417,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 
 	PMD_INIT_FUNC_TRACE();
 
+	eth_dev->rx_pkt_burst = &lio_dev_recv_pkts;
+
 	/* Primary does the initialization. */
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
@@ -448,6 +452,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 		lio_dev_err(lio_dev,
 			    "MAC addresses memory allocation failed\n");
 		eth_dev->dev_ops = NULL;
+		eth_dev->rx_pkt_burst = NULL;
 		return -ENOMEM;
 	}
 
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 9948023..9e4da3a 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -326,6 +326,386 @@
 	return 0;
 }
 
+static inline uint32_t
+lio_droq_get_bufcount(uint32_t buf_size, uint32_t total_len)
+{
+	uint32_t buf_cnt = 0;
+
+	while (total_len > (buf_size * buf_cnt))
+		buf_cnt++;
+
+	return buf_cnt;
+}
+
+/* If we were not able to refill all buffers, try to move around
+ * the buffers that were not dispatched.
+ */
+static inline uint32_t
+lio_droq_refill_pullup_descs(struct lio_droq *droq,
+			     struct lio_droq_desc *desc_ring)
+{
+	uint32_t refill_index = droq->refill_idx;
+	uint32_t desc_refilled = 0;
+
+	while (refill_index != droq->read_idx) {
+		if (droq->recv_buf_list[refill_index].buffer) {
+			droq->recv_buf_list[droq->refill_idx].buffer =
+				droq->recv_buf_list[refill_index].buffer;
+			desc_ring[droq->refill_idx].buffer_ptr =
+				desc_ring[refill_index].buffer_ptr;
+			droq->recv_buf_list[refill_index].buffer = NULL;
+			desc_ring[refill_index].buffer_ptr = 0;
+			do {
+				droq->refill_idx = lio_incr_index(
+							droq->refill_idx, 1,
+							droq->max_count);
+				desc_refilled++;
+				droq->refill_count--;
+			} while (droq->recv_buf_list[droq->refill_idx].buffer);
+		}
+		refill_index = lio_incr_index(refill_index, 1,
+					      droq->max_count);
+	}	/* while */
+
+	return desc_refilled;
+}
+
+/* lio_droq_refill
+ *
+ * @param lio_dev	- pointer to the lio device structure
+ * @param droq		- droq in which descriptors require new buffers.
+ *
+ * Description:
+ *  Called during normal DROQ processing in interrupt mode or by the poll
+ *  thread to refill the descriptors from which buffers were dispatched
+ *  to upper layers. Attempts to allocate new buffers. If that fails, moves
+ *  up buffers (that were not dispatched) to form a contiguous ring.
+ *
+ * Returns:
+ *  No of descriptors refilled.
+ *
+ * Locks:
+ * This routine is called with droq->lock held.
+ */
+static uint32_t
+lio_droq_refill(struct lio_device *lio_dev, struct lio_droq *droq)
+{
+	struct lio_droq_desc *desc_ring;
+	uint32_t desc_refilled = 0;
+	void *buf = NULL;
+
+	desc_ring = droq->desc_ring;
+
+	while (droq->refill_count && (desc_refilled < droq->max_count)) {
+		/* If a valid buffer exists (happens if there is no dispatch),
+		 * reuse the buffer, else allocate.
+		 */
+		if (droq->recv_buf_list[droq->refill_idx].buffer == NULL) {
+			buf = lio_recv_buffer_alloc(lio_dev, droq->q_no);
+			/* If a buffer could not be allocated, no point in
+			 * continuing
+			 */
+			if (buf == NULL)
+				break;
+
+			droq->recv_buf_list[droq->refill_idx].buffer = buf;
+		}
+
+		desc_ring[droq->refill_idx].buffer_ptr =
+		    lio_map_ring(droq->recv_buf_list[droq->refill_idx].buffer);
+		/* Reset any previous values in the length field. */
+		droq->info_list[droq->refill_idx].length = 0;
+
+		droq->refill_idx = lio_incr_index(droq->refill_idx, 1,
+						  droq->max_count);
+		desc_refilled++;
+		droq->refill_count--;
+	}
+
+	if (droq->refill_count)
+		desc_refilled += lio_droq_refill_pullup_descs(droq, desc_ring);
+
+	/* if droq->refill_count
+	 * The refill count would not change in pass two. We only moved buffers
+	 * to close the gap in the ring, but we would still have the same no. of
+	 * buffers to refill.
+	 */
+	return desc_refilled;
+}
+
+static int
+lio_droq_fast_process_packet(struct lio_device *lio_dev,
+			     struct lio_droq *droq,
+			     struct rte_mbuf **rx_pkts)
+{
+	struct rte_mbuf *nicbuf = NULL;
+	struct lio_droq_info *info;
+	uint32_t total_len = 0;
+	int data_total_len = 0;
+	uint32_t pkt_len = 0;
+	union octeon_rh *rh;
+	int data_pkts = 0;
+
+	info = &droq->info_list[droq->read_idx];
+	lio_swap_8B_data((uint64_t *)info, 2);
+
+	if (!info->length)
+		return -1;
+
+	/* Len of resp hdr in included in the received data len. */
+	info->length -= OCTEON_RH_SIZE;
+	rh = &info->rh;
+
+	total_len += (uint32_t)info->length;
+
+	if (lio_opcode_slow_path(rh)) {
+		uint32_t buf_cnt;
+
+		buf_cnt = lio_droq_get_bufcount(droq->buffer_size,
+						(uint32_t)info->length);
+		droq->read_idx = lio_incr_index(droq->read_idx, buf_cnt,
+						droq->max_count);
+		droq->refill_count += buf_cnt;
+	} else {
+		if (info->length <= droq->buffer_size) {
+			if (rh->r_dh.has_hash)
+				pkt_len = (uint32_t)(info->length - 8);
+			else
+				pkt_len = (uint32_t)info->length;
+
+			nicbuf = droq->recv_buf_list[droq->read_idx].buffer;
+			droq->recv_buf_list[droq->read_idx].buffer = NULL;
+			droq->read_idx = lio_incr_index(
+						droq->read_idx, 1,
+						droq->max_count);
+			droq->refill_count++;
+
+			if (likely(nicbuf != NULL)) {
+				nicbuf->data_off = RTE_PKTMBUF_HEADROOM;
+				nicbuf->nb_segs = 1;
+				nicbuf->next = NULL;
+				/* We don't have a way to pass flags yet */
+				nicbuf->ol_flags = 0;
+				if (rh->r_dh.has_hash) {
+					uint64_t *hash_ptr;
+
+					nicbuf->ol_flags |= PKT_RX_RSS_HASH;
+					hash_ptr = rte_pktmbuf_mtod(nicbuf,
+								    uint64_t *);
+					lio_swap_8B_data(hash_ptr, 1);
+					nicbuf->hash.rss = (uint32_t)*hash_ptr;
+					nicbuf->data_off += 8;
+				}
+
+				nicbuf->pkt_len = pkt_len;
+				nicbuf->data_len = pkt_len;
+				nicbuf->port = lio_dev->port_id;
+				/* Store the mbuf */
+				rx_pkts[data_pkts++] = nicbuf;
+				data_total_len += pkt_len;
+			}
+
+			/* Prefetch buffer pointers when on a cache line
+			 * boundary
+			 */
+			if ((droq->read_idx & 3) == 0) {
+				rte_prefetch0(
+				    &droq->recv_buf_list[droq->read_idx]);
+				rte_prefetch0(
+				    &droq->info_list[droq->read_idx]);
+			}
+		} else {
+			struct rte_mbuf *first_buf = NULL;
+			struct rte_mbuf *last_buf = NULL;
+
+			while (pkt_len < info->length) {
+				int cpy_len = 0;
+
+				cpy_len = ((pkt_len + droq->buffer_size) >
+						info->length)
+						? ((uint32_t)info->length -
+							pkt_len)
+						: droq->buffer_size;
+
+				nicbuf =
+				    droq->recv_buf_list[droq->read_idx].buffer;
+				droq->recv_buf_list[droq->read_idx].buffer =
+				    NULL;
+
+				if (likely(nicbuf != NULL)) {
+					/* Note the first seg */
+					if (!pkt_len)
+						first_buf = nicbuf;
+
+					nicbuf->data_off = RTE_PKTMBUF_HEADROOM;
+					nicbuf->nb_segs = 1;
+					nicbuf->next = NULL;
+					nicbuf->port = lio_dev->port_id;
+					/* We don't have a way to pass
+					 * flags yet
+					 */
+					nicbuf->ol_flags = 0;
+					if ((!pkt_len) && (rh->r_dh.has_hash)) {
+						uint64_t *hash_ptr;
+
+						nicbuf->ol_flags |=
+						    PKT_RX_RSS_HASH;
+						hash_ptr = rte_pktmbuf_mtod(
+						    nicbuf, uint64_t *);
+						lio_swap_8B_data(hash_ptr, 1);
+						nicbuf->hash.rss =
+						    (uint32_t)*hash_ptr;
+						nicbuf->data_off += 8;
+						nicbuf->pkt_len = cpy_len - 8;
+						nicbuf->data_len = cpy_len - 8;
+					} else {
+						nicbuf->pkt_len = cpy_len;
+						nicbuf->data_len = cpy_len;
+					}
+
+					if (pkt_len)
+						first_buf->nb_segs++;
+
+					if (last_buf)
+						last_buf->next = nicbuf;
+
+					last_buf = nicbuf;
+				} else {
+					PMD_RX_LOG(lio_dev, ERR, "no buf\n");
+				}
+
+				pkt_len += cpy_len;
+				droq->read_idx = lio_incr_index(
+							droq->read_idx,
+							1, droq->max_count);
+				droq->refill_count++;
+
+				/* Prefetch buffer pointers when on a
+				 * cache line boundary
+				 */
+				if ((droq->read_idx & 3) == 0) {
+					rte_prefetch0(&droq->recv_buf_list
+							      [droq->read_idx]);
+
+					rte_prefetch0(
+					    &droq->info_list[droq->read_idx]);
+				}
+			}
+			rx_pkts[data_pkts++] = first_buf;
+			if (rh->r_dh.has_hash)
+				data_total_len += (pkt_len - 8);
+			else
+				data_total_len += pkt_len;
+		}
+
+		/* Inform upper layer about packet checksum verification */
+		struct rte_mbuf *m = rx_pkts[data_pkts - 1];
+
+		if (rh->r_dh.csum_verified & LIO_IP_CSUM_VERIFIED)
+			m->ol_flags |= PKT_RX_IP_CKSUM_GOOD;
+
+		if (rh->r_dh.csum_verified & LIO_L4_CSUM_VERIFIED)
+			m->ol_flags |= PKT_RX_L4_CKSUM_GOOD;
+	}
+
+	if (droq->refill_count >= droq->refill_threshold) {
+		int desc_refilled = lio_droq_refill(lio_dev, droq);
+
+		/* Flush the droq descriptor data to memory to be sure
+		 * that when we update the credits the data in memory is
+		 * accurate.
+		 */
+		rte_wmb();
+		rte_write32(desc_refilled, droq->pkts_credit_reg);
+		/* make sure mmio write completes */
+		rte_wmb();
+	}
+
+	info->length = 0;
+	info->rh.rh64 = 0;
+
+	return data_pkts;
+}
+
+static uint32_t
+lio_droq_fast_process_packets(struct lio_device *lio_dev,
+			      struct lio_droq *droq,
+			      struct rte_mbuf **rx_pkts,
+			      uint32_t pkts_to_process)
+{
+	int ret, data_pkts = 0;
+	uint32_t pkt;
+
+	for (pkt = 0; pkt < pkts_to_process; pkt++) {
+		ret = lio_droq_fast_process_packet(lio_dev, droq,
+						   &rx_pkts[data_pkts]);
+		if (ret < 0) {
+			lio_dev_err(lio_dev, "Port[%d] DROQ[%d] idx: %d len:0, pkt_cnt: %d\n",
+				    lio_dev->port_id, droq->q_no,
+				    droq->read_idx, pkts_to_process);
+			break;
+		}
+		data_pkts += ret;
+	}
+
+	rte_atomic64_sub(&droq->pkts_pending, pkt);
+
+	return data_pkts;
+}
+
+static inline uint32_t
+lio_droq_check_hw_for_pkts(struct lio_droq *droq)
+{
+	uint32_t last_count;
+	uint32_t pkt_count;
+
+	pkt_count = rte_read32(droq->pkts_sent_reg);
+
+	last_count = pkt_count - droq->pkt_count;
+	droq->pkt_count = pkt_count;
+
+	if (last_count)
+		rte_atomic64_add(&droq->pkts_pending, last_count);
+
+	return last_count;
+}
+
+uint16_t
+lio_dev_recv_pkts(void *rx_queue,
+		  struct rte_mbuf **rx_pkts,
+		  uint16_t budget)
+{
+	struct lio_droq *droq = rx_queue;
+	struct lio_device *lio_dev = droq->lio_dev;
+	uint32_t pkts_processed = 0;
+	uint32_t pkt_count = 0;
+
+	lio_droq_check_hw_for_pkts(droq);
+
+	pkt_count = rte_atomic64_read(&droq->pkts_pending);
+	if (!pkt_count)
+		return 0;
+
+	if (pkt_count > budget)
+		pkt_count = budget;
+
+	/* Grab the lock */
+	rte_spinlock_lock(&droq->lock);
+	pkts_processed = lio_droq_fast_process_packets(lio_dev,
+						       droq, rx_pkts,
+						       pkt_count);
+
+	if (droq->pkt_count) {
+		rte_write32(droq->pkt_count, droq->pkts_sent_reg);
+		droq->pkt_count = 0;
+	}
+
+	/* Release the spin lock */
+	rte_spinlock_unlock(&droq->lock);
+
+	return pkts_processed;
+}
+
 /**
  *  lio_init_instr_queue()
  *  @param lio_dev	- pointer to the lio device structure.
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index fc623ad..420b893 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -515,6 +515,17 @@ enum {
 	return (uint64_t)dma_addr;
 }
 
+static inline int
+lio_opcode_slow_path(union octeon_rh *rh)
+{
+	uint16_t subcode1, subcode2;
+
+	subcode1 = LIO_OPCODE_SUBCODE(rh->r.opcode, rh->r.subcode);
+	subcode2 = LIO_OPCODE_SUBCODE(LIO_OPCODE, LIO_OPCODE_NW_DATA);
+
+	return subcode2 != subcode1;
+}
+
 /* Macro to increment index.
  * Index is incremented by count; if the sum exceeds
  * max, index is wrapped-around to the start.
@@ -533,6 +544,8 @@ enum {
 int lio_setup_droq(struct lio_device *lio_dev, int q_no, int num_descs,
 		   int desc_size, struct rte_mempool *mpool,
 		   unsigned int socket_id);
+uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+			   uint16_t budget);
 
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
-- 
1.8.3.1

[PATCH v3 23/46] net/liquidio: add API to release Rx queue

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 28 ++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.c   | 10 ++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  1 +
 3 files changed, 39 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index ebfdf7a..d5f650e 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -121,6 +121,33 @@
 	return 0;
 }
 
+/**
+ * Release the receive queue/ringbuffer. Called by
+ * the upper layers.
+ *
+ * @param rxq
+ *    Opaque pointer to the receive queue to release
+ *
+ * @return
+ *    - nothing
+ */
+static void
+lio_dev_rx_queue_release(void *rxq)
+{
+	struct lio_droq *droq = rxq;
+	struct lio_device *lio_dev = droq->lio_dev;
+	int oq_no;
+
+	/* Run time queue deletion not supported */
+	if (lio_dev->port_configured)
+		return;
+
+	if (droq != NULL) {
+		oq_no = droq->q_no;
+		lio_delete_droq_queue(droq->lio_dev, oq_no);
+	}
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -266,6 +293,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
+	.rx_queue_release	= lio_dev_rx_queue_release,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 9e4da3a..4e63a50 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -706,6 +706,16 @@
 	return pkts_processed;
 }
 
+void
+lio_delete_droq_queue(struct lio_device *lio_dev,
+		      int oq_no)
+{
+	lio_delete_droq(lio_dev, oq_no);
+	lio_dev->num_oqs--;
+	rte_free(lio_dev->droq[oq_no]);
+	lio_dev->droq[oq_no] = NULL;
+}
+
 /**
  *  lio_init_instr_queue()
  *  @param lio_dev	- pointer to the lio device structure.
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 420b893..76d067e 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -546,6 +546,7 @@ int lio_setup_droq(struct lio_device *lio_dev, int q_no, int num_descs,
 		   unsigned int socket_id);
 uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			   uint16_t budget);
+void lio_delete_droq_queue(struct lio_device *lio_dev, int oq_no);
 
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
-- 
1.8.3.1

[PATCH v3 24/46] net/liquidio: add API to setup Tx queue

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 60 +++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.c   | 39 +++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  3 ++
 3 files changed, 102 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index d5f650e..9e2d3f8 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -148,6 +148,65 @@
 	}
 }
 
+/**
+ * Allocate and initialize SW ring. Initialize associated HW registers.
+ *
+ * @param eth_dev
+ *   Pointer to structure rte_eth_dev
+ *
+ * @param q_no
+ *   Queue number
+ *
+ * @param num_tx_descs
+ *   Number of ringbuffer descriptors
+ *
+ * @param socket_id
+ *   NUMA socket id, used for memory allocations
+ *
+ * @param tx_conf
+ *   Pointer to the structure rte_eth_txconf
+ *
+ * @return
+ *   - On success, return 0
+ *   - On failure, return -errno value
+ */
+static int
+lio_dev_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t q_no,
+		       uint16_t num_tx_descs, unsigned int socket_id,
+		       const struct rte_eth_txconf *tx_conf __rte_unused)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	int fw_mapped_iq = lio_dev->linfo.txpciq[q_no].s.q_no;
+	int retval;
+
+	if (q_no >= lio_dev->nb_tx_queues) {
+		lio_dev_err(lio_dev, "Invalid tx queue number %u\n", q_no);
+		return -EINVAL;
+	}
+
+	lio_dev_dbg(lio_dev, "setting up tx queue %u\n", q_no);
+
+	if ((lio_dev->instr_queue[fw_mapped_iq] != NULL) &&
+	    (num_tx_descs != lio_dev->instr_queue[fw_mapped_iq]->max_count)) {
+		lio_dev_err(lio_dev,
+			    "Reconfiguring Tx descs not supported. Configure descs to same value %u or restart application\n",
+			    lio_dev->instr_queue[fw_mapped_iq]->max_count);
+		return -ENOTSUP;
+	}
+
+	retval = lio_setup_iq(lio_dev, q_no, lio_dev->linfo.txpciq[q_no],
+			      num_tx_descs, lio_dev, socket_id);
+
+	if (retval) {
+		lio_dev_err(lio_dev, "Runtime IQ(TxQ) creation failed.\n");
+		return retval;
+	}
+
+	eth_dev->data->tx_queues[q_no] = lio_dev->instr_queue[fw_mapped_iq];
+
+	return 0;
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -294,6 +353,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.dev_configure		= lio_dev_configure,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
+	.tx_queue_setup		= lio_dev_tx_queue_setup,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 4e63a50..b0dfc9b 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -861,6 +861,45 @@
 	lio_dev->num_iqs--;
 }
 
+/* Return 0 on success, -1 on failure */
+int
+lio_setup_iq(struct lio_device *lio_dev, int q_index,
+	     union octeon_txpciq txpciq, uint32_t num_descs, void *app_ctx,
+	     unsigned int socket_id)
+{
+	uint32_t iq_no = (uint32_t)txpciq.s.q_no;
+
+	if (lio_dev->instr_queue[iq_no]) {
+		lio_dev_dbg(lio_dev, "IQ is in use. Cannot create the IQ: %d again\n",
+			    iq_no);
+		lio_dev->instr_queue[iq_no]->txpciq.txpciq64 = txpciq.txpciq64;
+		lio_dev->instr_queue[iq_no]->app_ctx = app_ctx;
+		return 0;
+	}
+
+	lio_dev->instr_queue[iq_no] = rte_zmalloc_socket("ethdev TX queue",
+						sizeof(struct lio_instr_queue),
+						RTE_CACHE_LINE_SIZE, socket_id);
+	if (lio_dev->instr_queue[iq_no] == NULL)
+		return -1;
+
+	lio_dev->instr_queue[iq_no]->q_index = q_index;
+	lio_dev->instr_queue[iq_no]->app_ctx = app_ctx;
+
+	if (lio_init_instr_queue(lio_dev, txpciq, num_descs, socket_id))
+		goto release_lio_iq;
+
+	lio_dev->num_iqs++;
+
+	return 0;
+
+release_lio_iq:
+	rte_free(lio_dev->instr_queue[iq_no]);
+	lio_dev->instr_queue[iq_no] = NULL;
+
+	return -1;
+}
+
 static inline void
 lio_ring_doorbell(struct lio_device *lio_dev,
 		  struct lio_instr_queue *iq)
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 76d067e..86d5864 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -548,6 +548,9 @@ uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			   uint16_t budget);
 void lio_delete_droq_queue(struct lio_device *lio_dev, int oq_no);
 
+int lio_setup_iq(struct lio_device *lio_dev, int q_index,
+		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
+		 unsigned int socket_id);
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
-- 
1.8.3.1

[PATCH v3 25/46] net/liquidio: add APIs for SG list

[Shijith Thotton]

Add APIs to setup and free Scatter-Gather list. SG list is used while
sending packets with multiple segments.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c |  23 ++++++++
 drivers/net/liquidio/lio_rxtx.c   | 107 ++++++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |   7 +++
 drivers/net/liquidio/lio_struct.h |  40 ++++++++++++++
 4 files changed, 177 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 9e2d3f8..77106f6 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -202,6 +202,15 @@
 		return retval;
 	}
 
+	retval = lio_setup_sglists(lio_dev, q_no, fw_mapped_iq,
+				lio_dev->instr_queue[fw_mapped_iq]->max_count,
+				socket_id);
+
+	if (retval) {
+		lio_delete_instruction_queue(lio_dev, fw_mapped_iq);
+		return retval;
+	}
+
 	eth_dev->data->tx_queues[q_no] = lio_dev->instr_queue[fw_mapped_iq];
 
 	return 0;
@@ -334,6 +343,20 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	/* Copy the permanent MAC address */
 	ether_addr_copy((struct ether_addr *)mac, &eth_dev->data->mac_addrs[0]);
 
+	lio_dev->glist_lock =
+	    rte_zmalloc(NULL, sizeof(*lio_dev->glist_lock) * num_iqueues, 0);
+	if (lio_dev->glist_lock == NULL)
+		return -ENOMEM;
+
+	lio_dev->glist_head =
+		rte_zmalloc(NULL, sizeof(*lio_dev->glist_head) * num_iqueues,
+			    0);
+	if (lio_dev->glist_head == NULL) {
+		rte_free(lio_dev->glist_lock);
+		lio_dev->glist_lock = NULL;
+		return -ENOMEM;
+	}
+
 	lio_dev->port_configured = 1;
 
 	lio_free_soft_command(sc);
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index b0dfc9b..50deaba 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -40,6 +40,8 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+#define LIO_MAX_SG 12
+
 static void
 lio_droq_compute_max_packet_bufs(struct lio_droq *droq)
 {
@@ -1272,3 +1274,108 @@ struct lio_soft_command *
 
 	return 0;
 }
+
+static inline struct lio_stailq_node *
+list_delete_first_node(struct lio_stailq_head *head)
+{
+	struct lio_stailq_node *node;
+
+	if (STAILQ_EMPTY(head))
+		node = NULL;
+	else
+		node = STAILQ_FIRST(head);
+
+	if (node)
+		STAILQ_REMOVE(head, node, lio_stailq_node, entries);
+
+	return node;
+}
+
+static void
+lio_delete_sglist(struct lio_instr_queue *txq)
+{
+	struct lio_device *lio_dev = txq->lio_dev;
+	int iq_no = txq->q_index;
+	struct lio_gather *g;
+
+	if (lio_dev->glist_head == NULL)
+		return;
+
+	do {
+		g = (struct lio_gather *)list_delete_first_node(
+						&lio_dev->glist_head[iq_no]);
+		if (g) {
+			if (g->sg)
+				rte_free(
+				    (void *)((unsigned long)g->sg - g->adjust));
+			rte_free(g);
+		}
+	} while (g);
+}
+
+/**
+ * \brief Setup gather lists
+ * @param lio per-network private data
+ */
+int
+lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
+		  int fw_mapped_iq, int num_descs, unsigned int socket_id)
+{
+	struct lio_gather *g;
+	int i;
+
+	rte_spinlock_init(&lio_dev->glist_lock[iq_no]);
+
+	STAILQ_INIT(&lio_dev->glist_head[iq_no]);
+
+	for (i = 0; i < num_descs; i++) {
+		g = rte_zmalloc_socket(NULL, sizeof(*g), RTE_CACHE_LINE_SIZE,
+				       socket_id);
+		if (g == NULL) {
+			lio_dev_err(lio_dev,
+				    "lio_gather memory allocation failed for qno %d\n",
+				    iq_no);
+			break;
+		}
+
+		g->sg_size =
+		    ((ROUNDUP4(LIO_MAX_SG) >> 2) * LIO_SG_ENTRY_SIZE);
+
+		g->sg = rte_zmalloc_socket(NULL, g->sg_size + 8,
+					   RTE_CACHE_LINE_SIZE, socket_id);
+		if (g->sg == NULL) {
+			lio_dev_err(lio_dev,
+				    "sg list memory allocation failed for qno %d\n",
+				    iq_no);
+			rte_free(g);
+			break;
+		}
+
+		/* The gather component should be aligned on 64-bit boundary */
+		if (((unsigned long)g->sg) & 7) {
+			g->adjust = 8 - (((unsigned long)g->sg) & 7);
+			g->sg =
+			    (struct lio_sg_entry *)((unsigned long)g->sg +
+						       g->adjust);
+		}
+
+		STAILQ_INSERT_TAIL(&lio_dev->glist_head[iq_no], &g->list,
+				   entries);
+	}
+
+	if (i != num_descs) {
+		lio_delete_sglist(lio_dev->instr_queue[fw_mapped_iq]);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+void
+lio_delete_instruction_queue(struct lio_device *lio_dev, int iq_no)
+{
+	lio_delete_instr_queue(lio_dev, iq_no);
+	rte_free(lio_dev->instr_queue[iq_no]);
+	lio_dev->instr_queue[iq_no] = NULL;
+	lio_dev->num_iqs--;
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 86d5864..10bca4c 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -42,6 +42,10 @@
 
 #include "lio_struct.h"
 
+#ifndef ROUNDUP4
+#define ROUNDUP4(val) (((val) + 3) & 0xfffffffc)
+#endif
+
 #define LIO_STQUEUE_FIRST_ENTRY(ptr, type, elem)	\
 	(type *)((char *)((ptr)->stqh_first) - offsetof(type, elem))
 
@@ -548,9 +552,12 @@ uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			   uint16_t budget);
 void lio_delete_droq_queue(struct lio_device *lio_dev, int oq_no);
 
+int lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
+		      int fw_mapped_iq, int num_descs, unsigned int socket_id);
 int lio_setup_iq(struct lio_device *lio_dev, int q_index,
 		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
 		 unsigned int socket_id);
+void lio_delete_instruction_queue(struct lio_device *lio_dev, int iq_no);
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
  *
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 7a7a4a6..4d67eb6 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -298,6 +298,41 @@ struct lio_instr_queue {
 	const struct rte_memzone *iq_mz;
 };
 
+/* The Scatter-Gather List Entry. The scatter or gather component used with
+ * input instruction has this format.
+ */
+struct lio_sg_entry {
+	/** The first 64 bit gives the size of data in each dptr. */
+	union {
+		uint16_t size[4];
+		uint64_t size64;
+	} u;
+
+	/** The 4 dptr pointers for this entry. */
+	uint64_t ptr[4];
+};
+
+#define LIO_SG_ENTRY_SIZE	(sizeof(struct lio_sg_entry))
+
+/** Structure of a node in list of gather components maintained by
+ *  driver for each network device.
+ */
+struct lio_gather {
+	/** List manipulation. Next and prev pointers. */
+	struct lio_stailq_node list;
+
+	/** Size of the gather component at sg in bytes. */
+	int sg_size;
+
+	/** Number of bytes that sg was adjusted to make it 8B-aligned. */
+	int adjust;
+
+	/** Gather component that can accommodate max sized fragment list
+	 *  received from the IP layer.
+	 */
+	struct lio_sg_entry *sg;
+};
+
 struct lio_io_enable {
 	uint64_t iq;
 	uint64_t oq;
@@ -516,6 +551,11 @@ struct lio_device {
 
 	uint32_t num_iqs;
 
+	/** Guards each glist */
+	rte_spinlock_t *glist_lock;
+	/** Array of gather component linked lists */
+	struct lio_stailq_head *glist_head;
+
 	/* The pool containing pre allocated buffers used for soft commands */
 	struct rte_mempool *sc_buf_pool;
 
-- 
1.8.3.1

[PATCH v3 26/46] net/liquidio: add APIs to enable and disable IO queues

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_23xx_vf.c | 70 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.c       | 13 ++++++
 drivers/net/liquidio/lio_rxtx.c         |  5 +++
 drivers/net/liquidio/lio_struct.h       |  2 +
 4 files changed, 90 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_23xx_vf.c b/drivers/net/liquidio/base/lio_23xx_vf.c
index 44d90c0..6ff5b69 100644
--- a/drivers/net/liquidio/base/lio_23xx_vf.c
+++ b/drivers/net/liquidio/base/lio_23xx_vf.c
@@ -310,6 +310,73 @@
 	return 0;
 }
 
+static int
+cn23xx_vf_enable_io_queues(struct lio_device *lio_dev)
+{
+	uint32_t q_no;
+
+	PMD_INIT_FUNC_TRACE();
+
+	for (q_no = 0; q_no < lio_dev->num_iqs; q_no++) {
+		uint64_t reg_val;
+
+		/* set the corresponding IQ IS_64B bit */
+		if (lio_dev->io_qmask.iq64B & (1ULL << q_no)) {
+			reg_val = lio_read_csr64(
+					lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+			reg_val = reg_val | CN23XX_PKT_INPUT_CTL_IS_64B;
+			lio_write_csr64(lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+					reg_val);
+		}
+
+		/* set the corresponding IQ ENB bit */
+		if (lio_dev->io_qmask.iq & (1ULL << q_no)) {
+			reg_val = lio_read_csr64(
+					lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no));
+			reg_val = reg_val | CN23XX_PKT_INPUT_CTL_RING_ENB;
+			lio_write_csr64(lio_dev,
+					CN23XX_SLI_IQ_PKT_CONTROL64(q_no),
+					reg_val);
+		}
+	}
+	for (q_no = 0; q_no < lio_dev->num_oqs; q_no++) {
+		uint32_t reg_val;
+
+		/* set the corresponding OQ ENB bit */
+		if (lio_dev->io_qmask.oq & (1ULL << q_no)) {
+			reg_val = lio_read_csr(
+					lio_dev,
+					CN23XX_SLI_OQ_PKT_CONTROL(q_no));
+			reg_val = reg_val | CN23XX_PKT_OUTPUT_CTL_RING_ENB;
+			lio_write_csr(lio_dev,
+				      CN23XX_SLI_OQ_PKT_CONTROL(q_no),
+				      reg_val);
+		}
+	}
+
+	return 0;
+}
+
+static void
+cn23xx_vf_disable_io_queues(struct lio_device *lio_dev)
+{
+	uint32_t num_queues;
+
+	PMD_INIT_FUNC_TRACE();
+
+	/* per HRM, rings can only be disabled via reset operation,
+	 * NOT via SLI_PKT()_INPUT/OUTPUT_CONTROL[ENB]
+	 */
+	num_queues = lio_dev->num_iqs;
+	if (num_queues < lio_dev->num_oqs)
+		num_queues = lio_dev->num_oqs;
+
+	cn23xx_vf_reset_io_queues(lio_dev, num_queues);
+}
+
 void
 cn23xx_vf_ask_pf_to_do_flr(struct lio_device *lio_dev)
 {
@@ -463,6 +530,9 @@
 
 	lio_dev->fn_list.setup_device_regs	= cn23xx_vf_setup_device_regs;
 
+	lio_dev->fn_list.enable_io_queues	= cn23xx_vf_enable_io_queues;
+	lio_dev->fn_list.disable_io_queues	= cn23xx_vf_disable_io_queues;
+
 	return 0;
 }
 
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 77106f6..b2e7a29 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -361,6 +361,15 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 
 	lio_free_soft_command(sc);
 
+	/* Disable iq_0 for reconf */
+	lio_dev->fn_list.disable_io_queues(lio_dev);
+
+	/* Reset ioq regs */
+	lio_dev->fn_list.setup_device_regs(lio_dev);
+
+	/* Free iq_0 used during init */
+	lio_free_instr_queue0(lio_dev);
+
 	return 0;
 
 nic_config_fail:
@@ -487,6 +496,10 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	lio_dev->max_tx_queues = dpdk_queues;
 	lio_dev->max_rx_queues = dpdk_queues;
 
+	/* Enable input and output queues for this device */
+	if (lio_dev->fn_list.enable_io_queues(lio_dev))
+		goto error;
+
 	return 0;
 
 error:
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 50deaba..63d8896 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -892,9 +892,14 @@
 		goto release_lio_iq;
 
 	lio_dev->num_iqs++;
+	if (lio_dev->fn_list.enable_io_queues(lio_dev))
+		goto delete_lio_iq;
 
 	return 0;
 
+delete_lio_iq:
+	lio_delete_instr_queue(lio_dev, iq_no);
+	lio_dev->num_iqs--;
 release_lio_iq:
 	rte_free(lio_dev->instr_queue[iq_no]);
 	lio_dev->instr_queue[iq_no] = NULL;
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 4d67eb6..906553c 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -347,6 +347,8 @@ struct lio_fn_list {
 	void (*free_mbox)(struct lio_device *);
 
 	int (*setup_device_regs)(struct lio_device *);
+	int (*enable_io_queues)(struct lio_device *);
+	void (*disable_io_queues)(struct lio_device *);
 };
 
 struct lio_pf_vf_hs_word {
-- 
1.8.3.1

[PATCH v3 27/46] net/liquidio: add Tx data path for single segment

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |   1 +
 drivers/net/liquidio/lio_ethdev.c       |   3 +
 drivers/net/liquidio/lio_rxtx.c         |  89 ++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         | 138 ++++++++++++++++++++++++++++++++
 4 files changed, 231 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 2db7085..1c1ad8e 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -108,6 +108,7 @@ enum octeon_tag_type {
 
 /* pre-defined host->NIC tag values */
 #define LIO_CONTROL	(0x11111110)
+#define LIO_DATA(i)	(0x11111111 + (i))
 
 /* used for NIC operations */
 #define LIO_OPCODE	1
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index b2e7a29..5c5aade 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -529,6 +529,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	eth_dev->data->mac_addrs = NULL;
 
 	eth_dev->rx_pkt_burst = NULL;
+	eth_dev->tx_pkt_burst = NULL;
 
 	return 0;
 }
@@ -542,6 +543,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	PMD_INIT_FUNC_TRACE();
 
 	eth_dev->rx_pkt_burst = &lio_dev_recv_pkts;
+	eth_dev->tx_pkt_burst = &lio_dev_xmit_pkts;
 
 	/* Primary does the initialization. */
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
@@ -577,6 +579,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 			    "MAC addresses memory allocation failed\n");
 		eth_dev->dev_ops = NULL;
 		eth_dev->rx_pkt_burst = NULL;
+		eth_dev->tx_pkt_burst = NULL;
 		return -ENOMEM;
 	}
 
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 63d8896..14113e1 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1384,3 +1384,92 @@ struct lio_soft_command *
 	lio_dev->instr_queue[iq_no] = NULL;
 	lio_dev->num_iqs--;
 }
+
+/** Send data packet to the device
+ *  @param lio_dev - lio device pointer
+ *  @param ndata   - control structure with queueing, and buffer information
+ *
+ *  @returns IQ_FAILED if it failed to add to the input queue. IQ_STOP if it the
+ *  queue should be stopped, and LIO_IQ_SEND_OK if it sent okay.
+ */
+static inline int
+lio_send_data_pkt(struct lio_device *lio_dev, struct lio_data_pkt *ndata)
+{
+	return lio_send_command(lio_dev, ndata->q_no, &ndata->cmd,
+				ndata->buf, ndata->datasize, ndata->reqtype);
+}
+
+uint16_t
+lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts, uint16_t nb_pkts)
+{
+	struct lio_instr_queue *txq = tx_queue;
+	union lio_cmd_setup cmdsetup;
+	struct lio_device *lio_dev;
+	struct lio_data_pkt ndata;
+	int i, processed = 0;
+	struct rte_mbuf *m;
+	uint32_t tag = 0;
+	int status = 0;
+	int iq_no;
+
+	lio_dev = txq->lio_dev;
+	iq_no = txq->txpciq.s.q_no;
+
+	if (!lio_dev->linfo.link.s.link_up) {
+		PMD_TX_LOG(lio_dev, ERR, "Transmit failed link_status : %d\n",
+			   lio_dev->linfo.link.s.link_up);
+		goto xmit_failed;
+	}
+
+	for (i = 0; i < nb_pkts; i++) {
+		uint32_t pkt_len = 0;
+
+		m = pkts[i];
+
+		/* Prepare the attributes for the data to be passed to BASE. */
+		memset(&ndata, 0, sizeof(struct lio_data_pkt));
+
+		ndata.buf = m;
+
+		ndata.q_no = iq_no;
+
+		cmdsetup.cmd_setup64 = 0;
+		cmdsetup.s.iq_no = iq_no;
+
+		/* check checksum offload flags to form cmd */
+		if (m->ol_flags & PKT_TX_IP_CKSUM)
+			cmdsetup.s.ip_csum = 1;
+
+		if ((m->ol_flags & PKT_TX_TCP_CKSUM) ||
+				(m->ol_flags & PKT_TX_UDP_CKSUM))
+			cmdsetup.s.transport_csum = 1;
+
+		if (m->nb_segs == 1) {
+			pkt_len = rte_pktmbuf_data_len(m);
+			cmdsetup.s.u.datasize = pkt_len;
+			lio_prepare_pci_cmd(lio_dev, &ndata.cmd,
+					    &cmdsetup, tag);
+			ndata.cmd.cmd3.dptr = rte_mbuf_data_dma_addr(m);
+			ndata.reqtype = LIO_REQTYPE_NORESP_NET;
+		}
+
+		ndata.datasize = pkt_len;
+
+		status = lio_send_data_pkt(lio_dev, &ndata);
+
+		if (unlikely(status == LIO_IQ_SEND_FAILED)) {
+			PMD_TX_LOG(lio_dev, ERR, "send failed\n");
+			break;
+		}
+
+		if (unlikely(status == LIO_IQ_SEND_STOP))
+			PMD_TX_LOG(lio_dev, DEBUG, "iq full\n");
+
+		processed++;
+	}
+
+xmit_failed:
+
+	return processed;
+}
+
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 10bca4c..6813aea 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -249,6 +249,50 @@ struct lio_iq_post_status {
 
 #define OCTEON_CMD_SIZE (sizeof(union octeon_cmd))
 
+/** Structure of data information passed by driver to the BASE
+ *  layer when forwarding data to Octeon device software.
+ */
+struct lio_data_pkt {
+	/** Pointer to information maintained by NIC module for this packet. The
+	 *  BASE layer passes this as-is to the driver.
+	 */
+	void *buf;
+
+	/** Type of buffer passed in "buf" above. */
+	uint32_t reqtype;
+
+	/** Total data bytes to be transferred in this command. */
+	uint32_t datasize;
+
+	/** Command to be passed to the Octeon device software. */
+	union lio_instr_64B cmd;
+
+	/** Input queue to use to send this command. */
+	uint32_t q_no;
+};
+
+/** Structure passed by driver to BASE layer to prepare a command to send
+ *  network data to Octeon.
+ */
+union lio_cmd_setup {
+	struct {
+		uint32_t iq_no : 8;
+		uint32_t gather : 1;
+		uint32_t timestamp : 1;
+		uint32_t ip_csum : 1;
+		uint32_t transport_csum : 1;
+		uint32_t tnl_csum : 1;
+		uint32_t rsvd : 19;
+
+		union {
+			uint32_t datasize;
+			uint32_t gatherptrs;
+		} u;
+	} s;
+
+	uint64_t cmd_setup64;
+};
+
 /* Instruction Header */
 struct octeon_instr_ih3 {
 #if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
@@ -418,6 +462,98 @@ struct octeon_instr_rdp {
 #endif
 };
 
+union octeon_packet_params {
+	uint32_t pkt_params32;
+	struct {
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint32_t reserved : 24;
+		uint32_t ip_csum : 1; /* Perform IP header checksum(s) */
+		/* Perform Outer transport header checksum */
+		uint32_t transport_csum : 1;
+		/* Find tunnel, and perform transport csum. */
+		uint32_t tnl_csum : 1;
+		uint32_t tsflag : 1;   /* Timestamp this packet */
+		uint32_t ipsec_ops : 4; /* IPsec operation */
+#else
+		uint32_t ipsec_ops : 4;
+		uint32_t tsflag : 1;
+		uint32_t tnl_csum : 1;
+		uint32_t transport_csum : 1;
+		uint32_t ip_csum : 1;
+		uint32_t reserved : 7;
+#endif
+	} s;
+};
+
+/** Utility function to prepare a 64B NIC instruction based on a setup command
+ * @param cmd - pointer to instruction to be filled in.
+ * @param setup - pointer to the setup structure
+ * @param q_no - which queue for back pressure
+ *
+ * Assumes the cmd instruction is pre-allocated, but no fields are filled in.
+ */
+static inline void
+lio_prepare_pci_cmd(struct lio_device *lio_dev,
+		    union lio_instr_64B *cmd,
+		    union lio_cmd_setup *setup,
+		    uint32_t tag)
+{
+	union octeon_packet_params packet_params;
+	struct octeon_instr_pki_ih3 *pki_ih3;
+	struct octeon_instr_irh *irh;
+	struct octeon_instr_ih3 *ih3;
+	int port;
+
+	memset(cmd, 0, sizeof(union lio_instr_64B));
+
+	ih3 = (struct octeon_instr_ih3 *)&cmd->cmd3.ih3;
+	pki_ih3 = (struct octeon_instr_pki_ih3 *)&cmd->cmd3.pki_ih3;
+
+	/* assume that rflag is cleared so therefore front data will only have
+	 * irh and ossp[1] and ossp[2] for a total of 24 bytes
+	 */
+	ih3->pkind = lio_dev->instr_queue[setup->s.iq_no]->txpciq.s.pkind;
+	/* PKI IH */
+	ih3->fsz = OCTEON_PCI_CMD_O3;
+
+	if (!setup->s.gather) {
+		ih3->dlengsz = setup->s.u.datasize;
+	} else {
+		ih3->gather = 1;
+		ih3->dlengsz = setup->s.u.gatherptrs;
+	}
+
+	pki_ih3->w = 1;
+	pki_ih3->raw = 0;
+	pki_ih3->utag = 0;
+	pki_ih3->utt = 1;
+	pki_ih3->uqpg = lio_dev->instr_queue[setup->s.iq_no]->txpciq.s.use_qpg;
+
+	port = (int)lio_dev->instr_queue[setup->s.iq_no]->txpciq.s.port;
+
+	if (tag)
+		pki_ih3->tag = tag;
+	else
+		pki_ih3->tag = LIO_DATA(port);
+
+	pki_ih3->tagtype = OCTEON_ORDERED_TAG;
+	pki_ih3->qpg = lio_dev->instr_queue[setup->s.iq_no]->txpciq.s.qpg;
+	pki_ih3->pm = 0x0; /* parse from L2 */
+	pki_ih3->sl = 32;  /* sl will be sizeof(pki_ih3) + irh + ossp0 + ossp1*/
+
+	irh = (struct octeon_instr_irh *)&cmd->cmd3.irh;
+
+	irh->opcode = LIO_OPCODE;
+	irh->subcode = LIO_OPCODE_NW_DATA;
+
+	packet_params.pkt_params32 = 0;
+	packet_params.s.ip_csum = setup->s.ip_csum;
+	packet_params.s.transport_csum = setup->s.transport_csum;
+	packet_params.s.tsflag = setup->s.timestamp;
+
+	irh->ossp = packet_params.pkt_params32;
+}
+
 int lio_setup_sc_buffer_pool(struct lio_device *lio_dev);
 void lio_free_sc_buffer_pool(struct lio_device *lio_dev);
 
@@ -554,6 +690,8 @@ uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 
 int lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
 		      int fw_mapped_iq, int num_descs, unsigned int socket_id);
+uint16_t lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts,
+			   uint16_t nb_pkts);
 int lio_setup_iq(struct lio_device *lio_dev, int q_index,
 		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
 		 unsigned int socket_id);
-- 
1.8.3.1

[PATCH v3 28/46] net/liquidio: add Tx data path for multiple segments

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 doc/guides/nics/features/liquidio.ini |  1 +
 drivers/net/liquidio/lio_rxtx.c       | 62 +++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h       | 11 +++++++
 drivers/net/liquidio/lio_struct.h     | 24 ++++++++++++++
 4 files changed, 98 insertions(+)

diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
index 554d921..d4bbea1 100644
--- a/doc/guides/nics/features/liquidio.ini
+++ b/doc/guides/nics/features/liquidio.ini
@@ -4,6 +4,7 @@
 ; Refer to default.ini for the full list of available PMD features.
 ;
 [Features]
+Jumbo frame          = Y
 Scattered Rx         = Y
 CRC offload          = Y
 L3 checksum offload  = Y
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 14113e1..75ecbd5 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1451,6 +1451,68 @@ struct lio_soft_command *
 					    &cmdsetup, tag);
 			ndata.cmd.cmd3.dptr = rte_mbuf_data_dma_addr(m);
 			ndata.reqtype = LIO_REQTYPE_NORESP_NET;
+		} else {
+			struct lio_buf_free_info *finfo;
+			struct lio_gather *g;
+			phys_addr_t phyaddr;
+			int i, frags;
+
+			finfo = (struct lio_buf_free_info *)rte_malloc(NULL,
+							sizeof(*finfo), 0);
+			if (finfo == NULL) {
+				PMD_TX_LOG(lio_dev, ERR,
+					   "free buffer alloc failed\n");
+				goto xmit_failed;
+			}
+
+			rte_spinlock_lock(&lio_dev->glist_lock[iq_no]);
+			g = (struct lio_gather *)list_delete_first_node(
+						&lio_dev->glist_head[iq_no]);
+			rte_spinlock_unlock(&lio_dev->glist_lock[iq_no]);
+			if (g == NULL) {
+				PMD_TX_LOG(lio_dev, ERR,
+					   "Transmit scatter gather: glist null!\n");
+				goto xmit_failed;
+			}
+
+			cmdsetup.s.gather = 1;
+			cmdsetup.s.u.gatherptrs = m->nb_segs;
+			lio_prepare_pci_cmd(lio_dev, &ndata.cmd,
+					    &cmdsetup, tag);
+
+			memset(g->sg, 0, g->sg_size);
+			g->sg[0].ptr[0] = rte_mbuf_data_dma_addr(m);
+			lio_add_sg_size(&g->sg[0], m->data_len, 0);
+			pkt_len = m->data_len;
+			finfo->mbuf = m;
+
+			/* First seg taken care above */
+			frags = m->nb_segs - 1;
+			i = 1;
+			m = m->next;
+			while (frags--) {
+				g->sg[(i >> 2)].ptr[(i & 3)] =
+						rte_mbuf_data_dma_addr(m);
+				lio_add_sg_size(&g->sg[(i >> 2)],
+						m->data_len, (i & 3));
+				pkt_len += m->data_len;
+				i++;
+				m = m->next;
+			}
+
+			phyaddr = rte_mem_virt2phy(g->sg);
+			if (phyaddr == RTE_BAD_PHYS_ADDR) {
+				PMD_TX_LOG(lio_dev, ERR, "bad phys addr\n");
+				goto xmit_failed;
+			}
+
+			ndata.cmd.cmd3.dptr = phyaddr;
+			ndata.reqtype = LIO_REQTYPE_NORESP_NET_SG;
+
+			finfo->g = g;
+			finfo->lio_dev = lio_dev;
+			finfo->iq_no = (uint64_t)iq_no;
+			ndata.buf = finfo;
 		}
 
 		ndata.datasize = pkt_len;
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 6813aea..b555bde 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -666,6 +666,17 @@ enum {
 	return subcode2 != subcode1;
 }
 
+static inline void
+lio_add_sg_size(struct lio_sg_entry *sg_entry,
+		uint16_t size, uint32_t pos)
+{
+#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+	sg_entry->u.size[pos] = size;
+#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+	sg_entry->u.size[3 - pos] = size;
+#endif
+}
+
 /* Macro to increment index.
  * Index is incremented by count; if the sum exceeds
  * max, index is wrapped-around to the start.
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 906553c..478a290 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -298,6 +298,30 @@ struct lio_instr_queue {
 	const struct rte_memzone *iq_mz;
 };
 
+/** This structure is used by driver to store information required
+ *  to free the mbuff when the packet has been fetched by Octeon.
+ *  Bytes offset below assume worst-case of a 64-bit system.
+ */
+struct lio_buf_free_info {
+	/** Bytes 1-8. Pointer to network device private structure. */
+	struct lio_device *lio_dev;
+
+	/** Bytes 9-16. Pointer to mbuff. */
+	struct rte_mbuf *mbuf;
+
+	/** Bytes 17-24. Pointer to gather list. */
+	struct lio_gather *g;
+
+	/** Bytes 25-32. Physical address of mbuf->data or gather list. */
+	uint64_t dptr;
+
+	/** Bytes 33-47. Piggybacked soft command, if any */
+	struct lio_soft_command *sc;
+
+	/** Bytes 48-63. iq no */
+	uint64_t iq_no;
+};
+
 /* The Scatter-Gather List Entry. The scatter or gather component used with
  * input instruction has this format.
  */
-- 
1.8.3.1

[PATCH v3 29/46] net/liquidio: add API to flush IQ

[Shijith Thotton]

API to flush instruction queue checks how many packets reached device
and frees associated host buffers using request list.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c |   1 +
 drivers/net/liquidio/lio_rxtx.c   | 187 +++++++++++++++++++++++++++++++++++++-
 drivers/net/liquidio/lio_rxtx.h   |   1 +
 3 files changed, 188 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 5c5aade..b8baa4f 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -281,6 +281,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	 * response arrived or timed-out.
 	 */
 	while ((*sc->status_word == LIO_COMPLETION_WORD_INIT) && --timeout) {
+		lio_flush_iq(lio_dev, lio_dev->instr_queue[sc->iq_no]);
 		lio_process_ordered_list(lio_dev);
 		rte_delay_ms(1);
 	}
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 75ecbd5..64c0385 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -41,6 +41,9 @@
 #include "lio_rxtx.h"
 
 #define LIO_MAX_SG 12
+/* Flush iq if available tx_desc fall below LIO_FLUSH_WM */
+#define LIO_FLUSH_WM(_iq) ((_iq)->max_count / 2)
+#define LIO_PKT_IN_DONE_CNT_MASK 0x00000000FFFFFFFFULL
 
 static void
 lio_droq_compute_max_packet_bufs(struct lio_droq *droq)
@@ -977,6 +980,146 @@
 	iq->request_list[idx].reqtype = reqtype;
 }
 
+static inline void
+lio_free_netsgbuf(void *buf)
+{
+	struct lio_buf_free_info *finfo = buf;
+	struct lio_device *lio_dev = finfo->lio_dev;
+	struct rte_mbuf *m = finfo->mbuf;
+	struct lio_gather *g = finfo->g;
+	uint8_t iq = finfo->iq_no;
+
+	/* This will take care of multiple segments also */
+	rte_pktmbuf_free(m);
+
+	rte_spinlock_lock(&lio_dev->glist_lock[iq]);
+	STAILQ_INSERT_TAIL(&lio_dev->glist_head[iq], &g->list, entries);
+	rte_spinlock_unlock(&lio_dev->glist_lock[iq]);
+	rte_free(finfo);
+}
+
+/* Can only run in process context */
+static int
+lio_process_iq_request_list(struct lio_device *lio_dev,
+			    struct lio_instr_queue *iq)
+{
+	struct octeon_instr_irh *irh = NULL;
+	uint32_t old = iq->flush_index;
+	struct lio_soft_command *sc;
+	uint32_t inst_count = 0;
+	int reqtype;
+	void *buf;
+
+	while (old != iq->lio_read_index) {
+		reqtype = iq->request_list[old].reqtype;
+		buf     = iq->request_list[old].buf;
+
+		if (reqtype == LIO_REQTYPE_NONE)
+			goto skip_this;
+
+		switch (reqtype) {
+		case LIO_REQTYPE_NORESP_NET:
+			rte_pktmbuf_free((struct rte_mbuf *)buf);
+			break;
+		case LIO_REQTYPE_NORESP_NET_SG:
+			lio_free_netsgbuf(buf);
+			break;
+		case LIO_REQTYPE_SOFT_COMMAND:
+			sc = buf;
+			irh = (struct octeon_instr_irh *)&sc->cmd.cmd3.irh;
+			if (irh->rflag) {
+				/* We're expecting a response from Octeon.
+				 * It's up to lio_process_ordered_list() to
+				 * process sc. Add sc to the ordered soft
+				 * command response list because we expect
+				 * a response from Octeon.
+				 */
+				rte_spinlock_lock(&lio_dev->response_list.lock);
+				rte_atomic64_inc(
+				    &lio_dev->response_list.pending_req_count);
+				STAILQ_INSERT_TAIL(
+					&lio_dev->response_list.head,
+					&sc->node, entries);
+				rte_spinlock_unlock(
+						&lio_dev->response_list.lock);
+			} else {
+				if (sc->callback) {
+					/* This callback must not sleep */
+					sc->callback(LIO_REQUEST_DONE,
+						     sc->callback_arg);
+				}
+			}
+			break;
+		default:
+			lio_dev_err(lio_dev,
+				    "Unknown reqtype: %d buf: %p at idx %d\n",
+				    reqtype, buf, old);
+		}
+
+		iq->request_list[old].buf = NULL;
+		iq->request_list[old].reqtype = 0;
+
+skip_this:
+		inst_count++;
+		old = lio_incr_index(old, 1, iq->max_count);
+	}
+
+	iq->flush_index = old;
+
+	return inst_count;
+}
+
+static void
+lio_update_read_index(struct lio_instr_queue *iq)
+{
+	uint32_t pkt_in_done = rte_read32(iq->inst_cnt_reg);
+	uint32_t last_done;
+
+	last_done = pkt_in_done - iq->pkt_in_done;
+	iq->pkt_in_done = pkt_in_done;
+
+	/* Add last_done and modulo with the IQ size to get new index */
+	iq->lio_read_index = (iq->lio_read_index +
+			(uint32_t)(last_done & LIO_PKT_IN_DONE_CNT_MASK)) %
+			iq->max_count;
+}
+
+int
+lio_flush_iq(struct lio_device *lio_dev, struct lio_instr_queue *iq)
+{
+	uint32_t tot_inst_processed = 0;
+	uint32_t inst_processed = 0;
+	int tx_done = 1;
+
+	if (rte_atomic64_test_and_set(&iq->iq_flush_running) == 0)
+		return tx_done;
+
+	rte_spinlock_lock(&iq->lock);
+
+	lio_update_read_index(iq);
+
+	do {
+		/* Process any outstanding IQ packets. */
+		if (iq->flush_index == iq->lio_read_index)
+			break;
+
+		inst_processed = lio_process_iq_request_list(lio_dev, iq);
+
+		if (inst_processed)
+			rte_atomic64_sub(&iq->instr_pending, inst_processed);
+
+		tot_inst_processed += inst_processed;
+		inst_processed = 0;
+
+	} while (1);
+
+	rte_spinlock_unlock(&iq->lock);
+
+	rte_atomic64_clear(&iq->iq_flush_running);
+
+	return tx_done;
+}
+
 static int
 lio_send_command(struct lio_device *lio_dev, uint32_t iq_no, void *cmd,
 		 void *buf, uint32_t datasize __rte_unused, uint32_t reqtype)
@@ -1385,6 +1528,35 @@ struct lio_soft_command *
 	lio_dev->num_iqs--;
 }
 
+static inline uint32_t
+lio_iq_get_available(struct lio_device *lio_dev, uint32_t q_no)
+{
+	return ((lio_dev->instr_queue[q_no]->max_count - 1) -
+		(uint32_t)rte_atomic64_read(
+				&lio_dev->instr_queue[q_no]->instr_pending));
+}
+
+static inline int
+lio_iq_is_full(struct lio_device *lio_dev, uint32_t q_no)
+{
+	return ((uint32_t)rte_atomic64_read(
+				&lio_dev->instr_queue[q_no]->instr_pending) >=
+				(lio_dev->instr_queue[q_no]->max_count - 2));
+}
+
+static int
+lio_dev_cleanup_iq(struct lio_device *lio_dev, int iq_no)
+{
+	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
+	uint32_t count = 10000;
+
+	while ((lio_iq_get_available(lio_dev, iq_no) < LIO_FLUSH_WM(iq)) &&
+			--count)
+		lio_flush_iq(lio_dev, iq);
+
+	return count ? 0 : 1;
+}
+
 /** Send data packet to the device
  *  @param lio_dev - lio device pointer
  *  @param ndata   - control structure with queueing, and buffer information
@@ -1421,6 +1593,8 @@ struct lio_soft_command *
 		goto xmit_failed;
 	}
 
+	lio_dev_cleanup_iq(lio_dev, iq_no);
+
 	for (i = 0; i < nb_pkts; i++) {
 		uint32_t pkt_len = 0;
 
@@ -1432,6 +1606,14 @@ struct lio_soft_command *
 		ndata.buf = m;
 
 		ndata.q_no = iq_no;
+		if (lio_iq_is_full(lio_dev, ndata.q_no)) {
+			if (lio_dev_cleanup_iq(lio_dev, iq_no)) {
+				PMD_TX_LOG(lio_dev, ERR,
+					   "Transmit failed iq:%d full\n",
+					   ndata.q_no);
+				break;
+			}
+		}
 
 		cmdsetup.cmd_setup64 = 0;
 		cmdsetup.s.iq_no = iq_no;
@@ -1524,8 +1706,11 @@ struct lio_soft_command *
 			break;
 		}
 
-		if (unlikely(status == LIO_IQ_SEND_STOP))
+		if (unlikely(status == LIO_IQ_SEND_STOP)) {
 			PMD_TX_LOG(lio_dev, DEBUG, "iq full\n");
+			/* create space as iq is full */
+			lio_dev_cleanup_iq(lio_dev, iq_no);
+		}
 
 		processed++;
 	}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index b555bde..0a4cc2b 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -706,6 +706,7 @@ uint16_t lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts,
 int lio_setup_iq(struct lio_device *lio_dev, int q_index,
 		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
 		 unsigned int socket_id);
+int lio_flush_iq(struct lio_device *lio_dev, struct lio_instr_queue *iq);
 void lio_delete_instruction_queue(struct lio_device *lio_dev, int iq_no);
 /** Setup instruction queue zero for the device
  *  @param lio_dev which lio device to setup
-- 
1.8.3.1

[PATCH v3 30/46] net/liquidio: add API to release Tx queue

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 31 +++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.c   |  2 +-
 drivers/net/liquidio/lio_rxtx.h   |  1 +
 3 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index b8baa4f..97acfcc 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -216,6 +216,36 @@
 	return 0;
 }
 
+/**
+ * Release the transmit queue/ringbuffer. Called by
+ * the upper layers.
+ *
+ * @param txq
+ *    Opaque pointer to the transmit queue to release
+ *
+ * @return
+ *    - nothing
+ */
+static void
+lio_dev_tx_queue_release(void *txq)
+{
+	struct lio_instr_queue *tq = txq;
+	struct lio_device *lio_dev = tq->lio_dev;
+	uint32_t fw_mapped_iq_no;
+
+	/* Run time queue deletion not supported */
+	if (lio_dev->port_configured)
+		return;
+
+	if (tq != NULL) {
+		/* Free sg_list */
+		lio_delete_sglist(tq);
+
+		fw_mapped_iq_no = tq->txpciq.s.q_no;
+		lio_delete_instruction_queue(tq->lio_dev, fw_mapped_iq_no);
+	}
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -387,6 +417,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
+	.tx_queue_release	= lio_dev_tx_queue_release,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 64c0385..6a1d265 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1439,7 +1439,7 @@ struct lio_soft_command *
 	return node;
 }
 
-static void
+void
 lio_delete_sglist(struct lio_instr_queue *txq)
 {
 	struct lio_device *lio_dev = txq->lio_dev;
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 0a4cc2b..964a884 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -699,6 +699,7 @@ uint16_t lio_dev_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			   uint16_t budget);
 void lio_delete_droq_queue(struct lio_device *lio_dev, int oq_no);
 
+void lio_delete_sglist(struct lio_instr_queue *txq);
 int lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
 		      int fw_mapped_iq, int num_descs, unsigned int socket_id);
 uint16_t lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts,
-- 
1.8.3.1

[PATCH v3 31/46] net/liquidio: add APIs to start device and update link

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 doc/guides/nics/features/liquidio.ini   |   2 +
 drivers/net/liquidio/base/lio_hw_defs.h |   1 +
 drivers/net/liquidio/lio_ethdev.c       | 182 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  13 +++
 drivers/net/liquidio/lio_rxtx.c         |   2 +-
 drivers/net/liquidio/lio_struct.h       |   2 +
 6 files changed, 201 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
index d4bbea1..29df586 100644
--- a/doc/guides/nics/features/liquidio.ini
+++ b/doc/guides/nics/features/liquidio.ini
@@ -4,6 +4,8 @@
 ; Refer to default.ini for the full list of available PMD features.
 ;
 [Features]
+Link status          = Y
+Link status event    = Y
 Jumbo frame          = Y
 Scattered Rx         = Y
 CRC offload          = Y
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 1c1ad8e..e3f18e3 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -122,6 +122,7 @@ enum octeon_tag_type {
 /** LIO_OPCODE subcodes */
 /* This subcode is sent by core PCI driver to indicate cores are ready. */
 #define LIO_OPCODE_NW_DATA		0x02 /* network packet data */
+#define LIO_OPCODE_INFO			0x04
 #define LIO_OPCODE_IF_CFG		0x09
 
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 97acfcc..4962cad 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -41,6 +41,32 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+/**
+ * Atomically writes the link status information into global
+ * structure rte_eth_dev.
+ *
+ * @param eth_dev
+ *   - Pointer to the structure rte_eth_dev to read from.
+ *   - Pointer to the buffer to be saved with the link status.
+ *
+ * @return
+ *   - On success, zero.
+ *   - On failure, negative value.
+ */
+static inline int
+lio_dev_atomic_write_link_status(struct rte_eth_dev *eth_dev,
+				 struct rte_eth_link *link)
+{
+	struct rte_eth_link *dst = &eth_dev->data->dev_link;
+	struct rte_eth_link *src = link;
+
+	if (rte_atomic64_cmpset((uint64_t *)dst, *(uint64_t *)dst,
+				*(uint64_t *)src) == 0)
+		return -1;
+
+	return 0;
+}
+
 static uint64_t
 lio_hweight64(uint64_t w)
 {
@@ -55,6 +81,49 @@
 	return (res + (res >> 32)) & 0x00000000000000FFul;
 }
 
+static int
+lio_dev_link_update(struct rte_eth_dev *eth_dev,
+		    int wait_to_complete __rte_unused)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct rte_eth_link link, old;
+
+	/* Initialize */
+	link.link_status = ETH_LINK_DOWN;
+	link.link_speed = ETH_SPEED_NUM_NONE;
+	link.link_duplex = ETH_LINK_HALF_DUPLEX;
+	memset(&old, 0, sizeof(old));
+
+	/* Return what we found */
+	if (lio_dev->linfo.link.s.link_up == 0) {
+		/* Interface is down */
+		if (lio_dev_atomic_write_link_status(eth_dev, &link))
+			return -1;
+		if (link.link_status == old.link_status)
+			return -1;
+		return 0;
+	}
+
+	link.link_status = ETH_LINK_UP; /* Interface is up */
+	link.link_duplex = ETH_LINK_FULL_DUPLEX;
+	switch (lio_dev->linfo.link.s.speed) {
+	case LIO_LINK_SPEED_10000:
+		link.link_speed = ETH_SPEED_NUM_10G;
+		break;
+	default:
+		link.link_speed = ETH_SPEED_NUM_NONE;
+		link.link_duplex = ETH_LINK_HALF_DUPLEX;
+	}
+
+	if (lio_dev_atomic_write_link_status(eth_dev, &link))
+		return -1;
+
+	if (link.link_status == old.link_status)
+		return -1;
+
+	return 0;
+}
+
 /**
  * Setup our receive queue/ringbuffer. This is the
  * queue the Octeon uses to send us packets and
@@ -246,6 +315,115 @@
 	}
 }
 
+/**
+ * Api to check link state.
+ */
+static void
+lio_dev_get_link_status(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
+	struct lio_link_status_resp *resp;
+	union octeon_link_status *ls;
+	struct lio_soft_command *sc;
+	uint32_t resp_size;
+
+	if (!lio_dev->intf_open)
+		return;
+
+	resp_size = sizeof(struct lio_link_status_resp);
+	sc = lio_alloc_soft_command(lio_dev, 0, resp_size, 0);
+	if (sc == NULL)
+		return;
+
+	resp = (struct lio_link_status_resp *)sc->virtrptr;
+	lio_prepare_soft_command(lio_dev, sc, LIO_OPCODE,
+				 LIO_OPCODE_INFO, 0, 0, 0);
+
+	/* Setting wait time in seconds */
+	sc->wait_time = LIO_MAX_CMD_TIMEOUT / 1000;
+
+	if (lio_send_soft_command(lio_dev, sc) == LIO_IQ_SEND_FAILED)
+		goto get_status_fail;
+
+	while ((*sc->status_word == LIO_COMPLETION_WORD_INIT) && --timeout) {
+		lio_flush_iq(lio_dev, lio_dev->instr_queue[sc->iq_no]);
+		rte_delay_ms(1);
+	}
+
+	if (resp->status)
+		goto get_status_fail;
+
+	ls = &resp->link_info.link;
+
+	lio_swap_8B_data((uint64_t *)ls, sizeof(union octeon_link_status) >> 3);
+
+	if (lio_dev->linfo.link.link_status64 != ls->link_status64) {
+		lio_dev->linfo.link.link_status64 = ls->link_status64;
+		lio_dev_link_update(eth_dev, 0);
+	}
+
+	lio_free_soft_command(sc);
+
+	return;
+
+get_status_fail:
+	lio_free_soft_command(sc);
+}
+
+/* This function will be invoked every LSC_TIMEOUT ns (100ms)
+ * and will update link state if it changes.
+ */
+static void
+lio_sync_link_state_check(void *eth_dev)
+{
+	struct lio_device *lio_dev =
+		(((struct rte_eth_dev *)eth_dev)->data->dev_private);
+
+	if (lio_dev->port_configured)
+		lio_dev_get_link_status(eth_dev);
+
+	/* Schedule periodic link status check.
+	 * Stop check if interface is close and start again while opening.
+	 */
+	if (lio_dev->intf_open)
+		rte_eal_alarm_set(LIO_LSC_TIMEOUT, lio_sync_link_state_check,
+				  eth_dev);
+}
+
+static int
+lio_dev_start(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	int ret = 0;
+
+	lio_dev_info(lio_dev, "Starting port %d\n", eth_dev->data->port_id);
+
+	if (lio_dev->fn_list.enable_io_queues(lio_dev))
+		return -1;
+
+	/* Ready for link status updates */
+	lio_dev->intf_open = 1;
+	rte_mb();
+
+	/* start polling for lsc */
+	ret = rte_eal_alarm_set(LIO_LSC_TIMEOUT,
+				lio_sync_link_state_check,
+				eth_dev);
+	if (ret) {
+		lio_dev_err(lio_dev,
+			    "link state check handler creation failed\n");
+		goto dev_lsc_handle_error;
+	}
+
+	return 0;
+
+dev_lsc_handle_error:
+	lio_dev->intf_open = 0;
+
+	return ret;
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -388,6 +566,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
+	lio_dev_link_update(eth_dev, 0);
+
 	lio_dev->port_configured = 1;
 
 	lio_free_soft_command(sc);
@@ -414,6 +594,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 /* Define our ethernet definitions */
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
+	.dev_start		= lio_dev_start,
+	.link_update		= lio_dev_link_update,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 22e3d83..98ff493 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -38,10 +38,17 @@
 
 #include "lio_struct.h"
 
+/* timeout to check link state updates from firmware in us */
+#define LIO_LSC_TIMEOUT		100000 /* 100000us (100ms) */
 #define LIO_MAX_CMD_TIMEOUT     10000 /* 10000ms (10s) */
 
 #define LIO_DEV(_eth_dev)		((_eth_dev)->data->dev_private)
 
+enum lio_bus_speed {
+	LIO_LINK_SPEED_UNKNOWN  = 0,
+	LIO_LINK_SPEED_10000    = 10000
+};
+
 struct octeon_if_cfg_info {
 	uint64_t iqmask;	/** mask for IQs enabled for the port */
 	uint64_t oqmask;	/** mask for OQs enabled for the port */
@@ -73,4 +80,10 @@ struct lio_if_cfg_resp {
 	struct octeon_if_cfg_info cfg_info;
 	uint64_t status;
 };
+
+struct lio_link_status_resp {
+	uint64_t rh;
+	struct octeon_link_info link_info;
+	uint64_t status;
+};
 #endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 6a1d265..d7e17bf 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1587,7 +1587,7 @@ struct lio_soft_command *
 	lio_dev = txq->lio_dev;
 	iq_no = txq->txpciq.s.q_no;
 
-	if (!lio_dev->linfo.link.s.link_up) {
+	if (!lio_dev->intf_open || !lio_dev->linfo.link.s.link_up) {
 		PMD_TX_LOG(lio_dev, ERR, "Transmit failed link_status : %d\n",
 			   lio_dev->linfo.link.s.link_up);
 		goto xmit_failed;
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 478a290..da08fe4 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -569,6 +569,8 @@ struct lio_device {
 	/** The state of this device */
 	rte_atomic64_t status;
 
+	uint8_t intf_open;
+
 	struct octeon_link_info linfo;
 
 	uint8_t *hw_addr;
-- 
1.8.3.1

[PATCH v3 32/46] net/liquidio: add APIs to alloc and send control command

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  1 +
 drivers/net/liquidio/lio_ethdev.h       |  6 +++
 drivers/net/liquidio/lio_rxtx.c         | 82 +++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h         | 44 ++++++++++++++++++
 4 files changed, 133 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index e3f18e3..d38c835 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -122,6 +122,7 @@ enum octeon_tag_type {
 /** LIO_OPCODE subcodes */
 /* This subcode is sent by core PCI driver to indicate cores are ready. */
 #define LIO_OPCODE_NW_DATA		0x02 /* network packet data */
+#define LIO_OPCODE_CMD			0x03
 #define LIO_OPCODE_INFO			0x04
 #define LIO_OPCODE_IF_CFG		0x09
 
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 98ff493..7b5343a 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -44,6 +44,12 @@
 
 #define LIO_DEV(_eth_dev)		((_eth_dev)->data->dev_private)
 
+/* LIO Response condition variable */
+struct lio_dev_ctrl_cmd {
+	struct rte_eth_dev *eth_dev;
+	uint64_t cond;
+};
+
 enum lio_bus_speed {
 	LIO_LINK_SPEED_UNKNOWN  = 0,
 	LIO_LINK_SPEED_10000    = 10000
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index d7e17bf..c12960c 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1557,6 +1557,88 @@ struct lio_soft_command *
 	return count ? 0 : 1;
 }
 
+static void
+lio_ctrl_cmd_callback(uint32_t status __rte_unused, void *sc_ptr)
+{
+	struct lio_soft_command *sc = sc_ptr;
+	struct lio_dev_ctrl_cmd *ctrl_cmd;
+	struct lio_ctrl_pkt *ctrl_pkt;
+
+	ctrl_pkt = (struct lio_ctrl_pkt *)sc->ctxptr;
+	ctrl_cmd = ctrl_pkt->ctrl_cmd;
+	ctrl_cmd->cond = 1;
+
+	lio_free_soft_command(sc);
+}
+
+static inline struct lio_soft_command *
+lio_alloc_ctrl_pkt_sc(struct lio_device *lio_dev,
+		      struct lio_ctrl_pkt *ctrl_pkt)
+{
+	struct lio_soft_command *sc = NULL;
+	uint32_t uddsize, datasize;
+	uint32_t rdatasize;
+	uint8_t *data;
+
+	uddsize = (uint32_t)(ctrl_pkt->ncmd.s.more * 8);
+
+	datasize = OCTEON_CMD_SIZE + uddsize;
+	rdatasize = (ctrl_pkt->wait_time) ? 16 : 0;
+
+	sc = lio_alloc_soft_command(lio_dev, datasize,
+				    rdatasize, sizeof(struct lio_ctrl_pkt));
+	if (sc == NULL)
+		return NULL;
+
+	rte_memcpy(sc->ctxptr, ctrl_pkt, sizeof(struct lio_ctrl_pkt));
+
+	data = (uint8_t *)sc->virtdptr;
+
+	rte_memcpy(data, &ctrl_pkt->ncmd, OCTEON_CMD_SIZE);
+
+	lio_swap_8B_data((uint64_t *)data, OCTEON_CMD_SIZE >> 3);
+
+	if (uddsize) {
+		/* Endian-Swap for UDD should have been done by caller. */
+		rte_memcpy(data + OCTEON_CMD_SIZE, ctrl_pkt->udd, uddsize);
+	}
+
+	sc->iq_no = (uint32_t)ctrl_pkt->iq_no;
+
+	lio_prepare_soft_command(lio_dev, sc,
+				 LIO_OPCODE, LIO_OPCODE_CMD,
+				 0, 0, 0);
+
+	sc->callback = lio_ctrl_cmd_callback;
+	sc->callback_arg = sc;
+	sc->wait_time = ctrl_pkt->wait_time;
+
+	return sc;
+}
+
+int
+lio_send_ctrl_pkt(struct lio_device *lio_dev, struct lio_ctrl_pkt *ctrl_pkt)
+{
+	struct lio_soft_command *sc = NULL;
+	int retval;
+
+	sc = lio_alloc_ctrl_pkt_sc(lio_dev, ctrl_pkt);
+	if (sc == NULL) {
+		lio_dev_err(lio_dev, "soft command allocation failed\n");
+		return -1;
+	}
+
+	retval = lio_send_soft_command(lio_dev, sc);
+	if (retval == LIO_IQ_SEND_FAILED) {
+		lio_free_soft_command(sc);
+		lio_dev_err(lio_dev, "Port: %d soft command: %d send failed status: %x\n",
+			    lio_dev->port_id, ctrl_pkt->ncmd.s.cmd, retval);
+		return -1;
+	}
+
+	return retval;
+}
+
 /** Send data packet to the device
  *  @param lio_dev - lio device pointer
  *  @param ndata   - control structure with queueing, and buffer information
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 964a884..95d0007 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -249,6 +249,40 @@ struct lio_iq_post_status {
 
 #define OCTEON_CMD_SIZE (sizeof(union octeon_cmd))
 
+/* Maximum number of 8-byte words can be
+ * sent in a NIC control message.
+ */
+#define LIO_MAX_NCTRL_UDD	32
+
+/* Structure of control information passed by driver to the BASE
+ * layer when sending control commands to Octeon device software.
+ */
+struct lio_ctrl_pkt {
+	/** Command to be passed to the Octeon device software. */
+	union octeon_cmd ncmd;
+
+	/** Send buffer */
+	void *data;
+	uint64_t dmadata;
+
+	/** Response buffer */
+	void *rdata;
+	uint64_t dmardata;
+
+	/** Additional data that may be needed by some commands. */
+	uint64_t udd[LIO_MAX_NCTRL_UDD];
+
+	/** Input queue to use to send this command. */
+	uint64_t iq_no;
+
+	/** Time to wait for Octeon software to respond to this control command.
+	 *  If wait_time is 0, BASE assumes no response is expected.
+	 */
+	size_t wait_time;
+
+	struct lio_dev_ctrl_cmd *ctrl_cmd;
+};
+
 /** Structure of data information passed by driver to the BASE
  *  layer when forwarding data to Octeon device software.
  */
@@ -570,6 +604,16 @@ int lio_send_soft_command(struct lio_device *lio_dev,
 			  struct lio_soft_command *sc);
 void lio_free_soft_command(struct lio_soft_command *sc);
 
+/** Send control packet to the device
+ *  @param lio_dev - lio device pointer
+ *  @param nctrl   - control structure with command, timeout, and callback info
+ *
+ *  @returns IQ_FAILED if it failed to add to the input queue. IQ_STOP if it the
+ *  queue should be stopped, and LIO_IQ_SEND_OK if it sent okay.
+ */
+int lio_send_ctrl_pkt(struct lio_device *lio_dev,
+		      struct lio_ctrl_pkt *ctrl_pkt);
+
 /** Maximum ordered requests to process in every invocation of
  *  lio_process_ordered_list(). The function will continue to process requests
  *  as long as it can find one that has finished processing. If it keeps
-- 
1.8.3.1

[PATCH v3 33/46] net/liquidio: add API to control Rx

[Shijith Thotton]

Enable or disable packet reception.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  3 ++
 drivers/net/liquidio/lio_ethdev.c       | 59 +++++++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index d38c835..59668c0 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -128,6 +128,9 @@ enum octeon_tag_type {
 
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
 
+/* NIC Command types */
+#define LIO_CMD_RX_CTL			0x4
+
 /* RX(packets coming from wire) Checksum verification flags */
 /* TCP/UDP csum */
 #define LIO_L4_CSUM_VERIFIED		0x1
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 4962cad..c698c70 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -41,6 +41,61 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+/* Wait for control command to reach nic. */
+static uint16_t
+lio_wait_for_ctrl_cmd(struct lio_device *lio_dev,
+		      struct lio_dev_ctrl_cmd *ctrl_cmd)
+{
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
+
+	while ((ctrl_cmd->cond == 0) && --timeout) {
+		lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+		rte_delay_ms(1);
+	}
+
+	return !timeout;
+}
+
+/**
+ * \brief Send Rx control command
+ * @param eth_dev Pointer to the structure rte_eth_dev
+ * @param start_stop whether to start or stop
+ */
+static int
+lio_send_rx_ctrl_cmd(struct rte_eth_dev *eth_dev, int start_stop)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_RX_CTL;
+	ctrl_pkt.ncmd.s.param1 = start_stop;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send RX Control message\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "RX Control command timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Atomically writes the link status information into global
  * structure rte_eth_dev.
@@ -402,6 +457,9 @@
 	if (lio_dev->fn_list.enable_io_queues(lio_dev))
 		return -1;
 
+	if (lio_send_rx_ctrl_cmd(eth_dev, 1))
+		return -1;
+
 	/* Ready for link status updates */
 	lio_dev->intf_open = 1;
 	rte_mb();
@@ -420,6 +478,7 @@
 
 dev_lsc_handle_error:
 	lio_dev->intf_open = 0;
+	lio_send_rx_ctrl_cmd(eth_dev, 0);
 
 	return ret;
 }
-- 
1.8.3.1

[PATCH v3 34/46] net/liquidio: add RSS support

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 doc/guides/nics/features/liquidio.ini   |   3 +
 drivers/net/liquidio/base/lio_hw_defs.h |  26 +++
 drivers/net/liquidio/lio_ethdev.c       | 336 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  21 ++
 drivers/net/liquidio/lio_struct.h       |  16 ++
 5 files changed, 402 insertions(+)

diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
index 29df586..8760849 100644
--- a/doc/guides/nics/features/liquidio.ini
+++ b/doc/guides/nics/features/liquidio.ini
@@ -8,6 +8,9 @@ Link status          = Y
 Link status event    = Y
 Jumbo frame          = Y
 Scattered Rx         = Y
+RSS hash             = Y
+RSS key update       = Y
+RSS reta update      = Y
 CRC offload          = Y
 L3 checksum offload  = Y
 L4 checksum offload  = Y
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 59668c0..8272162 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -130,12 +130,38 @@ enum octeon_tag_type {
 
 /* NIC Command types */
 #define LIO_CMD_RX_CTL			0x4
+#define LIO_CMD_SET_RSS			0xD
 
 /* RX(packets coming from wire) Checksum verification flags */
 /* TCP/UDP csum */
 #define LIO_L4_CSUM_VERIFIED		0x1
 #define LIO_IP_CSUM_VERIFIED		0x2
 
+/* RSS */
+#define LIO_RSS_PARAM_DISABLE_RSS		0x10
+#define LIO_RSS_PARAM_HASH_KEY_UNCHANGED	0x08
+#define LIO_RSS_PARAM_ITABLE_UNCHANGED		0x04
+#define LIO_RSS_PARAM_HASH_INFO_UNCHANGED	0x02
+
+#define LIO_RSS_HASH_IPV4			0x100
+#define LIO_RSS_HASH_TCP_IPV4			0x200
+#define LIO_RSS_HASH_IPV6			0x400
+#define LIO_RSS_HASH_TCP_IPV6			0x1000
+#define LIO_RSS_HASH_IPV6_EX			0x800
+#define LIO_RSS_HASH_TCP_IPV6_EX		0x2000
+
+#define LIO_RSS_OFFLOAD_ALL (		\
+		LIO_RSS_HASH_IPV4 |	\
+		LIO_RSS_HASH_TCP_IPV4 |	\
+		LIO_RSS_HASH_IPV6 |	\
+		LIO_RSS_HASH_TCP_IPV6 |	\
+		LIO_RSS_HASH_IPV6_EX |	\
+		LIO_RSS_HASH_TCP_IPV6_EX)
+
+#define LIO_RSS_MAX_TABLE_SZ		128
+#define LIO_RSS_MAX_KEY_SZ		40
+#define LIO_RSS_PARAM_SIZE		16
+
 /* Interface flags communicated between host driver and core app. */
 enum lio_ifflags {
 	LIO_IFFLAG_UNICAST	= 0x10
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index c698c70..58a932e 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -41,6 +41,15 @@
 #include "lio_ethdev.h"
 #include "lio_rxtx.h"
 
+/* Default RSS key in use */
+static uint8_t lio_rss_key[40] = {
+	0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2,
+	0x41, 0x67, 0x25, 0x3D, 0x43, 0xA3, 0x8F, 0xB0,
+	0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+	0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C,
+	0x6A, 0x42, 0xB7, 0x3B, 0xBE, 0xAC, 0x01, 0xFA,
+};
+
 /* Wait for control command to reach nic. */
 static uint16_t
 lio_wait_for_ctrl_cmd(struct lio_device *lio_dev,
@@ -96,6 +105,267 @@
 	return 0;
 }
 
+static int
+lio_dev_rss_reta_update(struct rte_eth_dev *eth_dev,
+			struct rte_eth_rss_reta_entry64 *reta_conf,
+			uint16_t reta_size)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	struct lio_rss_set *rss_param;
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+	int i, j, index;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't update reta\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	if (reta_size != LIO_RSS_MAX_TABLE_SZ) {
+		lio_dev_err(lio_dev,
+			    "The size of hash lookup table configured (%d) doesn't match the number hardware can supported (%d)\n",
+			    reta_size, LIO_RSS_MAX_TABLE_SZ);
+		return -EINVAL;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	rss_param = (struct lio_rss_set *)&ctrl_pkt.udd[0];
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_SET_RSS;
+	ctrl_pkt.ncmd.s.more = sizeof(struct lio_rss_set) >> 3;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	rss_param->param.flags = 0xF;
+	rss_param->param.flags &= ~LIO_RSS_PARAM_ITABLE_UNCHANGED;
+	rss_param->param.itablesize = LIO_RSS_MAX_TABLE_SZ;
+
+	for (i = 0; i < (reta_size / RTE_RETA_GROUP_SIZE); i++) {
+		for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) {
+			if ((reta_conf[i].mask) & ((uint64_t)1 << j)) {
+				index = (i * RTE_RETA_GROUP_SIZE) + j;
+				rss_state->itable[index] = reta_conf[i].reta[j];
+			}
+		}
+	}
+
+	rss_state->itable_size = LIO_RSS_MAX_TABLE_SZ;
+	memcpy(rss_param->itable, rss_state->itable, rss_state->itable_size);
+
+	lio_swap_8B_data((uint64_t *)rss_param, LIO_RSS_PARAM_SIZE >> 3);
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to set rss hash\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "Set rss hash timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+static int
+lio_dev_rss_reta_query(struct rte_eth_dev *eth_dev,
+		       struct rte_eth_rss_reta_entry64 *reta_conf,
+		       uint16_t reta_size)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	int i, num;
+
+	if (reta_size != LIO_RSS_MAX_TABLE_SZ) {
+		lio_dev_err(lio_dev,
+			    "The size of hash lookup table configured (%d) doesn't match the number hardware can supported (%d)\n",
+			    reta_size, LIO_RSS_MAX_TABLE_SZ);
+		return -EINVAL;
+	}
+
+	num = reta_size / RTE_RETA_GROUP_SIZE;
+
+	for (i = 0; i < num; i++) {
+		memcpy(reta_conf->reta,
+		       &rss_state->itable[i * RTE_RETA_GROUP_SIZE],
+		       RTE_RETA_GROUP_SIZE);
+		reta_conf++;
+	}
+
+	return 0;
+}
+
+static int
+lio_dev_rss_hash_conf_get(struct rte_eth_dev *eth_dev,
+			  struct rte_eth_rss_conf *rss_conf)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	uint8_t *hash_key = NULL;
+	uint64_t rss_hf = 0;
+
+	if (rss_state->hash_disable) {
+		lio_dev_info(lio_dev, "RSS disabled in nic\n");
+		rss_conf->rss_hf = 0;
+		return 0;
+	}
+
+	/* Get key value */
+	hash_key = rss_conf->rss_key;
+	if (hash_key != NULL)
+		memcpy(hash_key, rss_state->hash_key, rss_state->hash_key_size);
+
+	if (rss_state->ip)
+		rss_hf |= ETH_RSS_IPV4;
+	if (rss_state->tcp_hash)
+		rss_hf |= ETH_RSS_NONFRAG_IPV4_TCP;
+	if (rss_state->ipv6)
+		rss_hf |= ETH_RSS_IPV6;
+	if (rss_state->ipv6_tcp_hash)
+		rss_hf |= ETH_RSS_NONFRAG_IPV6_TCP;
+	if (rss_state->ipv6_ex)
+		rss_hf |= ETH_RSS_IPV6_EX;
+	if (rss_state->ipv6_tcp_ex_hash)
+		rss_hf |= ETH_RSS_IPV6_TCP_EX;
+
+	rss_conf->rss_hf = rss_hf;
+
+	return 0;
+}
+
+static int
+lio_dev_rss_hash_update(struct rte_eth_dev *eth_dev,
+			struct rte_eth_rss_conf *rss_conf)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	struct lio_rss_set *rss_param;
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't update hash\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	rss_param = (struct lio_rss_set *)&ctrl_pkt.udd[0];
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_SET_RSS;
+	ctrl_pkt.ncmd.s.more = sizeof(struct lio_rss_set) >> 3;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	rss_param->param.flags = 0xF;
+
+	if (rss_conf->rss_key) {
+		rss_param->param.flags &= ~LIO_RSS_PARAM_HASH_KEY_UNCHANGED;
+		rss_state->hash_key_size = LIO_RSS_MAX_KEY_SZ;
+		rss_param->param.hashkeysize = LIO_RSS_MAX_KEY_SZ;
+		memcpy(rss_state->hash_key, rss_conf->rss_key,
+		       rss_state->hash_key_size);
+		memcpy(rss_param->key, rss_state->hash_key,
+		       rss_state->hash_key_size);
+	}
+
+	if ((rss_conf->rss_hf & LIO_RSS_OFFLOAD_ALL) == 0) {
+		/* Can't disable rss through hash flags,
+		 * if it is enabled by default during init
+		 */
+		if (!rss_state->hash_disable)
+			return -EINVAL;
+
+		/* This is for --disable-rss during testpmd launch */
+		rss_param->param.flags |= LIO_RSS_PARAM_DISABLE_RSS;
+	} else {
+		uint32_t hashinfo = 0;
+
+		/* Can't enable rss if disabled by default during init */
+		if (rss_state->hash_disable)
+			return -EINVAL;
+
+		if (rss_conf->rss_hf & ETH_RSS_IPV4) {
+			hashinfo |= LIO_RSS_HASH_IPV4;
+			rss_state->ip = 1;
+		} else {
+			rss_state->ip = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_NONFRAG_IPV4_TCP) {
+			hashinfo |= LIO_RSS_HASH_TCP_IPV4;
+			rss_state->tcp_hash = 1;
+		} else {
+			rss_state->tcp_hash = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_IPV6) {
+			hashinfo |= LIO_RSS_HASH_IPV6;
+			rss_state->ipv6 = 1;
+		} else {
+			rss_state->ipv6 = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_NONFRAG_IPV6_TCP) {
+			hashinfo |= LIO_RSS_HASH_TCP_IPV6;
+			rss_state->ipv6_tcp_hash = 1;
+		} else {
+			rss_state->ipv6_tcp_hash = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_IPV6_EX) {
+			hashinfo |= LIO_RSS_HASH_IPV6_EX;
+			rss_state->ipv6_ex = 1;
+		} else {
+			rss_state->ipv6_ex = 0;
+		}
+
+		if (rss_conf->rss_hf & ETH_RSS_IPV6_TCP_EX) {
+			hashinfo |= LIO_RSS_HASH_TCP_IPV6_EX;
+			rss_state->ipv6_tcp_ex_hash = 1;
+		} else {
+			rss_state->ipv6_tcp_ex_hash = 0;
+		}
+
+		rss_param->param.flags &= ~LIO_RSS_PARAM_HASH_INFO_UNCHANGED;
+		rss_param->param.hashinfo = hashinfo;
+	}
+
+	lio_swap_8B_data((uint64_t *)rss_param, LIO_RSS_PARAM_SIZE >> 3);
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to set rss hash\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "Set rss hash timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Atomically writes the link status information into global
  * structure rte_eth_dev.
@@ -179,6 +449,65 @@
 	return 0;
 }
 
+static void
+lio_dev_rss_configure(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	struct rte_eth_rss_reta_entry64 reta_conf[8];
+	struct rte_eth_rss_conf rss_conf;
+	uint16_t i;
+
+	/* Configure the RSS key and the RSS protocols used to compute
+	 * the RSS hash of input packets.
+	 */
+	rss_conf = eth_dev->data->dev_conf.rx_adv_conf.rss_conf;
+	if ((rss_conf.rss_hf & LIO_RSS_OFFLOAD_ALL) == 0) {
+		rss_state->hash_disable = 1;
+		lio_dev_rss_hash_update(eth_dev, &rss_conf);
+		return;
+	}
+
+	if (rss_conf.rss_key == NULL)
+		rss_conf.rss_key = lio_rss_key; /* Default hash key */
+
+	lio_dev_rss_hash_update(eth_dev, &rss_conf);
+
+	memset(reta_conf, 0, sizeof(reta_conf));
+	for (i = 0; i < LIO_RSS_MAX_TABLE_SZ; i++) {
+		uint8_t q_idx, conf_idx, reta_idx;
+
+		q_idx = (uint8_t)((eth_dev->data->nb_rx_queues > 1) ?
+				  i % eth_dev->data->nb_rx_queues : 0);
+		conf_idx = i / RTE_RETA_GROUP_SIZE;
+		reta_idx = i % RTE_RETA_GROUP_SIZE;
+		reta_conf[conf_idx].reta[reta_idx] = q_idx;
+		reta_conf[conf_idx].mask |= ((uint64_t)1 << reta_idx);
+	}
+
+	lio_dev_rss_reta_update(eth_dev, reta_conf, LIO_RSS_MAX_TABLE_SZ);
+}
+
+static void
+lio_dev_mq_rx_configure(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_rss_ctx *rss_state = &lio_dev->rss_state;
+	struct rte_eth_rss_conf rss_conf;
+
+	switch (eth_dev->data->dev_conf.rxmode.mq_mode) {
+	case ETH_MQ_RX_RSS:
+		lio_dev_rss_configure(eth_dev);
+		break;
+	case ETH_MQ_RX_NONE:
+	/* if mq_mode is none, disable rss mode. */
+	default:
+		memset(&rss_conf, 0, sizeof(rss_conf));
+		rss_state->hash_disable = 1;
+		lio_dev_rss_hash_update(eth_dev, &rss_conf);
+	}
+}
+
 /**
  * Setup our receive queue/ringbuffer. This is the
  * queue the Octeon uses to send us packets and
@@ -464,6 +793,9 @@
 	lio_dev->intf_open = 1;
 	rte_mb();
 
+	/* Configure RSS if device configured with multiple RX queues. */
+	lio_dev_mq_rx_configure(eth_dev);
+
 	/* start polling for lsc */
 	ret = rte_eal_alarm_set(LIO_LSC_TIMEOUT,
 				lio_sync_link_state_check,
@@ -659,6 +991,10 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
 	.tx_queue_release	= lio_dev_tx_queue_release,
+	.reta_update		= lio_dev_rss_reta_update,
+	.reta_query		= lio_dev_rss_reta_query,
+	.rss_hash_conf_get	= lio_dev_rss_hash_conf_get,
+	.rss_hash_update	= lio_dev_rss_hash_update,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 7b5343a..6543061 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -92,4 +92,25 @@ struct lio_link_status_resp {
 	struct octeon_link_info link_info;
 	uint64_t status;
 };
+
+struct lio_rss_set {
+	struct param {
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+		uint64_t flags : 16;
+		uint64_t hashinfo : 32;
+		uint64_t itablesize : 16;
+		uint64_t hashkeysize : 16;
+		uint64_t reserved : 48;
+#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+		uint64_t itablesize : 16;
+		uint64_t hashinfo : 32;
+		uint64_t flags : 16;
+		uint64_t reserved : 48;
+		uint64_t hashkeysize : 16;
+#endif
+	} param;
+
+	uint8_t itable[LIO_RSS_MAX_TABLE_SZ];
+	uint8_t key[LIO_RSS_MAX_KEY_SZ];
+};
 #endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index da08fe4..c2293f7 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -357,6 +357,21 @@ struct lio_gather {
 	struct lio_sg_entry *sg;
 };
 
+struct lio_rss_ctx {
+	uint16_t hash_key_size;
+	uint8_t  hash_key[LIO_RSS_MAX_KEY_SZ];
+	/* Ideally a factor of number of queues */
+	uint8_t  itable[LIO_RSS_MAX_TABLE_SZ];
+	uint8_t  itable_size;
+	uint8_t  ip;
+	uint8_t  tcp_hash;
+	uint8_t  ipv6;
+	uint8_t  ipv6_tcp_hash;
+	uint8_t  ipv6_ex;
+	uint8_t  ipv6_tcp_ex_hash;
+	uint8_t  hash_disable;
+};
+
 struct lio_io_enable {
 	uint64_t iq;
 	uint64_t oq;
@@ -619,6 +634,7 @@ struct lio_device {
 	uint8_t nb_rx_queues;
 	uint8_t nb_tx_queues;
 	uint8_t port_configured;
+	struct lio_rss_ctx rss_state;
 	uint8_t port_id;
 };
 #endif /* _LIO_STRUCT_H_ */
-- 
1.8.3.1

[PATCH v3 35/46] net/liquidio: add API to get device info

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/base/lio_hw_defs.h |  1 +
 drivers/net/liquidio/lio_ethdev.c       | 47 +++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+)

diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 8272162..3ea2e0f 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -126,6 +126,7 @@ enum octeon_tag_type {
 #define LIO_OPCODE_INFO			0x04
 #define LIO_OPCODE_IF_CFG		0x09
 
+#define LIO_MIN_RX_BUF_SIZE		64
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
 
 /* NIC Command types */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 58a932e..e2040b9 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -50,6 +50,18 @@
 	0x6A, 0x42, 0xB7, 0x3B, 0xBE, 0xAC, 0x01, 0xFA,
 };
 
+static const struct rte_eth_desc_lim lio_rx_desc_lim = {
+	.nb_max		= CN23XX_MAX_OQ_DESCRIPTORS,
+	.nb_min		= CN23XX_MIN_OQ_DESCRIPTORS,
+	.nb_align	= 1,
+};
+
+static const struct rte_eth_desc_lim lio_tx_desc_lim = {
+	.nb_max		= CN23XX_MAX_IQ_DESCRIPTORS,
+	.nb_min		= CN23XX_MIN_IQ_DESCRIPTORS,
+	.nb_align	= 1,
+};
+
 /* Wait for control command to reach nic. */
 static uint16_t
 lio_wait_for_ctrl_cmd(struct lio_device *lio_dev,
@@ -105,6 +117,40 @@
 	return 0;
 }
 
+static void
+lio_dev_info_get(struct rte_eth_dev *eth_dev,
+		 struct rte_eth_dev_info *devinfo)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	devinfo->max_rx_queues = lio_dev->max_rx_queues;
+	devinfo->max_tx_queues = lio_dev->max_tx_queues;
+
+	devinfo->min_rx_bufsize = LIO_MIN_RX_BUF_SIZE;
+	devinfo->max_rx_pktlen = LIO_MAX_RX_PKTLEN;
+
+	devinfo->max_mac_addrs = 1;
+
+	devinfo->rx_offload_capa = (DEV_RX_OFFLOAD_IPV4_CKSUM		|
+				    DEV_RX_OFFLOAD_UDP_CKSUM		|
+				    DEV_RX_OFFLOAD_TCP_CKSUM);
+	devinfo->tx_offload_capa = (DEV_TX_OFFLOAD_IPV4_CKSUM		|
+				    DEV_TX_OFFLOAD_UDP_CKSUM		|
+				    DEV_TX_OFFLOAD_TCP_CKSUM);
+
+	devinfo->rx_desc_lim = lio_rx_desc_lim;
+	devinfo->tx_desc_lim = lio_tx_desc_lim;
+
+	devinfo->reta_size = LIO_RSS_MAX_TABLE_SZ;
+	devinfo->hash_key_size = LIO_RSS_MAX_KEY_SZ;
+	devinfo->flow_type_rss_offloads = (ETH_RSS_IPV4			|
+					   ETH_RSS_NONFRAG_IPV4_TCP	|
+					   ETH_RSS_IPV6			|
+					   ETH_RSS_NONFRAG_IPV6_TCP	|
+					   ETH_RSS_IPV6_EX		|
+					   ETH_RSS_IPV6_TCP_EX);
+}
+
 static int
 lio_dev_rss_reta_update(struct rte_eth_dev *eth_dev,
 			struct rte_eth_rss_reta_entry64 *reta_conf,
@@ -987,6 +1033,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.dev_configure		= lio_dev_configure,
 	.dev_start		= lio_dev_start,
 	.link_update		= lio_dev_link_update,
+	.dev_infos_get		= lio_dev_info_get,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
-- 
1.8.3.1

[PATCH v3 36/46] net/liquidio: add API to validate VF MTU

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 46 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index e2040b9..560f195 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -152,6 +152,33 @@
 }
 
 static int
+lio_dev_validate_vf_mtu(struct rte_eth_dev *eth_dev, uint16_t new_mtu)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	PMD_INIT_FUNC_TRACE();
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't check MTU\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	/* Limit the MTU to make sure the ethernet packets are between
+	 * ETHER_MIN_MTU bytes and PF's MTU
+	 */
+	if ((new_mtu < ETHER_MIN_MTU) ||
+			(new_mtu > lio_dev->linfo.link.s.mtu)) {
+		lio_dev_err(lio_dev, "Invalid MTU: %d\n", new_mtu);
+		lio_dev_err(lio_dev, "Valid range %d and %d\n",
+			    ETHER_MIN_MTU, lio_dev->linfo.link.s.mtu);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int
 lio_dev_rss_reta_update(struct rte_eth_dev *eth_dev,
 			struct rte_eth_rss_reta_entry64 *reta_conf,
 			uint16_t reta_size)
@@ -824,7 +851,9 @@
 static int
 lio_dev_start(struct rte_eth_dev *eth_dev)
 {
+	uint16_t mtu = eth_dev->data->dev_conf.rxmode.max_rx_pkt_len;
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
 	int ret = 0;
 
 	lio_dev_info(lio_dev, "Starting port %d\n", eth_dev->data->port_id);
@@ -852,8 +881,25 @@
 		goto dev_lsc_handle_error;
 	}
 
+	while ((lio_dev->linfo.link.link_status64 == 0) && (--timeout))
+		rte_delay_ms(1);
+
+	if (lio_dev->linfo.link.link_status64 == 0) {
+		ret = -1;
+		goto dev_mtu_check_error;
+	}
+
+	if (lio_dev->linfo.link.s.mtu != mtu) {
+		ret = lio_dev_validate_vf_mtu(eth_dev, mtu);
+		if (ret)
+			goto dev_mtu_check_error;
+	}
+
 	return 0;
 
+dev_mtu_check_error:
+	rte_eal_alarm_cancel(lio_sync_link_state_check, eth_dev);
+
 dev_lsc_handle_error:
 	lio_dev->intf_open = 0;
 	lio_send_rx_ctrl_cmd(eth_dev, 0);
-- 
1.8.3.1

[PATCH v3 37/46] net/liquidio: add APIs to enable and disable multicast

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 doc/guides/nics/features/liquidio.ini   |  1 +
 drivers/net/liquidio/base/lio_hw_defs.h |  2 +
 drivers/net/liquidio/lio_ethdev.c       | 68 +++++++++++++++++++++++++++++++++
 3 files changed, 71 insertions(+)

diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
index 8760849..11e5df3 100644
--- a/doc/guides/nics/features/liquidio.ini
+++ b/doc/guides/nics/features/liquidio.ini
@@ -8,6 +8,7 @@ Link status          = Y
 Link status event    = Y
 Jumbo frame          = Y
 Scattered Rx         = Y
+Allmulticast mode    = Y
 RSS hash             = Y
 RSS key update       = Y
 RSS reta update      = Y
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 3ea2e0f..781e929 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -130,6 +130,7 @@ enum octeon_tag_type {
 #define LIO_MAX_RX_PKTLEN		(64 * 1024)
 
 /* NIC Command types */
+#define LIO_CMD_CHANGE_DEVFLAGS		0x3
 #define LIO_CMD_RX_CTL			0x4
 #define LIO_CMD_SET_RSS			0xD
 
@@ -165,6 +166,7 @@ enum octeon_tag_type {
 
 /* Interface flags communicated between host driver and core app. */
 enum lio_ifflags {
+	LIO_IFFLAG_ALLMULTI	= 0x02,
 	LIO_IFFLAG_UNICAST	= 0x10
 };
 
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 560f195..ab97977 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -522,6 +522,72 @@
 	return 0;
 }
 
+/**
+ * \brief Net device enable, disable allmulticast
+ * @param eth_dev Pointer to the structure rte_eth_dev
+ */
+static void
+lio_change_dev_flag(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	/* Create a ctrl pkt command to be sent to core app. */
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_CHANGE_DEVFLAGS;
+	ctrl_pkt.ncmd.s.param1 = lio_dev->ifflags;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send change flag message\n");
+		return;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd))
+		lio_dev_err(lio_dev, "Change dev flag command timed out\n");
+}
+
+static void
+lio_dev_allmulticast_enable(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't enable multicast\n",
+			    lio_dev->port_id);
+		return;
+	}
+
+	lio_dev->ifflags |= LIO_IFFLAG_ALLMULTI;
+	lio_change_dev_flag(eth_dev);
+}
+
+static void
+lio_dev_allmulticast_disable(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down, can't disable multicast\n",
+			    lio_dev->port_id);
+		return;
+	}
+
+	lio_dev->ifflags &= ~LIO_IFFLAG_ALLMULTI;
+	lio_change_dev_flag(eth_dev);
+}
+
 static void
 lio_dev_rss_configure(struct rte_eth_dev *eth_dev)
 {
@@ -1078,6 +1144,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
 	.dev_start		= lio_dev_start,
+	.allmulticast_enable	= lio_dev_allmulticast_enable,
+	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
 	.dev_infos_get		= lio_dev_info_get,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
-- 
1.8.3.1

[PATCH v3 38/46] net/liquidio: add APIs to set link up and down

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 56 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index ab97977..ef4d794 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -973,6 +973,60 @@
 	return ret;
 }
 
+static int
+lio_dev_set_link_up(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	if (!lio_dev->intf_open) {
+		lio_dev_info(lio_dev, "Port is stopped, Start the port first\n");
+		return 0;
+	}
+
+	if (lio_dev->linfo.link.s.link_up) {
+		lio_dev_info(lio_dev, "Link is already UP\n");
+		return 0;
+	}
+
+	if (lio_send_rx_ctrl_cmd(eth_dev, 1)) {
+		lio_dev_err(lio_dev, "Unable to set Link UP\n");
+		return -1;
+	}
+
+	lio_dev->linfo.link.s.link_up = 1;
+	eth_dev->data->dev_link.link_status = ETH_LINK_UP;
+
+	return 0;
+}
+
+static int
+lio_dev_set_link_down(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	if (!lio_dev->intf_open) {
+		lio_dev_info(lio_dev, "Port is stopped, Start the port first\n");
+		return 0;
+	}
+
+	if (!lio_dev->linfo.link.s.link_up) {
+		lio_dev_info(lio_dev, "Link is already DOWN\n");
+		return 0;
+	}
+
+	lio_dev->linfo.link.s.link_up = 0;
+	eth_dev->data->dev_link.link_status = ETH_LINK_DOWN;
+
+	if (lio_send_rx_ctrl_cmd(eth_dev, 0)) {
+		lio_dev->linfo.link.s.link_up = 1;
+		eth_dev->data->dev_link.link_status = ETH_LINK_UP;
+		lio_dev_err(lio_dev, "Unable to set Link Down\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -1144,6 +1198,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
 	.dev_start		= lio_dev_start,
+	.dev_set_link_up	= lio_dev_set_link_up,
+	.dev_set_link_down	= lio_dev_set_link_down,
 	.allmulticast_enable	= lio_dev_allmulticast_enable,
 	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
-- 
1.8.3.1

[PATCH v3 39/46] net/liquidio: add APIs to configure UDP tunnel port

[Shijith Thotton]

Add APIs to configure VXLAN port and enable tunnel checksum.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 doc/guides/nics/features/liquidio.ini   |   2 +
 drivers/net/liquidio/base/lio_hw_defs.h |   8 ++
 drivers/net/liquidio/lio_ethdev.c       | 191 +++++++++++++++++++++++++++++++-
 drivers/net/liquidio/lio_rxtx.c         |   4 +-
 drivers/net/liquidio/lio_rxtx.h         |   1 +
 5 files changed, 204 insertions(+), 2 deletions(-)

diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
index 11e5df3..2d6a49e 100644
--- a/doc/guides/nics/features/liquidio.ini
+++ b/doc/guides/nics/features/liquidio.ini
@@ -15,6 +15,8 @@ RSS reta update      = Y
 CRC offload          = Y
 L3 checksum offload  = Y
 L4 checksum offload  = Y
+Inner L3 checksum    = Y
+Inner L4 checksum    = Y
 Multiprocess aware   = Y
 Linux UIO            = Y
 Linux VFIO           = Y
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 781e929..cc189ad 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -133,6 +133,14 @@ enum octeon_tag_type {
 #define LIO_CMD_CHANGE_DEVFLAGS		0x3
 #define LIO_CMD_RX_CTL			0x4
 #define LIO_CMD_SET_RSS			0xD
+#define LIO_CMD_TNL_RX_CSUM_CTL		0x10
+#define LIO_CMD_TNL_TX_CSUM_CTL		0x11
+#define LIO_CMD_VXLAN_PORT_CONFIG	0x19
+
+#define LIO_CMD_VXLAN_PORT_ADD		0x0
+#define LIO_CMD_VXLAN_PORT_DEL		0x1
+#define LIO_CMD_RXCSUM_ENABLE		0x0
+#define LIO_CMD_TXCSUM_ENABLE		0x0
 
 /* RX(packets coming from wire) Checksum verification flags */
 /* TCP/UDP csum */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index ef4d794..3911864 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -136,7 +136,8 @@
 				    DEV_RX_OFFLOAD_TCP_CKSUM);
 	devinfo->tx_offload_capa = (DEV_TX_OFFLOAD_IPV4_CKSUM		|
 				    DEV_TX_OFFLOAD_UDP_CKSUM		|
-				    DEV_TX_OFFLOAD_TCP_CKSUM);
+				    DEV_TX_OFFLOAD_TCP_CKSUM		|
+				    DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM);
 
 	devinfo->rx_desc_lim = lio_rx_desc_lim;
 	devinfo->tx_desc_lim = lio_tx_desc_lim;
@@ -440,6 +441,120 @@
 }
 
 /**
+ * Add vxlan dest udp port for an interface.
+ *
+ * @param eth_dev
+ *  Pointer to the structure rte_eth_dev
+ * @param udp_tnl
+ *  udp tunnel conf
+ *
+ * @return
+ *  On success return 0
+ *  On failure return -1
+ */
+static int
+lio_dev_udp_tunnel_add(struct rte_eth_dev *eth_dev,
+		       struct rte_eth_udp_tunnel *udp_tnl)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (udp_tnl == NULL)
+		return -EINVAL;
+
+	if (udp_tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN) {
+		lio_dev_err(lio_dev, "Unsupported tunnel type\n");
+		return -1;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_VXLAN_PORT_CONFIG;
+	ctrl_pkt.ncmd.s.param1 = udp_tnl->udp_port;
+	ctrl_pkt.ncmd.s.more = LIO_CMD_VXLAN_PORT_ADD;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send VXLAN_PORT_ADD command\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "VXLAN_PORT_ADD command timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+/**
+ * Remove vxlan dest udp port for an interface.
+ *
+ * @param eth_dev
+ *  Pointer to the structure rte_eth_dev
+ * @param udp_tnl
+ *  udp tunnel conf
+ *
+ * @return
+ *  On success return 0
+ *  On failure return -1
+ */
+static int
+lio_dev_udp_tunnel_del(struct rte_eth_dev *eth_dev,
+		       struct rte_eth_udp_tunnel *udp_tnl)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (udp_tnl == NULL)
+		return -EINVAL;
+
+	if (udp_tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN) {
+		lio_dev_err(lio_dev, "Unsupported tunnel type\n");
+		return -1;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_VXLAN_PORT_CONFIG;
+	ctrl_pkt.ncmd.s.param1 = udp_tnl->udp_port;
+	ctrl_pkt.ncmd.s.more = LIO_CMD_VXLAN_PORT_DEL;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send VXLAN_PORT_DEL command\n");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "VXLAN_PORT_DEL command timed out\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+/**
  * Atomically writes the link status information into global
  * structure rte_eth_dev.
  *
@@ -1027,6 +1142,74 @@
 	return 0;
 }
 
+/**
+ * Enable tunnel rx checksum verification from firmware.
+ */
+static void
+lio_enable_hw_tunnel_rx_checksum(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_TNL_RX_CSUM_CTL;
+	ctrl_pkt.ncmd.s.param1 = LIO_CMD_RXCSUM_ENABLE;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send TNL_RX_CSUM command\n");
+		return;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd))
+		lio_dev_err(lio_dev, "TNL_RX_CSUM command timed out\n");
+}
+
+/**
+ * Enable checksum calculation for inner packet in a tunnel.
+ */
+static void
+lio_enable_hw_tunnel_tx_checksum(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_TNL_TX_CSUM_CTL;
+	ctrl_pkt.ncmd.s.param1 = LIO_CMD_TXCSUM_ENABLE;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send TNL_TX_CSUM command\n");
+		return;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd))
+		lio_dev_err(lio_dev, "TNL_TX_CSUM command timed out\n");
+}
+
 static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -1155,6 +1338,10 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	/* Copy the permanent MAC address */
 	ether_addr_copy((struct ether_addr *)mac, &eth_dev->data->mac_addrs[0]);
 
+	/* enable firmware checksum support for tunnel packets */
+	lio_enable_hw_tunnel_rx_checksum(eth_dev);
+	lio_enable_hw_tunnel_tx_checksum(eth_dev);
+
 	lio_dev->glist_lock =
 	    rte_zmalloc(NULL, sizeof(*lio_dev->glist_lock) * num_iqueues, 0);
 	if (lio_dev->glist_lock == NULL)
@@ -1212,6 +1399,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.reta_query		= lio_dev_rss_reta_query,
 	.rss_hash_conf_get	= lio_dev_rss_hash_conf_get,
 	.rss_hash_update	= lio_dev_rss_hash_update,
+	.udp_tunnel_port_add	= lio_dev_udp_tunnel_add,
+	.udp_tunnel_port_del	= lio_dev_udp_tunnel_del,
 };
 
 static void
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index c12960c..f105457 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1704,7 +1704,9 @@ struct lio_soft_command *
 		if (m->ol_flags & PKT_TX_IP_CKSUM)
 			cmdsetup.s.ip_csum = 1;
 
-		if ((m->ol_flags & PKT_TX_TCP_CKSUM) ||
+		if (m->ol_flags & PKT_TX_OUTER_IP_CKSUM)
+			cmdsetup.s.tnl_csum = 1;
+		else if ((m->ol_flags & PKT_TX_TCP_CKSUM) ||
 				(m->ol_flags & PKT_TX_UDP_CKSUM))
 			cmdsetup.s.transport_csum = 1;
 
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 95d0007..6835430 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -583,6 +583,7 @@ struct octeon_instr_rdp {
 	packet_params.pkt_params32 = 0;
 	packet_params.s.ip_csum = setup->s.ip_csum;
 	packet_params.s.transport_csum = setup->s.transport_csum;
+	packet_params.s.tnl_csum = setup->s.tnl_csum;
 	packet_params.s.tsflag = setup->s.timestamp;
 
 	irh->ossp = packet_params.pkt_params32;
-- 
1.8.3.1

[PATCH v3 40/46] net/liquidio: add support for Rx stats

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 doc/guides/nics/features/liquidio.ini |  1 +
 drivers/net/liquidio/lio_ethdev.c     | 50 +++++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.c       | 10 ++++++-
 drivers/net/liquidio/lio_struct.h     | 34 ++++++++++++++++++++++++
 4 files changed, 94 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
index 2d6a49e..3195515 100644
--- a/doc/guides/nics/features/liquidio.ini
+++ b/doc/guides/nics/features/liquidio.ini
@@ -17,6 +17,7 @@ L3 checksum offload  = Y
 L4 checksum offload  = Y
 Inner L3 checksum    = Y
 Inner L4 checksum    = Y
+Basic stats          = Y
 Multiprocess aware   = Y
 Linux UIO            = Y
 Linux VFIO           = Y
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 3911864..adf0db2 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -117,6 +117,54 @@
 	return 0;
 }
 
+/* Retrieve the device statistics (# packets in/out, # bytes in/out, etc */
+static void
+lio_dev_stats_get(struct rte_eth_dev *eth_dev,
+		  struct rte_eth_stats *stats)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_droq_stats *oq_stats;
+	struct lio_droq *droq;
+	uint64_t bytes = 0;
+	uint64_t pkts = 0;
+	uint64_t drop = 0;
+	int i, oq_no;
+
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
+		oq_no = lio_dev->linfo.rxpciq[i].s.q_no;
+		droq = lio_dev->droq[oq_no];
+		if (droq != NULL) {
+			oq_stats = &droq->stats;
+			pkts += oq_stats->rx_pkts_received;
+			drop += (oq_stats->rx_dropped +
+					oq_stats->dropped_toomany +
+					oq_stats->dropped_nomem);
+			bytes += oq_stats->rx_bytes_received;
+		}
+	}
+	stats->ibytes = bytes;
+	stats->ipackets = pkts;
+	stats->ierrors = drop;
+}
+
+static void
+lio_dev_stats_reset(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_droq_stats *oq_stats;
+	struct lio_droq *droq;
+	int i, oq_no;
+
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
+		oq_no = lio_dev->linfo.rxpciq[i].s.q_no;
+		droq = lio_dev->droq[oq_no];
+		if (droq != NULL) {
+			oq_stats = &droq->stats;
+			memset(oq_stats, 0, sizeof(struct lio_droq_stats));
+		}
+	}
+}
+
 static void
 lio_dev_info_get(struct rte_eth_dev *eth_dev,
 		 struct rte_eth_dev_info *devinfo)
@@ -1390,6 +1438,8 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.allmulticast_enable	= lio_dev_allmulticast_enable,
 	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
+	.stats_get		= lio_dev_stats_get,
+	.stats_reset		= lio_dev_stats_reset,
 	.dev_infos_get		= lio_dev_info_get,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index f105457..adbd990 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -115,6 +115,7 @@
 		buf = lio_recv_buffer_alloc(lio_dev, droq->q_no);
 		if (buf == NULL) {
 			lio_dev_err(lio_dev, "buffer alloc failed\n");
+			droq->stats.rx_alloc_failure++;
 			lio_droq_destroy_ring_buffers(droq);
 			return -ENOMEM;
 		}
@@ -410,8 +411,10 @@
 			/* If a buffer could not be allocated, no point in
 			 * continuing
 			 */
-			if (buf == NULL)
+			if (buf == NULL) {
+				droq->stats.rx_alloc_failure++;
 				break;
+			}
 
 			droq->recv_buf_list[droq->refill_idx].buffer = buf;
 		}
@@ -629,6 +632,11 @@
 	info->length = 0;
 	info->rh.rh64 = 0;
 
+	droq->stats.pkts_received++;
+	droq->stats.rx_pkts_received += data_pkts;
+	droq->stats.rx_bytes_received += data_total_len;
+	droq->stats.bytes_received += total_len;
+
 	return data_pkts;
 }
 
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index c2293f7..50d5e86 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -56,6 +56,37 @@ struct lio_version {
 	uint16_t reserved;
 };
 
+/** Output Queue statistics. Each output queue has four stats fields. */
+struct lio_droq_stats {
+	/** Number of packets received in this queue. */
+	uint64_t pkts_received;
+
+	/** Bytes received by this queue. */
+	uint64_t bytes_received;
+
+	/** Packets dropped due to no memory available. */
+	uint64_t dropped_nomem;
+
+	/** Packets dropped due to large number of pkts to process. */
+	uint64_t dropped_toomany;
+
+	/** Number of packets  sent to stack from this queue. */
+	uint64_t rx_pkts_received;
+
+	/** Number of Bytes sent to stack from this queue. */
+	uint64_t rx_bytes_received;
+
+	/** Num of Packets dropped due to receive path failures. */
+	uint64_t rx_dropped;
+
+	/** Num of vxlan packets received; */
+	uint64_t rx_vxlan;
+
+	/** Num of failures of lio_recv_buffer_alloc() */
+	uint64_t rx_alloc_failure;
+
+};
+
 /** The Descriptor Ring Output Queue structure.
  *  This structure has all the information required to implement a
  *  DROQ.
@@ -117,6 +148,9 @@ struct lio_droq {
 	 */
 	void *pkts_sent_reg;
 
+	/** Statistics for this DROQ. */
+	struct lio_droq_stats stats;
+
 	/** DMA mapped address of the DROQ descriptor ring. */
 	size_t desc_ring_dma;
 
-- 
1.8.3.1

[PATCH v3 41/46] net/liquidio: add support for Tx stats

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 36 ++++++++++++++++++++++++++++++++++--
 drivers/net/liquidio/lio_rxtx.c   | 18 ++++++++++++++++--
 drivers/net/liquidio/lio_rxtx.h   |  3 +++
 drivers/net/liquidio/lio_struct.h | 15 +++++++++++++++
 4 files changed, 68 insertions(+), 4 deletions(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index adf0db2..2f3bb3d 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -124,11 +124,32 @@
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
 	struct lio_droq_stats *oq_stats;
+	struct lio_iq_stats *iq_stats;
+	struct lio_instr_queue *txq;
 	struct lio_droq *droq;
+	int i, iq_no, oq_no;
 	uint64_t bytes = 0;
 	uint64_t pkts = 0;
 	uint64_t drop = 0;
-	int i, oq_no;
+
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
+		iq_no = lio_dev->linfo.txpciq[i].s.q_no;
+		txq = lio_dev->instr_queue[iq_no];
+		if (txq != NULL) {
+			iq_stats = &txq->stats;
+			pkts += iq_stats->tx_done;
+			drop += iq_stats->tx_dropped;
+			bytes += iq_stats->tx_tot_bytes;
+		}
+	}
+
+	stats->opackets = pkts;
+	stats->obytes = bytes;
+	stats->oerrors = drop;
+
+	pkts = 0;
+	drop = 0;
+	bytes = 0;
 
 	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
 		oq_no = lio_dev->linfo.rxpciq[i].s.q_no;
@@ -152,8 +173,19 @@
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
 	struct lio_droq_stats *oq_stats;
+	struct lio_iq_stats *iq_stats;
+	struct lio_instr_queue *txq;
 	struct lio_droq *droq;
-	int i, oq_no;
+	int i, iq_no, oq_no;
+
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
+		iq_no = lio_dev->linfo.txpciq[i].s.q_no;
+		txq = lio_dev->instr_queue[iq_no];
+		if (txq != NULL) {
+			iq_stats = &txq->stats;
+			memset(iq_stats, 0, sizeof(struct lio_iq_stats));
+		}
+	}
 
 	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
 		oq_no = lio_dev->linfo.rxpciq[i].s.q_no;
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index adbd990..470131c 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -1113,8 +1113,10 @@
 
 		inst_processed = lio_process_iq_request_list(lio_dev, iq);
 
-		if (inst_processed)
+		if (inst_processed) {
 			rte_atomic64_sub(&iq->instr_pending, inst_processed);
+			iq->stats.instr_processed += inst_processed;
+		}
 
 		tot_inst_processed += inst_processed;
 		inst_processed = 0;
@@ -1130,7 +1132,7 @@
 
 static int
 lio_send_command(struct lio_device *lio_dev, uint32_t iq_no, void *cmd,
-		 void *buf, uint32_t datasize __rte_unused, uint32_t reqtype)
+		 void *buf, uint32_t datasize, uint32_t reqtype)
 {
 	struct lio_instr_queue *iq = lio_dev->instr_queue[iq_no];
 	struct lio_iq_post_status st;
@@ -1141,7 +1143,13 @@
 
 	if (st.status != LIO_IQ_SEND_FAILED) {
 		lio_add_to_request_list(iq, st.index, buf, reqtype);
+		LIO_INCR_INSTRQUEUE_PKT_COUNT(lio_dev, iq_no, bytes_sent,
+					      datasize);
+		LIO_INCR_INSTRQUEUE_PKT_COUNT(lio_dev, iq_no, instr_posted, 1);
+
 		lio_ring_doorbell(lio_dev, iq);
+	} else {
+		LIO_INCR_INSTRQUEUE_PKT_COUNT(lio_dev, iq_no, instr_dropped, 1);
 	}
 
 	rte_spinlock_unlock(&iq->post_lock);
@@ -1667,6 +1675,7 @@ struct lio_soft_command *
 	struct lio_instr_queue *txq = tx_queue;
 	union lio_cmd_setup cmdsetup;
 	struct lio_device *lio_dev;
+	struct lio_iq_stats *stats;
 	struct lio_data_pkt ndata;
 	int i, processed = 0;
 	struct rte_mbuf *m;
@@ -1676,6 +1685,7 @@ struct lio_soft_command *
 
 	lio_dev = txq->lio_dev;
 	iq_no = txq->txpciq.s.q_no;
+	stats = &lio_dev->instr_queue[iq_no]->stats;
 
 	if (!lio_dev->intf_open || !lio_dev->linfo.link.s.link_up) {
 		PMD_TX_LOG(lio_dev, ERR, "Transmit failed link_status : %d\n",
@@ -1697,6 +1707,7 @@ struct lio_soft_command *
 
 		ndata.q_no = iq_no;
 		if (lio_iq_is_full(lio_dev, ndata.q_no)) {
+			stats->tx_iq_busy++;
 			if (lio_dev_cleanup_iq(lio_dev, iq_no)) {
 				PMD_TX_LOG(lio_dev, ERR,
 					   "Transmit failed iq:%d full\n",
@@ -1804,10 +1815,13 @@ struct lio_soft_command *
 			lio_dev_cleanup_iq(lio_dev, iq_no);
 		}
 
+		stats->tx_done++;
+		stats->tx_tot_bytes += pkt_len;
 		processed++;
 	}
 
 xmit_failed:
+	stats->tx_dropped += (nb_pkts - processed);
 
 	return processed;
 }
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index 6835430..d5aed6a 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -670,6 +670,9 @@ enum {
  */
 int lio_process_ordered_list(struct lio_device *lio_dev);
 
+#define LIO_INCR_INSTRQUEUE_PKT_COUNT(lio_dev, iq_no, field, count)	\
+	(((lio_dev)->instr_queue[iq_no]->stats.field) += count)
+
 static inline void
 lio_swap_8B_data(uint64_t *data, uint32_t blocks)
 {
diff --git a/drivers/net/liquidio/lio_struct.h b/drivers/net/liquidio/lio_struct.h
index 50d5e86..26f803f 100644
--- a/drivers/net/liquidio/lio_struct.h
+++ b/drivers/net/liquidio/lio_struct.h
@@ -56,6 +56,18 @@ struct lio_version {
 	uint16_t reserved;
 };
 
+/** Input Queue statistics. Each input queue has four stats fields. */
+struct lio_iq_stats {
+	uint64_t instr_posted; /**< Instructions posted to this queue. */
+	uint64_t instr_processed; /**< Instructions processed in this queue. */
+	uint64_t instr_dropped; /**< Instructions that could not be processed */
+	uint64_t bytes_sent; /**< Bytes sent through this queue. */
+	uint64_t tx_done; /**< Num of packets sent to network. */
+	uint64_t tx_iq_busy; /**< Num of times this iq was found to be full. */
+	uint64_t tx_dropped; /**< Num of pkts dropped due to xmitpath errors. */
+	uint64_t tx_tot_bytes; /**< Total count of bytes sent to network. */
+};
+
 /** Output Queue statistics. Each output queue has four stats fields. */
 struct lio_droq_stats {
 	/** Number of packets received in this queue. */
@@ -319,6 +331,9 @@ struct lio_instr_queue {
 	/** Number of instructions pending to be posted to Octeon. */
 	uint32_t fill_cnt;
 
+	/** Statistics for this input queue. */
+	struct lio_iq_stats stats;
+
 	/** DMA mapped base address of the input descriptor ring. */
 	uint64_t base_addr_dma;
 
-- 
1.8.3.1

[PATCH v3 42/46] net/liquidio: add APIs for hardware stats

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 doc/guides/nics/features/liquidio.ini   |   1 +
 drivers/net/liquidio/base/lio_hw_defs.h |   2 +
 drivers/net/liquidio/lio_ethdev.c       | 194 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_ethdev.h       |  83 ++++++++++++++
 4 files changed, 280 insertions(+)

diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
index 3195515..d6d9e92 100644
--- a/doc/guides/nics/features/liquidio.ini
+++ b/doc/guides/nics/features/liquidio.ini
@@ -18,6 +18,7 @@ L4 checksum offload  = Y
 Inner L3 checksum    = Y
 Inner L4 checksum    = Y
 Basic stats          = Y
+Extended stats       = Y
 Multiprocess aware   = Y
 Linux UIO            = Y
 Linux VFIO           = Y
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index cc189ad..8a22d10 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -124,6 +124,7 @@ enum octeon_tag_type {
 #define LIO_OPCODE_NW_DATA		0x02 /* network packet data */
 #define LIO_OPCODE_CMD			0x03
 #define LIO_OPCODE_INFO			0x04
+#define LIO_OPCODE_PORT_STATS		0x05
 #define LIO_OPCODE_IF_CFG		0x09
 
 #define LIO_MIN_RX_BUF_SIZE		64
@@ -132,6 +133,7 @@ enum octeon_tag_type {
 /* NIC Command types */
 #define LIO_CMD_CHANGE_DEVFLAGS		0x3
 #define LIO_CMD_RX_CTL			0x4
+#define LIO_CMD_CLEAR_STATS		0x6
 #define LIO_CMD_SET_RSS			0xD
 #define LIO_CMD_TNL_RX_CSUM_CTL		0x10
 #define LIO_CMD_TNL_TX_CSUM_CTL		0x11
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 2f3bb3d..4d0977a 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -117,6 +117,197 @@
 	return 0;
 }
 
+/* store statistics names and its offset in stats structure */
+struct rte_lio_xstats_name_off {
+	char name[RTE_ETH_XSTATS_NAME_SIZE];
+	unsigned int offset;
+};
+
+static const struct rte_lio_xstats_name_off rte_lio_stats_strings[] = {
+	{"rx_pkts", offsetof(struct octeon_rx_stats, total_rcvd)},
+	{"rx_bytes", offsetof(struct octeon_rx_stats, bytes_rcvd)},
+	{"rx_broadcast_pkts", offsetof(struct octeon_rx_stats, total_bcst)},
+	{"rx_multicast_pkts", offsetof(struct octeon_rx_stats, total_mcst)},
+	{"rx_flow_ctrl_pkts", offsetof(struct octeon_rx_stats, ctl_rcvd)},
+	{"rx_fifo_err", offsetof(struct octeon_rx_stats, fifo_err)},
+	{"rx_dmac_drop", offsetof(struct octeon_rx_stats, dmac_drop)},
+	{"rx_fcs_err", offsetof(struct octeon_rx_stats, fcs_err)},
+	{"rx_jabber_err", offsetof(struct octeon_rx_stats, jabber_err)},
+	{"rx_l2_err", offsetof(struct octeon_rx_stats, l2_err)},
+	{"rx_vxlan_pkts", offsetof(struct octeon_rx_stats, fw_rx_vxlan)},
+	{"rx_vxlan_err", offsetof(struct octeon_rx_stats, fw_rx_vxlan_err)},
+	{"rx_lro_pkts", offsetof(struct octeon_rx_stats, fw_lro_pkts)},
+	{"tx_pkts", (offsetof(struct octeon_tx_stats, total_pkts_sent)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_bytes", (offsetof(struct octeon_tx_stats, total_bytes_sent)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_broadcast_pkts",
+		(offsetof(struct octeon_tx_stats, bcast_pkts_sent)) +
+			sizeof(struct octeon_rx_stats)},
+	{"tx_multicast_pkts",
+		(offsetof(struct octeon_tx_stats, mcast_pkts_sent)) +
+			sizeof(struct octeon_rx_stats)},
+	{"tx_flow_ctrl_pkts", (offsetof(struct octeon_tx_stats, ctl_sent)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_fifo_err", (offsetof(struct octeon_tx_stats, fifo_err)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_total_collisions", (offsetof(struct octeon_tx_stats,
+					  total_collisions)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_tso", (offsetof(struct octeon_tx_stats, fw_tso)) +
+						sizeof(struct octeon_rx_stats)},
+	{"tx_vxlan_pkts", (offsetof(struct octeon_tx_stats, fw_tx_vxlan)) +
+						sizeof(struct octeon_rx_stats)},
+};
+
+#define LIO_NB_XSTATS	RTE_DIM(rte_lio_stats_strings)
+
+/* Get hw stats of the port */
+static int
+lio_dev_xstats_get(struct rte_eth_dev *eth_dev, struct rte_eth_xstat *xstats,
+		   unsigned int n)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint16_t timeout = LIO_MAX_CMD_TIMEOUT;
+	struct octeon_link_stats *hw_stats;
+	struct lio_link_stats_resp *resp;
+	struct lio_soft_command *sc;
+	uint32_t resp_size;
+	unsigned int i;
+	int retval;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	if (n < LIO_NB_XSTATS)
+		return LIO_NB_XSTATS;
+
+	resp_size = sizeof(struct lio_link_stats_resp);
+	sc = lio_alloc_soft_command(lio_dev, 0, resp_size, 0);
+	if (sc == NULL)
+		return -ENOMEM;
+
+	resp = (struct lio_link_stats_resp *)sc->virtrptr;
+	lio_prepare_soft_command(lio_dev, sc, LIO_OPCODE,
+				 LIO_OPCODE_PORT_STATS, 0, 0, 0);
+
+	/* Setting wait time in seconds */
+	sc->wait_time = LIO_MAX_CMD_TIMEOUT / 1000;
+
+	retval = lio_send_soft_command(lio_dev, sc);
+	if (retval == LIO_IQ_SEND_FAILED) {
+		lio_dev_err(lio_dev, "failed to get port stats from firmware. status: %x\n",
+			    retval);
+		goto get_stats_fail;
+	}
+
+	while ((*sc->status_word == LIO_COMPLETION_WORD_INIT) && --timeout) {
+		lio_flush_iq(lio_dev, lio_dev->instr_queue[sc->iq_no]);
+		lio_process_ordered_list(lio_dev);
+		rte_delay_ms(1);
+	}
+
+	retval = resp->status;
+	if (retval) {
+		lio_dev_err(lio_dev, "failed to get port stats from firmware\n");
+		goto get_stats_fail;
+	}
+
+	lio_swap_8B_data((uint64_t *)(&resp->link_stats),
+			 sizeof(struct octeon_link_stats) >> 3);
+
+	hw_stats = &resp->link_stats;
+
+	for (i = 0; i < LIO_NB_XSTATS; i++) {
+		xstats[i].id = i;
+		xstats[i].value =
+		    *(uint64_t *)(((char *)hw_stats) +
+					rte_lio_stats_strings[i].offset);
+	}
+
+	lio_free_soft_command(sc);
+
+	return LIO_NB_XSTATS;
+
+get_stats_fail:
+	lio_free_soft_command(sc);
+
+	return -1;
+}
+
+static int
+lio_dev_xstats_get_names(struct rte_eth_dev *eth_dev,
+			 struct rte_eth_xstat_name *xstats_names,
+			 unsigned limit __rte_unused)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	unsigned int i;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down\n",
+			    lio_dev->port_id);
+		return -EINVAL;
+	}
+
+	if (xstats_names == NULL)
+		return LIO_NB_XSTATS;
+
+	/* Note: limit checked in rte_eth_xstats_names() */
+
+	for (i = 0; i < LIO_NB_XSTATS; i++) {
+		snprintf(xstats_names[i].name, sizeof(xstats_names[i].name),
+			 "%s", rte_lio_stats_strings[i].name);
+	}
+
+	return LIO_NB_XSTATS;
+}
+
+/* Reset hw stats for the port */
+static void
+lio_dev_xstats_reset(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (!lio_dev->intf_open) {
+		lio_dev_err(lio_dev, "Port %d down\n",
+			    lio_dev->port_id);
+		return;
+	}
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = LIO_CMD_CLEAR_STATS;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to send clear stats command\n");
+		return;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "Clear stats command timed out\n");
+		return;
+	}
+
+	/* clear stored per queue stats */
+	RTE_FUNC_PTR_OR_RET(*eth_dev->dev_ops->stats_reset);
+	(*eth_dev->dev_ops->stats_reset)(eth_dev);
+}
+
 /* Retrieve the device statistics (# packets in/out, # bytes in/out, etc */
 static void
 lio_dev_stats_get(struct rte_eth_dev *eth_dev,
@@ -1471,7 +1662,10 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
 	.stats_get		= lio_dev_stats_get,
+	.xstats_get		= lio_dev_xstats_get,
+	.xstats_get_names	= lio_dev_xstats_get_names,
 	.stats_reset		= lio_dev_stats_reset,
+	.xstats_reset		= lio_dev_xstats_reset,
 	.dev_infos_get		= lio_dev_info_get,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 6543061..150e9c9 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -62,6 +62,83 @@ struct octeon_if_cfg_info {
 	char lio_firmware_version[LIO_FW_VERSION_LENGTH];
 };
 
+/** Stats for each NIC port in RX direction. */
+struct octeon_rx_stats {
+	/* link-level stats */
+	uint64_t total_rcvd;
+	uint64_t bytes_rcvd;
+	uint64_t total_bcst;
+	uint64_t total_mcst;
+	uint64_t runts;
+	uint64_t ctl_rcvd;
+	uint64_t fifo_err; /* Accounts for over/under-run of buffers */
+	uint64_t dmac_drop;
+	uint64_t fcs_err;
+	uint64_t jabber_err;
+	uint64_t l2_err;
+	uint64_t frame_err;
+
+	/* firmware stats */
+	uint64_t fw_total_rcvd;
+	uint64_t fw_total_fwd;
+	uint64_t fw_total_fwd_bytes;
+	uint64_t fw_err_pko;
+	uint64_t fw_err_link;
+	uint64_t fw_err_drop;
+	uint64_t fw_rx_vxlan;
+	uint64_t fw_rx_vxlan_err;
+
+	/* LRO */
+	uint64_t fw_lro_pkts;   /* Number of packets that are LROed */
+	uint64_t fw_lro_octs;   /* Number of octets that are LROed */
+	uint64_t fw_total_lro;  /* Number of LRO packets formed */
+	uint64_t fw_lro_aborts; /* Number of times lRO of packet aborted */
+	uint64_t fw_lro_aborts_port;
+	uint64_t fw_lro_aborts_seq;
+	uint64_t fw_lro_aborts_tsval;
+	uint64_t fw_lro_aborts_timer;
+	/* intrmod: packet forward rate */
+	uint64_t fwd_rate;
+};
+
+/** Stats for each NIC port in RX direction. */
+struct octeon_tx_stats {
+	/* link-level stats */
+	uint64_t total_pkts_sent;
+	uint64_t total_bytes_sent;
+	uint64_t mcast_pkts_sent;
+	uint64_t bcast_pkts_sent;
+	uint64_t ctl_sent;
+	uint64_t one_collision_sent;	/* Packets sent after one collision */
+	/* Packets sent after multiple collision */
+	uint64_t multi_collision_sent;
+	/* Packets not sent due to max collisions */
+	uint64_t max_collision_fail;
+	/* Packets not sent due to max deferrals */
+	uint64_t max_deferral_fail;
+	/* Accounts for over/under-run of buffers */
+	uint64_t fifo_err;
+	uint64_t runts;
+	uint64_t total_collisions; /* Total number of collisions detected */
+
+	/* firmware stats */
+	uint64_t fw_total_sent;
+	uint64_t fw_total_fwd;
+	uint64_t fw_total_fwd_bytes;
+	uint64_t fw_err_pko;
+	uint64_t fw_err_link;
+	uint64_t fw_err_drop;
+	uint64_t fw_err_tso;
+	uint64_t fw_tso;     /* number of tso requests */
+	uint64_t fw_tso_fwd; /* number of packets segmented in tso */
+	uint64_t fw_tx_vxlan;
+};
+
+struct octeon_link_stats {
+	struct octeon_rx_stats fromwire;
+	struct octeon_tx_stats fromhost;
+};
+
 union lio_if_cfg {
 	uint64_t if_cfg64;
 	struct {
@@ -87,6 +164,12 @@ struct lio_if_cfg_resp {
 	uint64_t status;
 };
 
+struct lio_link_stats_resp {
+	uint64_t rh;
+	struct octeon_link_stats link_stats;
+	uint64_t status;
+};
+
 struct lio_link_status_resp {
 	uint64_t rh;
 	struct octeon_link_info link_info;
-- 
1.8.3.1

[PATCH v3 43/46] net/liquidio: add API to stop device

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 4d0977a..76d775d 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1359,6 +1359,25 @@ struct rte_lio_xstats_name_off {
 	return ret;
 }
 
+/* Stop device and disable input/output functions */
+static void
+lio_dev_stop(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+
+	lio_dev_info(lio_dev, "Stopping port %d\n", eth_dev->data->port_id);
+	lio_dev->intf_open = 0;
+	rte_mb();
+
+	/* Cancel callback if still running. */
+	rte_eal_alarm_cancel(lio_sync_link_state_check, eth_dev);
+
+	lio_send_rx_ctrl_cmd(eth_dev, 0);
+
+	/* Clear recorded link status */
+	lio_dev->linfo.link.link_status64 = 0;
+}
+
 static int
 lio_dev_set_link_up(struct rte_eth_dev *eth_dev)
 {
@@ -1656,6 +1675,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 static const struct eth_dev_ops liovf_eth_dev_ops = {
 	.dev_configure		= lio_dev_configure,
 	.dev_start		= lio_dev_start,
+	.dev_stop		= lio_dev_stop,
 	.dev_set_link_up	= lio_dev_set_link_up,
 	.dev_set_link_down	= lio_dev_set_link_down,
 	.allmulticast_enable	= lio_dev_allmulticast_enable,
-- 
1.8.3.1

[PATCH v3 44/46] net/liquidio: add API to close device

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 drivers/net/liquidio/lio_ethdev.c | 68 +++++++++++++++++++++++++++++++++++++--
 drivers/net/liquidio/lio_ethdev.h |  5 +++
 drivers/net/liquidio/lio_rxtx.c   | 57 ++++++++++++++++++++++++++++++++
 drivers/net/liquidio/lio_rxtx.h   |  2 ++
 4 files changed, 130 insertions(+), 2 deletions(-)

diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 76d775d..60dfa61 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1109,7 +1109,7 @@ struct rte_lio_xstats_name_off {
  * @return
  *    - nothing
  */
-static void
+void
 lio_dev_rx_queue_release(void *rxq)
 {
 	struct lio_droq *droq = rxq;
@@ -1204,7 +1204,7 @@ struct rte_lio_xstats_name_off {
  * @return
  *    - nothing
  */
-static void
+void
 lio_dev_tx_queue_release(void *txq)
 {
 	struct lio_instr_queue *tq = txq;
@@ -1433,6 +1433,68 @@ struct rte_lio_xstats_name_off {
 }
 
 /**
+ * Reset and stop the device. This occurs on the first
+ * call to this routine. Subsequent calls will simply
+ * return. NB: This will require the NIC to be rebooted.
+ *
+ * @param eth_dev
+ *    Pointer to the structure rte_eth_dev
+ *
+ * @return
+ *    - nothing
+ */
+static void
+lio_dev_close(struct rte_eth_dev *eth_dev)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	uint32_t i;
+
+	lio_dev_info(lio_dev, "closing port %d\n", eth_dev->data->port_id);
+
+	if (lio_dev->intf_open)
+		lio_dev_stop(eth_dev);
+
+	lio_wait_for_instr_fetch(lio_dev);
+
+	lio_dev->fn_list.disable_io_queues(lio_dev);
+
+	cn23xx_vf_set_io_queues_off(lio_dev);
+
+	/* Reset iq regs (IQ_DBELL).
+	 * Clear sli_pktx_cnts (OQ_PKTS_SENT).
+	 */
+	for (i = 0; i < lio_dev->nb_rx_queues; i++) {
+		struct lio_droq *droq = lio_dev->droq[i];
+
+		if (droq == NULL)
+			break;
+
+		uint32_t pkt_count = rte_read32(droq->pkts_sent_reg);
+
+		lio_dev_dbg(lio_dev,
+			    "pending oq count %u\n", pkt_count);
+		rte_write32(pkt_count, droq->pkts_sent_reg);
+	}
+
+	/* Do FLR for the VF */
+	cn23xx_vf_ask_pf_to_do_flr(lio_dev);
+
+	/* lio_free_mbox */
+	lio_dev->fn_list.free_mbox(lio_dev);
+
+	/* Free glist resources */
+	rte_free(lio_dev->glist_head);
+	rte_free(lio_dev->glist_lock);
+	lio_dev->glist_head = NULL;
+	lio_dev->glist_lock = NULL;
+
+	lio_dev->port_configured = 0;
+
+	 /* Delete all queues */
+	lio_dev_clear_queues(eth_dev);
+}
+
+/**
  * Enable tunnel rx checksum verification from firmware.
  */
 static void
@@ -1678,6 +1740,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.dev_stop		= lio_dev_stop,
 	.dev_set_link_up	= lio_dev_set_link_up,
 	.dev_set_link_down	= lio_dev_set_link_down,
+	.dev_close		= lio_dev_close,
 	.allmulticast_enable	= lio_dev_allmulticast_enable,
 	.allmulticast_disable	= lio_dev_allmulticast_disable,
 	.link_update		= lio_dev_link_update,
@@ -1839,6 +1902,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	rte_free(eth_dev->data->mac_addrs);
 	eth_dev->data->mac_addrs = NULL;
 
+	eth_dev->dev_ops = NULL;
 	eth_dev->rx_pkt_burst = NULL;
 	eth_dev->tx_pkt_burst = NULL;
 
diff --git a/drivers/net/liquidio/lio_ethdev.h b/drivers/net/liquidio/lio_ethdev.h
index 150e9c9..ee30615 100644
--- a/drivers/net/liquidio/lio_ethdev.h
+++ b/drivers/net/liquidio/lio_ethdev.h
@@ -196,4 +196,9 @@ struct lio_rss_set {
 	uint8_t itable[LIO_RSS_MAX_TABLE_SZ];
 	uint8_t key[LIO_RSS_MAX_KEY_SZ];
 };
+
+void lio_dev_rx_queue_release(void *rxq);
+
+void lio_dev_tx_queue_release(void *txq);
+
 #endif	/* _LIO_ETHDEV_H_ */
diff --git a/drivers/net/liquidio/lio_rxtx.c b/drivers/net/liquidio/lio_rxtx.c
index 470131c..9533015 100644
--- a/drivers/net/liquidio/lio_rxtx.c
+++ b/drivers/net/liquidio/lio_rxtx.c
@@ -918,6 +918,40 @@
 	return -1;
 }
 
+int
+lio_wait_for_instr_fetch(struct lio_device *lio_dev)
+{
+	int pending, instr_cnt;
+	int i, retry = 1000;
+
+	do {
+		instr_cnt = 0;
+
+		for (i = 0; i < LIO_MAX_INSTR_QUEUES(lio_dev); i++) {
+			if (!(lio_dev->io_qmask.iq & (1ULL << i)))
+				continue;
+
+			if (lio_dev->instr_queue[i] == NULL)
+				break;
+
+			pending = rte_atomic64_read(
+			    &lio_dev->instr_queue[i]->instr_pending);
+			if (pending)
+				lio_flush_iq(lio_dev, lio_dev->instr_queue[i]);
+
+			instr_cnt += pending;
+		}
+
+		if (instr_cnt == 0)
+			break;
+
+		rte_delay_ms(1);
+
+	} while (retry-- && instr_cnt);
+
+	return instr_cnt;
+}
+
 static inline void
 lio_ring_doorbell(struct lio_device *lio_dev,
 		  struct lio_instr_queue *iq)
@@ -1826,3 +1860,26 @@ struct lio_soft_command *
 	return processed;
 }
 
+void
+lio_dev_clear_queues(struct rte_eth_dev *eth_dev)
+{
+	struct lio_instr_queue *txq;
+	struct lio_droq *rxq;
+	uint16_t i;
+
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
+		txq = eth_dev->data->tx_queues[i];
+		if (txq != NULL) {
+			lio_dev_tx_queue_release(txq);
+			eth_dev->data->tx_queues[i] = NULL;
+		}
+	}
+
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
+		rxq = eth_dev->data->rx_queues[i];
+		if (rxq != NULL) {
+			lio_dev_rx_queue_release(rxq);
+			eth_dev->data->rx_queues[i] = NULL;
+		}
+	}
+}
diff --git a/drivers/net/liquidio/lio_rxtx.h b/drivers/net/liquidio/lio_rxtx.h
index d5aed6a..85685dc 100644
--- a/drivers/net/liquidio/lio_rxtx.h
+++ b/drivers/net/liquidio/lio_rxtx.h
@@ -752,6 +752,7 @@ int lio_setup_sglists(struct lio_device *lio_dev, int iq_no,
 		      int fw_mapped_iq, int num_descs, unsigned int socket_id);
 uint16_t lio_dev_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts,
 			   uint16_t nb_pkts);
+int lio_wait_for_instr_fetch(struct lio_device *lio_dev);
 int lio_setup_iq(struct lio_device *lio_dev, int q_index,
 		 union octeon_txpciq iq_no, uint32_t num_descs, void *app_ctx,
 		 unsigned int socket_id);
@@ -764,4 +765,5 @@ int lio_setup_iq(struct lio_device *lio_dev, int q_index,
  */
 int lio_setup_instr_queue0(struct lio_device *lio_dev);
 void lio_free_instr_queue0(struct lio_device *lio_dev);
+void lio_dev_clear_queues(struct rte_eth_dev *eth_dev);
 #endif	/* _LIO_RXTX_H_ */
-- 
1.8.3.1

[PATCH v3 45/46] net/liquidio: add API to add and remove VLAN port

[Shijith Thotton]

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 doc/guides/nics/features/liquidio.ini   |  2 ++
 drivers/net/liquidio/base/lio_hw_defs.h |  2 ++
 drivers/net/liquidio/lio_ethdev.c       | 45 ++++++++++++++++++++++++++++++++-
 3 files changed, 48 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
index d6d9e92..6230033 100644
--- a/doc/guides/nics/features/liquidio.ini
+++ b/doc/guides/nics/features/liquidio.ini
@@ -12,7 +12,9 @@ Allmulticast mode    = Y
 RSS hash             = Y
 RSS key update       = Y
 RSS reta update      = Y
+VLAN filter          = Y
 CRC offload          = Y
+VLAN offload         = P
 L3 checksum offload  = Y
 L4 checksum offload  = Y
 Inner L3 checksum    = Y
diff --git a/drivers/net/liquidio/base/lio_hw_defs.h b/drivers/net/liquidio/base/lio_hw_defs.h
index 8a22d10..67eaa45 100644
--- a/drivers/net/liquidio/base/lio_hw_defs.h
+++ b/drivers/net/liquidio/base/lio_hw_defs.h
@@ -137,6 +137,8 @@ enum octeon_tag_type {
 #define LIO_CMD_SET_RSS			0xD
 #define LIO_CMD_TNL_RX_CSUM_CTL		0x10
 #define LIO_CMD_TNL_TX_CSUM_CTL		0x11
+#define LIO_CMD_ADD_VLAN_FILTER		0x17
+#define LIO_CMD_DEL_VLAN_FILTER		0x18
 #define LIO_CMD_VXLAN_PORT_CONFIG	0x19
 
 #define LIO_CMD_VXLAN_PORT_ADD		0x0
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 60dfa61..df91659 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -404,7 +404,8 @@ struct rte_lio_xstats_name_off {
 
 	devinfo->rx_offload_capa = (DEV_RX_OFFLOAD_IPV4_CKSUM		|
 				    DEV_RX_OFFLOAD_UDP_CKSUM		|
-				    DEV_RX_OFFLOAD_TCP_CKSUM);
+				    DEV_RX_OFFLOAD_TCP_CKSUM		|
+				    DEV_RX_OFFLOAD_VLAN_STRIP);
 	devinfo->tx_offload_capa = (DEV_TX_OFFLOAD_IPV4_CKSUM		|
 				    DEV_TX_OFFLOAD_UDP_CKSUM		|
 				    DEV_TX_OFFLOAD_TCP_CKSUM		|
@@ -825,6 +826,47 @@ struct rte_lio_xstats_name_off {
 	return 0;
 }
 
+static int
+lio_dev_vlan_filter_set(struct rte_eth_dev *eth_dev, uint16_t vlan_id, int on)
+{
+	struct lio_device *lio_dev = LIO_DEV(eth_dev);
+	struct lio_dev_ctrl_cmd ctrl_cmd;
+	struct lio_ctrl_pkt ctrl_pkt;
+
+	if (lio_dev->linfo.vlan_is_admin_assigned)
+		return -EPERM;
+
+	/* flush added to prevent cmd failure
+	 * incase the queue is full
+	 */
+	lio_flush_iq(lio_dev, lio_dev->instr_queue[0]);
+
+	memset(&ctrl_pkt, 0, sizeof(struct lio_ctrl_pkt));
+	memset(&ctrl_cmd, 0, sizeof(struct lio_dev_ctrl_cmd));
+
+	ctrl_cmd.eth_dev = eth_dev;
+	ctrl_cmd.cond = 0;
+
+	ctrl_pkt.ncmd.s.cmd = on ?
+			LIO_CMD_ADD_VLAN_FILTER : LIO_CMD_DEL_VLAN_FILTER;
+	ctrl_pkt.ncmd.s.param1 = vlan_id;
+	ctrl_pkt.ctrl_cmd = &ctrl_cmd;
+
+	if (lio_send_ctrl_pkt(lio_dev, &ctrl_pkt)) {
+		lio_dev_err(lio_dev, "Failed to %s VLAN port\n",
+			    on ? "add" : "remove");
+		return -1;
+	}
+
+	if (lio_wait_for_ctrl_cmd(lio_dev, &ctrl_cmd)) {
+		lio_dev_err(lio_dev, "Command to %s VLAN port timed out\n",
+			    on ? "add" : "remove");
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Atomically writes the link status information into global
  * structure rte_eth_dev.
@@ -1750,6 +1792,7 @@ static int lio_dev_configure(struct rte_eth_dev *eth_dev)
 	.stats_reset		= lio_dev_stats_reset,
 	.xstats_reset		= lio_dev_xstats_reset,
 	.dev_infos_get		= lio_dev_info_get,
+	.vlan_filter_set	= lio_dev_vlan_filter_set,
 	.rx_queue_setup		= lio_dev_rx_queue_setup,
 	.rx_queue_release	= lio_dev_rx_queue_release,
 	.tx_queue_setup		= lio_dev_tx_queue_setup,
-- 
1.8.3.1

[PATCH v3 46/46] doc: add doc for liquidio and update release notes

[Shijith Thotton]

Add liquidio driver documentation and update 17.05 release notes.

Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
---
 MAINTAINERS                            |   1 +
 doc/guides/nics/features/liquidio.ini  |   1 +
 doc/guides/nics/index.rst              |   1 +
 doc/guides/nics/liquidio.rst           | 280 +++++++++++++++++++++++++++++++++
 doc/guides/rel_notes/release_17_05.rst |   4 +
 5 files changed, 287 insertions(+)
 create mode 100644 doc/guides/nics/liquidio.rst

diff --git a/MAINTAINERS b/MAINTAINERS
index 1421c85..e245d33 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -295,6 +295,7 @@ Cavium LiquidIO
 M: Shijith Thotton <shijith.thotton@cavium.com>
 M: Srisivasubramanian Srinivasan <ssrinivasan@cavium.com>
 F: drivers/net/liquidio/
+F: doc/guides/nics/liquidio.rst
 F: doc/guides/nics/features/liquidio.ini
 
 Chelsio cxgbe
diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini
index 6230033..49cc356 100644
--- a/doc/guides/nics/features/liquidio.ini
+++ b/doc/guides/nics/features/liquidio.ini
@@ -25,3 +25,4 @@ Multiprocess aware   = Y
 Linux UIO            = Y
 Linux VFIO           = Y
 x86-64               = Y
+Usage doc            = Y
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 5139ca6..819fd35 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -48,6 +48,7 @@ Network Interface Controller Drivers
     ixgbe
     intel_vf
     kni
+    liquidio
     mlx4
     mlx5
     nfp
diff --git a/doc/guides/nics/liquidio.rst b/doc/guides/nics/liquidio.rst
new file mode 100644
index 0000000..9ffdc35
--- /dev/null
+++ b/doc/guides/nics/liquidio.rst
@@ -0,0 +1,280 @@
+..  BSD LICENSE
+    Copyright(c) 2017 Cavium, Inc.. All rights reserved.
+    All rights reserved.
+
+    Redistribution and use in source and binary forms, with or without
+    modification, are permitted provided that the following conditions
+    are met:
+
+    * Redistributions of source code must retain the above copyright
+    notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright
+    notice, this list of conditions and the following disclaimer in
+    the documentation and/or other materials provided with the
+    distribution.
+    * Neither the name of Cavium, Inc. nor the names of its
+    contributors may be used to endorse or promote products derived
+    from this software without specific prior written permission.
+
+    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+    OWNER(S) OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+LiquidIO VF Poll Mode Driver
+============================
+
+The LiquidIO VF PMD library (librte_pmd_lio) provides poll mode driver support for
+Cavium LiquidIO® II server adapter VFs. PF management and VF creation can be
+done using kernel driver.
+
+More information can be found at `Cavium Official Website
+<http://cavium.com/LiquidIO_Adapters.html>`_.
+
+Supported LiquidIO Adapters
+-----------------------------
+
+- LiquidIO II CN2350 210SV
+- LiquidIO II CN2360 210SV
+
+
+Pre-Installation Configuration
+------------------------------
+
+The following options can be modified in the ``config`` file.
+Please note that enabling debugging options may affect system performance.
+
+- ``CONFIG_RTE_LIBRTE_LIO_PMD`` (default ``y``)
+
+  Toggle compilation of LiquidIO PMD.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_DRIVER`` (default ``n``)
+
+  Toggle display of generic debugging messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_INIT`` (default ``n``)
+
+  Toggle display of initialization related messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_RX`` (default ``n``)
+
+  Toggle display of receive fast path run-time messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_TX`` (default ``n``)
+
+  Toggle display of transmit fast path run-time messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_MBOX`` (default ``n``)
+
+  Toggle display of mailbox messages.
+
+- ``CONFIG_RTE_LIBRTE_LIO_DEBUG_REGS`` (default ``n``)
+
+  Toggle display of register reads and writes.
+
+
+.. _lio_driver-compilation:
+
+Driver Compilation
+------------------
+
+To compile LiquidIO PMD for Linux x86_64 gcc target, run the following "make"
+command:
+
+.. code-block:: console
+
+   cd <DPDK-source-directory>
+   make install T=x86_64-native-linuxapp-gcc
+
+
+Sample Application Notes
+------------------------
+
+This section demonstrates how to launch ``testpmd`` with LiquidIO® CN23XX
+device managed by ``librte_pmd_lio`` in Linux operating system.
+
+#. Mount huge pages:
+
+   .. code-block:: console
+
+      mkdir /mnt/huge
+      mount -t hugetlbfs nodev /mnt/huge
+
+#. Request huge pages:
+
+   .. code-block:: console
+
+      echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages/nr_hugepages
+
+#. Load ``vfio-pci`` driver:
+
+   .. code-block:: console
+
+      modprobe vfio-pci
+
+#. Bind the LiquidIO VFs to ``vfio-pci`` loaded in previous step:
+
+   Setup VFIO permissions for regular users and then bind to ``vfio-pci``:
+
+   .. code-block:: console
+
+      sudo chmod a+x /dev/vfio
+
+      sudo chmod 0666 /dev/vfio/*
+
+      ./usertools/dpdk-devbind.py --bind vfio-pci 0000:03:00.3 0000:03:08.3
+
+#. Start ``testpmd`` with basic parameters:
+
+   .. code-block:: console
+
+      ./build/app/testpmd -c 0xf -n 4 -- -i
+
+   Example output:
+
+   .. code-block:: console
+
+      [...]
+      EAL: PCI device 0000:03:00.3 on NUMA socket 0
+      EAL:   probe driver: 177d:9712 net_liovf
+      EAL:   using IOMMU type 1 (Type 1)
+      PMD: net_liovf[03:00.3]INFO: DEVICE : CN23XX VF
+      EAL: PCI device 0000:03:08.3 on NUMA socket 0
+      EAL:   probe driver: 177d:9712 net_liovf
+      PMD: net_liovf[03:08.3]INFO: DEVICE : CN23XX VF
+      Interactive-mode selected
+      USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=171456, size=2176, socket=0
+      Configuring Port 0 (socket 0)
+      PMD: net_liovf[03:00.3]INFO: Starting port 0
+      Port 0: F2:A8:1B:5E:B4:66
+      Configuring Port 1 (socket 0)
+      PMD: net_liovf[03:08.3]INFO: Starting port 1
+      Port 1: 32:76:CC:EE:56:D7
+      Checking link statuses...
+      Port 0 Link Up - speed 10000 Mbps - full-duplex
+      Port 1 Link Up - speed 10000 Mbps - full-duplex
+      Done
+      testpmd>
+
+
+SR-IOV: Prerequisites and Sample Application Notes
+--------------------------------------------------
+
+This section provides instructions to configure SR-IOV with Linux OS.
+
+#. Verify SR-IOV and ARI capabilities are enabled on the adapter using ``lspci``:
+
+   .. code-block:: console
+
+      lspci -s <slot> -vvv
+
+   Example output:
+
+   .. code-block:: console
+
+      [...]
+      Capabilities: [148 v1] Alternative Routing-ID Interpretation (ARI)
+      [...]
+      Capabilities: [178 v1] Single Root I/O Virtualization (SR-IOV)
+      [...]
+      Kernel driver in use: LiquidIO
+
+#. Load the kernel module:
+
+   .. code-block:: console
+
+      modprobe liquidio
+
+#. Bring up the PF ports:
+
+   .. code-block:: console
+
+      ifconfig p4p1 up
+      ifconfig p4p2 up
+
+#. Change PF MTU if required:
+
+   .. code-block:: console
+
+      ifconfig p4p1 mtu 9000
+      ifconfig p4p2 mtu 9000
+
+#. Create VF device(s):
+
+   Echo number of VFs to be created into ``"sriov_numvfs"`` sysfs entry
+   of the parent PF.
+
+   .. code-block:: console
+
+      echo 1 > /sys/bus/pci/devices/0000:03:00.0/sriov_numvfs
+      echo 1 > /sys/bus/pci/devices/0000:03:00.1/sriov_numvfs
+
+
+#. Assign VF MAC address:
+
+   Assign MAC address to the VF using iproute2 utility. The syntax is::
+
+      ip link set <PF iface> vf <VF id> mac <macaddr>
+
+   Example output:
+
+   .. code-block:: console
+
+      ip link set p4p1 vf 0 mac F2:A8:1B:5E:B4:66
+
+
+#. Assign VF(s) to VM.
+
+   The VF devices may be passed through to the guest VM using qemu or
+   virt-manager or virsh etc.
+
+   Example qemu guest launch command:
+
+   .. code-block:: console
+
+      ./qemu-system-x86_64 -name lio-vm -machine accel=kvm \
+      -cpu host -m 4096 -smp 4 \
+      -drive file=<disk_file>,if=none,id=disk1,format=<type> \
+      -device virtio-blk-pci,scsi=off,drive=disk1,id=virtio-disk1,bootindex=1 \
+      -device vfio-pci,host=03:00.3 -device vfio-pci,host=03:08.3
+
+
+#. Running testpmd
+
+   Refer :ref:`notes above <lio_driver-compilation>`
+   to compile and run ``testpmd`` application.
+   Use ``igb_uio`` instead of ``vfio-pci`` in VM.
+
+
+Limitations
+-----------
+
+VF MTU
+~~~~~~
+
+VF MTU is limited by PF MTU. Raise PF value before configuring VF for larger packet size.
+
+VLAN offload
+~~~~~~~~~~~~
+
+Tx VLAN insertion is not supported and consequently VLAN offload feature is
+marked partial.
+
+Ring size
+~~~~~~~~~
+
+Number of descriptors for Rx/Tx ring should be in the range 128 to 512.
+
+CRC striping
+~~~~~~~~~~~~
+
+LiquidIO adapters strip ethernet FCS of every packet coming to the host
+interface. So, CRC will be stripped even when the ``rxmode.hw_strip_crc``
+member is set to 0 in ``struct rte_eth_conf``.
diff --git a/doc/guides/rel_notes/release_17_05.rst b/doc/guides/rel_notes/release_17_05.rst
index dee6391..9cee6a3 100644
--- a/doc/guides/rel_notes/release_17_05.rst
+++ b/doc/guides/rel_notes/release_17_05.rst
@@ -96,6 +96,10 @@ New Features
   performance enhancements viz. configurable TX data ring, Receive
   Data Ring, ability to register memory regions.
 
+* **Added LiquidIO network PMD.**
+
+  Added poll mode driver support for Cavium LiquidIO II server adapter VFs.
+
 
 Resolved Issues
 ---------------
-- 
1.8.3.1

Re: [PATCH v3 00/46] LiquidIO PMD

[Ferruh Yigit]

On 3/25/2017 6:24 AM, Shijith Thotton wrote:
> The patch series provides initial version of virtual function poll mode
> driver for Cavium LiquidIO II server adapters. This version adds support
> for LiquidIO II CN23XX 210SV adapters.
> 
> Patch series includes driver documentation doc/guides/nics/liquidio.rst
> and list of supported features doc/guides/nics/features/liquidio.ini.
> Updated release notes to notify the addition of new PMD.
> 
> v3 changes:
> * Addressed review comments from Ferruh.
>  - http://dpdk.org/ml/archives/dev/2017-March/060778.html
>  - http://dpdk.org/ml/archives/dev/2017-March/060772.html
> 
> v2 changes:
> * Restructured patches as suggested by Ferruh.
>  - http://dpdk.org/ml/archives/dev/2017-February/058186.html
> * Addressed review comments on driver from Ferruh.
>  - http://dpdk.org/ml/archives/dev/2017-February/058188.html
>  - http://dpdk.org/ml/archives/dev/2017-February/058194.html
> * Modified commit logs as suggested by Ferruh.
>  - http://dpdk.org/ml/archives/dev/2017-February/058189.html
>  - http://dpdk.org/ml/archives/dev/2017-February/058190.html
>  - http://dpdk.org/ml/archives/dev/2017-February/058193.html
> * Made changes to documentation as per review comments from John.
>  - http://dpdk.org/ml/archives/dev/2017-February/058206.html
> * Fixed FreeBSD build failure.
>  - http://dpdk.org/ml/archives/test-report/2017-February/011272.html
> * Updated driver documentation:
>  - Added CN2360 under supported LiquidIO adapters.
>  - Added CRC strip under limitations.
> 
> Shijith Thotton (46):
>   net/liquidio: add liquidio PMD skeleton
>   net/liquidio/base: hardware register definitions
>   net/liquidio: definitions for log
>   net/liquidio: liquidio VF PMD driver registration
>   net/liquidio/base: macros to read and write register
>   net/liquidio: liquidio device init
>   net/liquidio: add API to disable IO queues
>   net/liquidio: add API to setup IO queue registers
>   net/liquidio: add mbox APIs for PF VF communication
>   net/liquidio: add API to setup mbox registers
>   net/liquidio: add API for PF VF handshake
>   net/liquidio: add API for VF FLR
>   net/liquidio: add APIs to allocate and free IQ
>   net/liquidio: add API to setup IQ
>   net/liquidio: add APIs to allocate and free SC buffer pool
>   net/liquidio: add APIs to allocate and free soft command
>   net/liquidio: add APIs for response list
>   net/liquidio: add API to send packet to device
>   net/liquidio: add API to configure device
>   net/liquidio: add API to setup Rx queue
>   net/liquidio: initialize Rx queue
>   net/liquidio: add Rx data path
>   net/liquidio: add API to release Rx queue
>   net/liquidio: add API to setup Tx queue
>   net/liquidio: add APIs for SG list
>   net/liquidio: add APIs to enable and disable IO queues
>   net/liquidio: add Tx data path for single segment
>   net/liquidio: add Tx data path for multiple segments
>   net/liquidio: add API to flush IQ
>   net/liquidio: add API to release Tx queue
>   net/liquidio: add APIs to start device and update link
>   net/liquidio: add APIs to alloc and send control command
>   net/liquidio: add API to control Rx
>   net/liquidio: add RSS support
>   net/liquidio: add API to get device info
>   net/liquidio: add API to validate VF MTU
>   net/liquidio: add APIs to enable and disable multicast
>   net/liquidio: add APIs to set link up and down
>   net/liquidio: add APIs to configure UDP tunnel port
>   net/liquidio: add support for Rx stats
>   net/liquidio: add support for Tx stats
>   net/liquidio: add APIs for hardware stats
>   net/liquidio: add API to stop device
>   net/liquidio: add API to close device
>   net/liquidio: add API to add and remove VLAN port
>   doc: add doc for liquidio and update release notes

Series applied to dpdk-next-net/master, thanks.