[PATCH net-next 0/4] qed: iWARP fixes and enhancements

[PATCH net-next 0/4] qed: iWARP fixes and enhancements

[Michal Kalderon]

This patch series includes several fixes and enhancements
related to iWARP.

Patch #1 is actually the last of the initial iWARP submission.
It has been delayed until now as I wanted to make sure that qedr
supports iWARP prior to enabling iWARP device detection.

iWARP changes in RDMA tree have been accepted and targeted at
kernel 4.15, therefore, all iWARP fixes for this cycle are
submitted to net-next.

Signed-off by: Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org
Signed-off-by: Ariel Elior <Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>

Michal Kalderon (4):
  qed: Add iWARP enablement support
  qed: Add iWARP out of order support
  qed: Fix maximum number of CQs for iWARP
  qed: iWARP - Add check for errors on a SYN packet

 drivers/net/ethernet/qlogic/qed/qed_cxt.c         |  6 +++
 drivers/net/ethernet/qlogic/qed/qed_iwarp.c       | 52 +++++++++++++++++++++++
 drivers/net/ethernet/qlogic/qed/qed_iwarp.h       | 11 ++++-
 drivers/net/ethernet/qlogic/qed/qed_ll2.c         |  1 +
 drivers/net/ethernet/qlogic/qed/qed_mcp.c         | 10 ++---
 drivers/net/ethernet/qlogic/qed/qed_rdma.c        | 24 +++++++----
 drivers/net/ethernet/qlogic/qed/qed_sp_commands.c |  1 +
 include/linux/qed/qed_ll2_if.h                    |  1 +
 8 files changed, 91 insertions(+), 15 deletions(-)

-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH net-next 1/4] qed: Add iWARP enablement support

[Michal Kalderon]

This patch is the last of the initial iWARP patch series. It
adds the possiblity to actually detect iWARP from the device and enable
it in the critical locations which basically make iWARP available.

It wasn't submitted until now as iWARP hadn't been accepted into
the rdma tree.

Signed-off-by: Michal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: Ariel Elior <Ariel.Elior@cavium.com>
---
 drivers/net/ethernet/qlogic/qed/qed_cxt.c         |  6 ++++++
 drivers/net/ethernet/qlogic/qed/qed_mcp.c         | 10 +++++-----
 drivers/net/ethernet/qlogic/qed/qed_rdma.c        |  5 ++++-
 drivers/net/ethernet/qlogic/qed/qed_sp_commands.c |  1 +
 4 files changed, 16 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/qlogic/qed/qed_cxt.c b/drivers/net/ethernet/qlogic/qed/qed_cxt.c
index af106be..afd07ad 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_cxt.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_cxt.c
@@ -2069,6 +2069,12 @@ static void qed_rdma_set_pf_params(struct qed_hwfn *p_hwfn,
 
 	num_srqs = min_t(u32, 32 * 1024, p_params->num_srqs);
 
+	if (p_hwfn->mcp_info->func_info.protocol == QED_PCI_ETH_RDMA) {
+		DP_NOTICE(p_hwfn,
+			  "Current day drivers don't support RoCE & iWARP simultaneously on the same PF. Default to RoCE-only\n");
+		p_hwfn->hw_info.personality = QED_PCI_ETH_ROCE;
+	}
+
 	switch (p_hwfn->hw_info.personality) {
 	case QED_PCI_ETH_IWARP:
 		/* Each QP requires one connection */
diff --git a/drivers/net/ethernet/qlogic/qed/qed_mcp.c b/drivers/net/ethernet/qlogic/qed/qed_mcp.c
index 376485d..8b99c7d 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_mcp.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_mcp.c
@@ -1691,12 +1691,12 @@ int qed_mcp_get_media_type(struct qed_dev *cdev, u32 *p_media_type)
 	case FW_MB_PARAM_GET_PF_RDMA_ROCE:
 		*p_proto = QED_PCI_ETH_ROCE;
 		break;
-	case FW_MB_PARAM_GET_PF_RDMA_BOTH:
-		DP_NOTICE(p_hwfn,
-			  "Current day drivers don't support RoCE & iWARP. Default to RoCE-only\n");
-		*p_proto = QED_PCI_ETH_ROCE;
-		break;
 	case FW_MB_PARAM_GET_PF_RDMA_IWARP:
+		*p_proto = QED_PCI_ETH_IWARP;
+		break;
+	case FW_MB_PARAM_GET_PF_RDMA_BOTH:
+		*p_proto = QED_PCI_ETH_RDMA;
+		break;
 	default:
 		DP_NOTICE(p_hwfn,
 			  "MFW answers GET_PF_RDMA_PROTOCOL but param is %08x\n",
diff --git a/drivers/net/ethernet/qlogic/qed/qed_rdma.c b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
index 6fb9951..06715f7 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_rdma.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
@@ -156,7 +156,10 @@ static int qed_rdma_alloc(struct qed_hwfn *p_hwfn,
 		return rc;
 
 	p_hwfn->p_rdma_info = p_rdma_info;
-	p_rdma_info->proto = PROTOCOLID_ROCE;
+	if (QED_IS_IWARP_PERSONALITY(p_hwfn))
+		p_rdma_info->proto = PROTOCOLID_IWARP;
+	else
+		p_rdma_info->proto = PROTOCOLID_ROCE;
 
 	num_cons = qed_cxt_get_proto_cid_count(p_hwfn, p_rdma_info->proto,
 					       NULL);
diff --git a/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c b/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c
index 46d0c3c..a1d33f3 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c
@@ -377,6 +377,7 @@ int qed_sp_pf_start(struct qed_hwfn *p_hwfn,
 		p_ramrod->personality = PERSONALITY_ISCSI;
 		break;
 	case QED_PCI_ETH_ROCE:
+	case QED_PCI_ETH_IWARP:
 		p_ramrod->personality = PERSONALITY_RDMA_AND_ETH;
 		break;
 	default:
-- 
1.8.3.1

[PATCH net-next 2/4] qed: Add iWARP out of order support

[Michal Kalderon]

iWARP requires OOO support which is already provided by the ll2
interface (until now was used only for iSCSI offload).
The changes mostly include opening a ll2 dedicated connection for
OOO and notifiying the FW about the handle id.

Signed-off-by: Michal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: Ariel Elior <Ariel.Elior@cavium.com>
---
 drivers/net/ethernet/qlogic/qed/qed_iwarp.c | 44 +++++++++++++++++++++++++++++
 drivers/net/ethernet/qlogic/qed/qed_iwarp.h | 11 +++++++-
 drivers/net/ethernet/qlogic/qed/qed_rdma.c  |  7 +++--
 3 files changed, 59 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/qlogic/qed/qed_iwarp.c b/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
index 9d989c9..568e985 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
@@ -41,6 +41,7 @@
 #include "qed_rdma.h"
 #include "qed_reg_addr.h"
 #include "qed_sp.h"
+#include "qed_ooo.h"
 
 #define QED_IWARP_ORD_DEFAULT		32
 #define QED_IWARP_IRD_DEFAULT		32
@@ -119,6 +120,13 @@ static void qed_iwarp_cid_cleaned(struct qed_hwfn *p_hwfn, u32 cid)
 	spin_unlock_bh(&p_hwfn->p_rdma_info->lock);
 }
 
+void qed_iwarp_init_fw_ramrod(struct qed_hwfn *p_hwfn,
+			      struct iwarp_init_func_params *p_ramrod)
+{
+	p_ramrod->ll2_ooo_q_index = RESC_START(p_hwfn, QED_LL2_QUEUE) +
+				    p_hwfn->p_rdma_info->iwarp.ll2_ooo_handle;
+}
+
 static int qed_iwarp_alloc_cid(struct qed_hwfn *p_hwfn, u32 *cid)
 {
 	int rc;
@@ -1876,6 +1884,16 @@ static int qed_iwarp_ll2_stop(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt)
 		iwarp_info->ll2_syn_handle = QED_IWARP_HANDLE_INVAL;
 	}
 
+	if (iwarp_info->ll2_ooo_handle != QED_IWARP_HANDLE_INVAL) {
+		rc = qed_ll2_terminate_connection(p_hwfn,
+						  iwarp_info->ll2_ooo_handle);
+		if (rc)
+			DP_INFO(p_hwfn, "Failed to terminate ooo connection\n");
+
+		qed_ll2_release_connection(p_hwfn, iwarp_info->ll2_ooo_handle);
+		iwarp_info->ll2_ooo_handle = QED_IWARP_HANDLE_INVAL;
+	}
+
 	qed_llh_remove_mac_filter(p_hwfn,
 				  p_ptt, p_hwfn->p_rdma_info->iwarp.mac_addr);
 	return rc;
@@ -1927,10 +1945,12 @@ static int qed_iwarp_ll2_stop(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt)
 	struct qed_iwarp_info *iwarp_info;
 	struct qed_ll2_acquire_data data;
 	struct qed_ll2_cbs cbs;
+	u16 n_ooo_bufs;
 	int rc = 0;
 
 	iwarp_info = &p_hwfn->p_rdma_info->iwarp;
 	iwarp_info->ll2_syn_handle = QED_IWARP_HANDLE_INVAL;
+	iwarp_info->ll2_ooo_handle = QED_IWARP_HANDLE_INVAL;
 
 	iwarp_info->max_mtu = params->max_mtu;
 
@@ -1978,6 +1998,29 @@ static int qed_iwarp_ll2_stop(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt)
 	if (rc)
 		goto err;
 
+	/* Start OOO connection */
+	data.input.conn_type = QED_LL2_TYPE_OOO;
+	data.input.mtu = params->max_mtu;
+
+	n_ooo_bufs = (QED_IWARP_MAX_OOO * QED_IWARP_RCV_WND_SIZE_DEF) /
+		     iwarp_info->max_mtu;
+	n_ooo_bufs = min_t(u32, n_ooo_bufs, QED_IWARP_LL2_OOO_MAX_RX_SIZE);
+
+	data.input.rx_num_desc = n_ooo_bufs;
+	data.input.rx_num_ooo_buffers = n_ooo_bufs;
+
+	data.input.tx_max_bds_per_packet = 1;	/* will never be fragmented */
+	data.input.tx_num_desc = QED_IWARP_LL2_OOO_DEF_TX_SIZE;
+	data.p_connection_handle = &iwarp_info->ll2_ooo_handle;
+
+	rc = qed_ll2_acquire_connection(p_hwfn, &data);
+	if (rc)
+		goto err;
+
+	rc = qed_ll2_establish_connection(p_hwfn, iwarp_info->ll2_ooo_handle);
+	if (rc)
+		goto err;
+
 	return rc;
 err:
 	qed_iwarp_ll2_stop(p_hwfn, p_ptt);
@@ -2014,6 +2057,7 @@ int qed_iwarp_setup(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt,
 
 	qed_spq_register_async_cb(p_hwfn, PROTOCOLID_IWARP,
 				  qed_iwarp_async_event);
+	qed_ooo_setup(p_hwfn);
 
 	return qed_iwarp_ll2_start(p_hwfn, params, p_ptt);
 }
diff --git a/drivers/net/ethernet/qlogic/qed/qed_iwarp.h b/drivers/net/ethernet/qlogic/qed/qed_iwarp.h
index 148ef3c..9e2bfde 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_iwarp.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_iwarp.h
@@ -47,7 +47,12 @@ enum qed_iwarp_qp_state {
 #define QED_IWARP_LL2_SYN_TX_SIZE       (128)
 #define QED_IWARP_LL2_SYN_RX_SIZE       (256)
 #define QED_IWARP_MAX_SYN_PKT_SIZE      (128)
-#define QED_IWARP_HANDLE_INVAL			(0xff)
+
+#define QED_IWARP_LL2_OOO_DEF_TX_SIZE   (256)
+#define QED_IWARP_MAX_OOO		(16)
+#define QED_IWARP_LL2_OOO_MAX_RX_SIZE   (16384)
+
+#define QED_IWARP_HANDLE_INVAL		(0xff)
 
 struct qed_iwarp_ll2_buff {
 	void *data;
@@ -67,6 +72,7 @@ struct qed_iwarp_info {
 	u8 crc_needed;
 	u8 tcp_flags;
 	u8 ll2_syn_handle;
+	u8 ll2_ooo_handle;
 	u8 peer2peer;
 	enum mpa_negotiation_mode mpa_rev;
 	enum mpa_rtr_type rtr_type;
@@ -147,6 +153,9 @@ struct qed_iwarp_listener {
 int qed_iwarp_setup(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt,
 		    struct qed_rdma_start_in_params *params);
 
+void qed_iwarp_init_fw_ramrod(struct qed_hwfn *p_hwfn,
+			      struct iwarp_init_func_params *p_ramrod);
+
 int qed_iwarp_stop(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt);
 
 void qed_iwarp_resc_free(struct qed_hwfn *p_hwfn);
diff --git a/drivers/net/ethernet/qlogic/qed/qed_rdma.c b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
index 06715f7..4f46f28 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_rdma.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
@@ -551,10 +551,13 @@ static int qed_rdma_start_fw(struct qed_hwfn *p_hwfn,
 	if (rc)
 		return rc;
 
-	if (QED_IS_IWARP_PERSONALITY(p_hwfn))
+	if (QED_IS_IWARP_PERSONALITY(p_hwfn)) {
+		qed_iwarp_init_fw_ramrod(p_hwfn,
+					 &p_ent->ramrod.iwarp_init_func.iwarp);
 		p_ramrod = &p_ent->ramrod.iwarp_init_func.rdma;
-	else
+	} else {
 		p_ramrod = &p_ent->ramrod.roce_init_func.rdma;
+	}
 
 	p_params_header = &p_ramrod->params_header;
 	p_params_header->cnq_start_offset = (u8)RESC_START(p_hwfn,
-- 
1.8.3.1

[PATCH net-next 3/4] qed: Fix maximum number of CQs for iWARP

[Michal Kalderon]

The maximum number of CQs supported is bound to the number
of connections supported, which differs between RoCE and iWARP.

This fixes a crash that occurred in iWARP when running 1000 sessions
using perftest.

Signed-off-by: Michal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: Ariel Elior <Ariel.Elior@cavium.com>
---
 drivers/net/ethernet/qlogic/qed/qed_rdma.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/qlogic/qed/qed_rdma.c b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
index 4f46f28..c8c4b39 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_rdma.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
@@ -209,11 +209,11 @@ static int qed_rdma_alloc(struct qed_hwfn *p_hwfn,
 		goto free_pd_map;
 	}
 
-	/* Allocate bitmap for cq's. The maximum number of CQs is bounded to
-	 * twice the number of QPs.
+	/* Allocate bitmap for cq's. The maximum number of CQs is bound to
+	 * the number of connections we support. (num_qps in iWARP or
+	 * num_qps/2 in RoCE).
 	 */
-	rc = qed_rdma_bmap_alloc(p_hwfn, &p_rdma_info->cq_map,
-				 p_rdma_info->num_qps * 2, "CQ");
+	rc = qed_rdma_bmap_alloc(p_hwfn, &p_rdma_info->cq_map, num_cons, "CQ");
 	if (rc) {
 		DP_VERBOSE(p_hwfn, QED_MSG_RDMA,
 			   "Failed to allocate cq bitmap, rc = %d\n", rc);
@@ -222,10 +222,10 @@ static int qed_rdma_alloc(struct qed_hwfn *p_hwfn,
 
 	/* Allocate bitmap for toggle bit for cq icids
 	 * We toggle the bit every time we create or resize cq for a given icid.
-	 * The maximum number of CQs is bounded to  twice the number of QPs.
+	 * Size needs to equal the size of the cq bmap.
 	 */
 	rc = qed_rdma_bmap_alloc(p_hwfn, &p_rdma_info->toggle_bits,
-				 p_rdma_info->num_qps * 2, "Toggle");
+				 num_cons, "Toggle");
 	if (rc) {
 		DP_VERBOSE(p_hwfn, QED_MSG_RDMA,
 			   "Failed to allocate toogle bits, rc = %d\n", rc);
-- 
1.8.3.1

[PATCH net-next 4/4] qed: iWARP - Add check for errors on a SYN packet

[Michal Kalderon]

A SYN packet which arrives with errors from FW should be dropped.
This required adding an additional field to the ll2
rx completion data.

Signed-off-by: Michal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: Ariel Elior <Ariel.Elior@cavium.com>
---
 drivers/net/ethernet/qlogic/qed/qed_iwarp.c | 8 ++++++++
 drivers/net/ethernet/qlogic/qed/qed_ll2.c   | 1 +
 include/linux/qed/qed_ll2_if.h              | 1 +
 3 files changed, 10 insertions(+)

diff --git a/drivers/net/ethernet/qlogic/qed/qed_iwarp.c b/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
index 568e985..8fc9c811 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
@@ -1733,6 +1733,14 @@ int qed_iwarp_reject(void *rdma_cxt, struct qed_iwarp_reject_in *iparams)
 
 	memset(&cm_info, 0, sizeof(cm_info));
 	ll2_syn_handle = p_hwfn->p_rdma_info->iwarp.ll2_syn_handle;
+
+	/* Check if packet was received with errors... */
+	if (data->err_flags) {
+		DP_NOTICE(p_hwfn, "Error received on SYN packet: 0x%x\n",
+			  data->err_flags);
+		goto err;
+	}
+
 	if (GET_FIELD(data->parse_flags,
 		      PARSING_AND_ERR_FLAGS_L4CHKSMWASCALCULATED) &&
 	    GET_FIELD(data->parse_flags, PARSING_AND_ERR_FLAGS_L4CHKSMERROR)) {
diff --git a/drivers/net/ethernet/qlogic/qed/qed_ll2.c b/drivers/net/ethernet/qlogic/qed/qed_ll2.c
index c06ad4f..250afa5 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_ll2.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_ll2.c
@@ -413,6 +413,7 @@ static void qed_ll2_rxq_parse_reg(struct qed_hwfn *p_hwfn,
 				  struct qed_ll2_comp_rx_data *data)
 {
 	data->parse_flags = le16_to_cpu(p_cqe->rx_cqe_fp.parse_flags.flags);
+	data->err_flags = le16_to_cpu(p_cqe->rx_cqe_fp.err_flags.flags);
 	data->length.packet_length =
 	    le16_to_cpu(p_cqe->rx_cqe_fp.packet_length);
 	data->vlan = le16_to_cpu(p_cqe->rx_cqe_fp.vlan);
diff --git a/include/linux/qed/qed_ll2_if.h b/include/linux/qed/qed_ll2_if.h
index dd7a3b8..89fa0bb 100644
--- a/include/linux/qed/qed_ll2_if.h
+++ b/include/linux/qed/qed_ll2_if.h
@@ -101,6 +101,7 @@ struct qed_ll2_comp_rx_data {
 	void *cookie;
 	dma_addr_t rx_buf_addr;
 	u16 parse_flags;
+	u16 err_flags;
 	u16 vlan;
 	bool b_last_packet;
 	u8 connection_handle;
-- 
1.8.3.1

Re: [PATCH net-next 2/4] qed: Add iWARP out of order support

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 6380 bytes --]

On Tue, Sep 19, 2017 at 08:26:17PM +0300, Michal Kalderon wrote:
> iWARP requires OOO support which is already provided by the ll2
> interface (until now was used only for iSCSI offload).
> The changes mostly include opening a ll2 dedicated connection for
> OOO and notifiying the FW about the handle id.
>
> Signed-off-by: Michal Kalderon <Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Ariel Elior <Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> ---
>  drivers/net/ethernet/qlogic/qed/qed_iwarp.c | 44 +++++++++++++++++++++++++++++
>  drivers/net/ethernet/qlogic/qed/qed_iwarp.h | 11 +++++++-
>  drivers/net/ethernet/qlogic/qed/qed_rdma.c  |  7 +++--
>  3 files changed, 59 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/ethernet/qlogic/qed/qed_iwarp.c b/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
> index 9d989c9..568e985 100644
> --- a/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
> +++ b/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
> @@ -41,6 +41,7 @@
>  #include "qed_rdma.h"
>  #include "qed_reg_addr.h"
>  #include "qed_sp.h"
> +#include "qed_ooo.h"
>
>  #define QED_IWARP_ORD_DEFAULT		32
>  #define QED_IWARP_IRD_DEFAULT		32
> @@ -119,6 +120,13 @@ static void qed_iwarp_cid_cleaned(struct qed_hwfn *p_hwfn, u32 cid)
>  	spin_unlock_bh(&p_hwfn->p_rdma_info->lock);
>  }
>
> +void qed_iwarp_init_fw_ramrod(struct qed_hwfn *p_hwfn,
> +			      struct iwarp_init_func_params *p_ramrod)
> +{
> +	p_ramrod->ll2_ooo_q_index = RESC_START(p_hwfn, QED_LL2_QUEUE) +
> +				    p_hwfn->p_rdma_info->iwarp.ll2_ooo_handle;
> +}
> +
>  static int qed_iwarp_alloc_cid(struct qed_hwfn *p_hwfn, u32 *cid)
>  {
>  	int rc;
> @@ -1876,6 +1884,16 @@ static int qed_iwarp_ll2_stop(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt)
>  		iwarp_info->ll2_syn_handle = QED_IWARP_HANDLE_INVAL;
>  	}
>
> +	if (iwarp_info->ll2_ooo_handle != QED_IWARP_HANDLE_INVAL) {
> +		rc = qed_ll2_terminate_connection(p_hwfn,
> +						  iwarp_info->ll2_ooo_handle);
> +		if (rc)
> +			DP_INFO(p_hwfn, "Failed to terminate ooo connection\n");

What exactly will you do with this knowledge? Anyway you are not
interested in return values of qed_ll2_terminate_connection function in
this place and other places too.

Why don't you handle EAGAIN returned from the qed_ll2_terminate_connection()?

Thanks

> +
> +		qed_ll2_release_connection(p_hwfn, iwarp_info->ll2_ooo_handle);
> +		iwarp_info->ll2_ooo_handle = QED_IWARP_HANDLE_INVAL;
> +	}
> +
>  	qed_llh_remove_mac_filter(p_hwfn,
>  				  p_ptt, p_hwfn->p_rdma_info->iwarp.mac_addr);
>  	return rc;
> @@ -1927,10 +1945,12 @@ static int qed_iwarp_ll2_stop(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt)
>  	struct qed_iwarp_info *iwarp_info;
>  	struct qed_ll2_acquire_data data;
>  	struct qed_ll2_cbs cbs;
> +	u16 n_ooo_bufs;
>  	int rc = 0;
>
>  	iwarp_info = &p_hwfn->p_rdma_info->iwarp;
>  	iwarp_info->ll2_syn_handle = QED_IWARP_HANDLE_INVAL;
> +	iwarp_info->ll2_ooo_handle = QED_IWARP_HANDLE_INVAL;
>
>  	iwarp_info->max_mtu = params->max_mtu;
>
> @@ -1978,6 +1998,29 @@ static int qed_iwarp_ll2_stop(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt)
>  	if (rc)
>  		goto err;
>
> +	/* Start OOO connection */
> +	data.input.conn_type = QED_LL2_TYPE_OOO;
> +	data.input.mtu = params->max_mtu;
> +
> +	n_ooo_bufs = (QED_IWARP_MAX_OOO * QED_IWARP_RCV_WND_SIZE_DEF) /
> +		     iwarp_info->max_mtu;
> +	n_ooo_bufs = min_t(u32, n_ooo_bufs, QED_IWARP_LL2_OOO_MAX_RX_SIZE);
> +
> +	data.input.rx_num_desc = n_ooo_bufs;
> +	data.input.rx_num_ooo_buffers = n_ooo_bufs;
> +
> +	data.input.tx_max_bds_per_packet = 1;	/* will never be fragmented */
> +	data.input.tx_num_desc = QED_IWARP_LL2_OOO_DEF_TX_SIZE;
> +	data.p_connection_handle = &iwarp_info->ll2_ooo_handle;
> +
> +	rc = qed_ll2_acquire_connection(p_hwfn, &data);
> +	if (rc)
> +		goto err;
> +
> +	rc = qed_ll2_establish_connection(p_hwfn, iwarp_info->ll2_ooo_handle);
> +	if (rc)
> +		goto err;
> +
>  	return rc;
>  err:
>  	qed_iwarp_ll2_stop(p_hwfn, p_ptt);
> @@ -2014,6 +2057,7 @@ int qed_iwarp_setup(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt,
>
>  	qed_spq_register_async_cb(p_hwfn, PROTOCOLID_IWARP,
>  				  qed_iwarp_async_event);
> +	qed_ooo_setup(p_hwfn);
>
>  	return qed_iwarp_ll2_start(p_hwfn, params, p_ptt);
>  }
> diff --git a/drivers/net/ethernet/qlogic/qed/qed_iwarp.h b/drivers/net/ethernet/qlogic/qed/qed_iwarp.h
> index 148ef3c..9e2bfde 100644
> --- a/drivers/net/ethernet/qlogic/qed/qed_iwarp.h
> +++ b/drivers/net/ethernet/qlogic/qed/qed_iwarp.h
> @@ -47,7 +47,12 @@ enum qed_iwarp_qp_state {
>  #define QED_IWARP_LL2_SYN_TX_SIZE       (128)
>  #define QED_IWARP_LL2_SYN_RX_SIZE       (256)
>  #define QED_IWARP_MAX_SYN_PKT_SIZE      (128)
> -#define QED_IWARP_HANDLE_INVAL			(0xff)
> +
> +#define QED_IWARP_LL2_OOO_DEF_TX_SIZE   (256)
> +#define QED_IWARP_MAX_OOO		(16)
> +#define QED_IWARP_LL2_OOO_MAX_RX_SIZE   (16384)
> +
> +#define QED_IWARP_HANDLE_INVAL		(0xff)
>
>  struct qed_iwarp_ll2_buff {
>  	void *data;
> @@ -67,6 +72,7 @@ struct qed_iwarp_info {
>  	u8 crc_needed;
>  	u8 tcp_flags;
>  	u8 ll2_syn_handle;
> +	u8 ll2_ooo_handle;
>  	u8 peer2peer;
>  	enum mpa_negotiation_mode mpa_rev;
>  	enum mpa_rtr_type rtr_type;
> @@ -147,6 +153,9 @@ struct qed_iwarp_listener {
>  int qed_iwarp_setup(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt,
>  		    struct qed_rdma_start_in_params *params);
>
> +void qed_iwarp_init_fw_ramrod(struct qed_hwfn *p_hwfn,
> +			      struct iwarp_init_func_params *p_ramrod);
> +
>  int qed_iwarp_stop(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt);
>
>  void qed_iwarp_resc_free(struct qed_hwfn *p_hwfn);
> diff --git a/drivers/net/ethernet/qlogic/qed/qed_rdma.c b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
> index 06715f7..4f46f28 100644
> --- a/drivers/net/ethernet/qlogic/qed/qed_rdma.c
> +++ b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
> @@ -551,10 +551,13 @@ static int qed_rdma_start_fw(struct qed_hwfn *p_hwfn,
>  	if (rc)
>  		return rc;
>
> -	if (QED_IS_IWARP_PERSONALITY(p_hwfn))
> +	if (QED_IS_IWARP_PERSONALITY(p_hwfn)) {
> +		qed_iwarp_init_fw_ramrod(p_hwfn,
> +					 &p_ent->ramrod.iwarp_init_func.iwarp);
>  		p_ramrod = &p_ent->ramrod.iwarp_init_func.rdma;
> -	else
> +	} else {
>  		p_ramrod = &p_ent->ramrod.roce_init_func.rdma;
> +	}
>
>  	p_params_header = &p_ramrod->params_header;
>  	p_params_header->cnq_start_offset = (u8)RESC_START(p_hwfn,
> --
> 1.8.3.1
>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH net-next 3/4] qed: Fix maximum number of CQs for iWARP

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 505 bytes --]

On Tue, Sep 19, 2017 at 08:26:18PM +0300, Michal Kalderon wrote:
> The maximum number of CQs supported is bound to the number
> of connections supported, which differs between RoCE and iWARP.
>
> This fixes a crash that occurred in iWARP when running 1000 sessions
> using perftest.
>
> Signed-off-by: Michal Kalderon <Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Ariel Elior <Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> ---

It is worth to add Fixes line.

Thanks

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH net-next 3/4] qed: Fix maximum number of CQs for iWARP

[Kalderon, Michal]

From: Leon Romanovsky <leon@kernel.org>
Sent: Tuesday, September 19, 2017 8:46 PM
On Tue, Sep 19, 2017 at 08:26:18PM +0300, Michal Kalderon wrote:
>> The maximum number of CQs supported is bound to the number
>> of connections supported, which differs between RoCE and iWARP.
>>
>> This fixes a crash that occurred in iWARP when running 1000 sessions
>> using perftest.
>>
>> Signed-off-by: Michal Kalderon <Michal.Kalderon@cavium.com>
>> Signed-off-by: Ariel Elior <Ariel.Elior@cavium.com>
>> ---
>
>It is worth to add Fixes line.
>
>Thanks
The original code was there before we had iWARP support, so this doesn't
exactly fix an older commit, but fixes iWARP code in general.

Re: [PATCH net-next 2/4] qed: Add iWARP out of order support

[Kalderon, Michal]

From: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Sent: Tuesday, September 19, 2017 8:45 PM
On Tue, Sep 19, 2017 at 08:26:17PM +0300, Michal Kalderon wrote:
>> iWARP requires OOO support which is already provided by the ll2
>> interface (until now was used only for iSCSI offload).
>> The changes mostly include opening a ll2 dedicated connection for
>> OOO and notifiying the FW about the handle id.
>>
>> Signed-off-by: Michal Kalderon <Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
>> Signed-off-by: Ariel Elior <Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
>> ---
>>  drivers/net/ethernet/qlogic/qed/qed_iwarp.c | 44 +++++++++++++++++++++++++++++
>>  drivers/net/ethernet/qlogic/qed/qed_iwarp.h | 11 +++++++-
>>  drivers/net/ethernet/qlogic/qed/qed_rdma.c  |  7 +++--
>>  3 files changed, 59 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/qlogic/qed/qed_iwarp.c b/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
>> index 9d989c9..568e985 100644
>> --- a/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
>> +++ b/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
>> @@ -41,6 +41,7 @@
>>  #include "qed_rdma.h"
>>  #include "qed_reg_addr.h"
>>  #include "qed_sp.h"
>> +#include "qed_ooo.h"
>>
>>  #define QED_IWARP_ORD_DEFAULT                32
>>  #define QED_IWARP_IRD_DEFAULT                32
>> @@ -119,6 +120,13 @@ static void qed_iwarp_cid_cleaned(struct qed_hwfn *p_hwfn, u32 cid)
>>       spin_unlock_bh(&p_hwfn->p_rdma_info->lock);
>>  }
>>
>> +void qed_iwarp_init_fw_ramrod(struct qed_hwfn *p_hwfn,
>> +                           struct iwarp_init_func_params *p_ramrod)
>> +{
>> +     p_ramrod->ll2_ooo_q_index = RESC_START(p_hwfn, QED_LL2_QUEUE) +
>> +                                 p_hwfn->p_rdma_info->iwarp.ll2_ooo_handle;
>> +}
>> +
>>  static int qed_iwarp_alloc_cid(struct qed_hwfn *p_hwfn, u32 *cid)
>>  {
>>       int rc;
>> @@ -1876,6 +1884,16 @@ static int qed_iwarp_ll2_stop(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt)
>>               iwarp_info->ll2_syn_handle = QED_IWARP_HANDLE_INVAL;
>>       }
>>
>> +     if (iwarp_info->ll2_ooo_handle != QED_IWARP_HANDLE_INVAL) {
>> +             rc = qed_ll2_terminate_connection(p_hwfn,
>> +                                               iwarp_info->ll2_ooo_handle);
>> +             if (rc)
>> +                     DP_INFO(p_hwfn, "Failed to terminate ooo connection\n");
>
>What exactly will you do with this knowledge? Anyway you are not
>interested in return values of qed_ll2_terminate_connection function in
>this place and other places too.
>
>Why don't you handle EAGAIN returned from the qed_ll2_terminate_connection()?
>
>Thanks
Thanks for pointing this out, you're right we could have ignored the return code, as there's
not much we can do at this point if it failed. But I still feel failures are worth knowing about,
and could help in analysis if they unexpectedly lead to another issue.
As for EAGAIN, it is very unlikely that we'll get this return code. Will consider adding generic
handling for this as a separate patch, as this currently isn't handled in any of the ll2 flows.
thanks,--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH net-next 3/4] qed: Fix maximum number of CQs for iWARP

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1158 bytes --]

On Wed, Sep 20, 2017 at 05:46:35AM +0000, Kalderon, Michal wrote:
> From: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> Sent: Tuesday, September 19, 2017 8:46 PM
> On Tue, Sep 19, 2017 at 08:26:18PM +0300, Michal Kalderon wrote:
> >> The maximum number of CQs supported is bound to the number
> >> of connections supported, which differs between RoCE and iWARP.
> >>
> >> This fixes a crash that occurred in iWARP when running 1000 sessions
> >> using perftest.
> >>
> >> Signed-off-by: Michal Kalderon <Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> >> Signed-off-by: Ariel Elior <Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> >> ---
> >
> >It is worth to add Fixes line.
> >
> >Thanks
> The original code was there before we had iWARP support, so this doesn't
> exactly fix an older commit, but fixes iWARP code in general.

So add Fixes which points to iWARP ennoblement patch.

Thanks

>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]