[PATCH rdma-core 0/7] Add mlx4 direct verbs

[PATCH rdma-core 0/7] Add mlx4 direct verbs

[Yishai Hadas]

This patchset from Maor adds direct access to mlx4 devices, it follows same
solution from both API and packaging perspectives as was already accepted for
mlx5.

The libibverbs API is an abstract API. It is agnostic to any underlying
provider specific implementation. While this abstraction has the advantage of
user applications portability it has a performance penalty. For some
applications optimizing performance is more important than portability.

The mlx4 direct verbs API introduced in this patchset is intended for such
applications. It exposes mlx4 specific low level data path
(send/receive/completion) operations, allowing the application to bypass the
libibverbs data path API.

The proposed interface consists from one hardware specific header with relevant
inline functions and conversion logic from ibverbs structures to mlx4 related
structures.

New header is going to be installed in below path location:
/usr/inlcude/infiniband/mlx4dv.h

The direct include of mlx4dv.h together with linkage to mlx4 library will
allow usage of this new interface.

Pull request was sent:
https://github.com/linux-rdma/rdma-core/pull/155

Yishai

Maor Gottlieb (7):
  mlx4: Add mlx4 direct verbs
  mlx4: Add CQE fields
  mlx4: Convert datagram segment to use mlx4_av struct
  mlx4: Add WQE segments implementation
  mlx4: Convert to use predefined get CQE opcodes
  mlx4: Add direct verbs man pages
  mlx4: Export mlx4 direct verbs interface

 CMakeLists.txt                       |   1 +
 debian/ibverbs-providers.install     |   1 +
 debian/ibverbs-providers.symbols     |   4 +
 debian/libibverbs-dev.install        |   4 +
 providers/mlx4/CMakeLists.txt        |   7 +-
 providers/mlx4/cq.c                  |  65 +----
 providers/mlx4/libmlx4.map           |   7 +
 providers/mlx4/man/CMakeLists.txt    |   4 +
 providers/mlx4/man/mlx4dv.7          |  39 +++
 providers/mlx4/man/mlx4dv_init_obj.3 | 117 +++++++++
 providers/mlx4/mlx4.c                |  76 ++++++
 providers/mlx4/mlx4.h                |  73 +-----
 providers/mlx4/mlx4dv.h              | 465 +++++++++++++++++++++++++++++++++++
 providers/mlx4/qp.c                  |   3 +-
 providers/mlx4/srq.c                 |   1 -
 providers/mlx4/verbs.c               |   1 -
 providers/mlx4/wqe.h                 | 149 -----------
 redhat/rdma-core.spec                |   3 +
 18 files changed, 743 insertions(+), 277 deletions(-)
 create mode 100644 providers/mlx4/libmlx4.map
 create mode 100644 providers/mlx4/man/CMakeLists.txt
 create mode 100644 providers/mlx4/man/mlx4dv.7
 create mode 100644 providers/mlx4/man/mlx4dv_init_obj.3
 create mode 100644 providers/mlx4/mlx4dv.h
 delete mode 100644 providers/mlx4/wqe.h

-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 1/7] mlx4: Add mlx4 direct verbs

[Yishai Hadas]

From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Direct verbs feature provides fast data path execution
by implementing verbs directly over hardware specification.

It consists from the two parts:
 * Exported functions to translate internal mlx4 structures to be
   accessible by user applications.
 * Inline functions, enums and defines to simplify programmer's life.

This patch adds one header file (mlx4dv.h) with general, CQE and WQE data.

Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 providers/mlx4/cq.c     |  47 +------
 providers/mlx4/mlx4.c   |  76 +++++++++++
 providers/mlx4/mlx4.h   |  60 +--------
 providers/mlx4/mlx4dv.h | 338 ++++++++++++++++++++++++++++++++++++++++++++++++
 providers/mlx4/qp.c     |   1 -
 providers/mlx4/srq.c    |   1 -
 providers/mlx4/verbs.c  |   1 -
 providers/mlx4/wqe.h    | 149 ---------------------
 8 files changed, 420 insertions(+), 253 deletions(-)
 create mode 100644 providers/mlx4/mlx4dv.h
 delete mode 100644 providers/mlx4/wqe.h

diff --git a/providers/mlx4/cq.c b/providers/mlx4/cq.c
index 8ebc2ae..068e0d2 100644
--- a/providers/mlx4/cq.c
+++ b/providers/mlx4/cq.c
@@ -46,55 +46,11 @@
 #include "mlx4.h"
 
 enum {
-	MLX4_CQ_DOORBELL			= 0x20
-};
-
-enum {
 	CQ_OK					=  0,
 	CQ_EMPTY				= -1,
 	CQ_POLL_ERR				= -2
 };
 
-#define MLX4_CQ_DB_REQ_NOT_SOL			(1 << 24)
-#define MLX4_CQ_DB_REQ_NOT			(2 << 24)
-
-enum {
-	MLX4_CQE_VLAN_PRESENT_MASK		= 1 << 29,
-	MLX4_CQE_QPN_MASK			= 0xffffff,
-};
-
-enum {
-	MLX4_CQE_OWNER_MASK			= 0x80,
-	MLX4_CQE_IS_SEND_MASK			= 0x40,
-	MLX4_CQE_OPCODE_MASK			= 0x1f
-};
-
-enum {
-	MLX4_CQE_SYNDROME_LOCAL_LENGTH_ERR		= 0x01,
-	MLX4_CQE_SYNDROME_LOCAL_QP_OP_ERR		= 0x02,
-	MLX4_CQE_SYNDROME_LOCAL_PROT_ERR		= 0x04,
-	MLX4_CQE_SYNDROME_WR_FLUSH_ERR			= 0x05,
-	MLX4_CQE_SYNDROME_MW_BIND_ERR			= 0x06,
-	MLX4_CQE_SYNDROME_BAD_RESP_ERR			= 0x10,
-	MLX4_CQE_SYNDROME_LOCAL_ACCESS_ERR		= 0x11,
-	MLX4_CQE_SYNDROME_REMOTE_INVAL_REQ_ERR		= 0x12,
-	MLX4_CQE_SYNDROME_REMOTE_ACCESS_ERR		= 0x13,
-	MLX4_CQE_SYNDROME_REMOTE_OP_ERR			= 0x14,
-	MLX4_CQE_SYNDROME_TRANSPORT_RETRY_EXC_ERR	= 0x15,
-	MLX4_CQE_SYNDROME_RNR_RETRY_EXC_ERR		= 0x16,
-	MLX4_CQE_SYNDROME_REMOTE_ABORTED_ERR		= 0x22,
-};
-
-struct mlx4_err_cqe {
-	uint32_t	vlan_my_qpn;
-	uint32_t	reserved1[5];
-	uint16_t	wqe_index;
-	uint8_t		vendor_err;
-	uint8_t		syndrome;
-	uint8_t		reserved2[3];
-	uint8_t		owner_sr_opcode;
-};
-
 static struct mlx4_cqe *get_cqe(struct mlx4_cq *cq, int entry)
 {
 	return cq->buf.buf + entry * cq->cqe_size;
@@ -722,6 +678,9 @@ void __mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq)
 	int nfreed = 0;
 	int cqe_inc = cq->cqe_size == 64 ? 1 : 0;
 
+	if (!cq || cq->flags & MLX4_CQ_FLAGS_DV_OWNED)
+		return;
+
 	/*
 	 * First we need to find the current producer index, so we
 	 * know where to start cleaning from.  It doesn't matter if HW
diff --git a/providers/mlx4/mlx4.c b/providers/mlx4/mlx4.c
index b798b37..db3f903 100644
--- a/providers/mlx4/mlx4.c
+++ b/providers/mlx4/mlx4.c
@@ -324,3 +324,79 @@ static __attribute__((constructor)) void mlx4_register_driver(void)
 {
 	verbs_register_driver("mlx4", mlx4_driver_init);
 }
+
+static int mlx4dv_get_qp(struct ibv_qp *qp_in,
+			 struct mlx4dv_qp *qp_out)
+{
+	struct mlx4_qp *mqp = to_mqp(qp_in);
+	struct mlx4_context *ctx = to_mctx(qp_in->context);
+
+	qp_out->comp_mask = 0;
+
+	qp_out->buf.buf = mqp->buf.buf;
+	qp_out->buf.length = mqp->buf.length;
+
+	qp_out->rdb = mqp->db;
+	qp_out->sdb = (uint32_t *) (ctx->uar + MLX4_SEND_DOORBELL);
+	qp_out->doorbell_qpn = mqp->doorbell_qpn;
+
+	qp_out->sq.wqe_cnt = mqp->sq.wqe_cnt;
+	qp_out->sq.wqe_shift = mqp->sq.wqe_shift;
+	qp_out->sq.offset = mqp->sq.offset;
+
+	qp_out->rq.wqe_cnt = mqp->rq.wqe_cnt;
+	qp_out->rq.wqe_shift = mqp->rq.wqe_shift;
+	qp_out->rq.offset = mqp->rq.offset;
+
+	return 0;
+}
+
+static int mlx4dv_get_cq(struct ibv_cq *cq_in,
+			 struct mlx4dv_cq *cq_out)
+{
+	struct mlx4_cq *mcq = to_mcq(cq_in);
+
+	cq_out->comp_mask = 0;
+	cq_out->buf.buf = mcq->buf.buf;
+	cq_out->buf.length = mcq->buf.length;
+	cq_out->cqn = mcq->cqn;
+	cq_out->set_ci_db = mcq->set_ci_db;
+	cq_out->arm_db = mcq->arm_db;
+	cq_out->arm_sn = mcq->arm_sn;
+	cq_out->cqe_size = mcq->cqe_size;
+	cq_out->cqe_cnt = mcq->ibv_cq.cqe + 1;
+
+	mcq->flags |= MLX4_CQ_FLAGS_DV_OWNED;
+
+	return 0;
+}
+
+static int mlx4dv_get_srq(struct ibv_srq *srq_in,
+			  struct mlx4dv_srq *srq_out)
+{
+	struct mlx4_srq *msrq = to_msrq(srq_in);
+
+	srq_out->comp_mask = 0;
+	srq_out->buf.buf = msrq->buf.buf;
+	srq_out->buf.length = msrq->buf.length;
+	srq_out->wqe_shift = msrq->wqe_shift;
+	srq_out->head = msrq->head;
+	srq_out->tail = msrq->tail;
+	srq_out->db = msrq->db;
+
+	return 0;
+}
+
+int mlx4dv_init_obj(struct mlx4dv_obj *obj, uint64_t obj_type)
+{
+	int ret = 0;
+
+	if (obj_type & MLX4DV_OBJ_QP)
+		ret = mlx4dv_get_qp(obj->qp.in, obj->qp.out);
+	if (!ret && (obj_type & MLX4DV_OBJ_CQ))
+		ret = mlx4dv_get_cq(obj->cq.in, obj->cq.out);
+	if (!ret && (obj_type & MLX4DV_OBJ_SRQ))
+		ret = mlx4dv_get_srq(obj->srq.in, obj->srq.out);
+
+	return ret;
+}
diff --git a/providers/mlx4/mlx4.h b/providers/mlx4/mlx4.h
index b4f6e86..8f0d6f2 100644
--- a/providers/mlx4/mlx4.h
+++ b/providers/mlx4/mlx4.h
@@ -42,6 +42,8 @@
 #include <util/udma_barrier.h>
 #include <infiniband/verbs.h>
 
+#include "mlx4dv.h"
+
 #define MLX4_PORTS_NUM 2
 
 #include <valgrind/memcheck.h>
@@ -88,33 +90,6 @@ enum mlx4_db_type {
 	MLX4_NUM_DB_TYPE
 };
 
-enum {
-	MLX4_OPCODE_NOP			= 0x00,
-	MLX4_OPCODE_SEND_INVAL		= 0x01,
-	MLX4_OPCODE_RDMA_WRITE		= 0x08,
-	MLX4_OPCODE_RDMA_WRITE_IMM	= 0x09,
-	MLX4_OPCODE_SEND		= 0x0a,
-	MLX4_OPCODE_SEND_IMM		= 0x0b,
-	MLX4_OPCODE_LSO			= 0x0e,
-	MLX4_OPCODE_RDMA_READ		= 0x10,
-	MLX4_OPCODE_ATOMIC_CS		= 0x11,
-	MLX4_OPCODE_ATOMIC_FA		= 0x12,
-	MLX4_OPCODE_MASKED_ATOMIC_CS	= 0x14,
-	MLX4_OPCODE_MASKED_ATOMIC_FA	= 0x15,
-	MLX4_OPCODE_BIND_MW		= 0x18,
-	MLX4_OPCODE_FMR			= 0x19,
-	MLX4_OPCODE_LOCAL_INVAL		= 0x1b,
-	MLX4_OPCODE_CONFIG_CMD		= 0x1f,
-
-	MLX4_RECV_OPCODE_RDMA_WRITE_IMM	= 0x00,
-	MLX4_RECV_OPCODE_SEND		= 0x01,
-	MLX4_RECV_OPCODE_SEND_IMM	= 0x02,
-	MLX4_RECV_OPCODE_SEND_INVAL	= 0x03,
-
-	MLX4_CQE_OPCODE_ERROR		= 0x1e,
-	MLX4_CQE_OPCODE_RESIZE		= 0x16,
-};
-
 struct mlx4_device {
 	struct verbs_device		verbs_dev;
 	int				page_size;
@@ -174,6 +149,7 @@ enum {
 	MLX4_CQ_FLAGS_RX_CSUM_VALID = 1 << 0,
 	MLX4_CQ_FLAGS_EXTENDED = 1 << 1,
 	MLX4_CQ_FLAGS_SINGLE_THREADED = 1 << 2,
+	MLX4_CQ_FLAGS_DV_OWNED = 1 << 3,
 };
 
 struct mlx4_cq {
@@ -265,36 +241,6 @@ enum {
 	MLX4_RX_CSUM_VALID		= (1 <<  16),
 };
 
-enum mlx4_cqe_status {
-	MLX4_CQE_STATUS_TCP_UDP_CSUM_OK	= (1 <<  2),
-	MLX4_CQE_STATUS_IPV4_PKT	= (1 << 22),
-	MLX4_CQE_STATUS_IP_HDR_CSUM_OK	= (1 << 28),
-	MLX4_CQE_STATUS_IPV4_CSUM_OK	= MLX4_CQE_STATUS_IPV4_PKT |
-					MLX4_CQE_STATUS_IP_HDR_CSUM_OK |
-					MLX4_CQE_STATUS_TCP_UDP_CSUM_OK
-};
-
-struct mlx4_cqe {
-	uint32_t	vlan_my_qpn;
-	uint32_t	immed_rss_invalid;
-	uint32_t	g_mlpath_rqpn;
-	union {
-		struct {
-			uint16_t	sl_vid;
-			uint16_t	rlid;
-		};
-		uint32_t ts_47_16;
-	};
-	uint32_t	status;
-	uint32_t	byte_cnt;
-	uint16_t	wqe_index;
-	uint16_t	checksum;
-	uint8_t		reserved3;
-	uint8_t		ts_15_8;
-	uint8_t		ts_7_0;
-	uint8_t		owner_sr_opcode;
-};
-
 static inline unsigned long align(unsigned long val, unsigned long align)
 {
 	return (val + align - 1) & ~(align - 1);
diff --git a/providers/mlx4/mlx4dv.h b/providers/mlx4/mlx4dv.h
new file mode 100644
index 0000000..86b4e13
--- /dev/null
+++ b/providers/mlx4/mlx4dv.h
@@ -0,0 +1,338 @@
+/*
+ * Copyright (c) 2017 Mellanox Technologies, Inc.  All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses.  You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ *     Redistribution and use in source and binary forms, with or
+ *     without modification, are permitted provided that the following
+ *     conditions are met:
+ *
+ *      - Redistributions of source code must retain the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer.
+ *
+ *      - Redistributions in binary form must reproduce the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer in the documentation and/or other materials
+ *        provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _MLX4DV_H_
+#define _MLX4DV_H_
+
+#include <linux/types.h>
+#include <endian.h>
+#include <infiniband/verbs.h>
+
+enum {
+	MLX4_OPCODE_NOP			= 0x00,
+	MLX4_OPCODE_SEND_INVAL		= 0x01,
+	MLX4_OPCODE_RDMA_WRITE		= 0x08,
+	MLX4_OPCODE_RDMA_WRITE_IMM	= 0x09,
+	MLX4_OPCODE_SEND		= 0x0a,
+	MLX4_OPCODE_SEND_IMM		= 0x0b,
+	MLX4_OPCODE_LSO			= 0x0e,
+	MLX4_OPCODE_RDMA_READ		= 0x10,
+	MLX4_OPCODE_ATOMIC_CS		= 0x11,
+	MLX4_OPCODE_ATOMIC_FA		= 0x12,
+	MLX4_OPCODE_MASKED_ATOMIC_CS	= 0x14,
+	MLX4_OPCODE_MASKED_ATOMIC_FA	= 0x15,
+	MLX4_OPCODE_BIND_MW		= 0x18,
+	MLX4_OPCODE_FMR			= 0x19,
+	MLX4_OPCODE_LOCAL_INVAL		= 0x1b,
+	MLX4_OPCODE_CONFIG_CMD		= 0x1f,
+
+	MLX4_RECV_OPCODE_RDMA_WRITE_IMM	= 0x00,
+	MLX4_RECV_OPCODE_SEND		= 0x01,
+	MLX4_RECV_OPCODE_SEND_IMM	= 0x02,
+	MLX4_RECV_OPCODE_SEND_INVAL	= 0x03,
+
+	MLX4_CQE_OPCODE_ERROR		= 0x1e,
+	MLX4_CQE_OPCODE_RESIZE		= 0x16,
+};
+
+enum {
+	MLX4_CQ_DOORBELL			= 0x20
+};
+
+#define MLX4_CQ_DB_REQ_NOT_SOL			(1 << 24)
+#define MLX4_CQ_DB_REQ_NOT			(2 << 24)
+
+enum {
+	MLX4_CQE_VLAN_PRESENT_MASK		= 1 << 29,
+	MLX4_CQE_QPN_MASK			= 0xffffff,
+};
+
+enum {
+	MLX4_CQE_OWNER_MASK			= 0x80,
+	MLX4_CQE_IS_SEND_MASK			= 0x40,
+	MLX4_CQE_OPCODE_MASK			= 0x1f
+};
+
+enum {
+	MLX4_CQE_SYNDROME_LOCAL_LENGTH_ERR		= 0x01,
+	MLX4_CQE_SYNDROME_LOCAL_QP_OP_ERR		= 0x02,
+	MLX4_CQE_SYNDROME_LOCAL_PROT_ERR		= 0x04,
+	MLX4_CQE_SYNDROME_WR_FLUSH_ERR			= 0x05,
+	MLX4_CQE_SYNDROME_MW_BIND_ERR			= 0x06,
+	MLX4_CQE_SYNDROME_BAD_RESP_ERR			= 0x10,
+	MLX4_CQE_SYNDROME_LOCAL_ACCESS_ERR		= 0x11,
+	MLX4_CQE_SYNDROME_REMOTE_INVAL_REQ_ERR		= 0x12,
+	MLX4_CQE_SYNDROME_REMOTE_ACCESS_ERR		= 0x13,
+	MLX4_CQE_SYNDROME_REMOTE_OP_ERR			= 0x14,
+	MLX4_CQE_SYNDROME_TRANSPORT_RETRY_EXC_ERR	= 0x15,
+	MLX4_CQE_SYNDROME_RNR_RETRY_EXC_ERR		= 0x16,
+	MLX4_CQE_SYNDROME_REMOTE_ABORTED_ERR		= 0x22,
+};
+
+struct mlx4_err_cqe {
+	uint32_t	vlan_my_qpn;
+	uint32_t	reserved1[5];
+	uint16_t	wqe_index;
+	uint8_t		vendor_err;
+	uint8_t		syndrome;
+	uint8_t		reserved2[3];
+	uint8_t		owner_sr_opcode;
+};
+
+enum mlx4_cqe_status {
+	MLX4_CQE_STATUS_TCP_UDP_CSUM_OK	= (1 <<  2),
+	MLX4_CQE_STATUS_IPV4_PKT	= (1 << 22),
+	MLX4_CQE_STATUS_IP_HDR_CSUM_OK	= (1 << 28),
+	MLX4_CQE_STATUS_IPV4_CSUM_OK	= MLX4_CQE_STATUS_IPV4_PKT |
+					MLX4_CQE_STATUS_IP_HDR_CSUM_OK |
+					MLX4_CQE_STATUS_TCP_UDP_CSUM_OK
+};
+
+struct mlx4_cqe {
+	uint32_t	vlan_my_qpn;
+	uint32_t	immed_rss_invalid;
+	uint32_t	g_mlpath_rqpn;
+	union {
+		struct {
+			uint16_t	sl_vid;
+			uint16_t	rlid;
+		};
+		uint32_t ts_47_16;
+	};
+	uint32_t	status;
+	uint32_t	byte_cnt;
+	uint16_t	wqe_index;
+	uint16_t	checksum;
+	uint8_t		reserved3;
+	uint8_t		ts_15_8;
+	uint8_t		ts_7_0;
+	uint8_t		owner_sr_opcode;
+};
+
+struct mlx4dv_qp {
+	uint32_t		*rdb;
+	uint32_t		*sdb;
+	uint32_t		doorbell_qpn;
+	struct {
+		uint32_t	wqe_cnt;
+		int		wqe_shift;
+		int		offset;
+	} sq;
+	struct {
+		uint32_t	wqe_cnt;
+		int		wqe_shift;
+		int		offset;
+	} rq;
+	struct {
+		void			*buf;
+		size_t			length;
+	} buf;
+	uint64_t		comp_mask;
+};
+
+struct mlx4dv_cq {
+	struct {
+		void			*buf;
+		size_t			length;
+	} buf;
+	uint32_t			cqe_cnt;
+	uint32_t			cqn;
+	uint32_t		       *set_ci_db;
+	uint32_t		       *arm_db;
+	int				arm_sn;
+	int				cqe_size;
+	uint64_t			comp_mask;
+};
+struct mlx4dv_srq {
+	struct {
+		void			*buf;
+		size_t			length;
+	} buf;
+	int				wqe_shift;
+	int				head;
+	int				tail;
+	uint32_t		       *db;
+	uint64_t			comp_mask;
+};
+
+struct mlx4dv_obj {
+	struct {
+		struct ibv_qp		*in;
+		struct mlx4dv_qp	*out;
+	} qp;
+	struct {
+		struct ibv_cq		*in;
+		struct mlx4dv_cq	*out;
+	} cq;
+	struct {
+		struct ibv_srq		*in;
+		struct mlx4dv_srq	*out;
+	} srq;
+};
+
+enum mlx4dv_obj_type {
+	MLX4DV_OBJ_QP	= 1 << 0,
+	MLX4DV_OBJ_CQ	= 1 << 1,
+	MLX4DV_OBJ_SRQ	= 1 << 2,
+};
+
+/*
+ * This function will initialize mlx4dv_xxx structs based on supplied type.
+ * The information for initialization is taken from ibv_xx structs supplied
+ * as part of input.
+ *
+ * Request information of CQ marks its owned by DV for all consumer index
+ * related actions.
+ *
+ * The initialization type can be combination of several types together.
+ *
+ * Return: 0 in case of success.
+ */
+int mlx4dv_init_obj(struct mlx4dv_obj *obj, uint64_t obj_type);
+
+/*
+ * WQE related part
+ */
+
+enum {
+	MLX4_SEND_DOORBELL	= 0x14,
+};
+
+enum {
+	MLX4_WQE_CTRL_SOLICIT		= 1 << 1,
+	MLX4_WQE_CTRL_CQ_UPDATE		= 3 << 2,
+	MLX4_WQE_CTRL_IP_HDR_CSUM	= 1 << 4,
+	MLX4_WQE_CTRL_TCP_UDP_CSUM	= 1 << 5,
+	MLX4_WQE_CTRL_FENCE		= 1 << 6,
+	MLX4_WQE_CTRL_STRONG_ORDER	= 1 << 7
+};
+
+enum {
+	MLX4_WQE_BIND_TYPE_2		= (1<<31),
+	MLX4_WQE_BIND_ZERO_BASED	= (1<<30),
+};
+
+enum {
+	MLX4_INLINE_SEG		= 1 << 31,
+	MLX4_INLINE_ALIGN	= 64,
+};
+
+enum {
+	MLX4_INVALID_LKEY	= 0x100,
+};
+
+enum {
+	MLX4_WQE_MW_REMOTE_READ   = 1 << 29,
+	MLX4_WQE_MW_REMOTE_WRITE  = 1 << 30,
+	MLX4_WQE_MW_ATOMIC        = 1 << 31
+};
+
+struct mlx4_wqe_local_inval_seg {
+	uint64_t		reserved1;
+	uint32_t		mem_key;
+	uint32_t		reserved2;
+	uint64_t		reserved3[2];
+};
+
+struct mlx4_wqe_bind_seg {
+	uint32_t		flags1;
+	uint32_t		flags2;
+	uint32_t		new_rkey;
+	uint32_t		lkey;
+	uint64_t		addr;
+	uint64_t		length;
+};
+
+struct mlx4_wqe_ctrl_seg {
+	uint32_t		owner_opcode;
+	union {
+		struct {
+			uint8_t			reserved[3];
+			uint8_t			fence_size;
+		};
+		uint32_t	bf_qpn;
+	};
+	/*
+	 * High 24 bits are SRC remote buffer; low 8 bits are flags:
+	 * [7]   SO (strong ordering)
+	 * [5]   TCP/UDP checksum
+	 * [4]   IP checksum
+	 * [3:2] C (generate completion queue entry)
+	 * [1]   SE (solicited event)
+	 * [0]   FL (force loopback)
+	 */
+	uint32_t		srcrb_flags;
+	/*
+	 * imm is immediate data for send/RDMA write w/ immediate;
+	 * also invalidation key for send with invalidate; input
+	 * modifier for WQEs on CCQs.
+	 */
+	uint32_t		imm;
+};
+
+struct mlx4_wqe_datagram_seg {
+	uint32_t		av[8];
+	uint32_t		dqpn;
+	uint32_t		qkey;
+	uint16_t		vlan;
+	uint8_t			mac[6];
+};
+
+struct mlx4_wqe_data_seg {
+	uint32_t		byte_count;
+	uint32_t		lkey;
+	uint64_t		addr;
+};
+
+struct mlx4_wqe_inline_seg {
+	uint32_t		byte_count;
+};
+
+struct mlx4_wqe_srq_next_seg {
+	uint16_t		reserved1;
+	uint16_t		next_wqe_index;
+	uint32_t		reserved2[3];
+};
+
+struct mlx4_wqe_raddr_seg {
+	uint64_t		raddr;
+	uint32_t		rkey;
+	uint32_t		reserved;
+};
+
+struct mlx4_wqe_atomic_seg {
+	uint64_t		swap_add;
+	uint64_t		compare;
+};
+
+#endif /* _MLX4DV_H_ */
+
diff --git a/providers/mlx4/qp.c b/providers/mlx4/qp.c
index e7f10b9..b7e3669 100644
--- a/providers/mlx4/qp.c
+++ b/providers/mlx4/qp.c
@@ -42,7 +42,6 @@
 #include <util/compiler.h>
 
 #include "mlx4.h"
-#include "wqe.h"
 
 static const uint32_t mlx4_ib_opcode[] = {
 	[IBV_WR_SEND]			= MLX4_OPCODE_SEND,
diff --git a/providers/mlx4/srq.c b/providers/mlx4/srq.c
index f30cc2e..9c566cd 100644
--- a/providers/mlx4/srq.c
+++ b/providers/mlx4/srq.c
@@ -37,7 +37,6 @@
 #include <string.h>
 
 #include "mlx4.h"
-#include "wqe.h"
 #include "mlx4-abi.h"
 
 static void *get_wqe(struct mlx4_srq *srq, int n)
diff --git a/providers/mlx4/verbs.c b/providers/mlx4/verbs.c
index 80efd9a..fce6f22 100644
--- a/providers/mlx4/verbs.c
+++ b/providers/mlx4/verbs.c
@@ -43,7 +43,6 @@
 
 #include "mlx4.h"
 #include "mlx4-abi.h"
-#include "wqe.h"
 
 int mlx4_query_device(struct ibv_context *context, struct ibv_device_attr *attr)
 {
diff --git a/providers/mlx4/wqe.h b/providers/mlx4/wqe.h
deleted file mode 100644
index 6f833d9..0000000
--- a/providers/mlx4/wqe.h
+++ /dev/null
@@ -1,149 +0,0 @@
-/*
- * Copyright (c) 2007 Cisco, Inc.  All rights reserved.
- *
- * This software is available to you under a choice of one of two
- * licenses.  You may choose to be licensed under the terms of the GNU
- * General Public License (GPL) Version 2, available from the file
- * COPYING in the main directory of this source tree, or the
- * OpenIB.org BSD license below:
- *
- *     Redistribution and use in source and binary forms, with or
- *     without modification, are permitted provided that the following
- *     conditions are met:
- *
- *      - Redistributions of source code must retain the above
- *        copyright notice, this list of conditions and the following
- *        disclaimer.
- *
- *      - Redistributions in binary form must reproduce the above
- *        copyright notice, this list of conditions and the following
- *        disclaimer in the documentation and/or other materials
- *        provided with the distribution.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
- * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
- * SOFTWARE.
- */
-
-#ifndef WQE_H
-#define WQE_H
-
-#include <stdint.h>
-
-enum {
-	MLX4_SEND_DOORBELL	= 0x14,
-};
-
-enum {
-	MLX4_WQE_CTRL_SOLICIT		= 1 << 1,
-	MLX4_WQE_CTRL_CQ_UPDATE		= 3 << 2,
-	MLX4_WQE_CTRL_IP_HDR_CSUM	= 1 << 4,
-	MLX4_WQE_CTRL_TCP_UDP_CSUM	= 1 << 5,
-	MLX4_WQE_CTRL_FENCE		= 1 << 6,
-	MLX4_WQE_CTRL_STRONG_ORDER	= 1 << 7
-};
-
-enum {
-	MLX4_WQE_BIND_TYPE_2		= (1<<31),
-	MLX4_WQE_BIND_ZERO_BASED	= (1<<30),
-};
-
-enum {
-	MLX4_INLINE_SEG		= 1 << 31,
-	MLX4_INLINE_ALIGN	= 64,
-};
-
-enum {
-	MLX4_INVALID_LKEY	= 0x100,
-};
-
-struct mlx4_wqe_ctrl_seg {
-	uint32_t		owner_opcode;
-	union {
-		struct {
-			uint8_t			reserved[3];
-			uint8_t			fence_size;
-		};
-		uint32_t	bf_qpn;
-	};
-	/*
-	 * High 24 bits are SRC remote buffer; low 8 bits are flags:
-	 * [7]   SO (strong ordering)
-	 * [5]   TCP/UDP checksum
-	 * [4]   IP checksum
-	 * [3:2] C (generate completion queue entry)
-	 * [1]   SE (solicited event)
-	 * [0]   FL (force loopback)
-	 */
-	uint32_t		srcrb_flags;
-	/*
-	 * imm is immediate data for send/RDMA write w/ immediate;
-	 * also invalidation key for send with invalidate; input
-	 * modifier for WQEs on CCQs.
-	 */
-	uint32_t		imm;
-};
-
-struct mlx4_wqe_datagram_seg {
-	uint32_t		av[8];
-	uint32_t		dqpn;
-	uint32_t		qkey;
-	uint16_t		vlan;
-	uint8_t			mac[6];
-};
-
-struct mlx4_wqe_data_seg {
-	uint32_t		byte_count;
-	uint32_t		lkey;
-	uint64_t		addr;
-};
-
-struct mlx4_wqe_inline_seg {
-	uint32_t		byte_count;
-};
-
-struct mlx4_wqe_srq_next_seg {
-	uint16_t		reserved1;
-	uint16_t		next_wqe_index;
-	uint32_t		reserved2[3];
-};
-
-struct mlx4_wqe_local_inval_seg {
-	uint64_t		reserved1;
-	uint32_t		mem_key;
-	uint32_t		reserved2;
-	uint64_t		reserved3[2];
-};
-
-enum {
-	MLX4_WQE_MW_REMOTE_READ   = 1 << 29,
-	MLX4_WQE_MW_REMOTE_WRITE  = 1 << 30,
-	MLX4_WQE_MW_ATOMIC        = 1 << 31
-};
-
-struct mlx4_wqe_raddr_seg {
-	uint64_t		raddr;
-	uint32_t		rkey;
-	uint32_t		reserved;
-};
-
-struct mlx4_wqe_atomic_seg {
-	uint64_t		swap_add;
-	uint64_t		compare;
-};
-
-struct mlx4_wqe_bind_seg {
-	uint32_t		flags1;
-	uint32_t		flags2;
-	uint32_t		new_rkey;
-	uint32_t		lkey;
-	uint64_t		addr;
-	uint64_t		length;
-};
-
-#endif /* WQE_H */
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 2/7] mlx4: Add CQE fields

[Yishai Hadas]

From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Introduce inline calls to get/set for most
commonly used CQE data path fields.

Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 providers/mlx4/mlx4dv.h | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/providers/mlx4/mlx4dv.h b/providers/mlx4/mlx4dv.h
index 86b4e13..e677fe1 100644
--- a/providers/mlx4/mlx4dv.h
+++ b/providers/mlx4/mlx4dv.h
@@ -37,6 +37,13 @@
 #include <endian.h>
 #include <infiniband/verbs.h>
 
+/* Always inline the functions */
+#ifdef __GNUC__
+#define MLX4DV_ALWAYS_INLINE inline __attribute__((always_inline))
+#else
+#define MLX4DV_ALWAYS_INLINE inline
+#endif
+
 enum {
 	MLX4_OPCODE_NOP			= 0x00,
 	MLX4_OPCODE_SEND_INVAL		= 0x01,
@@ -219,6 +226,25 @@ enum mlx4dv_obj_type {
  */
 int mlx4dv_init_obj(struct mlx4dv_obj *obj, uint64_t obj_type);
 
+static MLX4DV_ALWAYS_INLINE
+uint8_t mlx4dv_get_cqe_owner(struct mlx4_cqe *cqe)
+{
+	return cqe->owner_sr_opcode & MLX4_CQE_OWNER_MASK;
+}
+
+static MLX4DV_ALWAYS_INLINE
+void mlx4dv_set_cqe_owner(struct mlx4_cqe *cqe, uint8_t val)
+{
+	cqe->owner_sr_opcode = (val & MLX4_CQE_OWNER_MASK) |
+		(cqe->owner_sr_opcode & ~MLX4_CQE_OWNER_MASK);
+}
+
+static MLX4DV_ALWAYS_INLINE
+uint8_t mlx4dv_get_cqe_opcode(struct mlx4_cqe *cqe)
+{
+	return cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK;
+}
+
 /*
  * WQE related part
  */
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 3/7] mlx4: Convert datagram segment to use mlx4_av struct

[Yishai Hadas]

From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

The datagram segment struct contains array which
represents the address vector, convert the array
to mlx4_av.

mlx4dv will use this struct in downstream patches.

Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 providers/mlx4/mlx4.h   | 13 -------------
 providers/mlx4/mlx4dv.h | 15 ++++++++++++++-
 providers/mlx4/qp.c     |  2 +-
 3 files changed, 15 insertions(+), 15 deletions(-)

diff --git a/providers/mlx4/mlx4.h b/providers/mlx4/mlx4.h
index 8f0d6f2..bdb7f16 100644
--- a/providers/mlx4/mlx4.h
+++ b/providers/mlx4/mlx4.h
@@ -214,19 +214,6 @@ struct mlx4_qp {
 	uint32_t			qp_cap_cache;
 };
 
-struct mlx4_av {
-	uint32_t			port_pd;
-	uint8_t				reserved1;
-	uint8_t				g_slid;
-	uint16_t			dlid;
-	uint8_t				reserved2;
-	uint8_t				gid_index;
-	uint8_t				stat_rate;
-	uint8_t				hop_limit;
-	uint32_t			sl_tclass_flowlabel;
-	uint8_t				dgid[16];
-};
-
 struct mlx4_ah {
 	struct ibv_ah			ibv_ah;
 	struct mlx4_av			av;
diff --git a/providers/mlx4/mlx4dv.h b/providers/mlx4/mlx4dv.h
index e677fe1..7679ece 100644
--- a/providers/mlx4/mlx4dv.h
+++ b/providers/mlx4/mlx4dv.h
@@ -325,8 +325,21 @@ struct mlx4_wqe_ctrl_seg {
 	uint32_t		imm;
 };
 
+struct mlx4_av {
+	uint32_t			port_pd;
+	uint8_t				reserved1;
+	uint8_t				g_slid;
+	uint16_t			dlid;
+	uint8_t				reserved2;
+	uint8_t				gid_index;
+	uint8_t				stat_rate;
+	uint8_t				hop_limit;
+	uint32_t			sl_tclass_flowlabel;
+	uint8_t				dgid[16];
+};
+
 struct mlx4_wqe_datagram_seg {
-	uint32_t		av[8];
+	struct mlx4_av		av;
 	uint32_t		dqpn;
 	uint32_t		qkey;
 	uint16_t		vlan;
diff --git a/providers/mlx4/qp.c b/providers/mlx4/qp.c
index b7e3669..1be100b 100644
--- a/providers/mlx4/qp.c
+++ b/providers/mlx4/qp.c
@@ -175,7 +175,7 @@ static void set_atomic_seg(struct mlx4_wqe_atomic_seg *aseg, struct ibv_send_wr
 static void set_datagram_seg(struct mlx4_wqe_datagram_seg *dseg,
 			     struct ibv_send_wr *wr)
 {
-	memcpy(dseg->av, &to_mah(wr->wr.ud.ah)->av, sizeof (struct mlx4_av));
+	memcpy(&dseg->av, &to_mah(wr->wr.ud.ah)->av, sizeof (struct mlx4_av));
 	dseg->dqpn = htobe32(wr->wr.ud.remote_qpn);
 	dseg->qkey = htobe32(wr->wr.ud.remote_qkey);
 	dseg->vlan = htobe16(to_mah(wr->wr.ud.ah)->vlan);
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 4/7] mlx4: Add WQE segments implementation

[Yishai Hadas]

From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Add the following WQE segments implementation:
1. Control segment.
2. Datagram segment.
3. Data segment.

Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 providers/mlx4/mlx4dv.h | 88 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)

diff --git a/providers/mlx4/mlx4dv.h b/providers/mlx4/mlx4dv.h
index 7679ece..f15c56c 100644
--- a/providers/mlx4/mlx4dv.h
+++ b/providers/mlx4/mlx4dv.h
@@ -373,5 +373,93 @@ struct mlx4_wqe_atomic_seg {
 	uint64_t		compare;
 };
 
+/*
+ * Control segment - contains some control information for the current WQE.
+ *
+ * Output:
+ *	seg	  - control segment to be filled
+ * Input:
+ *	opcode		- Opcode of this WQE. Encodes the type of operation
+ *	                  to be executed on the QP.
+ *	wqe_cnt		- Number of queue entries.
+ *	ind		- WQEBB number of the first block of this WQE.
+ *	fence_size	- Fence bit and WQE size in octowords.
+ *	srcrb_flags	- High 24 bits are SRC remote buffer; low 8 bits are
+ *	                  flags which described in mlx4_wqe_ctrl_seg struct.
+ *	imm		- Immediate data/Invalidation key.
+ */
+static MLX4DV_ALWAYS_INLINE
+void mlx4dv_set_ctrl_seg(struct mlx4_wqe_ctrl_seg *seg, uint8_t opcode, uint32_t
+			 wqe_cnt, uint32_t ind, uint8_t fence_size,
+			 uint32_t srcrb_flags, uint32_t imm)
+{
+	seg->owner_opcode = htobe32(opcode) |
+		(ind & wqe_cnt ? htobe32(1 << 31) : 0);
+	seg->fence_size = fence_size;
+	seg->srcrb_flags = srcrb_flags;
+	/*
+	 * The caller should prepare "imm" in advance based on WR opcode.
+	 * For IBV_WR_SEND_WITH_IMM and IBV_WR_RDMA_WRITE_WITH_IMM,
+	 * the "imm" should be assigned as is.
+	 * For the IBV_WR_SEND_WITH_INV, it should be htobe32(imm).
+	 */
+	seg->imm = imm;
+}
+
+/*
+ * Datagram Segment - contains address information required in order
+ * to form a datagram message.
+ *
+ * Output:
+ *	seg - datagram segment to be filled.
+ * Input:
+ *	port_pd			- Port number and protection domain.
+ *	g_slid			- GRH and source LID for IB port only.
+ *	dlid			- Remote LID.
+ *	gid_index		- Index to port GID table.
+ *	state_rate		- Maximum static rate control.
+ *	hop_limit		- IPv6 hop limit.
+ *	sl_tclass_flowlabel	- Service Level, IPv6 TClass and flow table.
+ *	dgid			- Remote GID for IB port only.
+ *	dqpn			- Destination QP.
+ *	qkey			- QKey.
+ *	vlan			- VLAN for RAW ETHERNET QP only.
+ *	mac			- Destination MAC for RAW ETHERNET QP only.
+ */
+static MLX4DV_ALWAYS_INLINE
+void mlx4dv_set_dgram_seg(struct mlx4_wqe_datagram_seg *seg, uint32_t port_pd,
+			  uint8_t g_slid, uint16_t dlid, uint8_t gid_index,
+			  uint8_t stat_rate, uint8_t hop_limit, uint32_t
+			  sl_tclass_flowlabel, uint8_t *dgid, uint32_t dqpn,
+			  uint32_t qkey, uint16_t vlan, uint8_t *mac)
+{
+	seg->av.port_pd = htobe32(port_pd);
+	seg->av.g_slid = g_slid;
+	seg->av.dlid = htobe16(dlid);
+	seg->av.gid_index = gid_index;
+	seg->av.stat_rate = stat_rate;
+	seg->av.hop_limit = hop_limit;
+	seg->av.sl_tclass_flowlabel = htobe32(sl_tclass_flowlabel);
+	memcpy(seg->av.dgid, dgid, 16);
+	seg->dqpn = htobe32(dqpn);
+	seg->qkey = htobe32(qkey);
+	seg->vlan = htobe16(vlan);
+	memcpy(seg->mac, mac, 6);
+}
+
+/*
+ * Data Segments - contain pointers and a byte count for the scatter/gather list.
+ * They can optionally contain data, which will save a memory read access for
+ * gather Work Requests.
+ */
+static MLX4DV_ALWAYS_INLINE
+void mlx4dv_set_data_seg(struct mlx4_wqe_data_seg *seg,
+			 uint32_t length, uint32_t lkey,
+			 uintptr_t address)
+{
+	seg->byte_count = htobe32(length);
+	seg->lkey       = htobe32(lkey);
+	seg->addr       = htobe64(address);
+}
 #endif /* _MLX4DV_H_ */
 
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 5/7] mlx4: Convert to use predefined get CQE opcodes

[Yishai Hadas]

From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 providers/mlx4/cq.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/providers/mlx4/cq.c b/providers/mlx4/cq.c
index 068e0d2..50adebb 100644
--- a/providers/mlx4/cq.c
+++ b/providers/mlx4/cq.c
@@ -115,7 +115,7 @@ static enum ibv_wc_status mlx4_handle_error_cqe(struct mlx4_err_cqe *cqe)
 static inline void handle_good_req(struct ibv_wc *wc, struct mlx4_cqe *cqe)
 {
 	wc->wc_flags = 0;
-	switch (cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) {
+	switch (mlx4dv_get_cqe_opcode(cqe)) {
 	case MLX4_OPCODE_RDMA_WRITE_IMM:
 		wc->wc_flags |= IBV_WC_WITH_IMM;
 		SWITCH_FALLTHROUGH;
@@ -215,7 +215,7 @@ static inline int mlx4_parse_cqe(struct mlx4_cq *cq,
 		wc->qp_num = qpn;
 
 	is_send  = cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK;
-	is_error = (cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) ==
+	is_error = (mlx4dv_get_cqe_opcode(cqe)) ==
 		MLX4_CQE_OPCODE_ERROR;
 
 	if ((qpn & MLX4_XRC_QPN_BIT) && !is_send) {
@@ -278,7 +278,7 @@ static inline int mlx4_parse_cqe(struct mlx4_cq *cq,
 	} else {
 		wc->byte_len = be32toh(cqe->byte_cnt);
 
-		switch (cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) {
+		switch (mlx4dv_get_cqe_opcode(cqe)) {
 		case MLX4_RECV_OPCODE_RDMA_WRITE_IMM:
 			wc->opcode   = IBV_WC_RECV_RDMA_WITH_IMM;
 			wc->wc_flags = IBV_WC_WITH_IMM;
@@ -460,7 +460,7 @@ static enum ibv_wc_opcode mlx4_cq_read_wc_opcode(struct ibv_cq_ex *ibcq)
 	struct mlx4_cq *cq = to_mcq(ibv_cq_ex_to_cq(ibcq));
 
 	if (cq->cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK) {
-		switch (cq->cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) {
+		switch (mlx4dv_get_cqe_opcode(cq->cqe)) {
 		case MLX4_OPCODE_RDMA_WRITE_IMM:
 		case MLX4_OPCODE_RDMA_WRITE:
 			return IBV_WC_RDMA_WRITE;
@@ -480,7 +480,7 @@ static enum ibv_wc_opcode mlx4_cq_read_wc_opcode(struct ibv_cq_ex *ibcq)
 			return IBV_WC_BIND_MW;
 		}
 	} else {
-		switch (cq->cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) {
+		switch (mlx4dv_get_cqe_opcode(cq->cqe)) {
 		case MLX4_RECV_OPCODE_RDMA_WRITE_IMM:
 			return IBV_WC_RECV_RDMA_WITH_IMM;
 		case MLX4_RECV_OPCODE_SEND_INVAL:
@@ -507,7 +507,7 @@ static int mlx4_cq_read_wc_flags(struct ibv_cq_ex *ibcq)
 	int wc_flags = 0;
 
 	if (is_send) {
-		switch (cq->cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) {
+		switch (mlx4dv_get_cqe_opcode(cq->cqe)) {
 		case MLX4_OPCODE_RDMA_WRITE_IMM:
 		case MLX4_OPCODE_SEND_IMM:
 			wc_flags |= IBV_WC_WITH_IMM;
@@ -520,7 +520,7 @@ static int mlx4_cq_read_wc_flags(struct ibv_cq_ex *ibcq)
 				htobe32(MLX4_CQE_STATUS_IPV4_CSUM_OK)) <<
 				IBV_WC_IP_CSUM_OK_SHIFT;
 
-		switch (cq->cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) {
+		switch (mlx4dv_get_cqe_opcode(cq->cqe)) {
 		case MLX4_RECV_OPCODE_RDMA_WRITE_IMM:
 		case MLX4_RECV_OPCODE_SEND_IMM:
 			wc_flags |= IBV_WC_WITH_IMM;
@@ -554,7 +554,7 @@ static uint32_t mlx4_cq_read_wc_imm_data(struct ibv_cq_ex *ibcq)
 {
 	struct mlx4_cq *cq = to_mcq(ibv_cq_ex_to_cq(ibcq));
 
-	switch (cq->cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) {
+	switch (mlx4dv_get_cqe_opcode(cq->cqe)) {
 	case MLX4_RECV_OPCODE_SEND_INVAL:
 		return be32toh(cq->cqe->immed_rss_invalid);
 	default:
@@ -756,7 +756,7 @@ void mlx4_cq_resize_copy_cqes(struct mlx4_cq *cq, void *buf, int old_cqe)
 	cqe = get_cqe(cq, (i & old_cqe));
 	cqe += cqe_inc;
 
-	while ((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) != MLX4_CQE_OPCODE_RESIZE) {
+	while ((mlx4dv_get_cqe_opcode(cqe)) != MLX4_CQE_OPCODE_RESIZE) {
 		cqe->owner_sr_opcode = (cqe->owner_sr_opcode & ~MLX4_CQE_OWNER_MASK) |
 			(((i + 1) & (cq->ibv_cq.cqe + 1)) ? MLX4_CQE_OWNER_MASK : 0);
 		memcpy(buf + ((i + 1) & cq->ibv_cq.cqe) * cq->cqe_size,
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 6/7] mlx4: Add direct verbs man pages

[Yishai Hadas]

From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Document mlx4 direct verbs feature and exposed related functions.

Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 CMakeLists.txt                       |   1 +
 providers/mlx4/man/CMakeLists.txt    |   4 ++
 providers/mlx4/man/mlx4dv.7          |  39 ++++++++++++
 providers/mlx4/man/mlx4dv_init_obj.3 | 117 +++++++++++++++++++++++++++++++++++
 4 files changed, 161 insertions(+)
 create mode 100644 providers/mlx4/man/CMakeLists.txt
 create mode 100644 providers/mlx4/man/mlx4dv.7
 create mode 100644 providers/mlx4/man/mlx4dv_init_obj.3

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 634f327..5dcdb90 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -415,6 +415,7 @@ add_subdirectory(providers/cxgb4) # NO SPARSE
 add_subdirectory(providers/hns) # NO SPARSE
 add_subdirectory(providers/i40iw) # NO SPARSE
 add_subdirectory(providers/mlx4) # NO SPARSE
+add_subdirectory(providers/mlx4/man) # NO SPARSE
 add_subdirectory(providers/mlx5) # NO SPARSE
 add_subdirectory(providers/mlx5/man) # NO SPARSE
 add_subdirectory(providers/mthca) # NO SPARSE
diff --git a/providers/mlx4/man/CMakeLists.txt b/providers/mlx4/man/CMakeLists.txt
new file mode 100644
index 0000000..c44c7c3
--- /dev/null
+++ b/providers/mlx4/man/CMakeLists.txt
@@ -0,0 +1,4 @@
+rdma_man_pages(
+  mlx4dv_init_obj.3
+  mlx4dv.7
+)
diff --git a/providers/mlx4/man/mlx4dv.7 b/providers/mlx4/man/mlx4dv.7
new file mode 100644
index 0000000..92ced99
--- /dev/null
+++ b/providers/mlx4/man/mlx4dv.7
@@ -0,0 +1,39 @@
+.\" -*- nroff -*-
+.\" Licensed under the OpenIB.org (MIT) - See COPYING.md
+.\"
+.TH MLX4DV 7 2017-04-19 1.0.0
+.SH "NAME"
+mlx4dv \- Direct verbs for mlx4 devices
+.br
+This is low level access to mlx4 devices to perform data path operations,
+without general branching performed by \fBibv_post_send\fR(3).
+
+.SH "DESCRIPTION"
+The libibverbs API is an abstract one. It is agnostic to any underlying
+provider specific implementation. While this abstraction has the advantage
+of user applications portability it has a performance penalty. For some
+applications optimizing performance is more important than portability.
+
+The mlx4 direct verbs API is intended for such applications.
+It exposes mlx4 specific low level data path (send/receive/completion)
+operations, allowing the application to bypass the libibverbs data path API.
+
+This interface consists from one hardware specific header file
+with relevant inline functions and conversion logic from ibverbs structures
+to mlx4 specific structures.
+
+The direct include of mlx4dv.h together with linkage to mlx4 library will
+allow usage of this new interface.
+
+Once an application uses the direct flow the locking scheme is fully managed
+by itself. There is an expectation that no mixed flows in the data path for both
+direct/non-direct access will be by same application.
+
+.SH "NOTES"
+.SH "SEE ALSO"
+.BR ibv_post_send (3),
+.BR verbs (7)
+
+.SH "AUTHORS"
+.TP
+Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
diff --git a/providers/mlx4/man/mlx4dv_init_obj.3 b/providers/mlx4/man/mlx4dv_init_obj.3
new file mode 100644
index 0000000..0d0a4a5
--- /dev/null
+++ b/providers/mlx4/man/mlx4dv_init_obj.3
@@ -0,0 +1,117 @@
+.\" -*- nroff -*-
+.\" Licensed under the OpenIB.org (MIT) - See COPYING.md
+.\"
+.TH MLX4DV_INIT_OBJ 3 2017-02-02 1.0.0
+.SH "NAME"
+mlx4dv_init_obj \- Initialize mlx4 direct verbs object from ibv_xxx structures
+.SH "SYNOPSIS"
+.nf
+.B #include <infiniband/mlx4dv.h>
+.sp
+.BI "int mlx4dv_init_obj(struct mlx4dv_obj *obj, uint64_t obj_type);
+.fi
+.SH "DESCRIPTION"
+.B mlx4dv_init_obj()
+This function will initialize mlx4dv_xxx structs based on supplied type. The information
+for initialization is taken from ibv_xx structs supplied as part of input.
+
+Request information of CQ marks its owned by direct verbs for all consumer index
+related actions. The initialization type can be combination of several types together.
+.PP
+.nf
+struct mlx4dv_qp {
+.in +8
+uint32_t         *rdb;
+uint32_t         *sdb;
+struct {
+.in +8
+uint32_t        wqe_cnt;
+int             wqe_shift;
+int             offset;
+.in -8
+} sq;
+struct {
+.in +8
+uint32_t        wqe_cnt;
+int             wqe_shift;
+int             offset;
+.in -8
+} rq;
+struct {
+.in +8
+void            *buf;
+size_t          length;
+.in -8
+} buf;
+uint64_t        comp_mask;
+.in -8
+};
+
+struct mlx4dv_cq {
+.in +8
+struct {
+.in +8
+void            *buf;
+size_t          length;
+.in -8
+} buf;
+uint32_t        cqe_cnt;
+uint32_t        cqn;
+uint32_t        *set_ci_db;
+uint32_t        *arm_db;
+int             arm_sn;
+int             cqe_size;
+uint64_t        comp_mask;
+.in -8
+};
+
+struct mlx4dv_srq {
+.in +8
+struct {
+.in +8
+void            *buf;
+size_t          length;
+.in -8
+} buf;
+int             wqe_shift;
+int             head;
+int             tail;
+uint32_t        *db;
+uint64_t        comp_mask;
+.in -8
+};
+
+struct mlx4dv_obj {
+.in +8
+struct {
+.in +8
+struct ibv_qp      *in;
+struct mlx4dv_qp   *out;
+.in -8
+} qp;
+struct {
+.in +8
+struct ibv_cq      *in;
+struct mlx4dv_cq   *out;
+.in -8
+} cq;
+.in -8
+};
+
+enum mlx4dv_obj_type {
+.in +8
+MLX4DV_OBJ_QP   = 1 << 0,
+MLX4DV_OBJ_CQ   = 1 << 1,
+MLX4DV_OBJ_SRQ  = 1 << 2,
+.in -8
+};
+.fi
+.SH "RETURN VALUE"
+0 on success or the value of errno on failure (which indicates the failure reason).
+.SH "NOTES"
+ * Compatibility masks (comp_mask) are in/out fields.
+.SH "SEE ALSO"
+.BR mlx4dv (7)
+.SH "AUTHORS"
+.TP
+Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 7/7] mlx4: Export mlx4 direct verbs interface

[Yishai Hadas]

From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Install direct verbs header file into /usr/include/infiniband/
folder and allow possible users to explicitly include it
into their applications.
In addition, make the required packaging changes needed for both
RedHat and Debian users to link with libmlx4 directly.

Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 debian/ibverbs-providers.install | 1 +
 debian/ibverbs-providers.symbols | 4 ++++
 debian/libibverbs-dev.install    | 4 ++++
 providers/mlx4/CMakeLists.txt    | 7 ++++++-
 providers/mlx4/libmlx4.map       | 7 +++++++
 redhat/rdma-core.spec            | 3 +++
 6 files changed, 25 insertions(+), 1 deletion(-)
 create mode 100644 providers/mlx4/libmlx4.map

diff --git a/debian/ibverbs-providers.install b/debian/ibverbs-providers.install
index 1d234ba..1d693fe 100644
--- a/debian/ibverbs-providers.install
+++ b/debian/ibverbs-providers.install
@@ -3,6 +3,7 @@ etc/modprobe.d/truescale.conf
 usr/bin/rxe_cfg
 usr/lib/*/libibverbs/lib*-rdmav2.so
 usr/lib/*/libmlx5.so.*
+usr/lib/*/libmlx4.so.*
 usr/lib/truescale-serdes.cmds
 usr/share/doc/rdma-core/rxe.md usr/share/doc/ibverbs-providers/
 usr/share/man/man7/rxe.7
diff --git a/debian/ibverbs-providers.symbols b/debian/ibverbs-providers.symbols
index ea63da8..d64e3ef 100644
--- a/debian/ibverbs-providers.symbols
+++ b/debian/ibverbs-providers.symbols
@@ -4,3 +4,7 @@ libmlx5.so.1 ibverbs-providers #MINVER#
  mlx5dv_init_obj@MLX5_1.0 13
  mlx5dv_query_device@MLX5_1.0 13
  mlx5dv_create_cq@MLX5_1.1 14
+
+libmlx4.so.1 ibverbs-providers #MINVER#
+ MLX4_1.0@MLX4_1.0 15
+ mlx4dv_init_obj@MLX4_1.0 15
diff --git a/debian/libibverbs-dev.install b/debian/libibverbs-dev.install
index ed7a276..3773d00 100644
--- a/debian/libibverbs-dev.install
+++ b/debian/libibverbs-dev.install
@@ -1,14 +1,18 @@
 usr/include/infiniband/arch.h
 usr/include/infiniband/kern-abi.h
 usr/include/infiniband/mlx5dv.h
+usr/include/infiniband/mlx4dv.h
 usr/include/infiniband/opcode.h
 usr/include/infiniband/sa-kern-abi.h
 usr/include/infiniband/sa.h
 usr/include/infiniband/verbs.h
 usr/lib/*/libibverbs*.so
 usr/lib/*/libmlx5.so
+usr/lib/*/libmlx4.so
 usr/share/man/man3/ibv_*
 usr/share/man/man3/mbps_to_ibv_rate.3
 usr/share/man/man3/mlx5dv_*.3
+usr/share/man/man3/mlx4dv_*.3
 usr/share/man/man3/mult_to_ibv_rate.3
 usr/share/man/man7/mlx5dv.7
+usr/share/man/man7/mlx4dv.7
diff --git a/providers/mlx4/CMakeLists.txt b/providers/mlx4/CMakeLists.txt
index d64d817..f6739d2 100644
--- a/providers/mlx4/CMakeLists.txt
+++ b/providers/mlx4/CMakeLists.txt
@@ -1,4 +1,5 @@
-rdma_provider(mlx4
+rdma_shared_provider(mlx4 libmlx4.map
+  1 1.0.${PACKAGE_VERSION}
   buf.c
   cq.c
   dbrec.c
@@ -7,3 +8,7 @@ rdma_provider(mlx4
   srq.c
   verbs.c
 )
+
+publish_headers(infiniband
+  mlx4dv.h
+)
diff --git a/providers/mlx4/libmlx4.map b/providers/mlx4/libmlx4.map
new file mode 100644
index 0000000..0e53f83
--- /dev/null
+++ b/providers/mlx4/libmlx4.map
@@ -0,0 +1,7 @@
+/* Export symbols should be added below according to
+   Documentation/versioning.md document. */
+MLX4_1.0 {
+	global:
+		mlx4dv_init_obj;
+	local: *;
+};
diff --git a/redhat/rdma-core.spec b/redhat/rdma-core.spec
index 1f5cd14..5cebc25 100644
--- a/redhat/rdma-core.spec
+++ b/redhat/rdma-core.spec
@@ -355,7 +355,9 @@ rm -rf %{buildroot}/%{_initrddir}/
 %{_mandir}/man3/*_to_ibv_rate.*
 %{_mandir}/man7/rdma_cm.*
 %{_mandir}/man3/mlx5dv*
+%{_mandir}/man3/mlx4dv*
 %{_mandir}/man7/mlx5dv*
+%{_mandir}/man7/mlx4dv*
 
 %files -n libibverbs
 %dir %{_sysconfdir}/libibverbs.d
@@ -363,6 +365,7 @@ rm -rf %{buildroot}/%{_initrddir}/
 %{_libdir}/libibverbs*.so.*
 %{_libdir}/libibverbs/*.so
 %{_libdir}/libmlx5.so*
+%{_libdir}/libmlx4.so*
 %config(noreplace) %{_sysconfdir}/libibverbs.d/*.driver
 %doc %{_docdir}/%{name}-%{version}/libibverbs.md
 %doc %{_docdir}/%{name}-%{version}/rxe.md
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core 0/7] Add mlx4 direct verbs

[Yishai Hadas]

On 6/29/2017 2:47 PM, Yishai Hadas wrote:
> This patchset from Maor adds direct access to mlx4 devices, it follows same
> solution from both API and packaging perspectives as was already accepted for
> mlx5.
>
> The libibverbs API is an abstract API. It is agnostic to any underlying
> provider specific implementation. While this abstraction has the advantage of
> user applications portability it has a performance penalty. For some
> applications optimizing performance is more important than portability.
>
> The mlx4 direct verbs API introduced in this patchset is intended for such
> applications. It exposes mlx4 specific low level data path
> (send/receive/completion) operations, allowing the application to bypass the
> libibverbs data path API.
>
> The proposed interface consists from one hardware specific header with relevant
> inline functions and conversion logic from ibverbs structures to mlx4 related
> structures.
>
> New header is going to be installed in below path location:
> /usr/inlcude/infiniband/mlx4dv.h
>
> The direct include of mlx4dv.h together with linkage to mlx4 library will
> allow usage of this new interface.
>
> Pull request was sent:
> https://github.com/linux-rdma/rdma-core/pull/155
>
> Yishai
>

Applied, thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html