[PATCH rdma-core 0/7] Add mlx5 direct verbs

[PATCH rdma-core 0/7] Add mlx5 direct verbs

[Yishai Hadas]

This patchset from Leon adds direct access to mlx5 devices.

The libibverbs API is an abstract API. It is agnostic to any underlying
provider specific implementation. While this abstraction has the advantage of
user applications portability it has a performance penalty. For some
applications optimizing performance is more important than portability.

The mlx5 direct verbs API introduced in this patchset is intended for such
applications. It exposes mlx5 specific low level data path
(send/receive/completion) operations, allowing the application to bypass the
libibverbs data path API.

The proposed interface consists from small number of hardware specific headers
with relevant inline functions and conversion logic from ibverbs structures to
mlx5 related structures.

New headers are going to be installed to the /usr/include/infiniband/ folder in
such way that it will allow future extension and coexistence for other vendors.

* /usr/inlcude/infiniband/mlx5/dv.h               - shared code
* /usr/inlcude/infiniband/mlx5/dv-cqe.h           - CQE related functions
                                                    (general implementation)
* /usr/inlcude/infiniband/mlx5/dv-wqe.h           - WQE related functions
                                                    (general and SSE3 implementations)

The direct include of dv.h together with linkage to mlx5 library will
allow usage of this new interface.

Note: all Mellanox NIC devices starting from Connect-IB (Connect-IB,
ConnectX-4, ConnectX-4Lx, ConnectX-5 and more to come in the future)
implement the mlx5 API, thus using the mlx5 direct verbs does not limit
the applications to a single NIC HW device thus keeping some level of
portability.

Pull request was sent:
https://github.com/linux-rdma/rdma-core/pull/63

Yishai

Leon Romanovsky (7):
  mlx5: Add mlx5 direct verbs files
  mlx5: Add CQE fields
  mlx5: Add WQE segments implementation
  mlx5: Use macro for CQE version 1
  mlx5: Convert to use predefined get CQE opcodes
  mlx5: Unify exported and internal CQE format
  mlx5: Export mlx5 direct verbs interface

 buildlib/provider.map                 |  13 +-
 providers/mlx5/CMakeLists.txt         |   2 +
 providers/mlx5/cq.c                   | 120 +++----------
 providers/mlx5/include/CMakeLists.txt |   5 +
 providers/mlx5/include/dv-cqe.h       | 156 +++++++++++++++++
 providers/mlx5/include/dv-wqe.h       | 315 ++++++++++++++++++++++++++++++++++
 providers/mlx5/include/dv.h           | 181 +++++++++++++++++++
 providers/mlx5/mlx5.c                 | 111 +++++++++++-
 providers/mlx5/mlx5.h                 |  62 +------
 providers/mlx5/qp.c                   |   2 +-
 providers/mlx5/wqe.h                  |  97 +----------
 11 files changed, 806 insertions(+), 258 deletions(-)
 create mode 100644 providers/mlx5/include/CMakeLists.txt
 create mode 100644 providers/mlx5/include/dv-cqe.h
 create mode 100644 providers/mlx5/include/dv-wqe.h
 create mode 100644 providers/mlx5/include/dv.h

-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 1/7] mlx5: Add mlx5 direct verbs files

[Yishai Hadas]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Direct verbs feature provides fast data path execution by
implementing verbs directly over hardware specification.

It consists from the two parts:
 * Exported functions to translate internal mlx5 structures to be
   accessible by user applications.
 * Inline functions, enums and defines to simplify programmer's life.

This patch adds following files:
  * mlx5/dv.h        	- shared code
  * mlx5/dv-cqe.h    	- CQE related functions (general implementation)
  * mlx5/dv-wqe.h    	- WQE related functions (general implementation)

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Acked-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 providers/mlx5/CMakeLists.txt         |   2 +
 providers/mlx5/cq.c                   |  84 +---------------
 providers/mlx5/include/CMakeLists.txt |   1 +
 providers/mlx5/include/dv-cqe.h       | 125 +++++++++++++++++++++++
 providers/mlx5/include/dv-wqe.h       | 140 ++++++++++++++++++++++++++
 providers/mlx5/include/dv.h           | 181 ++++++++++++++++++++++++++++++++++
 providers/mlx5/mlx5.c                 | 109 ++++++++++++++++++++
 providers/mlx5/mlx5.h                 |  62 +-----------
 providers/mlx5/qp.c                   |   2 +-
 providers/mlx5/wqe.h                  |  97 +-----------------
 10 files changed, 568 insertions(+), 235 deletions(-)
 create mode 100644 providers/mlx5/include/CMakeLists.txt
 create mode 100644 providers/mlx5/include/dv-cqe.h
 create mode 100644 providers/mlx5/include/dv-wqe.h
 create mode 100644 providers/mlx5/include/dv.h

diff --git a/providers/mlx5/CMakeLists.txt b/providers/mlx5/CMakeLists.txt
index 44df83b..252243c 100644
--- a/providers/mlx5/CMakeLists.txt
+++ b/providers/mlx5/CMakeLists.txt
@@ -19,3 +19,5 @@ rdma_provider(mlx5
   srq.c
   verbs.c
 )
+
+add_subdirectory(include)
diff --git a/providers/mlx5/cq.c b/providers/mlx5/cq.c
index 7ad27a9..e5b50a3 100644
--- a/providers/mlx5/cq.c
+++ b/providers/mlx5/cq.c
@@ -47,97 +47,17 @@
 #include "doorbell.h"
 
 enum {
-	MLX5_CQ_DOORBELL			= 0x20
-};
-
-enum {
 	CQ_OK					=  0,
 	CQ_EMPTY				= -1,
 	CQ_POLL_ERR				= -2
 };
 
-#define MLX5_CQ_DB_REQ_NOT_SOL			(1 << 24)
-#define MLX5_CQ_DB_REQ_NOT			(0 << 24)
-
-enum {
-	MLX5_CQE_SYNDROME_LOCAL_LENGTH_ERR		= 0x01,
-	MLX5_CQE_SYNDROME_LOCAL_QP_OP_ERR		= 0x02,
-	MLX5_CQE_SYNDROME_LOCAL_PROT_ERR		= 0x04,
-	MLX5_CQE_SYNDROME_WR_FLUSH_ERR			= 0x05,
-	MLX5_CQE_SYNDROME_MW_BIND_ERR			= 0x06,
-	MLX5_CQE_SYNDROME_BAD_RESP_ERR			= 0x10,
-	MLX5_CQE_SYNDROME_LOCAL_ACCESS_ERR		= 0x11,
-	MLX5_CQE_SYNDROME_REMOTE_INVAL_REQ_ERR		= 0x12,
-	MLX5_CQE_SYNDROME_REMOTE_ACCESS_ERR		= 0x13,
-	MLX5_CQE_SYNDROME_REMOTE_OP_ERR			= 0x14,
-	MLX5_CQE_SYNDROME_TRANSPORT_RETRY_EXC_ERR	= 0x15,
-	MLX5_CQE_SYNDROME_RNR_RETRY_EXC_ERR		= 0x16,
-	MLX5_CQE_SYNDROME_REMOTE_ABORTED_ERR		= 0x22,
-};
-
-enum {
-	MLX5_CQE_OWNER_MASK	= 1,
-	MLX5_CQE_REQ		= 0,
-	MLX5_CQE_RESP_WR_IMM	= 1,
-	MLX5_CQE_RESP_SEND	= 2,
-	MLX5_CQE_RESP_SEND_IMM	= 3,
-	MLX5_CQE_RESP_SEND_INV	= 4,
-	MLX5_CQE_RESIZE_CQ	= 5,
-	MLX5_CQE_REQ_ERR	= 13,
-	MLX5_CQE_RESP_ERR	= 14,
-	MLX5_CQE_INVALID	= 15,
-};
-
 enum {
 	MLX5_CQ_MODIFY_RESEIZE = 0,
 	MLX5_CQ_MODIFY_MODER = 1,
 	MLX5_CQ_MODIFY_MAPPING = 2,
 };
 
-enum {
-	MLX5_CQE_L2_OK = 1 << 0,
-	MLX5_CQE_L3_OK = 1 << 1,
-	MLX5_CQE_L4_OK = 1 << 2,
-};
-
-enum {
-	MLX5_CQE_L3_HDR_TYPE_NONE = 0x0,
-	MLX5_CQE_L3_HDR_TYPE_IPV6 = 0x1,
-	MLX5_CQE_L3_HDR_TYPE_IPV4 = 0x2,
-};
-
-struct mlx5_err_cqe {
-	uint8_t		rsvd0[32];
-	uint32_t	srqn;
-	uint8_t		rsvd1[18];
-	uint8_t		vendor_err_synd;
-	uint8_t		syndrome;
-	uint32_t	s_wqe_opcode_qpn;
-	uint16_t	wqe_counter;
-	uint8_t		signature;
-	uint8_t		op_own;
-};
-
-struct mlx5_cqe64 {
-	uint8_t		rsvd0[17];
-	uint8_t		ml_path;
-	uint8_t		rsvd20[4];
-	uint16_t	slid;
-	uint32_t	flags_rqpn;
-	uint8_t		hds_ip_ext;
-	uint8_t		l4_hdr_type_etc;
-	uint16_t	vlan_info;
-	uint32_t	srqn_uidx;
-	uint32_t	imm_inval_pkey;
-	uint8_t		rsvd40[4];
-	uint32_t	byte_cnt;
-	__be64		timestamp;
-	uint32_t	sop_drop_qpn;
-	uint16_t	wqe_counter;
-	uint8_t		signature;
-	uint8_t		op_own;
-};
-
 int mlx5_stall_num_loop = 60;
 int mlx5_stall_cq_poll_min = 60;
 int mlx5_stall_cq_poll_max = 100000;
@@ -255,7 +175,7 @@ static inline int handle_responder_lazy(struct mlx5_cq *cq, struct mlx5_cqe64 *c
 		else if (cqe->op_own & MLX5_INLINE_SCATTER_64)
 			err = mlx5_copy_to_recv_wqe(qp, wqe_ctr, cqe - 1,
 						    ntohl(cqe->byte_cnt));
-	}
+}
 
 	return err;
 }
@@ -1432,7 +1352,7 @@ void __mlx5_cq_clean(struct mlx5_cq *cq, uint32_t rsn, struct mlx5_srq *srq)
 	uint8_t owner_bit;
 	int cqe_version;
 
-	if (!cq)
+	if (!cq || cq->flags & MLX5_CQ_FLAGS_DV_OWNED)
 		return;
 
 	/*
diff --git a/providers/mlx5/include/CMakeLists.txt b/providers/mlx5/include/CMakeLists.txt
new file mode 100644
index 0000000..11a2839
--- /dev/null
+++ b/providers/mlx5/include/CMakeLists.txt
@@ -0,0 +1 @@
+# Empty CMakelists.txt to avoid compilation error
diff --git a/providers/mlx5/include/dv-cqe.h b/providers/mlx5/include/dv-cqe.h
new file mode 100644
index 0000000..1ad33ad
--- /dev/null
+++ b/providers/mlx5/include/dv-cqe.h
@@ -0,0 +1,125 @@
+/*
+ * Copyright (c) 2017 Mellanox Technologies, Inc.  All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses.  You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ *     Redistribution and use in source and binary forms, with or
+ *     without modification, are permitted provided that the following
+ *     conditions are met:
+ *
+ *      - Redistributions of source code must retain the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer.
+ *
+ *      - Redistributions in binary form must reproduce the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer in the documentation and/or other materials
+ *        provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _MLX5DV_CQE_H_
+#define _MLX5DV_CQE_H_
+
+/* For __be64 type */
+#include <linux/types.h>
+
+enum {
+	MLX5_INLINE_SCATTER_32	= 0x4,
+	MLX5_INLINE_SCATTER_64	= 0x8,
+};
+
+enum {
+	MLX5_CQE_SYNDROME_LOCAL_LENGTH_ERR		= 0x01,
+	MLX5_CQE_SYNDROME_LOCAL_QP_OP_ERR		= 0x02,
+	MLX5_CQE_SYNDROME_LOCAL_PROT_ERR		= 0x04,
+	MLX5_CQE_SYNDROME_WR_FLUSH_ERR			= 0x05,
+	MLX5_CQE_SYNDROME_MW_BIND_ERR			= 0x06,
+	MLX5_CQE_SYNDROME_BAD_RESP_ERR			= 0x10,
+	MLX5_CQE_SYNDROME_LOCAL_ACCESS_ERR		= 0x11,
+	MLX5_CQE_SYNDROME_REMOTE_INVAL_REQ_ERR		= 0x12,
+	MLX5_CQE_SYNDROME_REMOTE_ACCESS_ERR		= 0x13,
+	MLX5_CQE_SYNDROME_REMOTE_OP_ERR			= 0x14,
+	MLX5_CQE_SYNDROME_TRANSPORT_RETRY_EXC_ERR	= 0x15,
+	MLX5_CQE_SYNDROME_RNR_RETRY_EXC_ERR		= 0x16,
+	MLX5_CQE_SYNDROME_REMOTE_ABORTED_ERR		= 0x22,
+};
+
+enum {
+	MLX5_CQE_L2_OK = 1 << 0,
+	MLX5_CQE_L3_OK = 1 << 1,
+	MLX5_CQE_L4_OK = 1 << 2,
+};
+
+enum {
+	MLX5_CQE_L3_HDR_TYPE_NONE = 0x0,
+	MLX5_CQE_L3_HDR_TYPE_IPV6 = 0x1,
+	MLX5_CQE_L3_HDR_TYPE_IPV4 = 0x2,
+};
+
+enum {
+	MLX5_CQE_OWNER_MASK	= 1,
+	MLX5_CQE_REQ		= 0,
+	MLX5_CQE_RESP_WR_IMM	= 1,
+	MLX5_CQE_RESP_SEND	= 2,
+	MLX5_CQE_RESP_SEND_IMM	= 3,
+	MLX5_CQE_RESP_SEND_INV	= 4,
+	MLX5_CQE_RESIZE_CQ	= 5,
+	MLX5_CQE_REQ_ERR	= 13,
+	MLX5_CQE_RESP_ERR	= 14,
+	MLX5_CQE_INVALID	= 15,
+};
+
+enum {
+	MLX5_CQ_DOORBELL			= 0x20
+};
+
+enum {
+	MLX5_CQ_DB_REQ_NOT_SOL	= 1 << 24,
+	MLX5_CQ_DB_REQ_NOT	= 0 << 24,
+};
+
+struct mlx5_err_cqe {
+	uint8_t		rsvd0[32];
+	uint32_t	srqn;
+	uint8_t		rsvd1[18];
+	uint8_t		vendor_err_synd;
+	uint8_t		syndrome;
+	uint32_t	s_wqe_opcode_qpn;
+	uint16_t	wqe_counter;
+	uint8_t		signature;
+	uint8_t		op_own;
+};
+
+struct mlx5_cqe64 {
+	uint8_t		rsvd0[17];
+	uint8_t		ml_path;
+	uint8_t		rsvd20[4];
+	uint16_t	slid;
+	uint32_t	flags_rqpn;
+	uint8_t		hds_ip_ext;
+	uint8_t		l4_hdr_type_etc;
+	uint16_t	vlan_info;
+	uint32_t	srqn_uidx;
+	uint32_t	imm_inval_pkey;
+	uint8_t		rsvd40[4];
+	uint32_t	byte_cnt;
+	__be64		timestamp;
+	uint32_t	sop_drop_qpn;
+	uint16_t	wqe_counter;
+	uint8_t		signature;
+	uint8_t		op_own;
+};
+#endif /* _MLX5DV_CQE_H_ */
diff --git a/providers/mlx5/include/dv-wqe.h b/providers/mlx5/include/dv-wqe.h
new file mode 100644
index 0000000..ce9bd86
--- /dev/null
+++ b/providers/mlx5/include/dv-wqe.h
@@ -0,0 +1,140 @@
+/*
+ * Copyright (c) 2017 Mellanox Technologies, Inc.  All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses.  You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ *     Redistribution and use in source and binary forms, with or
+ *     without modification, are permitted provided that the following
+ *     conditions are met:
+ *
+ *      - Redistributions of source code must retain the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer.
+ *
+ *      - Redistributions in binary form must reproduce the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer in the documentation and/or other materials
+ *        provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _MLX5DV_WQE_H_
+#define _MLX5DV_WQE_H_
+
+#include <arpa/inet.h>
+
+enum {
+	MLX5_INVALID_LKEY	= 0x100,
+};
+
+enum {
+	MLX5_EXTENDED_UD_AV	= 0x80000000,
+};
+
+enum {
+	MLX5_WQE_CTRL_CQ_UPDATE	= 2 << 2,
+	MLX5_WQE_CTRL_SOLICITED	= 1 << 1,
+	MLX5_WQE_CTRL_FENCE	= 4 << 5,
+	MLX5_WQE_CTRL_INITIATOR_SMALL_FENCE = 1 << 5,
+};
+
+enum {
+	MLX5_SEND_WQE_BB	= 64,
+	MLX5_SEND_WQE_SHIFT	= 6,
+};
+
+enum {
+	MLX5_INLINE_SEG	= 0x80000000,
+};
+
+enum {
+	MLX5_ETH_WQE_L3_CSUM = (1 << 6),
+	MLX5_ETH_WQE_L4_CSUM = (1 << 7),
+};
+
+struct mlx5_wqe_srq_next_seg {
+	uint8_t			rsvd0[2];
+	uint16_t		next_wqe_index;
+	uint8_t			signature;
+	uint8_t			rsvd1[11];
+};
+
+struct mlx5_wqe_data_seg {
+	uint32_t		byte_count;
+	uint32_t		lkey;
+	uint64_t		addr;
+};
+
+struct mlx5_wqe_ctrl_seg {
+	uint32_t	opmod_idx_opcode;
+	uint32_t	qpn_ds;
+	uint8_t		signature;
+	uint8_t		rsvd[2];
+	uint8_t		fm_ce_se;
+	uint32_t	imm;
+};
+
+struct mlx5_wqe_av {
+	union {
+		struct {
+			uint32_t	qkey;
+			uint32_t	reserved;
+		} qkey;
+		uint64_t	dc_key;
+	} key;
+	uint32_t	dqp_dct;
+	uint8_t		stat_rate_sl;
+	uint8_t		fl_mlid;
+	uint16_t	rlid;
+	uint8_t		reserved0[4];
+	uint8_t		rmac[6];
+	uint8_t		tclass;
+	uint8_t		hop_limit;
+	uint32_t	grh_gid_fl;
+	uint8_t		rgid[16];
+};
+
+struct mlx5_wqe_datagram_seg {
+	struct mlx5_wqe_av	av;
+};
+
+struct mlx5_wqe_raddr_seg {
+	uint64_t	raddr;
+	uint32_t	rkey;
+	uint32_t	reserved;
+};
+
+struct mlx5_wqe_atomic_seg {
+	uint64_t	swap_add;
+	uint64_t	compare;
+};
+
+struct mlx5_wqe_inl_data_seg {
+	uint32_t	byte_count;
+};
+
+struct mlx5_wqe_eth_seg {
+	uint32_t	rsvd0;
+	uint8_t		cs_flags;
+	uint8_t		rsvd1;
+	uint16_t	mss;
+	uint32_t	rsvd2;
+	uint16_t	inline_hdr_sz;
+	uint8_t		inline_hdr_start[2];
+	uint8_t		inline_hdr[16];
+};
+
+
+#endif /* _MLX5DV_WQE_H_ */
diff --git a/providers/mlx5/include/dv.h b/providers/mlx5/include/dv.h
new file mode 100644
index 0000000..ce4af76
--- /dev/null
+++ b/providers/mlx5/include/dv.h
@@ -0,0 +1,181 @@
+/*
+ * Copyright (c) 2017 Mellanox Technologies, Inc.  All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses.  You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ *     Redistribution and use in source and binary forms, with or
+ *     without modification, are permitted provided that the following
+ *     conditions are met:
+ *
+ *      - Redistributions of source code must retain the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer.
+ *
+ *      - Redistributions in binary form must reproduce the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer in the documentation and/or other materials
+ *        provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _MLX5DV_H_
+#define _MLX5DV_H_
+
+/* Always inline the function */
+#ifdef __GNUC__
+#define MLX5DV_ALWAYS_INLINE inline __attribute__((always_inline))
+#else
+#define MLX5DV_ALWAYS_INLINE inline
+#endif
+
+#include "dv-cqe.h"
+#include "dv-wqe.h"
+
+enum {
+	MLX5_RCV_DBR	= 0,
+	MLX5_SND_DBR	= 1,
+};
+
+/*
+ * Direct verbs device-specific attributes
+ */
+struct mlx5dv_context {
+	uint8_t		version;
+	uint64_t	flags;
+	uint64_t	comp_mask;
+};
+
+enum mlx5dv_context_flags {
+	/*
+	 * This flag indicates if CQE version 0 or 1 is needed.
+	 */
+	MLX5DV_CONTEXT_FLAGS_CQE_V1 = (1 << 0),
+};
+
+/*
+ * Most device capabilities are exported by ibv_query_device(...),
+ * but there is HW device-specific information which is important
+ * for data-path, but isn't provided.
+ *
+ * Return 0 on success.
+ */
+int mlx5dv_query_device(struct ibv_context *ctx_in,
+			struct mlx5dv_context *attrs_out);
+
+struct mlx5dv_qp {
+	uint32_t		*dbrec;
+	struct {
+		void		*buf;
+		uint32_t	wqe_cnt;
+		uint32_t	stride;
+	} sq;
+	struct {
+		void		*buf;
+		uint32_t	wqe_cnt;
+		uint32_t	stride;
+	} rq;
+	struct {
+		void		*reg;
+		uint32_t	size;
+	} bf;
+	uint64_t		comp_mask;
+};
+
+struct mlx5dv_cq {
+	void			*buf;
+	uint32_t		*dbrec;
+	uint32_t		cqe_cnt;
+	uint32_t		cqe_size;
+	void			*uar;
+	uint32_t		cqn;
+	uint64_t		comp_mask;
+};
+
+struct mlx5dv_srq {
+	void			*buf;
+	uint32_t		*dbrec;
+	uint32_t		stride;
+	uint32_t		head;
+	uint32_t		tail;
+	uint64_t		comp_mask;
+};
+
+struct mlx5dv_rwq {
+	void		*buf;
+	uint32_t	*dbrec;
+	uint32_t	wqe_cnt;
+	uint32_t	stride;
+	uint64_t	comp_mask;
+};
+
+struct mlx5dv_obj {
+	struct {
+		struct ibv_qp		*in;
+		struct mlx5dv_qp	*out;
+	} qp;
+	struct {
+		struct ibv_cq		*in;
+		struct mlx5dv_cq	*out;
+	} cq;
+	struct {
+		struct ibv_srq		*in;
+		struct mlx5dv_srq	*out;
+	} srq;
+	struct {
+		struct ibv_wq		*in;
+		struct mlx5dv_rwq	*out;
+	} rwq;
+};
+
+enum mlx5dv_obj_type {
+	MLX5DV_OBJ_QP	= 1 << 0,
+	MLX5DV_OBJ_CQ	= 1 << 1,
+	MLX5DV_OBJ_SRQ	= 1 << 2,
+	MLX5DV_OBJ_RWQ	= 1 << 3,
+};
+
+/*
+ * This function will initialize mlx5dv_xxx structs based on supplied type.
+ * The information for initialization is taken from ibv_xx structs supplied
+ * as part of input.
+ *
+ * Request information of CQ marks its owned by DV for all consumer index
+ * related actions.
+ *
+ * The initialization type can be combination of several types together.
+ *
+ * Return: 0 in case of success.
+ */
+int mlx5dv_init_obj(struct mlx5dv_obj *obj, uint64_t obj_type);
+
+enum {
+	MLX5_OPCODE_NOP			= 0x00,
+	MLX5_OPCODE_SEND_INVAL		= 0x01,
+	MLX5_OPCODE_RDMA_WRITE		= 0x08,
+	MLX5_OPCODE_RDMA_WRITE_IMM	= 0x09,
+	MLX5_OPCODE_SEND		= 0x0a,
+	MLX5_OPCODE_SEND_IMM		= 0x0b,
+	MLX5_OPCODE_TSO			= 0x0e,
+	MLX5_OPCODE_RDMA_READ		= 0x10,
+	MLX5_OPCODE_ATOMIC_CS		= 0x11,
+	MLX5_OPCODE_ATOMIC_FA		= 0x12,
+	MLX5_OPCODE_ATOMIC_MASKED_CS	= 0x14,
+	MLX5_OPCODE_ATOMIC_MASKED_FA	= 0x15,
+	MLX5_OPCODE_FMR			= 0x19,
+	MLX5_OPCODE_LOCAL_INVAL		= 0x1b,
+	MLX5_OPCODE_CONFIG_CMD		= 0x1f,
+	MLX5_OPCODE_UMR			= 0x25,
+};
+#endif /* _MLX5DV_H_ */
diff --git a/providers/mlx5/mlx5.c b/providers/mlx5/mlx5.c
index 747e242..1cf5a5f 100644
--- a/providers/mlx5/mlx5.c
+++ b/providers/mlx5/mlx5.c
@@ -578,6 +578,115 @@ static int mlx5_map_internal_clock(struct mlx5_device *mdev,
 	return 0;
 }
 
+int mlx5dv_query_device(struct ibv_context *ctx_in,
+			 struct mlx5dv_context *attrs_out)
+{
+	attrs_out->comp_mask = 0;
+	attrs_out->version   = 0;
+	attrs_out->flags     = 0;
+
+	if (to_mctx(ctx_in)->cqe_version == MLX5_CQE_VERSION_V1)
+		attrs_out->flags |= MLX5DV_CONTEXT_FLAGS_CQE_V1;
+
+	return 0;
+}
+
+static int mlx5dv_get_qp(struct ibv_qp *qp_in,
+			 struct mlx5dv_qp *qp_out)
+{
+	struct mlx5_qp *mqp = to_mqp(qp_in);
+
+	qp_out->comp_mask = 0;
+	qp_out->dbrec     = mqp->db;
+
+	if (mqp->sq_buf_size)
+		/* IBV_QPT_RAW_PACKET */
+		qp_out->sq.buf = (void *)((uintptr_t)mqp->sq_buf.buf);
+	else
+		qp_out->sq.buf = (void *)((uintptr_t)mqp->buf.buf + mqp->sq.offset);
+	qp_out->sq.wqe_cnt = mqp->sq.wqe_cnt;
+	qp_out->sq.stride  = 1 << mqp->sq.wqe_shift;
+
+	qp_out->rq.buf     = (void *)((uintptr_t)mqp->buf.buf + mqp->rq.offset);
+	qp_out->rq.wqe_cnt = mqp->rq.wqe_cnt;
+	qp_out->rq.stride  = 1 << mqp->rq.wqe_shift;
+
+	qp_out->bf.reg    = mqp->bf->reg;
+
+	if (mqp->bf->uuarn > 0)
+		qp_out->bf.size = mqp->bf->buf_size;
+	else
+		qp_out->bf.size = 0;
+
+	return 0;
+}
+
+static int mlx5dv_get_cq(struct ibv_cq *cq_in,
+			 struct mlx5dv_cq *cq_out)
+{
+	struct mlx5_cq *mcq = to_mcq(cq_in);
+	struct mlx5_context *mctx = to_mctx(cq_in->context);
+
+	cq_out->comp_mask = 0;
+	cq_out->cqn       = mcq->cqn;
+	cq_out->cqe_cnt   = mcq->ibv_cq.cqe + 1;
+	cq_out->cqe_size  = mcq->cqe_sz;
+	cq_out->buf       = mcq->active_buf->buf;
+	cq_out->dbrec     = mcq->dbrec;
+	cq_out->uar	  = mctx->uar;
+
+	mcq->flags	 |= MLX5_CQ_FLAGS_DV_OWNED;
+
+	return 0;
+}
+
+static int mlx5dv_get_rwq(struct ibv_wq *wq_in,
+			  struct mlx5dv_rwq *rwq_out)
+{
+	struct mlx5_rwq *mrwq = to_mrwq(wq_in);
+
+	rwq_out->comp_mask = 0;
+	rwq_out->buf       = mrwq->pbuff;
+	rwq_out->dbrec     = mrwq->recv_db;
+	rwq_out->wqe_cnt   = mrwq->rq.wqe_cnt;
+	rwq_out->stride    = 1 << mrwq->rq.wqe_shift;
+
+	return 0;
+}
+
+static int mlx5dv_get_srq(struct ibv_srq *srq_in,
+			  struct mlx5dv_srq *srq_out)
+{
+	struct mlx5_srq *msrq;
+
+	msrq = container_of(srq_in, struct mlx5_srq, vsrq.srq);
+
+	srq_out->comp_mask = 0;
+	srq_out->buf       = msrq->buf.buf;
+	srq_out->dbrec     = msrq->db;
+	srq_out->stride    = 1 << msrq->wqe_shift;
+	srq_out->head      = msrq->head;
+	srq_out->tail      = msrq->tail;
+
+	return 0;
+}
+
+int mlx5dv_init_obj(struct mlx5dv_obj *obj, uint64_t obj_type)
+{
+	int ret = 0;
+
+	if (obj_type & MLX5DV_OBJ_QP)
+		ret = mlx5dv_get_qp(obj->qp.in, obj->qp.out);
+	if (!ret && (obj_type & MLX5DV_OBJ_CQ))
+		ret = mlx5dv_get_cq(obj->cq.in, obj->cq.out);
+	if (!ret && (obj_type & MLX5DV_OBJ_SRQ))
+		ret = mlx5dv_get_srq(obj->srq.in, obj->srq.out);
+	if (!ret && (obj_type & MLX5DV_OBJ_RWQ))
+		ret = mlx5dv_get_rwq(obj->rwq.in, obj->rwq.out);
+
+	return ret;
+}
+
 static int mlx5_init_context(struct verbs_device *vdev,
 			     struct ibv_context *ctx, int cmd_fd)
 {
diff --git a/providers/mlx5/mlx5.h b/providers/mlx5/mlx5.h
index cf314fc..49ddc1d 100644
--- a/providers/mlx5/mlx5.h
+++ b/providers/mlx5/mlx5.h
@@ -42,6 +42,7 @@
 #include <ccan/list.h>
 #include "bitmap.h"
 #include <ccan/minmax.h>
+#include "include/dv.h"
 
 #ifdef __GNUC__
 #define likely(x)	__builtin_expect((x), 1)
@@ -110,11 +111,6 @@ static inline void mlx5_dbg(FILE *fp, uint32_t mask, const char *fmt, ...)
 #endif
 
 enum {
-	MLX5_RCV_DBR	= 0,
-	MLX5_SND_DBR	= 1,
-};
-
-enum {
 	MLX5_STAT_RATE_OFFSET		= 5
 };
 
@@ -137,36 +133,10 @@ enum {
 };
 
 enum {
-	MLX5_SEND_WQE_BB	= 64,
-	MLX5_SEND_WQE_SHIFT	= 6,
-};
-
-enum {
 	MLX5_BF_OFFSET	= 0x800
 };
 
 enum {
-	MLX5_INLINE_SCATTER_32	= 0x4,
-	MLX5_INLINE_SCATTER_64	= 0x8,
-};
-
-enum {
-	MLX5_OPCODE_NOP			= 0x00,
-	MLX5_OPCODE_SEND_INVAL		= 0x01,
-	MLX5_OPCODE_RDMA_WRITE		= 0x08,
-	MLX5_OPCODE_RDMA_WRITE_IMM	= 0x09,
-	MLX5_OPCODE_SEND		= 0x0a,
-	MLX5_OPCODE_SEND_IMM		= 0x0b,
-	MLX5_OPCODE_TSO			= 0x0e,
-	MLX5_OPCODE_RDMA_READ		= 0x10,
-	MLX5_OPCODE_ATOMIC_CS		= 0x11,
-	MLX5_OPCODE_ATOMIC_FA		= 0x12,
-	MLX5_OPCODE_ATOMIC_MASKED_CS	= 0x14,
-	MLX5_OPCODE_ATOMIC_MASKED_FA	= 0x15,
-	MLX5_OPCODE_FMR			= 0x19,
-	MLX5_OPCODE_LOCAL_INVAL		= 0x1b,
-	MLX5_OPCODE_CONFIG_CMD		= 0x1f,
-
 	MLX5_RECV_OPCODE_RDMA_WRITE_IMM	= 0x00,
 	MLX5_RECV_OPCODE_SEND		= 0x01,
 	MLX5_RECV_OPCODE_SEND_IMM	= 0x02,
@@ -174,7 +144,6 @@ enum {
 
 	MLX5_CQE_OPCODE_ERROR		= 0x1e,
 	MLX5_CQE_OPCODE_RESIZE		= 0x16,
-	MLX5_OPCODE_UMR			= 0x25,
 };
 
 enum {
@@ -182,10 +151,6 @@ enum {
 };
 
 enum {
-	MLX5_INLINE_SEG	= 0x80000000,
-};
-
-enum {
 	MLX5_MAX_PORTS_NUM = 2,
 };
 
@@ -249,7 +214,7 @@ struct mlx5_context {
 	int				prefer_bf;
 	int				shut_up_bf;
 	struct {
-		struct mlx5_qp	      **table;
+		struct mlx5_qp        **table;
 		int			refcnt;
 	}				qp_table[MLX5_QP_TABLE_SIZE];
 	pthread_mutex_t			qp_table_mutex;
@@ -338,6 +303,7 @@ enum {
 	MLX5_CQ_FLAGS_FOUND_CQES = 1 << 2,
 	MLX5_CQ_FLAGS_EXTENDED = 1 << 3,
 	MLX5_CQ_FLAGS_SINGLE_THREADED = 1 << 4,
+	MLX5_CQ_FLAGS_DV_OWNED = 1 << 5,
 };
 
 struct mlx5_cq {
@@ -448,29 +414,9 @@ struct mlx5_qp {
 	int                             rss_qp;
 };
 
-struct mlx5_av {
-	union {
-		struct {
-			uint32_t	qkey;
-			uint32_t	reserved;
-		} qkey;
-		uint64_t	dc_key;
-	} key;
-	uint32_t	dqp_dct;
-	uint8_t		stat_rate_sl;
-	uint8_t		fl_mlid;
-	uint16_t	rlid;
-	uint8_t		reserved0[4];
-	uint8_t		rmac[6];
-	uint8_t		tclass;
-	uint8_t		hop_limit;
-	uint32_t	grh_gid_fl;
-	uint8_t		rgid[16];
-};
-
 struct mlx5_ah {
 	struct ibv_ah			ibv_ah;
-	struct mlx5_av			av;
+	struct mlx5_wqe_av		av;
 	bool				kern_ah;
 };
 
diff --git a/providers/mlx5/qp.c b/providers/mlx5/qp.c
index e82b1a0..177b9ae 100644
--- a/providers/mlx5/qp.c
+++ b/providers/mlx5/qp.c
@@ -212,7 +212,7 @@ static void set_datagram_seg(struct mlx5_wqe_datagram_seg *dseg,
 			     struct ibv_send_wr *wr)
 {
 	memcpy(&dseg->av, &to_mah(wr->wr.ud.ah)->av, sizeof dseg->av);
-	dseg->av.dqp_dct = htonl(wr->wr.ud.remote_qpn | MLX5_EXTENED_UD_AV);
+	dseg->av.dqp_dct = htonl(wr->wr.ud.remote_qpn | MLX5_EXTENDED_UD_AV);
 	dseg->av.key.qkey.qkey = htonl(wr->wr.ud.remote_qkey);
 }
 
diff --git a/providers/mlx5/wqe.h b/providers/mlx5/wqe.h
index f097b77..13597bc 100644
--- a/providers/mlx5/wqe.h
+++ b/providers/mlx5/wqe.h
@@ -33,34 +33,6 @@
 #ifndef WQE_H
 #define WQE_H
 
-enum {
-	MLX5_WQE_CTRL_CQ_UPDATE	= 2 << 2,
-	MLX5_WQE_CTRL_SOLICITED	= 1 << 1,
-	MLX5_WQE_CTRL_FENCE	= 4 << 5,
-	MLX5_WQE_CTRL_INITIATOR_SMALL_FENCE = 1 << 5,
-};
-
-enum {
-	MLX5_INVALID_LKEY	= 0x100,
-};
-
-enum {
-	MLX5_EXTENED_UD_AV	= 0x80000000,
-};
-
-struct mlx5_wqe_srq_next_seg {
-	uint8_t			rsvd0[2];
-	uint16_t		next_wqe_index;
-	uint8_t			signature;
-	uint8_t			rsvd1[11];
-};
-
-struct mlx5_wqe_data_seg {
-	uint32_t		byte_count;
-	uint32_t		lkey;
-	uint64_t		addr;
-};
-
 struct mlx5_sg_copy_ptr {
 	int	index;
 	int	offset;
@@ -76,36 +48,6 @@ struct mlx5_eqe_qp_srq {
 	uint32_t	qp_srq_n;
 };
 
-enum {
-	MLX5_ETH_L2_INLINE_HEADER_SIZE	= 18,
-	MLX5_ETH_L2_MIN_HEADER_SIZE	= 14,
-};
-
-enum {
-	MLX5_ETH_WQE_L3_CSUM = (1 << 6),
-	MLX5_ETH_WQE_L4_CSUM = (1 << 7),
-};
-
-struct mlx5_wqe_eth_seg {
-	uint32_t	rsvd0;
-	uint8_t		cs_flags;
-	uint8_t		rsvd1;
-	uint16_t	mss;
-	uint32_t	rsvd2;
-	uint16_t	inline_hdr_sz;
-	uint8_t		inline_hdr_start[2];
-	uint8_t		inline_hdr[16];
-};
-
-struct mlx5_wqe_ctrl_seg {
-	uint32_t	opmod_idx_opcode;
-	uint32_t	qpn_ds;
-	uint8_t		signature;
-	uint8_t		rsvd[2];
-	uint8_t		fm_ce_se;
-	uint32_t	imm;
-};
-
 struct mlx5_wqe_xrc_seg {
 	uint32_t	xrc_srqn;
 	uint8_t		rsvd[12];
@@ -118,42 +60,9 @@ struct mlx5_wqe_masked_atomic_seg {
 	uint64_t	compare_mask;
 };
 
-struct mlx5_wqe_av {
-	union {
-		struct {
-			uint32_t	qkey;
-			uint32_t	reserved;
-		} qkey;
-		uint64_t	dc_key;
-	} key;
-	uint32_t	dqp_dct;
-	uint8_t		stat_rate_sl;
-	uint8_t		fl_mlid;
-	uint16_t	rlid;
-	uint8_t		reserved0[10];
-	uint8_t		tclass;
-	uint8_t		hop_limit;
-	uint32_t	grh_gid_fl;
-	uint8_t		rgid[16];
-};
-
-struct mlx5_wqe_datagram_seg {
-	struct mlx5_wqe_av	av;
-};
-
-struct mlx5_wqe_raddr_seg {
-	uint64_t	raddr;
-	uint32_t	rkey;
-	uint32_t	reserved;
-};
-
-struct mlx5_wqe_atomic_seg {
-	uint64_t	swap_add;
-	uint64_t	compare;
-};
-
-struct mlx5_wqe_inl_data_seg {
-	uint32_t	byte_count;
+enum {
+	MLX5_ETH_L2_INLINE_HEADER_SIZE	= 18,
+	MLX5_ETH_L2_MIN_HEADER_SIZE	= 14,
 };
 
 enum {
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 2/7] mlx5: Add CQE fields

[Yishai Hadas]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Introduce inline calls to get/set for most
commonly used CQE data path fields.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Acked-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 providers/mlx5/include/dv-cqe.h | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/providers/mlx5/include/dv-cqe.h b/providers/mlx5/include/dv-cqe.h
index 1ad33ad..c36b7e0 100644
--- a/providers/mlx5/include/dv-cqe.h
+++ b/providers/mlx5/include/dv-cqe.h
@@ -122,4 +122,35 @@ struct mlx5_cqe64 {
 	uint8_t		signature;
 	uint8_t		op_own;
 };
+
+static MLX5DV_ALWAYS_INLINE
+uint8_t mlx5dv_get_cqe_owner(struct mlx5_cqe64 *cqe)
+{
+	return cqe->op_own & 0x1;
+}
+
+static MLX5DV_ALWAYS_INLINE
+void mlx5dv_set_cqe_owner(struct mlx5_cqe64 *cqe, uint8_t val)
+{
+	cqe->op_own = (val & 0x1) | (cqe->op_own & ~0x1);
+}
+
+/* Solicited event */
+static MLX5DV_ALWAYS_INLINE
+uint8_t mlx5dv_get_cqe_se(struct mlx5_cqe64 *cqe)
+{
+	return (cqe->op_own >> 1) & 0x1;
+}
+
+static MLX5DV_ALWAYS_INLINE
+uint8_t mlx5dv_get_cqe_format(struct mlx5_cqe64 *cqe)
+{
+	return (cqe->op_own >> 2) & 0x3;
+}
+
+static MLX5DV_ALWAYS_INLINE
+uint8_t mlx5dv_get_cqe_opcode(struct mlx5_cqe64 *cqe)
+{
+	return cqe->op_own >> 4;
+}
 #endif /* _MLX5DV_CQE_H_ */
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 3/7] mlx5: Add WQE segments implementation

[Yishai Hadas]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Add generic and x86 SSE3 versions of WQE
segments implementations.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Acked-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 providers/mlx5/include/dv-wqe.h | 175 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 175 insertions(+)

diff --git a/providers/mlx5/include/dv-wqe.h b/providers/mlx5/include/dv-wqe.h
index ce9bd86..a819a6f 100644
--- a/providers/mlx5/include/dv-wqe.h
+++ b/providers/mlx5/include/dv-wqe.h
@@ -34,6 +34,10 @@
 #define _MLX5DV_WQE_H_
 
 #include <arpa/inet.h>
+#if defined(__SSE3__)
+#include <emmintrin.h>
+#include <tmmintrin.h>
+#endif /* defined(__SSE3__) */
 
 enum {
 	MLX5_INVALID_LKEY	= 0x100,
@@ -136,5 +140,176 @@ struct mlx5_wqe_eth_seg {
 	uint8_t		inline_hdr[16];
 };
 
+/*
+ * Control segment - contains some control information for the current WQE.
+ *
+ * Output:
+ *	seg	  - control segment to be filled
+ * Input:
+ *	pi	  - WQEBB number of the first block of this WQE.
+ *		    This number should wrap at 0xffff, regardless of
+ *		    size of the WQ.
+ *	opcode	  - Opcode of this WQE. Encodes the type of operation
+ *		    to be executed on the QP.
+ *	opmod	  - Opcode modifier.
+ *	qp_num	  - QP/SQ number this WQE is posted to.
+ *	fm_ce_se  - FM (fence mode), CE (completion and event mode)
+ *		    and SE (solicited event).
+ *	ds	  - WQE size in octowords (16-byte units). DS accounts for all
+ *		    the segments in the WQE as summarized in WQE construction.
+ *	signature - WQE signature.
+ *	imm	  - Immediate data/Invalidation key/UMR mkey.
+ */
+static MLX5DV_ALWAYS_INLINE
+void mlx5dv_set_ctrl_seg(struct mlx5_wqe_ctrl_seg *seg, uint16_t pi,
+			 uint8_t opcode, uint8_t opmod, uint32_t qp_num,
+			 uint8_t fm_ce_se, uint8_t ds,
+			 uint8_t signature, uint32_t imm)
+{
+	seg->opmod_idx_opcode	= htonl(((uint32_t)opmod << 24) | ((uint32_t)pi << 8) | opcode);
+	seg->qpn_ds		= htonl((qp_num << 8) | ds);
+	seg->fm_ce_se		= fm_ce_se;
+	seg->signature		= signature;
+	/*
+	 * The caller should prepare "imm" in advance based on WR opcode.
+	 * For IBV_WR_SEND_WITH_IMM and IBV_WR_RDMA_WRITE_WITH_IMM,
+	 * the "imm" should be assigned as is.
+	 * For the IBV_WR_SEND_WITH_INV, it should be htonl(imm).
+	 */
+	seg->imm		= imm;
+}
+
+/* x86 optimized version of mlx5dv_set_ctrl_seg()
+ *
+ * This is useful when doing calculations on large data sets
+ * for parallel calculations.
+ *
+ * It doesn't suit for serialized algorithms.
+ */
+#if defined(__SSE3__)
+static MLX5DV_ALWAYS_INLINE
+void mlx5dv_x86_set_ctrl_seg(struct mlx5_wqe_ctrl_seg *seg, uint16_t pi,
+			     uint8_t opcode, uint8_t opmod, uint32_t qp_num,
+			     uint8_t fm_ce_se, uint8_t ds,
+			     uint8_t signature, uint32_t imm)
+{
+	__m128i val  = _mm_set_epi32(imm, qp_num, (ds << 16) | pi,
+				     (signature << 24) | (opcode << 16) | (opmod << 8) | fm_ce_se);
+	__m128i mask = _mm_set_epi8(12, 13, 14, 15,	/* immediate */
+				     0,			/* signal/fence_mode */
+				     0x80, 0x80,	/* reserved */
+				     3,			/* signature */
+				     6,			/* data size */
+				     8, 9, 10,		/* QP num */
+				     2,			/* opcode */
+				     4, 5,		/* sw_pi in BE */
+				     1			/* opmod */
+				     );
+	*(__m128i *) seg = _mm_shuffle_epi8(val, mask);
+}
+#endif /* defined(__SSE3__) */
+
+/*
+ * Datagram Segment - contains address information required in order
+ * to form a datagram message.
+ *
+ * Output:
+ *	seg		- datagram segment to be filled.
+ * Input:
+ *	key		- Q_key/access key.
+ *	dqp_dct		- Destination QP number for UD and DCT for DC.
+ *	ext		- Address vector extension.
+ *	stat_rate_sl	- Maximum static rate control, SL/ethernet priority.
+ *	fl_mlid		- Force loopback and source LID for IB.
+ *	rlid		- Remote LID
+ *	rmac		- Remote MAC
+ *	tclass		- GRH tclass/IPv6 tclass/IPv4 ToS
+ *	hop_limit	- GRH hop limit/IPv6 hop limit/IPv4 TTL
+ *	grh_gid_fi	- GRH, source GID address and IPv6 flow label.
+ *	rgid		- Remote GID/IP address.
+ */
+static MLX5DV_ALWAYS_INLINE
+void mlx5dv_set_dgram_seg(struct mlx5_wqe_datagram_seg *seg,
+			  uint64_t key, uint32_t dqp_dct,
+			  uint8_t ext, uint8_t stat_rate_sl,
+			  uint8_t fl_mlid, uint16_t rlid,
+			  uint8_t *rmac, uint8_t tclass,
+			  uint8_t hop_limit, uint32_t grh_gid_fi,
+			  uint8_t *rgid)
+{
 
+	/* Always put 64 bits, in q_key, the reserved part will be 0 */
+	seg->av.key.dc_key	= htonll(key);
+	seg->av.dqp_dct		= htonl(((uint32_t)ext << 31) | dqp_dct);
+	seg->av.stat_rate_sl	= stat_rate_sl;
+	seg->av.fl_mlid		= fl_mlid;
+	seg->av.rlid		= htons(rlid);
+	memcpy(seg->av.rmac, rmac, 6);
+	seg->av.tclass		= tclass;
+	seg->av.hop_limit	= hop_limit;
+	seg->av.grh_gid_fl	= htonl(grh_gid_fi);
+	memcpy(seg->av.rgid, rgid, 16);
+}
+
+/*
+ * Data Segments - contain pointers and a byte count for the scatter/gather list.
+ * They can optionally contain data, which will save a memory read access for
+ * gather Work Requests.
+ */
+static MLX5DV_ALWAYS_INLINE
+void mlx5dv_set_data_seg(struct mlx5_wqe_data_seg *seg,
+			 uint32_t length, uint32_t lkey,
+			 uintptr_t address)
+{
+	seg->byte_count = htonl(length);
+	seg->lkey       = htonl(lkey);
+	seg->addr       = htonll(address);
+}
+/*
+ * x86 optimized version of mlx5dv_set_data_seg()
+ *
+ * This is useful when doing calculations on large data sets
+ * for parallel calculations.
+ *
+ * It doesn't suit for serialized algorithms.
+ */
+#if defined(__SSE3__)
+static MLX5DV_ALWAYS_INLINE
+void mlx5dv_x86_set_data_seg(struct mlx5_wqe_data_seg *seg,
+			     uint32_t length, uint32_t lkey,
+			     uintptr_t address)
+{
+	__m128i val  = _mm_set_epi32((uint32_t)address, (uint32_t)(address >> 32), lkey, length);
+	__m128i mask = _mm_set_epi8(12, 13, 14, 15,	/* local address low */
+				     8, 9, 10, 11,	/* local address high */
+				     4, 5, 6, 7,	/* l_key */
+				     0, 1, 2, 3		/* byte count */
+				     );
+	*(__m128i *) seg = _mm_shuffle_epi8(val, mask);
+}
+#endif /* defined(__SSE3__) */
+
+/*
+ * Eth Segment - contains packet headers and information for stateless L2, L3, L4 offloading.
+ *
+ * Output:
+ *	 seg		 - Eth segment to be filled.
+ * Input:
+ *	cs_flags	 - l3cs/l3cs_inner/l4cs/l4cs_inner.
+ *	mss		 - Maximum segment size. For TSO WQEs, the number of bytes
+ *			   in the TCP payload to be transmitted in each packet. Must
+ *			   be 0 on non TSO WQEs.
+ *	inline_hdr_sz	 - Length of the inlined packet headers.
+ *	inline_hdr_start - Inlined packet header.
+ */
+static MLX5DV_ALWAYS_INLINE
+void mlx5dv_set_eth_seg(struct mlx5_wqe_eth_seg *seg, uint8_t cs_flags,
+			uint16_t mss, uint16_t inline_hdr_sz,
+			uint8_t *inline_hdr_start)
+{
+	seg->cs_flags		= cs_flags;
+	seg->mss		= htons(mss);
+	seg->inline_hdr_sz	= htons(inline_hdr_sz);
+	memcpy(seg->inline_hdr_start, inline_hdr_start, inline_hdr_sz);
+}
 #endif /* _MLX5DV_WQE_H_ */
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 4/7] mlx5: Use macro for CQE version 1

[Yishai Hadas]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

There is macro defined to mark CQE version 1, so let's use it.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Acked-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 providers/mlx5/mlx5.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/providers/mlx5/mlx5.c b/providers/mlx5/mlx5.c
index 1cf5a5f..980e4df 100644
--- a/providers/mlx5/mlx5.c
+++ b/providers/mlx5/mlx5.c
@@ -771,7 +771,7 @@ static int mlx5_init_context(struct verbs_device *vdev,
 
 	context->cqe_version = resp.cqe_version;
 	if (context->cqe_version) {
-		if (context->cqe_version == 1)
+		if (context->cqe_version == MLX5_CQE_VERSION_V1)
 			mlx5_ctx_ops.poll_cq = mlx5_poll_cq_v1;
 		else
 			 goto err_free_bf;
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 5/7] mlx5: Convert to use predefined get CQE opcodes

[Yishai Hadas]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Acked-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 providers/mlx5/cq.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/providers/mlx5/cq.c b/providers/mlx5/cq.c
index e5b50a3..fe7379c 100644
--- a/providers/mlx5/cq.c
+++ b/providers/mlx5/cq.c
@@ -86,7 +86,7 @@ static void *get_sw_cqe(struct mlx5_cq *cq, int n)
 
 	cqe64 = (cq->cqe_sz == 64) ? cqe : cqe + 64;
 
-	if (likely((cqe64->op_own) >> 4 != MLX5_CQE_INVALID) &&
+	if (likely(mlx5dv_get_cqe_opcode(cqe64) != MLX5_CQE_INVALID) &&
 	    !((cqe64->op_own & MLX5_CQE_OWNER_MASK) ^ !!(n & (cq->ibv_cq.cqe + 1)))) {
 		return cqe;
 	} else {
@@ -544,7 +544,7 @@ static inline int mlx5_parse_cqe(struct mlx5_cq *cq,
 		wc->qp_num = qpn;
 	}
 
-	opcode = cqe64->op_own >> 4;
+	opcode = mlx5dv_get_cqe_opcode(cqe64);
 	switch (opcode) {
 	case MLX5_CQE_REQ:
 	{
@@ -1053,7 +1053,7 @@ static inline enum ibv_wc_opcode mlx5_cq_read_wc_opcode(struct ibv_cq_ex *ibcq)
 {
 	struct mlx5_cq *cq = to_mcq(ibv_cq_ex_to_cq(ibcq));
 
-	switch (cq->cqe64->op_own >> 4) {
+	switch (mlx5dv_get_cqe_opcode(cq->cqe64)) {
 	case MLX5_CQE_RESP_WR_IMM:
 		return IBV_WC_RECV_RDMA_WITH_IMM;
 	case MLX5_CQE_RESP_SEND:
@@ -1111,7 +1111,7 @@ static inline int mlx5_cq_read_wc_flags(struct ibv_cq_ex *ibcq)
 				  MLX5_CQE_L3_HDR_TYPE_IPV4)) <<
 				IBV_WC_IP_CSUM_OK_SHIFT;
 
-	switch (cq->cqe64->op_own >> 4) {
+	switch (mlx5dv_get_cqe_opcode(cq->cqe64)) {
 	case MLX5_CQE_RESP_WR_IMM:
 	case MLX5_CQE_RESP_SEND_IMM:
 		wc_flags	|= IBV_WC_WITH_IMM;
@@ -1144,7 +1144,7 @@ static inline uint32_t mlx5_cq_read_wc_imm_data(struct ibv_cq_ex *ibcq)
 {
 	struct mlx5_cq *cq = to_mcq(ibv_cq_ex_to_cq(ibcq));
 
-	switch (cq->cqe64->op_own >> 4) {
+	switch (mlx5dv_get_cqe_opcode(cq->cqe64)) {
 	case MLX5_CQE_RESP_SEND_INV:
 		return ntohl(cq->cqe64->imm_inval_pkey);
 	default:
@@ -1326,7 +1326,7 @@ static inline int free_res_cqe(struct mlx5_cqe64 *cqe64, uint32_t rsn,
 {
 	if (cqe_version) {
 		if (is_equal_uidx(cqe64, rsn)) {
-			if (srq && is_responder(cqe64->op_own >> 4))
+			if (srq && is_responder(mlx5dv_get_cqe_opcode(cqe64)))
 				mlx5_free_srq_wqe(srq,
 						  ntohs(cqe64->wqe_counter));
 			return 1;
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 6/7] mlx5: Unify exported and internal CQE format

[Yishai Hadas]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Direct verbs are exporting CQE format, which is mapped
to enum MLX5_INLINE_SCATTER_*.

The proposed change brings in sync the internal code,
exported enum and returned CQE format.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Acked-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 providers/mlx5/cq.c             | 24 ++++++++++++------------
 providers/mlx5/include/dv-cqe.h |  4 ++--
 2 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/providers/mlx5/cq.c b/providers/mlx5/cq.c
index fe7379c..0ae6a19 100644
--- a/providers/mlx5/cq.c
+++ b/providers/mlx5/cq.c
@@ -151,10 +151,10 @@ static inline int handle_responder_lazy(struct mlx5_cq *cq, struct mlx5_cqe64 *c
 		wqe_ctr = ntohs(cqe->wqe_counter);
 		cq->ibv_cq.wr_id = srq->wrid[wqe_ctr];
 		mlx5_free_srq_wqe(srq, wqe_ctr);
-		if (cqe->op_own & MLX5_INLINE_SCATTER_32)
+		if (mlx5dv_get_cqe_format(cqe) == MLX5_INLINE_SCATTER_32)
 			err = mlx5_copy_to_recv_srq(srq, wqe_ctr, cqe,
 						    ntohl(cqe->byte_cnt));
-		else if (cqe->op_own & MLX5_INLINE_SCATTER_64)
+		else if (mlx5dv_get_cqe_format(cqe) == MLX5_INLINE_SCATTER_64)
 			err = mlx5_copy_to_recv_srq(srq, wqe_ctr, cqe - 1,
 						    ntohl(cqe->byte_cnt));
 	} else {
@@ -169,10 +169,10 @@ static inline int handle_responder_lazy(struct mlx5_cq *cq, struct mlx5_cqe64 *c
 		wqe_ctr = wq->tail & (wq->wqe_cnt - 1);
 		cq->ibv_cq.wr_id = wq->wrid[wqe_ctr];
 		++wq->tail;
-		if (cqe->op_own & MLX5_INLINE_SCATTER_32)
+		if (mlx5dv_get_cqe_format(cqe) == MLX5_INLINE_SCATTER_32)
 			err = mlx5_copy_to_recv_wqe(qp, wqe_ctr, cqe,
 						    ntohl(cqe->byte_cnt));
-		else if (cqe->op_own & MLX5_INLINE_SCATTER_64)
+		else if (mlx5dv_get_cqe_format(cqe) == MLX5_INLINE_SCATTER_64)
 			err = mlx5_copy_to_recv_wqe(qp, wqe_ctr, cqe - 1,
 						    ntohl(cqe->byte_cnt));
 }
@@ -194,10 +194,10 @@ static inline int handle_responder(struct ibv_wc *wc, struct mlx5_cqe64 *cqe,
 		wqe_ctr = ntohs(cqe->wqe_counter);
 		wc->wr_id = srq->wrid[wqe_ctr];
 		mlx5_free_srq_wqe(srq, wqe_ctr);
-		if (cqe->op_own & MLX5_INLINE_SCATTER_32)
+		if (mlx5dv_get_cqe_format(cqe) == MLX5_INLINE_SCATTER_32)
 			err = mlx5_copy_to_recv_srq(srq, wqe_ctr, cqe,
 						    wc->byte_len);
-		else if (cqe->op_own & MLX5_INLINE_SCATTER_64)
+		else if (mlx5dv_get_cqe_format(cqe) == MLX5_INLINE_SCATTER_64)
 			err = mlx5_copy_to_recv_srq(srq, wqe_ctr, cqe - 1,
 						    wc->byte_len);
 	} else {
@@ -216,10 +216,10 @@ static inline int handle_responder(struct ibv_wc *wc, struct mlx5_cqe64 *cqe,
 		wqe_ctr = wq->tail & (wq->wqe_cnt - 1);
 		wc->wr_id = wq->wrid[wqe_ctr];
 		++wq->tail;
-		if (cqe->op_own & MLX5_INLINE_SCATTER_32)
+		if (mlx5dv_get_cqe_format(cqe) == MLX5_INLINE_SCATTER_32)
 			err = mlx5_copy_to_recv_wqe(qp, wqe_ctr, cqe,
 						    wc->byte_len);
-		else if (cqe->op_own & MLX5_INLINE_SCATTER_64)
+		else if (mlx5dv_get_cqe_format(cqe) == MLX5_INLINE_SCATTER_64)
 			err = mlx5_copy_to_recv_wqe(qp, wqe_ctr, cqe - 1,
 						    wc->byte_len);
 	}
@@ -572,10 +572,10 @@ static inline int mlx5_parse_cqe(struct mlx5_cq *cq,
 				wc_byte_len = 8;
 
 			scatter_out:
-				if (cqe64->op_own & MLX5_INLINE_SCATTER_32)
+				if (mlx5dv_get_cqe_format(cqe64) == MLX5_INLINE_SCATTER_32)
 					err = mlx5_copy_to_send_wqe(
 					    mqp, wqe_ctr, cqe, wc_byte_len);
-				else if (cqe64->op_own & MLX5_INLINE_SCATTER_64)
+				else if (mlx5dv_get_cqe_format(cqe64) == MLX5_INLINE_SCATTER_64)
 					err = mlx5_copy_to_send_wqe(
 					    mqp, wqe_ctr, cqe - 1, wc_byte_len);
 				break;
@@ -586,10 +586,10 @@ static inline int mlx5_parse_cqe(struct mlx5_cq *cq,
 		} else {
 			handle_good_req(wc, cqe64, wq, idx);
 
-			if (cqe64->op_own & MLX5_INLINE_SCATTER_32)
+			if (mlx5dv_get_cqe_format(cqe64) == MLX5_INLINE_SCATTER_32)
 				err = mlx5_copy_to_send_wqe(mqp, wqe_ctr, cqe,
 							    wc->byte_len);
-			else if (cqe64->op_own & MLX5_INLINE_SCATTER_64)
+			else if (mlx5dv_get_cqe_format(cqe64) == MLX5_INLINE_SCATTER_64)
 				err = mlx5_copy_to_send_wqe(
 				    mqp, wqe_ctr, cqe - 1, wc->byte_len);
 
diff --git a/providers/mlx5/include/dv-cqe.h b/providers/mlx5/include/dv-cqe.h
index c36b7e0..d7d3dba 100644
--- a/providers/mlx5/include/dv-cqe.h
+++ b/providers/mlx5/include/dv-cqe.h
@@ -37,8 +37,8 @@
 #include <linux/types.h>
 
 enum {
-	MLX5_INLINE_SCATTER_32	= 0x4,
-	MLX5_INLINE_SCATTER_64	= 0x8,
+	MLX5_INLINE_SCATTER_32	= 0x1,
+	MLX5_INLINE_SCATTER_64	= 0x2,
 };
 
 enum {
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-core 7/7] mlx5: Export mlx5 direct verbs interface

[Yishai Hadas]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Install direct verbs related files into
/usr/include/infiniband/mlx5 folder and allow
for possible users to explicitly include them
into their applications.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Acked-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 buildlib/provider.map                 | 13 +++++++++----
 providers/mlx5/include/CMakeLists.txt |  6 +++++-
 2 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/buildlib/provider.map b/buildlib/provider.map
index e985a6f..8c7875f 100644
--- a/buildlib/provider.map
+++ b/buildlib/provider.map
@@ -1,6 +1,11 @@
-/* The providers do not export any symbols at all. Instead they rely on
-   attribute(constructor) to cause their init function to run at dlopen
-   time. */
-{
+/* The providers usually do not export any symbols at all. Instead
+   they rely on attribute(constructor) to cause their init function to
+   run at dlopen time.
+   Providers that wish to export symbols should add them below according to
+   Documentation/versioning.md document. */
+MLX5_1.0 {
+	global:
+		mlx5dv_query_device;
+		mlx5dv_init_obj;
 	local: *;
 };
diff --git a/providers/mlx5/include/CMakeLists.txt b/providers/mlx5/include/CMakeLists.txt
index 11a2839..8e2dc4a 100644
--- a/providers/mlx5/include/CMakeLists.txt
+++ b/providers/mlx5/include/CMakeLists.txt
@@ -1 +1,5 @@
-# Empty CMakelists.txt to avoid compilation error
+publish_headers(infiniband/mlx5
+	dv.h
+	dv-wqe.h
+	dv-cqe.h
+)
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core 0/7] Add mlx5 direct verbs

[Jason Gunthorpe]

On Thu, Jan 26, 2017 at 05:56:15PM +0200, Yishai Hadas wrote:
> This patchset from Leon adds direct access to mlx5 devices.
> 
> The libibverbs API is an abstract API. It is agnostic to any underlying
> provider specific implementation. While this abstraction has the advantage of
> user applications portability it has a performance penalty. For some
> applications optimizing performance is more important than portability.
>
> The mlx5 direct verbs API introduced in this patchset is intended for such
> applications. It exposes mlx5 specific low level data path
> (send/receive/completion) operations, allowing the application to bypass the
> libibverbs data path API.

Applications need to link directly to the libraries that provide the
symbols they use - this approach you have where linking happens
implicitly at runtime is not OK.

Somehow libmlx needs to be in the standard ld.so search path, and
available for linking via gcc, as well as continuing to be a plugin..

The best choice might be to put it in /usr/lib/../libmlx5.so.1 and
symlink that file to /usr/lib/../libibverbs/libmlx5-rdmav2.so

This series also needs patches to the included packaging files to
bundle the new files. (buildlib/cbuild exists for a reason, be sure to
test the packaging)

These probably need a man page as well. It wasn't instantly obvious to
me how these were supposed to work.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core 1/7] mlx5: Add mlx5 direct verbs files

[Jason Gunthorpe]

On Thu, Jan 26, 2017 at 05:56:16PM +0200, Yishai Hadas wrote:

> +add_subdirectory(include)

No need for a subdirectory. I've been flattening things for a while

> +/* Always inline the function */
> +#ifdef __GNUC__
> +#define MLX5DV_ALWAYS_INLINE inline __attribute__((always_inline))
> +#else
> +#define MLX5DV_ALWAYS_INLINE inline
> +#endif

I doubt this is necessary... static inline is fine these days.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core 7/7] mlx5: Export mlx5 direct verbs interface

[Jason Gunthorpe]

On Thu, Jan 26, 2017 at 05:56:22PM +0200, Yishai Hadas wrote:
> From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> 
> Install direct verbs related files into
> /usr/include/infiniband/mlx5 folder and allow
> for possible users to explicitly include them
> into their applications.
> 
> Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Reviewed-by: Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Acked-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
>  buildlib/provider.map                 | 13 +++++++++----
>  providers/mlx5/include/CMakeLists.txt |  6 +++++-
>  2 files changed, 14 insertions(+), 5 deletions(-)
> 
> diff --git a/buildlib/provider.map b/buildlib/provider.map
> index e985a6f..8c7875f 100644
> +++ b/buildlib/provider.map
> @@ -1,6 +1,11 @@
> -/* The providers do not export any symbols at all. Instead they rely on
> -   attribute(constructor) to cause their init function to run at dlopen
> -   time. */
> -{
> +/* The providers usually do not export any symbols at all. Instead
> +   they rely on attribute(constructor) to cause their init function to
> +   run at dlopen time.
> +   Providers that wish to export symbols should add them below according to
> +   Documentation/versioning.md document. */
> +MLX5_1.0 {
> +	global:
> +		mlx5dv_query_device;
> +		mlx5dv_init_obj;
>  	local: *;

No, this is no longer just a provider so it doesn't get to use
provider.map

mlx5 will need its own map file and the various magic I described to
become a proper library and a plugin at once.

> +publish_headers(infiniband/mlx5
> +	dv.h
> +	dv-wqe.h
> +	dv-cqe.h
> +)

not ever dir needs a cmakefile, this is fine in the providers/mlx5
even with the include/ dir. It is easier to understand if things are
not spread out so much.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core 0/7] Add mlx5 direct verbs

[Yishai Hadas]

On 1/26/2017 7:25 PM, Jason Gunthorpe wrote:
> On Thu, Jan 26, 2017 at 05:56:15PM +0200, Yishai Hadas wrote:
>> This patchset from Leon adds direct access to mlx5 devices.
>>
>> The libibverbs API is an abstract API. It is agnostic to any underlying
>> provider specific implementation. While this abstraction has the advantage of
>> user applications portability it has a performance penalty. For some
>> applications optimizing performance is more important than portability.
>>
>> The mlx5 direct verbs API introduced in this patchset is intended for such
>> applications. It exposes mlx5 specific low level data path
>> (send/receive/completion) operations, allowing the application to bypass the
>> libibverbs data path API.
>
> Applications need to link directly to the libraries that provide the
> symbols they use - this approach you have where linking happens
> implicitly at runtime is not OK.

Correct, the link will be explicitly by the application at compile time 
so that the exported symbols will be resolved.

The below note from the cover letter planned to point on that:
"The direct include of dv.h together with linkage to mlx5 library will 
allow usage of this new interface"

> Somehow libmlx needs to be in the standard ld.so search path, and
> available for linking via gcc, as well as continuing to be a plugin..
> The best choice might be to put it in /usr/lib/../libmlx5.so.1 and
> symlink that file to /usr/lib/../libibverbs/libmlx5-rdmav2.so

You are pointing on same usage as done for libibverbs which has the 
symlink, correct ? Alternatively, application can use the linker flags 
as part of its Makefile to find the shared library location, isn't it ?

> This series also needs patches to the included packaging files to
> bundle the new files. (buildlib/cbuild exists for a reason, be sure to
> test the packaging)

Thanks for pointing on, we'll add and test the packaging.

> These probably need a man page as well. It wasn't instantly obvious to
> me how these were supposed to work.

Sure, we can add some man page here.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core 0/7] Add mlx5 direct verbs

[Jason Gunthorpe]

On Thu, Jan 26, 2017 at 11:54:53PM +0200, Yishai Hadas wrote:

> Correct, the link will be explicitly by the application at compile time so
> that the exported symbols will be resolved.

Okay, then you just need to make that possible.

> >Somehow libmlx needs to be in the standard ld.so search path, and
> >available for linking via gcc, as well as continuing to be a plugin..
> >The best choice might be to put it in /usr/lib/../libmlx5.so.1 and
> >symlink that file to /usr/lib/../libibverbs/libmlx5-rdmav2.so
> 
> You are pointing on same usage as done for libibverbs which has the symlink,
> correct ?


libibverbs has a symlink for a different purpose.

> Alternatively, application can use the linker flags as part of its
> Makefile to find the shared library location, isn't it ?

No, that would require applications to hardwire
/usr/lib/.../libibverbs/libmlx5-rdmav2.so as a path. Do not want that.

Make it so

gcc -libverbs_dv_mlx5 foo.c

Is all that is required.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core 0/7] Add mlx5 direct verbs

[Christoph Lameter]

Looks fairly straightforward from a user point of view. I hope you iterate
this patchset fast so that we can get it soon?

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core 0/7] Add mlx5 direct verbs

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 467 bytes --]

On Wed, Feb 01, 2017 at 09:56:56AM -0600, Christoph Lameter wrote:
>
> Looks fairly straightforward from a user point of view. I hope you iterate
> this patchset fast so that we can get it soon?

Yes, we have no plans to delay it.

Thanks
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH rdma-core 0/7] Add mlx5 direct verbs

[Sagi Grimberg]

> This patchset from Leon adds direct access to mlx5 devices.
>
> The libibverbs API is an abstract API. It is agnostic to any underlying
> provider specific implementation. While this abstraction has the advantage of
> user applications portability it has a performance penalty. For some
> applications optimizing performance is more important than portability.
>
> The mlx5 direct verbs API introduced in this patchset is intended for such
> applications. It exposes mlx5 specific low level data path
> (send/receive/completion) operations, allowing the application to bypass the
> libibverbs data path API.

Yishai and Leon this is good stuff !!

Couple of comments:

- The user of this is pretty intimate with the driver, which makes a
   somewhat strange combo of having both ibv_ and mlx5dv_ API. I
   understand that it can inflate the the implementation, but perhaps
   for completeness we can have also the control plane API as well (even
   as a trampoline to the generic equivalent where it's useless).

   The rational here is that extended features that needs allocation
   flags can be contained to mlx5 and not pollute/inflate the generic
   verbs implementation.

- This brings me to the next step of this. What is still missing is
   how we allow the user to have more control and information about how
   his db records (namely UARs) are allocated. How does the user
   know if he needs to protect the DB write? Is it BF? How can the user
   ask for more UARs than originally allocated to it by the kernel?
   The user of this can really use an API that allows more control
   of this area.

   Hopefully we can come up with something that is abstract enough to
   not expose how exactly UARs are strided on pages and the exact
   interface with the kernel, but still let the user choose if to use
   a dedicated db_rec and let the driver manage all the internal details.

Thoughts?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core 0/7] Add mlx5 direct verbs

[Yishai Hadas]

On 2/2/2017 10:52 AM, Sagi Grimberg wrote:
>
>> This patchset from Leon adds direct access to mlx5 devices.
>>
>> The libibverbs API is an abstract API. It is agnostic to any underlying
>> provider specific implementation. While this abstraction has the
>> advantage of
>> user applications portability it has a performance penalty. For some
>> applications optimizing performance is more important than portability.
>>
>> The mlx5 direct verbs API introduced in this patchset is intended for
>> such
>> applications. It exposes mlx5 specific low level data path
>> (send/receive/completion) operations, allowing the application to
>> bypass the
>> libibverbs data path API.
>
> Yishai and Leon this is good stuff !!

Thanks Sagi for pointing on, it's really appreciated.

> Couple of comments:
>
> - The user of this is pretty intimate with the driver, which makes a
>   somewhat strange combo of having both ibv_ and mlx5dv_ API. I
>   understand that it can inflate the the implementation, but perhaps
>   for completeness we can have also the control plane API as well (even
>   as a trampoline to the generic equivalent where it's useless).
>
>   The rational here is that extended features that needs allocation
>   flags can be contained to mlx5 and not pollute/inflate the generic
>   verbs implementation.

This series opens the door for optimizing the data path flows by 
exposing the internal driver stuff that is required in the general data 
path flow. It enables an application to optimize its own specific flow 
without considering the general case which involves branches that are 
not relevant to its use case.

Re control path:
Extending control path for driver specific configuration is really 
something that exists in our plans. It may be achieved by extending the 
above direct channel in a second phase with some extra APIs or by 
introducing some vendor channel mechanism to enable that on top of 
current control verbs. I believe that triggering that will be some 
feature request or a specific need that will come from the 
field/customers that will require this functionality.

> - This brings me to the next step of this. What is still missing is
>   how we allow the user to have more control and information about how
>   his db records (namely UARs) are allocated

Are you referring to some specific information that exists in mlx5 
driver which was not exposed by the direct API ? please have a look in 
dv.h for mlx5dv_init_obj and in mlx5 driver code and let us know if we 
missed something useful, we can add it.

Please consider that the above API was introduced with some extension 
mechanism (i.e. comp_mask) so that we can extend it in the future 
without breaking compatibility with applications that will use it.

>. How does the user know if he needs to protect the DB write?

Once an application uses the direct flow the locking scheme is fully 
managed by itself. We don't expect mixed flows in the data path for both 
direct/non-direct access by same application.

 > Is it BF?

This information is based on mlx5dv_qp->bf->size, in case of 0 it's not 
a BF. Please note that as part of V1 we plan to introduce a man page for 
mlx5 direct verbs which may point on that, in addition to other notes 
that I pointed on above.

>   How can the user ask for more UARs than originally allocated to it by the kernel?
>   The user of this can really use an API that allows more control
>   of this area.

This is some functionality that currently doesn't exist even in the 
*non* direct mode. It should be considered as some feature request 
regardless the above series. Please note that moving forward with this 
series and concept opens the door to give such an API which is some 
vendor specific more quickly and easily.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core 7/7] mlx5: Export mlx5 direct verbs interface

[Christoph Hellwig]

On Thu, Jan 26, 2017 at 10:30:05AM -0700, Jason Gunthorpe wrote:
> No, this is no longer just a provider so it doesn't get to use
> provider.map
> 
> mlx5 will need its own map file and the various magic I described to
> become a proper library and a plugin at once.

Honestly, I don't think it should be both at once.  The library for
direct access needs to be in /usr/lib and the provider hidden in the
providers directory, and not export anything not in the providers API.

I think what needs to happen is a new library for the direct access,
and the provider should be layered on top of it.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core 7/7] mlx5: Export mlx5 direct verbs interface

[Jason Gunthorpe]

On Fri, Feb 03, 2017 at 08:04:19AM -0800, Christoph Hellwig wrote:
> On Thu, Jan 26, 2017 at 10:30:05AM -0700, Jason Gunthorpe wrote:
> > No, this is no longer just a provider so it doesn't get to use
> > provider.map
> > 
> > mlx5 will need its own map file and the various magic I described to
> > become a proper library and a plugin at once.
> 
> Honestly, I don't think it should be both at once.  The library for
> direct access needs to be in /usr/lib and the provider hidden in the
> providers directory, and not export anything not in the providers API.

Provider libraries don't export any symbols at all. That is the main
reason having a library be both could potentially work out OK.. 

But it would have to be a proper /usr/lib/../ library first, and be
symlinked to a provider lib name second... And they will have to make
sure that dlopen figures this out and only loads one copy..

> I think what needs to happen is a new library for the direct access,
> and the provider should be layered on top of it.

I suspect that might just make a bigger mess, it looks like there is
some sharing of internal statics and what not - so the two libraries
would end up having their own private symbol interface, which is
looking like be a worse situation then the above...

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH rdma-core 0/7] Add mlx5 direct verbs

[Amrani, Ram]

> This patchset from Leon adds direct access to mlx5 devices.
> 
> The libibverbs API is an abstract API. It is agnostic to any underlying
> provider specific implementation. While this abstraction has the advantage of
> user applications portability it has a performance penalty. For some
> applications optimizing performance is more important than portability.
> 
> The mlx5 direct verbs API introduced in this patchset is intended for such
> applications. It exposes mlx5 specific low level data path
> (send/receive/completion) operations, allowing the application to bypass the
> libibverbs data path API.
> 
> The proposed interface consists from small number of hardware specific headers
> with relevant inline functions and conversion logic from ibverbs structures to
> mlx5 related structures.
>

Hi Yishay,
While I'm always for better performance, I'm not sure driver specific APIs is the way.
This will hurt portability for users.

Thanks,
Ram


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core 0/7] Add mlx5 direct verbs

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1458 bytes --]

On Mon, Feb 06, 2017 at 10:56:56AM +0000, Amrani, Ram wrote:
> > This patchset from Leon adds direct access to mlx5 devices.
> >
> > The libibverbs API is an abstract API. It is agnostic to any underlying
> > provider specific implementation. While this abstraction has the advantage of
> > user applications portability it has a performance penalty. For some
> > applications optimizing performance is more important than portability.
> >
> > The mlx5 direct verbs API introduced in this patchset is intended for such
> > applications. It exposes mlx5 specific low level data path
> > (send/receive/completion) operations, allowing the application to bypass the
> > libibverbs data path API.
> >
> > The proposed interface consists from small number of hardware specific headers
> > with relevant inline functions and conversion logic from ibverbs structures to
> > mlx5 related structures.
> >
>
> Hi Yishay,
> While I'm always for better performance, I'm not sure driver specific APIs is the way.
> This will hurt portability for users.

Hi Ram,

As it was expressed in cover letter, this feature is intended for users
who want performance and understand the disadvantages of such direct
approach.

Thanks

>
> Thanks,
> Ram
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH rdma-core 0/7] Add mlx5 direct verbs

[Jason Gunthorpe]

On Mon, Feb 06, 2017 at 01:46:09PM +0200, Leon Romanovsky wrote:
> On Mon, Feb 06, 2017 at 10:56:56AM +0000, Amrani, Ram wrote:
> > > This patchset from Leon adds direct access to mlx5 devices.
> > >
> > > The libibverbs API is an abstract API. It is agnostic to any underlying
> > > provider specific implementation. While this abstraction has the advantage of
> > > user applications portability it has a performance penalty. For some
> > > applications optimizing performance is more important than portability.
> > >
> > > The mlx5 direct verbs API introduced in this patchset is intended for such
> > > applications. It exposes mlx5 specific low level data path
> > > (send/receive/completion) operations, allowing the application to bypass the
> > > libibverbs data path API.
> > >
> > > The proposed interface consists from small number of hardware specific headers
> > > with relevant inline functions and conversion logic from ibverbs structures to
> > > mlx5 related structures.
> > >

> > While I'm always for better performance, I'm not sure driver specific APIs is the way.
> > This will hurt portability for users.
> 
> As it was expressed in cover letter, this feature is intended for
> users who want performance and understand the disadvantages of such
> direct approach.

To expound on that - the trade off we discussed in-person was we
either do something like this or continually take endless
vendor-specific patches to add 'common' core code for unstandardized
chip-specific features.

This seemed like the least bad trade off, if another vendor wants the
same things then they could work together and make an actual
multi-chip common API. I was expecting this interface to be used in
only a few places, like the next level of middleware libraries
(libfabric, mxm, etc) not for many end-users.

I'm still not completely sure what this API is intended to do, if it is
really just performance then I am less supportive and would rather see
an approach like libfabric where a user can build a single provider verbs
with the same source API, but optimized....

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core 0/7] Add mlx5 direct verbs

[Sagi Grimberg]

Hey Jason,

> To expound on that - the trade off we discussed in-person was we
> either do something like this or continually take endless
> vendor-specific patches to add 'common' core code for unstandardized
> chip-specific features.
>
> This seemed like the least bad trade off, if another vendor wants the
> same things then they could work together and make an actual
> multi-chip common API. I was expecting this interface to be used in
> only a few places, like the next level of middleware libraries
> (libfabric, mxm, etc) not for many end-users.

I think this is exactly the use-case. Something that looks like a
user-space driver on the back-side of a common API.

> I'm still not completely sure what this API is intended to do, if it is
> really just performance then I am less supportive and would rather see
> an approach like libfabric where a user can build a single provider verbs
> with the same source API, but optimized....

I'm also not in favor for having this code-path solely for performance
reasons, if the common stack is not good enough, let's fix it. What I
think this is trying to solve is the continuous inflation of
non-standard extensions to the common API.

These features raised constant complaints in code review before, which
in turn caused the vendors to place their extensions in an internal
packages (e.g. MLNX_OFED) and also educate their customers to use their
packaged drivers (which usually broke the rest of the world).

So I can understand Mellanox wanting to expose vendor specific
capabilities and move away from their in-house distribution of
user-space packages, and I think we should all encourage it.

And, this is also why I think that another set of a common API is
a good choice.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html