* [PATCH rdma-core 0/7] Add mlx4 direct verbs
@ 2017-06-29 11:47 Yishai Hadas
[not found] ` <1498736828-17875-1-git-send-email-yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
0 siblings, 1 reply; 9+ messages in thread
From: Yishai Hadas @ 2017-06-29 11:47 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, yishaih-VPRAkNaXOzVWk0Htik3J/w,
maorg-VPRAkNaXOzVWk0Htik3J/w, majd-VPRAkNaXOzVWk0Htik3J/w
This patchset from Maor adds direct access to mlx4 devices, it follows same
solution from both API and packaging perspectives as was already accepted for
mlx5.
The libibverbs API is an abstract API. It is agnostic to any underlying
provider specific implementation. While this abstraction has the advantage of
user applications portability it has a performance penalty. For some
applications optimizing performance is more important than portability.
The mlx4 direct verbs API introduced in this patchset is intended for such
applications. It exposes mlx4 specific low level data path
(send/receive/completion) operations, allowing the application to bypass the
libibverbs data path API.
The proposed interface consists from one hardware specific header with relevant
inline functions and conversion logic from ibverbs structures to mlx4 related
structures.
New header is going to be installed in below path location:
/usr/inlcude/infiniband/mlx4dv.h
The direct include of mlx4dv.h together with linkage to mlx4 library will
allow usage of this new interface.
Pull request was sent:
https://github.com/linux-rdma/rdma-core/pull/155
Yishai
Maor Gottlieb (7):
mlx4: Add mlx4 direct verbs
mlx4: Add CQE fields
mlx4: Convert datagram segment to use mlx4_av struct
mlx4: Add WQE segments implementation
mlx4: Convert to use predefined get CQE opcodes
mlx4: Add direct verbs man pages
mlx4: Export mlx4 direct verbs interface
CMakeLists.txt | 1 +
debian/ibverbs-providers.install | 1 +
debian/ibverbs-providers.symbols | 4 +
debian/libibverbs-dev.install | 4 +
providers/mlx4/CMakeLists.txt | 7 +-
providers/mlx4/cq.c | 65 +----
providers/mlx4/libmlx4.map | 7 +
providers/mlx4/man/CMakeLists.txt | 4 +
providers/mlx4/man/mlx4dv.7 | 39 +++
providers/mlx4/man/mlx4dv_init_obj.3 | 117 +++++++++
providers/mlx4/mlx4.c | 76 ++++++
providers/mlx4/mlx4.h | 73 +-----
providers/mlx4/mlx4dv.h | 465 +++++++++++++++++++++++++++++++++++
providers/mlx4/qp.c | 3 +-
providers/mlx4/srq.c | 1 -
providers/mlx4/verbs.c | 1 -
providers/mlx4/wqe.h | 149 -----------
redhat/rdma-core.spec | 3 +
18 files changed, 743 insertions(+), 277 deletions(-)
create mode 100644 providers/mlx4/libmlx4.map
create mode 100644 providers/mlx4/man/CMakeLists.txt
create mode 100644 providers/mlx4/man/mlx4dv.7
create mode 100644 providers/mlx4/man/mlx4dv_init_obj.3
create mode 100644 providers/mlx4/mlx4dv.h
delete mode 100644 providers/mlx4/wqe.h
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 9+ messages in thread[parent not found: <1498736828-17875-1-git-send-email-yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>]
* [PATCH rdma-core 1/7] mlx4: Add mlx4 direct verbs [not found] ` <1498736828-17875-1-git-send-email-yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> @ 2017-06-29 11:47 ` Yishai Hadas 2017-06-29 11:47 ` [PATCH rdma-core 2/7] mlx4: Add CQE fields Yishai Hadas ` (6 subsequent siblings) 7 siblings, 0 replies; 9+ messages in thread From: Yishai Hadas @ 2017-06-29 11:47 UTC (permalink / raw) To: linux-rdma-u79uwXL29TY76Z2rM5mHXA Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, yishaih-VPRAkNaXOzVWk0Htik3J/w, maorg-VPRAkNaXOzVWk0Htik3J/w, majd-VPRAkNaXOzVWk0Htik3J/w From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Direct verbs feature provides fast data path execution by implementing verbs directly over hardware specification. It consists from the two parts: * Exported functions to translate internal mlx4 structures to be accessible by user applications. * Inline functions, enums and defines to simplify programmer's life. This patch adds one header file (mlx4dv.h) with general, CQE and WQE data. Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> --- providers/mlx4/cq.c | 47 +------ providers/mlx4/mlx4.c | 76 +++++++++++ providers/mlx4/mlx4.h | 60 +-------- providers/mlx4/mlx4dv.h | 338 ++++++++++++++++++++++++++++++++++++++++++++++++ providers/mlx4/qp.c | 1 - providers/mlx4/srq.c | 1 - providers/mlx4/verbs.c | 1 - providers/mlx4/wqe.h | 149 --------------------- 8 files changed, 420 insertions(+), 253 deletions(-) create mode 100644 providers/mlx4/mlx4dv.h delete mode 100644 providers/mlx4/wqe.h diff --git a/providers/mlx4/cq.c b/providers/mlx4/cq.c index 8ebc2ae..068e0d2 100644 --- a/providers/mlx4/cq.c +++ b/providers/mlx4/cq.c @@ -46,55 +46,11 @@ #include "mlx4.h" enum { - MLX4_CQ_DOORBELL = 0x20 -}; - -enum { CQ_OK = 0, CQ_EMPTY = -1, CQ_POLL_ERR = -2 }; -#define MLX4_CQ_DB_REQ_NOT_SOL (1 << 24) -#define MLX4_CQ_DB_REQ_NOT (2 << 24) - -enum { - MLX4_CQE_VLAN_PRESENT_MASK = 1 << 29, - MLX4_CQE_QPN_MASK = 0xffffff, -}; - -enum { - MLX4_CQE_OWNER_MASK = 0x80, - MLX4_CQE_IS_SEND_MASK = 0x40, - MLX4_CQE_OPCODE_MASK = 0x1f -}; - -enum { - MLX4_CQE_SYNDROME_LOCAL_LENGTH_ERR = 0x01, - MLX4_CQE_SYNDROME_LOCAL_QP_OP_ERR = 0x02, - MLX4_CQE_SYNDROME_LOCAL_PROT_ERR = 0x04, - MLX4_CQE_SYNDROME_WR_FLUSH_ERR = 0x05, - MLX4_CQE_SYNDROME_MW_BIND_ERR = 0x06, - MLX4_CQE_SYNDROME_BAD_RESP_ERR = 0x10, - MLX4_CQE_SYNDROME_LOCAL_ACCESS_ERR = 0x11, - MLX4_CQE_SYNDROME_REMOTE_INVAL_REQ_ERR = 0x12, - MLX4_CQE_SYNDROME_REMOTE_ACCESS_ERR = 0x13, - MLX4_CQE_SYNDROME_REMOTE_OP_ERR = 0x14, - MLX4_CQE_SYNDROME_TRANSPORT_RETRY_EXC_ERR = 0x15, - MLX4_CQE_SYNDROME_RNR_RETRY_EXC_ERR = 0x16, - MLX4_CQE_SYNDROME_REMOTE_ABORTED_ERR = 0x22, -}; - -struct mlx4_err_cqe { - uint32_t vlan_my_qpn; - uint32_t reserved1[5]; - uint16_t wqe_index; - uint8_t vendor_err; - uint8_t syndrome; - uint8_t reserved2[3]; - uint8_t owner_sr_opcode; -}; - static struct mlx4_cqe *get_cqe(struct mlx4_cq *cq, int entry) { return cq->buf.buf + entry * cq->cqe_size; @@ -722,6 +678,9 @@ void __mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) int nfreed = 0; int cqe_inc = cq->cqe_size == 64 ? 1 : 0; + if (!cq || cq->flags & MLX4_CQ_FLAGS_DV_OWNED) + return; + /* * First we need to find the current producer index, so we * know where to start cleaning from. It doesn't matter if HW diff --git a/providers/mlx4/mlx4.c b/providers/mlx4/mlx4.c index b798b37..db3f903 100644 --- a/providers/mlx4/mlx4.c +++ b/providers/mlx4/mlx4.c @@ -324,3 +324,79 @@ static __attribute__((constructor)) void mlx4_register_driver(void) { verbs_register_driver("mlx4", mlx4_driver_init); } + +static int mlx4dv_get_qp(struct ibv_qp *qp_in, + struct mlx4dv_qp *qp_out) +{ + struct mlx4_qp *mqp = to_mqp(qp_in); + struct mlx4_context *ctx = to_mctx(qp_in->context); + + qp_out->comp_mask = 0; + + qp_out->buf.buf = mqp->buf.buf; + qp_out->buf.length = mqp->buf.length; + + qp_out->rdb = mqp->db; + qp_out->sdb = (uint32_t *) (ctx->uar + MLX4_SEND_DOORBELL); + qp_out->doorbell_qpn = mqp->doorbell_qpn; + + qp_out->sq.wqe_cnt = mqp->sq.wqe_cnt; + qp_out->sq.wqe_shift = mqp->sq.wqe_shift; + qp_out->sq.offset = mqp->sq.offset; + + qp_out->rq.wqe_cnt = mqp->rq.wqe_cnt; + qp_out->rq.wqe_shift = mqp->rq.wqe_shift; + qp_out->rq.offset = mqp->rq.offset; + + return 0; +} + +static int mlx4dv_get_cq(struct ibv_cq *cq_in, + struct mlx4dv_cq *cq_out) +{ + struct mlx4_cq *mcq = to_mcq(cq_in); + + cq_out->comp_mask = 0; + cq_out->buf.buf = mcq->buf.buf; + cq_out->buf.length = mcq->buf.length; + cq_out->cqn = mcq->cqn; + cq_out->set_ci_db = mcq->set_ci_db; + cq_out->arm_db = mcq->arm_db; + cq_out->arm_sn = mcq->arm_sn; + cq_out->cqe_size = mcq->cqe_size; + cq_out->cqe_cnt = mcq->ibv_cq.cqe + 1; + + mcq->flags |= MLX4_CQ_FLAGS_DV_OWNED; + + return 0; +} + +static int mlx4dv_get_srq(struct ibv_srq *srq_in, + struct mlx4dv_srq *srq_out) +{ + struct mlx4_srq *msrq = to_msrq(srq_in); + + srq_out->comp_mask = 0; + srq_out->buf.buf = msrq->buf.buf; + srq_out->buf.length = msrq->buf.length; + srq_out->wqe_shift = msrq->wqe_shift; + srq_out->head = msrq->head; + srq_out->tail = msrq->tail; + srq_out->db = msrq->db; + + return 0; +} + +int mlx4dv_init_obj(struct mlx4dv_obj *obj, uint64_t obj_type) +{ + int ret = 0; + + if (obj_type & MLX4DV_OBJ_QP) + ret = mlx4dv_get_qp(obj->qp.in, obj->qp.out); + if (!ret && (obj_type & MLX4DV_OBJ_CQ)) + ret = mlx4dv_get_cq(obj->cq.in, obj->cq.out); + if (!ret && (obj_type & MLX4DV_OBJ_SRQ)) + ret = mlx4dv_get_srq(obj->srq.in, obj->srq.out); + + return ret; +} diff --git a/providers/mlx4/mlx4.h b/providers/mlx4/mlx4.h index b4f6e86..8f0d6f2 100644 --- a/providers/mlx4/mlx4.h +++ b/providers/mlx4/mlx4.h @@ -42,6 +42,8 @@ #include <util/udma_barrier.h> #include <infiniband/verbs.h> +#include "mlx4dv.h" + #define MLX4_PORTS_NUM 2 #include <valgrind/memcheck.h> @@ -88,33 +90,6 @@ enum mlx4_db_type { MLX4_NUM_DB_TYPE }; -enum { - MLX4_OPCODE_NOP = 0x00, - MLX4_OPCODE_SEND_INVAL = 0x01, - MLX4_OPCODE_RDMA_WRITE = 0x08, - MLX4_OPCODE_RDMA_WRITE_IMM = 0x09, - MLX4_OPCODE_SEND = 0x0a, - MLX4_OPCODE_SEND_IMM = 0x0b, - MLX4_OPCODE_LSO = 0x0e, - MLX4_OPCODE_RDMA_READ = 0x10, - MLX4_OPCODE_ATOMIC_CS = 0x11, - MLX4_OPCODE_ATOMIC_FA = 0x12, - MLX4_OPCODE_MASKED_ATOMIC_CS = 0x14, - MLX4_OPCODE_MASKED_ATOMIC_FA = 0x15, - MLX4_OPCODE_BIND_MW = 0x18, - MLX4_OPCODE_FMR = 0x19, - MLX4_OPCODE_LOCAL_INVAL = 0x1b, - MLX4_OPCODE_CONFIG_CMD = 0x1f, - - MLX4_RECV_OPCODE_RDMA_WRITE_IMM = 0x00, - MLX4_RECV_OPCODE_SEND = 0x01, - MLX4_RECV_OPCODE_SEND_IMM = 0x02, - MLX4_RECV_OPCODE_SEND_INVAL = 0x03, - - MLX4_CQE_OPCODE_ERROR = 0x1e, - MLX4_CQE_OPCODE_RESIZE = 0x16, -}; - struct mlx4_device { struct verbs_device verbs_dev; int page_size; @@ -174,6 +149,7 @@ enum { MLX4_CQ_FLAGS_RX_CSUM_VALID = 1 << 0, MLX4_CQ_FLAGS_EXTENDED = 1 << 1, MLX4_CQ_FLAGS_SINGLE_THREADED = 1 << 2, + MLX4_CQ_FLAGS_DV_OWNED = 1 << 3, }; struct mlx4_cq { @@ -265,36 +241,6 @@ enum { MLX4_RX_CSUM_VALID = (1 << 16), }; -enum mlx4_cqe_status { - MLX4_CQE_STATUS_TCP_UDP_CSUM_OK = (1 << 2), - MLX4_CQE_STATUS_IPV4_PKT = (1 << 22), - MLX4_CQE_STATUS_IP_HDR_CSUM_OK = (1 << 28), - MLX4_CQE_STATUS_IPV4_CSUM_OK = MLX4_CQE_STATUS_IPV4_PKT | - MLX4_CQE_STATUS_IP_HDR_CSUM_OK | - MLX4_CQE_STATUS_TCP_UDP_CSUM_OK -}; - -struct mlx4_cqe { - uint32_t vlan_my_qpn; - uint32_t immed_rss_invalid; - uint32_t g_mlpath_rqpn; - union { - struct { - uint16_t sl_vid; - uint16_t rlid; - }; - uint32_t ts_47_16; - }; - uint32_t status; - uint32_t byte_cnt; - uint16_t wqe_index; - uint16_t checksum; - uint8_t reserved3; - uint8_t ts_15_8; - uint8_t ts_7_0; - uint8_t owner_sr_opcode; -}; - static inline unsigned long align(unsigned long val, unsigned long align) { return (val + align - 1) & ~(align - 1); diff --git a/providers/mlx4/mlx4dv.h b/providers/mlx4/mlx4dv.h new file mode 100644 index 0000000..86b4e13 --- /dev/null +++ b/providers/mlx4/mlx4dv.h @@ -0,0 +1,338 @@ +/* + * Copyright (c) 2017 Mellanox Technologies, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +#ifndef _MLX4DV_H_ +#define _MLX4DV_H_ + +#include <linux/types.h> +#include <endian.h> +#include <infiniband/verbs.h> + +enum { + MLX4_OPCODE_NOP = 0x00, + MLX4_OPCODE_SEND_INVAL = 0x01, + MLX4_OPCODE_RDMA_WRITE = 0x08, + MLX4_OPCODE_RDMA_WRITE_IMM = 0x09, + MLX4_OPCODE_SEND = 0x0a, + MLX4_OPCODE_SEND_IMM = 0x0b, + MLX4_OPCODE_LSO = 0x0e, + MLX4_OPCODE_RDMA_READ = 0x10, + MLX4_OPCODE_ATOMIC_CS = 0x11, + MLX4_OPCODE_ATOMIC_FA = 0x12, + MLX4_OPCODE_MASKED_ATOMIC_CS = 0x14, + MLX4_OPCODE_MASKED_ATOMIC_FA = 0x15, + MLX4_OPCODE_BIND_MW = 0x18, + MLX4_OPCODE_FMR = 0x19, + MLX4_OPCODE_LOCAL_INVAL = 0x1b, + MLX4_OPCODE_CONFIG_CMD = 0x1f, + + MLX4_RECV_OPCODE_RDMA_WRITE_IMM = 0x00, + MLX4_RECV_OPCODE_SEND = 0x01, + MLX4_RECV_OPCODE_SEND_IMM = 0x02, + MLX4_RECV_OPCODE_SEND_INVAL = 0x03, + + MLX4_CQE_OPCODE_ERROR = 0x1e, + MLX4_CQE_OPCODE_RESIZE = 0x16, +}; + +enum { + MLX4_CQ_DOORBELL = 0x20 +}; + +#define MLX4_CQ_DB_REQ_NOT_SOL (1 << 24) +#define MLX4_CQ_DB_REQ_NOT (2 << 24) + +enum { + MLX4_CQE_VLAN_PRESENT_MASK = 1 << 29, + MLX4_CQE_QPN_MASK = 0xffffff, +}; + +enum { + MLX4_CQE_OWNER_MASK = 0x80, + MLX4_CQE_IS_SEND_MASK = 0x40, + MLX4_CQE_OPCODE_MASK = 0x1f +}; + +enum { + MLX4_CQE_SYNDROME_LOCAL_LENGTH_ERR = 0x01, + MLX4_CQE_SYNDROME_LOCAL_QP_OP_ERR = 0x02, + MLX4_CQE_SYNDROME_LOCAL_PROT_ERR = 0x04, + MLX4_CQE_SYNDROME_WR_FLUSH_ERR = 0x05, + MLX4_CQE_SYNDROME_MW_BIND_ERR = 0x06, + MLX4_CQE_SYNDROME_BAD_RESP_ERR = 0x10, + MLX4_CQE_SYNDROME_LOCAL_ACCESS_ERR = 0x11, + MLX4_CQE_SYNDROME_REMOTE_INVAL_REQ_ERR = 0x12, + MLX4_CQE_SYNDROME_REMOTE_ACCESS_ERR = 0x13, + MLX4_CQE_SYNDROME_REMOTE_OP_ERR = 0x14, + MLX4_CQE_SYNDROME_TRANSPORT_RETRY_EXC_ERR = 0x15, + MLX4_CQE_SYNDROME_RNR_RETRY_EXC_ERR = 0x16, + MLX4_CQE_SYNDROME_REMOTE_ABORTED_ERR = 0x22, +}; + +struct mlx4_err_cqe { + uint32_t vlan_my_qpn; + uint32_t reserved1[5]; + uint16_t wqe_index; + uint8_t vendor_err; + uint8_t syndrome; + uint8_t reserved2[3]; + uint8_t owner_sr_opcode; +}; + +enum mlx4_cqe_status { + MLX4_CQE_STATUS_TCP_UDP_CSUM_OK = (1 << 2), + MLX4_CQE_STATUS_IPV4_PKT = (1 << 22), + MLX4_CQE_STATUS_IP_HDR_CSUM_OK = (1 << 28), + MLX4_CQE_STATUS_IPV4_CSUM_OK = MLX4_CQE_STATUS_IPV4_PKT | + MLX4_CQE_STATUS_IP_HDR_CSUM_OK | + MLX4_CQE_STATUS_TCP_UDP_CSUM_OK +}; + +struct mlx4_cqe { + uint32_t vlan_my_qpn; + uint32_t immed_rss_invalid; + uint32_t g_mlpath_rqpn; + union { + struct { + uint16_t sl_vid; + uint16_t rlid; + }; + uint32_t ts_47_16; + }; + uint32_t status; + uint32_t byte_cnt; + uint16_t wqe_index; + uint16_t checksum; + uint8_t reserved3; + uint8_t ts_15_8; + uint8_t ts_7_0; + uint8_t owner_sr_opcode; +}; + +struct mlx4dv_qp { + uint32_t *rdb; + uint32_t *sdb; + uint32_t doorbell_qpn; + struct { + uint32_t wqe_cnt; + int wqe_shift; + int offset; + } sq; + struct { + uint32_t wqe_cnt; + int wqe_shift; + int offset; + } rq; + struct { + void *buf; + size_t length; + } buf; + uint64_t comp_mask; +}; + +struct mlx4dv_cq { + struct { + void *buf; + size_t length; + } buf; + uint32_t cqe_cnt; + uint32_t cqn; + uint32_t *set_ci_db; + uint32_t *arm_db; + int arm_sn; + int cqe_size; + uint64_t comp_mask; +}; +struct mlx4dv_srq { + struct { + void *buf; + size_t length; + } buf; + int wqe_shift; + int head; + int tail; + uint32_t *db; + uint64_t comp_mask; +}; + +struct mlx4dv_obj { + struct { + struct ibv_qp *in; + struct mlx4dv_qp *out; + } qp; + struct { + struct ibv_cq *in; + struct mlx4dv_cq *out; + } cq; + struct { + struct ibv_srq *in; + struct mlx4dv_srq *out; + } srq; +}; + +enum mlx4dv_obj_type { + MLX4DV_OBJ_QP = 1 << 0, + MLX4DV_OBJ_CQ = 1 << 1, + MLX4DV_OBJ_SRQ = 1 << 2, +}; + +/* + * This function will initialize mlx4dv_xxx structs based on supplied type. + * The information for initialization is taken from ibv_xx structs supplied + * as part of input. + * + * Request information of CQ marks its owned by DV for all consumer index + * related actions. + * + * The initialization type can be combination of several types together. + * + * Return: 0 in case of success. + */ +int mlx4dv_init_obj(struct mlx4dv_obj *obj, uint64_t obj_type); + +/* + * WQE related part + */ + +enum { + MLX4_SEND_DOORBELL = 0x14, +}; + +enum { + MLX4_WQE_CTRL_SOLICIT = 1 << 1, + MLX4_WQE_CTRL_CQ_UPDATE = 3 << 2, + MLX4_WQE_CTRL_IP_HDR_CSUM = 1 << 4, + MLX4_WQE_CTRL_TCP_UDP_CSUM = 1 << 5, + MLX4_WQE_CTRL_FENCE = 1 << 6, + MLX4_WQE_CTRL_STRONG_ORDER = 1 << 7 +}; + +enum { + MLX4_WQE_BIND_TYPE_2 = (1<<31), + MLX4_WQE_BIND_ZERO_BASED = (1<<30), +}; + +enum { + MLX4_INLINE_SEG = 1 << 31, + MLX4_INLINE_ALIGN = 64, +}; + +enum { + MLX4_INVALID_LKEY = 0x100, +}; + +enum { + MLX4_WQE_MW_REMOTE_READ = 1 << 29, + MLX4_WQE_MW_REMOTE_WRITE = 1 << 30, + MLX4_WQE_MW_ATOMIC = 1 << 31 +}; + +struct mlx4_wqe_local_inval_seg { + uint64_t reserved1; + uint32_t mem_key; + uint32_t reserved2; + uint64_t reserved3[2]; +}; + +struct mlx4_wqe_bind_seg { + uint32_t flags1; + uint32_t flags2; + uint32_t new_rkey; + uint32_t lkey; + uint64_t addr; + uint64_t length; +}; + +struct mlx4_wqe_ctrl_seg { + uint32_t owner_opcode; + union { + struct { + uint8_t reserved[3]; + uint8_t fence_size; + }; + uint32_t bf_qpn; + }; + /* + * High 24 bits are SRC remote buffer; low 8 bits are flags: + * [7] SO (strong ordering) + * [5] TCP/UDP checksum + * [4] IP checksum + * [3:2] C (generate completion queue entry) + * [1] SE (solicited event) + * [0] FL (force loopback) + */ + uint32_t srcrb_flags; + /* + * imm is immediate data for send/RDMA write w/ immediate; + * also invalidation key for send with invalidate; input + * modifier for WQEs on CCQs. + */ + uint32_t imm; +}; + +struct mlx4_wqe_datagram_seg { + uint32_t av[8]; + uint32_t dqpn; + uint32_t qkey; + uint16_t vlan; + uint8_t mac[6]; +}; + +struct mlx4_wqe_data_seg { + uint32_t byte_count; + uint32_t lkey; + uint64_t addr; +}; + +struct mlx4_wqe_inline_seg { + uint32_t byte_count; +}; + +struct mlx4_wqe_srq_next_seg { + uint16_t reserved1; + uint16_t next_wqe_index; + uint32_t reserved2[3]; +}; + +struct mlx4_wqe_raddr_seg { + uint64_t raddr; + uint32_t rkey; + uint32_t reserved; +}; + +struct mlx4_wqe_atomic_seg { + uint64_t swap_add; + uint64_t compare; +}; + +#endif /* _MLX4DV_H_ */ + diff --git a/providers/mlx4/qp.c b/providers/mlx4/qp.c index e7f10b9..b7e3669 100644 --- a/providers/mlx4/qp.c +++ b/providers/mlx4/qp.c @@ -42,7 +42,6 @@ #include <util/compiler.h> #include "mlx4.h" -#include "wqe.h" static const uint32_t mlx4_ib_opcode[] = { [IBV_WR_SEND] = MLX4_OPCODE_SEND, diff --git a/providers/mlx4/srq.c b/providers/mlx4/srq.c index f30cc2e..9c566cd 100644 --- a/providers/mlx4/srq.c +++ b/providers/mlx4/srq.c @@ -37,7 +37,6 @@ #include <string.h> #include "mlx4.h" -#include "wqe.h" #include "mlx4-abi.h" static void *get_wqe(struct mlx4_srq *srq, int n) diff --git a/providers/mlx4/verbs.c b/providers/mlx4/verbs.c index 80efd9a..fce6f22 100644 --- a/providers/mlx4/verbs.c +++ b/providers/mlx4/verbs.c @@ -43,7 +43,6 @@ #include "mlx4.h" #include "mlx4-abi.h" -#include "wqe.h" int mlx4_query_device(struct ibv_context *context, struct ibv_device_attr *attr) { diff --git a/providers/mlx4/wqe.h b/providers/mlx4/wqe.h deleted file mode 100644 index 6f833d9..0000000 --- a/providers/mlx4/wqe.h +++ /dev/null @@ -1,149 +0,0 @@ -/* - * Copyright (c) 2007 Cisco, Inc. All rights reserved. - * - * This software is available to you under a choice of one of two - * licenses. You may choose to be licensed under the terms of the GNU - * General Public License (GPL) Version 2, available from the file - * COPYING in the main directory of this source tree, or the - * OpenIB.org BSD license below: - * - * Redistribution and use in source and binary forms, with or - * without modification, are permitted provided that the following - * conditions are met: - * - * - Redistributions of source code must retain the above - * copyright notice, this list of conditions and the following - * disclaimer. - * - * - Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following - * disclaimer in the documentation and/or other materials - * provided with the distribution. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, - * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND - * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS - * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN - * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN - * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE - * SOFTWARE. - */ - -#ifndef WQE_H -#define WQE_H - -#include <stdint.h> - -enum { - MLX4_SEND_DOORBELL = 0x14, -}; - -enum { - MLX4_WQE_CTRL_SOLICIT = 1 << 1, - MLX4_WQE_CTRL_CQ_UPDATE = 3 << 2, - MLX4_WQE_CTRL_IP_HDR_CSUM = 1 << 4, - MLX4_WQE_CTRL_TCP_UDP_CSUM = 1 << 5, - MLX4_WQE_CTRL_FENCE = 1 << 6, - MLX4_WQE_CTRL_STRONG_ORDER = 1 << 7 -}; - -enum { - MLX4_WQE_BIND_TYPE_2 = (1<<31), - MLX4_WQE_BIND_ZERO_BASED = (1<<30), -}; - -enum { - MLX4_INLINE_SEG = 1 << 31, - MLX4_INLINE_ALIGN = 64, -}; - -enum { - MLX4_INVALID_LKEY = 0x100, -}; - -struct mlx4_wqe_ctrl_seg { - uint32_t owner_opcode; - union { - struct { - uint8_t reserved[3]; - uint8_t fence_size; - }; - uint32_t bf_qpn; - }; - /* - * High 24 bits are SRC remote buffer; low 8 bits are flags: - * [7] SO (strong ordering) - * [5] TCP/UDP checksum - * [4] IP checksum - * [3:2] C (generate completion queue entry) - * [1] SE (solicited event) - * [0] FL (force loopback) - */ - uint32_t srcrb_flags; - /* - * imm is immediate data for send/RDMA write w/ immediate; - * also invalidation key for send with invalidate; input - * modifier for WQEs on CCQs. - */ - uint32_t imm; -}; - -struct mlx4_wqe_datagram_seg { - uint32_t av[8]; - uint32_t dqpn; - uint32_t qkey; - uint16_t vlan; - uint8_t mac[6]; -}; - -struct mlx4_wqe_data_seg { - uint32_t byte_count; - uint32_t lkey; - uint64_t addr; -}; - -struct mlx4_wqe_inline_seg { - uint32_t byte_count; -}; - -struct mlx4_wqe_srq_next_seg { - uint16_t reserved1; - uint16_t next_wqe_index; - uint32_t reserved2[3]; -}; - -struct mlx4_wqe_local_inval_seg { - uint64_t reserved1; - uint32_t mem_key; - uint32_t reserved2; - uint64_t reserved3[2]; -}; - -enum { - MLX4_WQE_MW_REMOTE_READ = 1 << 29, - MLX4_WQE_MW_REMOTE_WRITE = 1 << 30, - MLX4_WQE_MW_ATOMIC = 1 << 31 -}; - -struct mlx4_wqe_raddr_seg { - uint64_t raddr; - uint32_t rkey; - uint32_t reserved; -}; - -struct mlx4_wqe_atomic_seg { - uint64_t swap_add; - uint64_t compare; -}; - -struct mlx4_wqe_bind_seg { - uint32_t flags1; - uint32_t flags2; - uint32_t new_rkey; - uint32_t lkey; - uint64_t addr; - uint64_t length; -}; - -#endif /* WQE_H */ -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH rdma-core 2/7] mlx4: Add CQE fields [not found] ` <1498736828-17875-1-git-send-email-yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> 2017-06-29 11:47 ` [PATCH rdma-core 1/7] mlx4: " Yishai Hadas @ 2017-06-29 11:47 ` Yishai Hadas 2017-06-29 11:47 ` [PATCH rdma-core 3/7] mlx4: Convert datagram segment to use mlx4_av struct Yishai Hadas ` (5 subsequent siblings) 7 siblings, 0 replies; 9+ messages in thread From: Yishai Hadas @ 2017-06-29 11:47 UTC (permalink / raw) To: linux-rdma-u79uwXL29TY76Z2rM5mHXA Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, yishaih-VPRAkNaXOzVWk0Htik3J/w, maorg-VPRAkNaXOzVWk0Htik3J/w, majd-VPRAkNaXOzVWk0Htik3J/w From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Introduce inline calls to get/set for most commonly used CQE data path fields. Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> --- providers/mlx4/mlx4dv.h | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/providers/mlx4/mlx4dv.h b/providers/mlx4/mlx4dv.h index 86b4e13..e677fe1 100644 --- a/providers/mlx4/mlx4dv.h +++ b/providers/mlx4/mlx4dv.h @@ -37,6 +37,13 @@ #include <endian.h> #include <infiniband/verbs.h> +/* Always inline the functions */ +#ifdef __GNUC__ +#define MLX4DV_ALWAYS_INLINE inline __attribute__((always_inline)) +#else +#define MLX4DV_ALWAYS_INLINE inline +#endif + enum { MLX4_OPCODE_NOP = 0x00, MLX4_OPCODE_SEND_INVAL = 0x01, @@ -219,6 +226,25 @@ enum mlx4dv_obj_type { */ int mlx4dv_init_obj(struct mlx4dv_obj *obj, uint64_t obj_type); +static MLX4DV_ALWAYS_INLINE +uint8_t mlx4dv_get_cqe_owner(struct mlx4_cqe *cqe) +{ + return cqe->owner_sr_opcode & MLX4_CQE_OWNER_MASK; +} + +static MLX4DV_ALWAYS_INLINE +void mlx4dv_set_cqe_owner(struct mlx4_cqe *cqe, uint8_t val) +{ + cqe->owner_sr_opcode = (val & MLX4_CQE_OWNER_MASK) | + (cqe->owner_sr_opcode & ~MLX4_CQE_OWNER_MASK); +} + +static MLX4DV_ALWAYS_INLINE +uint8_t mlx4dv_get_cqe_opcode(struct mlx4_cqe *cqe) +{ + return cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK; +} + /* * WQE related part */ -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH rdma-core 3/7] mlx4: Convert datagram segment to use mlx4_av struct [not found] ` <1498736828-17875-1-git-send-email-yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> 2017-06-29 11:47 ` [PATCH rdma-core 1/7] mlx4: " Yishai Hadas 2017-06-29 11:47 ` [PATCH rdma-core 2/7] mlx4: Add CQE fields Yishai Hadas @ 2017-06-29 11:47 ` Yishai Hadas 2017-06-29 11:47 ` [PATCH rdma-core 4/7] mlx4: Add WQE segments implementation Yishai Hadas ` (4 subsequent siblings) 7 siblings, 0 replies; 9+ messages in thread From: Yishai Hadas @ 2017-06-29 11:47 UTC (permalink / raw) To: linux-rdma-u79uwXL29TY76Z2rM5mHXA Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, yishaih-VPRAkNaXOzVWk0Htik3J/w, maorg-VPRAkNaXOzVWk0Htik3J/w, majd-VPRAkNaXOzVWk0Htik3J/w From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> The datagram segment struct contains array which represents the address vector, convert the array to mlx4_av. mlx4dv will use this struct in downstream patches. Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> --- providers/mlx4/mlx4.h | 13 ------------- providers/mlx4/mlx4dv.h | 15 ++++++++++++++- providers/mlx4/qp.c | 2 +- 3 files changed, 15 insertions(+), 15 deletions(-) diff --git a/providers/mlx4/mlx4.h b/providers/mlx4/mlx4.h index 8f0d6f2..bdb7f16 100644 --- a/providers/mlx4/mlx4.h +++ b/providers/mlx4/mlx4.h @@ -214,19 +214,6 @@ struct mlx4_qp { uint32_t qp_cap_cache; }; -struct mlx4_av { - uint32_t port_pd; - uint8_t reserved1; - uint8_t g_slid; - uint16_t dlid; - uint8_t reserved2; - uint8_t gid_index; - uint8_t stat_rate; - uint8_t hop_limit; - uint32_t sl_tclass_flowlabel; - uint8_t dgid[16]; -}; - struct mlx4_ah { struct ibv_ah ibv_ah; struct mlx4_av av; diff --git a/providers/mlx4/mlx4dv.h b/providers/mlx4/mlx4dv.h index e677fe1..7679ece 100644 --- a/providers/mlx4/mlx4dv.h +++ b/providers/mlx4/mlx4dv.h @@ -325,8 +325,21 @@ struct mlx4_wqe_ctrl_seg { uint32_t imm; }; +struct mlx4_av { + uint32_t port_pd; + uint8_t reserved1; + uint8_t g_slid; + uint16_t dlid; + uint8_t reserved2; + uint8_t gid_index; + uint8_t stat_rate; + uint8_t hop_limit; + uint32_t sl_tclass_flowlabel; + uint8_t dgid[16]; +}; + struct mlx4_wqe_datagram_seg { - uint32_t av[8]; + struct mlx4_av av; uint32_t dqpn; uint32_t qkey; uint16_t vlan; diff --git a/providers/mlx4/qp.c b/providers/mlx4/qp.c index b7e3669..1be100b 100644 --- a/providers/mlx4/qp.c +++ b/providers/mlx4/qp.c @@ -175,7 +175,7 @@ static void set_atomic_seg(struct mlx4_wqe_atomic_seg *aseg, struct ibv_send_wr static void set_datagram_seg(struct mlx4_wqe_datagram_seg *dseg, struct ibv_send_wr *wr) { - memcpy(dseg->av, &to_mah(wr->wr.ud.ah)->av, sizeof (struct mlx4_av)); + memcpy(&dseg->av, &to_mah(wr->wr.ud.ah)->av, sizeof (struct mlx4_av)); dseg->dqpn = htobe32(wr->wr.ud.remote_qpn); dseg->qkey = htobe32(wr->wr.ud.remote_qkey); dseg->vlan = htobe16(to_mah(wr->wr.ud.ah)->vlan); -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH rdma-core 4/7] mlx4: Add WQE segments implementation [not found] ` <1498736828-17875-1-git-send-email-yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> ` (2 preceding siblings ...) 2017-06-29 11:47 ` [PATCH rdma-core 3/7] mlx4: Convert datagram segment to use mlx4_av struct Yishai Hadas @ 2017-06-29 11:47 ` Yishai Hadas 2017-06-29 11:47 ` [PATCH rdma-core 5/7] mlx4: Convert to use predefined get CQE opcodes Yishai Hadas ` (3 subsequent siblings) 7 siblings, 0 replies; 9+ messages in thread From: Yishai Hadas @ 2017-06-29 11:47 UTC (permalink / raw) To: linux-rdma-u79uwXL29TY76Z2rM5mHXA Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, yishaih-VPRAkNaXOzVWk0Htik3J/w, maorg-VPRAkNaXOzVWk0Htik3J/w, majd-VPRAkNaXOzVWk0Htik3J/w From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Add the following WQE segments implementation: 1. Control segment. 2. Datagram segment. 3. Data segment. Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> --- providers/mlx4/mlx4dv.h | 88 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) diff --git a/providers/mlx4/mlx4dv.h b/providers/mlx4/mlx4dv.h index 7679ece..f15c56c 100644 --- a/providers/mlx4/mlx4dv.h +++ b/providers/mlx4/mlx4dv.h @@ -373,5 +373,93 @@ struct mlx4_wqe_atomic_seg { uint64_t compare; }; +/* + * Control segment - contains some control information for the current WQE. + * + * Output: + * seg - control segment to be filled + * Input: + * opcode - Opcode of this WQE. Encodes the type of operation + * to be executed on the QP. + * wqe_cnt - Number of queue entries. + * ind - WQEBB number of the first block of this WQE. + * fence_size - Fence bit and WQE size in octowords. + * srcrb_flags - High 24 bits are SRC remote buffer; low 8 bits are + * flags which described in mlx4_wqe_ctrl_seg struct. + * imm - Immediate data/Invalidation key. + */ +static MLX4DV_ALWAYS_INLINE +void mlx4dv_set_ctrl_seg(struct mlx4_wqe_ctrl_seg *seg, uint8_t opcode, uint32_t + wqe_cnt, uint32_t ind, uint8_t fence_size, + uint32_t srcrb_flags, uint32_t imm) +{ + seg->owner_opcode = htobe32(opcode) | + (ind & wqe_cnt ? htobe32(1 << 31) : 0); + seg->fence_size = fence_size; + seg->srcrb_flags = srcrb_flags; + /* + * The caller should prepare "imm" in advance based on WR opcode. + * For IBV_WR_SEND_WITH_IMM and IBV_WR_RDMA_WRITE_WITH_IMM, + * the "imm" should be assigned as is. + * For the IBV_WR_SEND_WITH_INV, it should be htobe32(imm). + */ + seg->imm = imm; +} + +/* + * Datagram Segment - contains address information required in order + * to form a datagram message. + * + * Output: + * seg - datagram segment to be filled. + * Input: + * port_pd - Port number and protection domain. + * g_slid - GRH and source LID for IB port only. + * dlid - Remote LID. + * gid_index - Index to port GID table. + * state_rate - Maximum static rate control. + * hop_limit - IPv6 hop limit. + * sl_tclass_flowlabel - Service Level, IPv6 TClass and flow table. + * dgid - Remote GID for IB port only. + * dqpn - Destination QP. + * qkey - QKey. + * vlan - VLAN for RAW ETHERNET QP only. + * mac - Destination MAC for RAW ETHERNET QP only. + */ +static MLX4DV_ALWAYS_INLINE +void mlx4dv_set_dgram_seg(struct mlx4_wqe_datagram_seg *seg, uint32_t port_pd, + uint8_t g_slid, uint16_t dlid, uint8_t gid_index, + uint8_t stat_rate, uint8_t hop_limit, uint32_t + sl_tclass_flowlabel, uint8_t *dgid, uint32_t dqpn, + uint32_t qkey, uint16_t vlan, uint8_t *mac) +{ + seg->av.port_pd = htobe32(port_pd); + seg->av.g_slid = g_slid; + seg->av.dlid = htobe16(dlid); + seg->av.gid_index = gid_index; + seg->av.stat_rate = stat_rate; + seg->av.hop_limit = hop_limit; + seg->av.sl_tclass_flowlabel = htobe32(sl_tclass_flowlabel); + memcpy(seg->av.dgid, dgid, 16); + seg->dqpn = htobe32(dqpn); + seg->qkey = htobe32(qkey); + seg->vlan = htobe16(vlan); + memcpy(seg->mac, mac, 6); +} + +/* + * Data Segments - contain pointers and a byte count for the scatter/gather list. + * They can optionally contain data, which will save a memory read access for + * gather Work Requests. + */ +static MLX4DV_ALWAYS_INLINE +void mlx4dv_set_data_seg(struct mlx4_wqe_data_seg *seg, + uint32_t length, uint32_t lkey, + uintptr_t address) +{ + seg->byte_count = htobe32(length); + seg->lkey = htobe32(lkey); + seg->addr = htobe64(address); +} #endif /* _MLX4DV_H_ */ -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH rdma-core 5/7] mlx4: Convert to use predefined get CQE opcodes [not found] ` <1498736828-17875-1-git-send-email-yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> ` (3 preceding siblings ...) 2017-06-29 11:47 ` [PATCH rdma-core 4/7] mlx4: Add WQE segments implementation Yishai Hadas @ 2017-06-29 11:47 ` Yishai Hadas 2017-06-29 11:47 ` [PATCH rdma-core 6/7] mlx4: Add direct verbs man pages Yishai Hadas ` (2 subsequent siblings) 7 siblings, 0 replies; 9+ messages in thread From: Yishai Hadas @ 2017-06-29 11:47 UTC (permalink / raw) To: linux-rdma-u79uwXL29TY76Z2rM5mHXA Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, yishaih-VPRAkNaXOzVWk0Htik3J/w, maorg-VPRAkNaXOzVWk0Htik3J/w, majd-VPRAkNaXOzVWk0Htik3J/w From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> --- providers/mlx4/cq.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/providers/mlx4/cq.c b/providers/mlx4/cq.c index 068e0d2..50adebb 100644 --- a/providers/mlx4/cq.c +++ b/providers/mlx4/cq.c @@ -115,7 +115,7 @@ static enum ibv_wc_status mlx4_handle_error_cqe(struct mlx4_err_cqe *cqe) static inline void handle_good_req(struct ibv_wc *wc, struct mlx4_cqe *cqe) { wc->wc_flags = 0; - switch (cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) { + switch (mlx4dv_get_cqe_opcode(cqe)) { case MLX4_OPCODE_RDMA_WRITE_IMM: wc->wc_flags |= IBV_WC_WITH_IMM; SWITCH_FALLTHROUGH; @@ -215,7 +215,7 @@ static inline int mlx4_parse_cqe(struct mlx4_cq *cq, wc->qp_num = qpn; is_send = cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK; - is_error = (cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == + is_error = (mlx4dv_get_cqe_opcode(cqe)) == MLX4_CQE_OPCODE_ERROR; if ((qpn & MLX4_XRC_QPN_BIT) && !is_send) { @@ -278,7 +278,7 @@ static inline int mlx4_parse_cqe(struct mlx4_cq *cq, } else { wc->byte_len = be32toh(cqe->byte_cnt); - switch (cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) { + switch (mlx4dv_get_cqe_opcode(cqe)) { case MLX4_RECV_OPCODE_RDMA_WRITE_IMM: wc->opcode = IBV_WC_RECV_RDMA_WITH_IMM; wc->wc_flags = IBV_WC_WITH_IMM; @@ -460,7 +460,7 @@ static enum ibv_wc_opcode mlx4_cq_read_wc_opcode(struct ibv_cq_ex *ibcq) struct mlx4_cq *cq = to_mcq(ibv_cq_ex_to_cq(ibcq)); if (cq->cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK) { - switch (cq->cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) { + switch (mlx4dv_get_cqe_opcode(cq->cqe)) { case MLX4_OPCODE_RDMA_WRITE_IMM: case MLX4_OPCODE_RDMA_WRITE: return IBV_WC_RDMA_WRITE; @@ -480,7 +480,7 @@ static enum ibv_wc_opcode mlx4_cq_read_wc_opcode(struct ibv_cq_ex *ibcq) return IBV_WC_BIND_MW; } } else { - switch (cq->cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) { + switch (mlx4dv_get_cqe_opcode(cq->cqe)) { case MLX4_RECV_OPCODE_RDMA_WRITE_IMM: return IBV_WC_RECV_RDMA_WITH_IMM; case MLX4_RECV_OPCODE_SEND_INVAL: @@ -507,7 +507,7 @@ static int mlx4_cq_read_wc_flags(struct ibv_cq_ex *ibcq) int wc_flags = 0; if (is_send) { - switch (cq->cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) { + switch (mlx4dv_get_cqe_opcode(cq->cqe)) { case MLX4_OPCODE_RDMA_WRITE_IMM: case MLX4_OPCODE_SEND_IMM: wc_flags |= IBV_WC_WITH_IMM; @@ -520,7 +520,7 @@ static int mlx4_cq_read_wc_flags(struct ibv_cq_ex *ibcq) htobe32(MLX4_CQE_STATUS_IPV4_CSUM_OK)) << IBV_WC_IP_CSUM_OK_SHIFT; - switch (cq->cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) { + switch (mlx4dv_get_cqe_opcode(cq->cqe)) { case MLX4_RECV_OPCODE_RDMA_WRITE_IMM: case MLX4_RECV_OPCODE_SEND_IMM: wc_flags |= IBV_WC_WITH_IMM; @@ -554,7 +554,7 @@ static uint32_t mlx4_cq_read_wc_imm_data(struct ibv_cq_ex *ibcq) { struct mlx4_cq *cq = to_mcq(ibv_cq_ex_to_cq(ibcq)); - switch (cq->cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) { + switch (mlx4dv_get_cqe_opcode(cq->cqe)) { case MLX4_RECV_OPCODE_SEND_INVAL: return be32toh(cq->cqe->immed_rss_invalid); default: @@ -756,7 +756,7 @@ void mlx4_cq_resize_copy_cqes(struct mlx4_cq *cq, void *buf, int old_cqe) cqe = get_cqe(cq, (i & old_cqe)); cqe += cqe_inc; - while ((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) != MLX4_CQE_OPCODE_RESIZE) { + while ((mlx4dv_get_cqe_opcode(cqe)) != MLX4_CQE_OPCODE_RESIZE) { cqe->owner_sr_opcode = (cqe->owner_sr_opcode & ~MLX4_CQE_OWNER_MASK) | (((i + 1) & (cq->ibv_cq.cqe + 1)) ? MLX4_CQE_OWNER_MASK : 0); memcpy(buf + ((i + 1) & cq->ibv_cq.cqe) * cq->cqe_size, -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH rdma-core 6/7] mlx4: Add direct verbs man pages [not found] ` <1498736828-17875-1-git-send-email-yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> ` (4 preceding siblings ...) 2017-06-29 11:47 ` [PATCH rdma-core 5/7] mlx4: Convert to use predefined get CQE opcodes Yishai Hadas @ 2017-06-29 11:47 ` Yishai Hadas 2017-06-29 11:47 ` [PATCH rdma-core 7/7] mlx4: Export mlx4 direct verbs interface Yishai Hadas 2017-07-02 10:31 ` [PATCH rdma-core 0/7] Add mlx4 direct verbs Yishai Hadas 7 siblings, 0 replies; 9+ messages in thread From: Yishai Hadas @ 2017-06-29 11:47 UTC (permalink / raw) To: linux-rdma-u79uwXL29TY76Z2rM5mHXA Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, yishaih-VPRAkNaXOzVWk0Htik3J/w, maorg-VPRAkNaXOzVWk0Htik3J/w, majd-VPRAkNaXOzVWk0Htik3J/w From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Document mlx4 direct verbs feature and exposed related functions. Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> --- CMakeLists.txt | 1 + providers/mlx4/man/CMakeLists.txt | 4 ++ providers/mlx4/man/mlx4dv.7 | 39 ++++++++++++ providers/mlx4/man/mlx4dv_init_obj.3 | 117 +++++++++++++++++++++++++++++++++++ 4 files changed, 161 insertions(+) create mode 100644 providers/mlx4/man/CMakeLists.txt create mode 100644 providers/mlx4/man/mlx4dv.7 create mode 100644 providers/mlx4/man/mlx4dv_init_obj.3 diff --git a/CMakeLists.txt b/CMakeLists.txt index 634f327..5dcdb90 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -415,6 +415,7 @@ add_subdirectory(providers/cxgb4) # NO SPARSE add_subdirectory(providers/hns) # NO SPARSE add_subdirectory(providers/i40iw) # NO SPARSE add_subdirectory(providers/mlx4) # NO SPARSE +add_subdirectory(providers/mlx4/man) # NO SPARSE add_subdirectory(providers/mlx5) # NO SPARSE add_subdirectory(providers/mlx5/man) # NO SPARSE add_subdirectory(providers/mthca) # NO SPARSE diff --git a/providers/mlx4/man/CMakeLists.txt b/providers/mlx4/man/CMakeLists.txt new file mode 100644 index 0000000..c44c7c3 --- /dev/null +++ b/providers/mlx4/man/CMakeLists.txt @@ -0,0 +1,4 @@ +rdma_man_pages( + mlx4dv_init_obj.3 + mlx4dv.7 +) diff --git a/providers/mlx4/man/mlx4dv.7 b/providers/mlx4/man/mlx4dv.7 new file mode 100644 index 0000000..92ced99 --- /dev/null +++ b/providers/mlx4/man/mlx4dv.7 @@ -0,0 +1,39 @@ +.\" -*- nroff -*- +.\" Licensed under the OpenIB.org (MIT) - See COPYING.md +.\" +.TH MLX4DV 7 2017-04-19 1.0.0 +.SH "NAME" +mlx4dv \- Direct verbs for mlx4 devices +.br +This is low level access to mlx4 devices to perform data path operations, +without general branching performed by \fBibv_post_send\fR(3). + +.SH "DESCRIPTION" +The libibverbs API is an abstract one. It is agnostic to any underlying +provider specific implementation. While this abstraction has the advantage +of user applications portability it has a performance penalty. For some +applications optimizing performance is more important than portability. + +The mlx4 direct verbs API is intended for such applications. +It exposes mlx4 specific low level data path (send/receive/completion) +operations, allowing the application to bypass the libibverbs data path API. + +This interface consists from one hardware specific header file +with relevant inline functions and conversion logic from ibverbs structures +to mlx4 specific structures. + +The direct include of mlx4dv.h together with linkage to mlx4 library will +allow usage of this new interface. + +Once an application uses the direct flow the locking scheme is fully managed +by itself. There is an expectation that no mixed flows in the data path for both +direct/non-direct access will be by same application. + +.SH "NOTES" +.SH "SEE ALSO" +.BR ibv_post_send (3), +.BR verbs (7) + +.SH "AUTHORS" +.TP +Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> diff --git a/providers/mlx4/man/mlx4dv_init_obj.3 b/providers/mlx4/man/mlx4dv_init_obj.3 new file mode 100644 index 0000000..0d0a4a5 --- /dev/null +++ b/providers/mlx4/man/mlx4dv_init_obj.3 @@ -0,0 +1,117 @@ +.\" -*- nroff -*- +.\" Licensed under the OpenIB.org (MIT) - See COPYING.md +.\" +.TH MLX4DV_INIT_OBJ 3 2017-02-02 1.0.0 +.SH "NAME" +mlx4dv_init_obj \- Initialize mlx4 direct verbs object from ibv_xxx structures +.SH "SYNOPSIS" +.nf +.B #include <infiniband/mlx4dv.h> +.sp +.BI "int mlx4dv_init_obj(struct mlx4dv_obj *obj, uint64_t obj_type); +.fi +.SH "DESCRIPTION" +.B mlx4dv_init_obj() +This function will initialize mlx4dv_xxx structs based on supplied type. The information +for initialization is taken from ibv_xx structs supplied as part of input. + +Request information of CQ marks its owned by direct verbs for all consumer index +related actions. The initialization type can be combination of several types together. +.PP +.nf +struct mlx4dv_qp { +.in +8 +uint32_t *rdb; +uint32_t *sdb; +struct { +.in +8 +uint32_t wqe_cnt; +int wqe_shift; +int offset; +.in -8 +} sq; +struct { +.in +8 +uint32_t wqe_cnt; +int wqe_shift; +int offset; +.in -8 +} rq; +struct { +.in +8 +void *buf; +size_t length; +.in -8 +} buf; +uint64_t comp_mask; +.in -8 +}; + +struct mlx4dv_cq { +.in +8 +struct { +.in +8 +void *buf; +size_t length; +.in -8 +} buf; +uint32_t cqe_cnt; +uint32_t cqn; +uint32_t *set_ci_db; +uint32_t *arm_db; +int arm_sn; +int cqe_size; +uint64_t comp_mask; +.in -8 +}; + +struct mlx4dv_srq { +.in +8 +struct { +.in +8 +void *buf; +size_t length; +.in -8 +} buf; +int wqe_shift; +int head; +int tail; +uint32_t *db; +uint64_t comp_mask; +.in -8 +}; + +struct mlx4dv_obj { +.in +8 +struct { +.in +8 +struct ibv_qp *in; +struct mlx4dv_qp *out; +.in -8 +} qp; +struct { +.in +8 +struct ibv_cq *in; +struct mlx4dv_cq *out; +.in -8 +} cq; +.in -8 +}; + +enum mlx4dv_obj_type { +.in +8 +MLX4DV_OBJ_QP = 1 << 0, +MLX4DV_OBJ_CQ = 1 << 1, +MLX4DV_OBJ_SRQ = 1 << 2, +.in -8 +}; +.fi +.SH "RETURN VALUE" +0 on success or the value of errno on failure (which indicates the failure reason). +.SH "NOTES" + * Compatibility masks (comp_mask) are in/out fields. +.SH "SEE ALSO" +.BR mlx4dv (7) +.SH "AUTHORS" +.TP +Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH rdma-core 7/7] mlx4: Export mlx4 direct verbs interface [not found] ` <1498736828-17875-1-git-send-email-yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> ` (5 preceding siblings ...) 2017-06-29 11:47 ` [PATCH rdma-core 6/7] mlx4: Add direct verbs man pages Yishai Hadas @ 2017-06-29 11:47 ` Yishai Hadas 2017-07-02 10:31 ` [PATCH rdma-core 0/7] Add mlx4 direct verbs Yishai Hadas 7 siblings, 0 replies; 9+ messages in thread From: Yishai Hadas @ 2017-06-29 11:47 UTC (permalink / raw) To: linux-rdma-u79uwXL29TY76Z2rM5mHXA Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, yishaih-VPRAkNaXOzVWk0Htik3J/w, maorg-VPRAkNaXOzVWk0Htik3J/w, majd-VPRAkNaXOzVWk0Htik3J/w From: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Install direct verbs header file into /usr/include/infiniband/ folder and allow possible users to explicitly include it into their applications. In addition, make the required packaging changes needed for both RedHat and Debian users to link with libmlx4 directly. Signed-off-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> --- debian/ibverbs-providers.install | 1 + debian/ibverbs-providers.symbols | 4 ++++ debian/libibverbs-dev.install | 4 ++++ providers/mlx4/CMakeLists.txt | 7 ++++++- providers/mlx4/libmlx4.map | 7 +++++++ redhat/rdma-core.spec | 3 +++ 6 files changed, 25 insertions(+), 1 deletion(-) create mode 100644 providers/mlx4/libmlx4.map diff --git a/debian/ibverbs-providers.install b/debian/ibverbs-providers.install index 1d234ba..1d693fe 100644 --- a/debian/ibverbs-providers.install +++ b/debian/ibverbs-providers.install @@ -3,6 +3,7 @@ etc/modprobe.d/truescale.conf usr/bin/rxe_cfg usr/lib/*/libibverbs/lib*-rdmav2.so usr/lib/*/libmlx5.so.* +usr/lib/*/libmlx4.so.* usr/lib/truescale-serdes.cmds usr/share/doc/rdma-core/rxe.md usr/share/doc/ibverbs-providers/ usr/share/man/man7/rxe.7 diff --git a/debian/ibverbs-providers.symbols b/debian/ibverbs-providers.symbols index ea63da8..d64e3ef 100644 --- a/debian/ibverbs-providers.symbols +++ b/debian/ibverbs-providers.symbols @@ -4,3 +4,7 @@ libmlx5.so.1 ibverbs-providers #MINVER# mlx5dv_init_obj@MLX5_1.0 13 mlx5dv_query_device@MLX5_1.0 13 mlx5dv_create_cq@MLX5_1.1 14 + +libmlx4.so.1 ibverbs-providers #MINVER# + MLX4_1.0@MLX4_1.0 15 + mlx4dv_init_obj@MLX4_1.0 15 diff --git a/debian/libibverbs-dev.install b/debian/libibverbs-dev.install index ed7a276..3773d00 100644 --- a/debian/libibverbs-dev.install +++ b/debian/libibverbs-dev.install @@ -1,14 +1,18 @@ usr/include/infiniband/arch.h usr/include/infiniband/kern-abi.h usr/include/infiniband/mlx5dv.h +usr/include/infiniband/mlx4dv.h usr/include/infiniband/opcode.h usr/include/infiniband/sa-kern-abi.h usr/include/infiniband/sa.h usr/include/infiniband/verbs.h usr/lib/*/libibverbs*.so usr/lib/*/libmlx5.so +usr/lib/*/libmlx4.so usr/share/man/man3/ibv_* usr/share/man/man3/mbps_to_ibv_rate.3 usr/share/man/man3/mlx5dv_*.3 +usr/share/man/man3/mlx4dv_*.3 usr/share/man/man3/mult_to_ibv_rate.3 usr/share/man/man7/mlx5dv.7 +usr/share/man/man7/mlx4dv.7 diff --git a/providers/mlx4/CMakeLists.txt b/providers/mlx4/CMakeLists.txt index d64d817..f6739d2 100644 --- a/providers/mlx4/CMakeLists.txt +++ b/providers/mlx4/CMakeLists.txt @@ -1,4 +1,5 @@ -rdma_provider(mlx4 +rdma_shared_provider(mlx4 libmlx4.map + 1 1.0.${PACKAGE_VERSION} buf.c cq.c dbrec.c @@ -7,3 +8,7 @@ rdma_provider(mlx4 srq.c verbs.c ) + +publish_headers(infiniband + mlx4dv.h +) diff --git a/providers/mlx4/libmlx4.map b/providers/mlx4/libmlx4.map new file mode 100644 index 0000000..0e53f83 --- /dev/null +++ b/providers/mlx4/libmlx4.map @@ -0,0 +1,7 @@ +/* Export symbols should be added below according to + Documentation/versioning.md document. */ +MLX4_1.0 { + global: + mlx4dv_init_obj; + local: *; +}; diff --git a/redhat/rdma-core.spec b/redhat/rdma-core.spec index 1f5cd14..5cebc25 100644 --- a/redhat/rdma-core.spec +++ b/redhat/rdma-core.spec @@ -355,7 +355,9 @@ rm -rf %{buildroot}/%{_initrddir}/ %{_mandir}/man3/*_to_ibv_rate.* %{_mandir}/man7/rdma_cm.* %{_mandir}/man3/mlx5dv* +%{_mandir}/man3/mlx4dv* %{_mandir}/man7/mlx5dv* +%{_mandir}/man7/mlx4dv* %files -n libibverbs %dir %{_sysconfdir}/libibverbs.d @@ -363,6 +365,7 @@ rm -rf %{buildroot}/%{_initrddir}/ %{_libdir}/libibverbs*.so.* %{_libdir}/libibverbs/*.so %{_libdir}/libmlx5.so* +%{_libdir}/libmlx4.so* %config(noreplace) %{_sysconfdir}/libibverbs.d/*.driver %doc %{_docdir}/%{name}-%{version}/libibverbs.md %doc %{_docdir}/%{name}-%{version}/rxe.md -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH rdma-core 0/7] Add mlx4 direct verbs [not found] ` <1498736828-17875-1-git-send-email-yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> ` (6 preceding siblings ...) 2017-06-29 11:47 ` [PATCH rdma-core 7/7] mlx4: Export mlx4 direct verbs interface Yishai Hadas @ 2017-07-02 10:31 ` Yishai Hadas 7 siblings, 0 replies; 9+ messages in thread From: Yishai Hadas @ 2017-07-02 10:31 UTC (permalink / raw) To: maorg-VPRAkNaXOzVWk0Htik3J/w Cc: Yishai Hadas, linux-rdma-u79uwXL29TY76Z2rM5mHXA, dledford-H+wXaHxf7aLQT0dZR+AlfA, majd-VPRAkNaXOzVWk0Htik3J/w On 6/29/2017 2:47 PM, Yishai Hadas wrote: > This patchset from Maor adds direct access to mlx4 devices, it follows same > solution from both API and packaging perspectives as was already accepted for > mlx5. > > The libibverbs API is an abstract API. It is agnostic to any underlying > provider specific implementation. While this abstraction has the advantage of > user applications portability it has a performance penalty. For some > applications optimizing performance is more important than portability. > > The mlx4 direct verbs API introduced in this patchset is intended for such > applications. It exposes mlx4 specific low level data path > (send/receive/completion) operations, allowing the application to bypass the > libibverbs data path API. > > The proposed interface consists from one hardware specific header with relevant > inline functions and conversion logic from ibverbs structures to mlx4 related > structures. > > New header is going to be installed in below path location: > /usr/inlcude/infiniband/mlx4dv.h > > The direct include of mlx4dv.h together with linkage to mlx4 library will > allow usage of this new interface. > > Pull request was sent: > https://github.com/linux-rdma/rdma-core/pull/155 > > Yishai > Applied, thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2017-07-02 10:31 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-06-29 11:47 [PATCH rdma-core 0/7] Add mlx4 direct verbs Yishai Hadas
[not found] ` <1498736828-17875-1-git-send-email-yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2017-06-29 11:47 ` [PATCH rdma-core 1/7] mlx4: " Yishai Hadas
2017-06-29 11:47 ` [PATCH rdma-core 2/7] mlx4: Add CQE fields Yishai Hadas
2017-06-29 11:47 ` [PATCH rdma-core 3/7] mlx4: Convert datagram segment to use mlx4_av struct Yishai Hadas
2017-06-29 11:47 ` [PATCH rdma-core 4/7] mlx4: Add WQE segments implementation Yishai Hadas
2017-06-29 11:47 ` [PATCH rdma-core 5/7] mlx4: Convert to use predefined get CQE opcodes Yishai Hadas
2017-06-29 11:47 ` [PATCH rdma-core 6/7] mlx4: Add direct verbs man pages Yishai Hadas
2017-06-29 11:47 ` [PATCH rdma-core 7/7] mlx4: Export mlx4 direct verbs interface Yishai Hadas
2017-07-02 10:31 ` [PATCH rdma-core 0/7] Add mlx4 direct verbs Yishai Hadas
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.