* [PATCH v2 rdma for-rc 0/2] RDMA/qedr: A couple of patches
@ 2017-06-05 13:32 Ram Amrani
[not found] ` <1496669547-22503-1-git-send-email-Ram.Amrani-h88ZbnxC6KDQT0dZR+AlfA@public.gmane.org>
0 siblings, 1 reply; 4+ messages in thread
From: Ram Amrani @ 2017-06-05 13:32 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA,
Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, Ram Amrani
From: Ram Amrani <Ram.Amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Changes from v1:
I've rebased the patches agains 4.12-rc and request them to be added to
the rc and not for-next.
As a result I've used umem->shift instead of the now obsolete umem->page_size.
Michal Kalderon (1):
RDMA/qedr: Initialize byte_len in WC of READ and SEND commands
Ram Amrani (1):
RDMA/qedr: Add 64KB PAGE_SIZE support to user-space queues
drivers/infiniband/hw/qedr/qedr.h | 5 ++-
drivers/infiniband/hw/qedr/verbs.c | 68 +++++++++++++++++++++++---------------
2 files changed, 45 insertions(+), 28 deletions(-)
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 4+ messages in thread[parent not found: <1496669547-22503-1-git-send-email-Ram.Amrani-h88ZbnxC6KDQT0dZR+AlfA@public.gmane.org>]
* [PATCH v2 rdma for-rc 1/2] RDMA/qedr: Initialize byte_len in WC of READ and SEND commands [not found] ` <1496669547-22503-1-git-send-email-Ram.Amrani-h88ZbnxC6KDQT0dZR+AlfA@public.gmane.org> @ 2017-06-05 13:32 ` Ram Amrani 2017-06-05 13:32 ` [PATCH v2 rdma for-rc 2/2] RDMA/qedr: Add 64KB PAGE_SIZE support to user-space queues Ram Amrani 2017-06-13 20:41 ` [PATCH v2 rdma for-rc 0/2] RDMA/qedr: A couple of patches Doug Ledford 2 siblings, 0 replies; 4+ messages in thread From: Ram Amrani @ 2017-06-05 13:32 UTC (permalink / raw) To: dledford-H+wXaHxf7aLQT0dZR+AlfA Cc: Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA, Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA, linux-rdma-u79uwXL29TY76Z2rM5mHXA From: Michal Kalderon <Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> Initialize byte_len in work completion of RDMA_READ and RDMA_SEND. Exposed by uDAPL application. Signed-off-by: Michal Kalderon <Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> --- drivers/infiniband/hw/qedr/verbs.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c index 17685cf..7add0cd 100644 --- a/drivers/infiniband/hw/qedr/verbs.c +++ b/drivers/infiniband/hw/qedr/verbs.c @@ -3209,6 +3209,10 @@ static int process_req(struct qedr_dev *dev, struct qedr_qp *qp, case IB_WC_REG_MR: qp->wqe_wr_id[qp->sq.cons].mr->info.completed++; break; + case IB_WC_RDMA_READ: + case IB_WC_SEND: + wc->byte_len = qp->wqe_wr_id[qp->sq.cons].bytes_len; + break; default: break; } -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH v2 rdma for-rc 2/2] RDMA/qedr: Add 64KB PAGE_SIZE support to user-space queues [not found] ` <1496669547-22503-1-git-send-email-Ram.Amrani-h88ZbnxC6KDQT0dZR+AlfA@public.gmane.org> 2017-06-05 13:32 ` [PATCH v2 rdma for-rc 1/2] RDMA/qedr: Initialize byte_len in WC of READ and SEND commands Ram Amrani @ 2017-06-05 13:32 ` Ram Amrani 2017-06-13 20:41 ` [PATCH v2 rdma for-rc 0/2] RDMA/qedr: A couple of patches Doug Ledford 2 siblings, 0 replies; 4+ messages in thread From: Ram Amrani @ 2017-06-05 13:32 UTC (permalink / raw) To: dledford-H+wXaHxf7aLQT0dZR+AlfA Cc: Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA, Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA, linux-rdma-u79uwXL29TY76Z2rM5mHXA, Ram Amrani From: Ram Amrani <Ram.Amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> Add 64KB PAGE_SIZE support to user-space CQ, SQ and RQ queues. De-facto it means that code was added to translate 64KB pages to smaller 4KB pages that the FW can handle. Otherwise, the FW would wrap (or jump to the next page) when reaching 4KB while the user space library will continue on the same large page. Note that MR code remains as is since the FW supports larger pages for MRs. Signed-off-by: Ram Amrani <Ram.Amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> Signed-off-by: Michal Kalderon <Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> --- drivers/infiniband/hw/qedr/qedr.h | 5 ++- drivers/infiniband/hw/qedr/verbs.c | 64 ++++++++++++++++++++++---------------- 2 files changed, 41 insertions(+), 28 deletions(-) diff --git a/drivers/infiniband/hw/qedr/qedr.h b/drivers/infiniband/hw/qedr/qedr.h index aa08c76..d961f79 100644 --- a/drivers/infiniband/hw/qedr/qedr.h +++ b/drivers/infiniband/hw/qedr/qedr.h @@ -58,7 +58,10 @@ #define QEDR_MSG_QP " QP" #define QEDR_MSG_GSI " GSI" -#define QEDR_CQ_MAGIC_NUMBER (0x11223344) +#define QEDR_CQ_MAGIC_NUMBER (0x11223344) + +#define FW_PAGE_SIZE (RDMA_RING_PAGE_SIZE) +#define FW_PAGE_SHIFT (12) struct qedr_dev; diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c index 7add0cd..49bd74a 100644 --- a/drivers/infiniband/hw/qedr/verbs.c +++ b/drivers/infiniband/hw/qedr/verbs.c @@ -653,14 +653,15 @@ static int qedr_prepare_pbl_tbl(struct qedr_dev *dev, static void qedr_populate_pbls(struct qedr_dev *dev, struct ib_umem *umem, struct qedr_pbl *pbl, - struct qedr_pbl_info *pbl_info) + struct qedr_pbl_info *pbl_info, u32 pg_shift) { int shift, pg_cnt, pages, pbe_cnt, total_num_pbes = 0; + u32 fw_pg_cnt, fw_pg_per_umem_pg; struct qedr_pbl *pbl_tbl; struct scatterlist *sg; struct regpair *pbe; + u64 pg_addr; int entry; - u32 addr; if (!pbl_info->num_pbes) return; @@ -683,29 +684,35 @@ static void qedr_populate_pbls(struct qedr_dev *dev, struct ib_umem *umem, shift = umem->page_shift; + fw_pg_per_umem_pg = BIT(umem->page_shift - pg_shift); + for_each_sg(umem->sg_head.sgl, sg, umem->nmap, entry) { pages = sg_dma_len(sg) >> shift; + pg_addr = sg_dma_address(sg); for (pg_cnt = 0; pg_cnt < pages; pg_cnt++) { - /* store the page address in pbe */ - pbe->lo = cpu_to_le32(sg_dma_address(sg) + - (pg_cnt << shift)); - addr = upper_32_bits(sg_dma_address(sg) + - (pg_cnt << shift)); - pbe->hi = cpu_to_le32(addr); - pbe_cnt++; - total_num_pbes++; - pbe++; - - if (total_num_pbes == pbl_info->num_pbes) - return; - - /* If the given pbl is full storing the pbes, - * move to next pbl. - */ - if (pbe_cnt == (pbl_info->pbl_size / sizeof(u64))) { - pbl_tbl++; - pbe = (struct regpair *)pbl_tbl->va; - pbe_cnt = 0; + for (fw_pg_cnt = 0; fw_pg_cnt < fw_pg_per_umem_pg;) { + pbe->lo = cpu_to_le32(pg_addr); + pbe->hi = cpu_to_le32(upper_32_bits(pg_addr)); + + pg_addr += BIT(pg_shift); + pbe_cnt++; + total_num_pbes++; + pbe++; + + if (total_num_pbes == pbl_info->num_pbes) + return; + + /* If the given pbl is full storing the pbes, + * move to next pbl. + */ + if (pbe_cnt == + (pbl_info->pbl_size / sizeof(u64))) { + pbl_tbl++; + pbe = (struct regpair *)pbl_tbl->va; + pbe_cnt = 0; + } + + fw_pg_cnt++; } } } @@ -754,7 +761,7 @@ static inline int qedr_init_user_queue(struct ib_ucontext *ib_ctx, u64 buf_addr, size_t buf_len, int access, int dmasync) { - int page_cnt; + u32 fw_pages; int rc; q->buf_addr = buf_addr; @@ -766,8 +773,10 @@ static inline int qedr_init_user_queue(struct ib_ucontext *ib_ctx, return PTR_ERR(q->umem); } - page_cnt = ib_umem_page_count(q->umem); - rc = qedr_prepare_pbl_tbl(dev, &q->pbl_info, page_cnt, 0); + fw_pages = ib_umem_page_count(q->umem) << + (q->umem->page_shift - FW_PAGE_SHIFT); + + rc = qedr_prepare_pbl_tbl(dev, &q->pbl_info, fw_pages, 0); if (rc) goto err0; @@ -777,7 +786,8 @@ static inline int qedr_init_user_queue(struct ib_ucontext *ib_ctx, goto err0; } - qedr_populate_pbls(dev, q->umem, q->pbl_tbl, &q->pbl_info); + qedr_populate_pbls(dev, q->umem, q->pbl_tbl, &q->pbl_info, + FW_PAGE_SHIFT); return 0; @@ -2226,7 +2236,7 @@ struct ib_mr *qedr_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len, goto err1; qedr_populate_pbls(dev, mr->umem, mr->info.pbl_table, - &mr->info.pbl_info); + &mr->info.pbl_info, mr->umem->page_shift); rc = dev->ops->rdma_alloc_tid(dev->rdma_ctx, &mr->hw_mr.itid); if (rc) { -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH v2 rdma for-rc 0/2] RDMA/qedr: A couple of patches [not found] ` <1496669547-22503-1-git-send-email-Ram.Amrani-h88ZbnxC6KDQT0dZR+AlfA@public.gmane.org> 2017-06-05 13:32 ` [PATCH v2 rdma for-rc 1/2] RDMA/qedr: Initialize byte_len in WC of READ and SEND commands Ram Amrani 2017-06-05 13:32 ` [PATCH v2 rdma for-rc 2/2] RDMA/qedr: Add 64KB PAGE_SIZE support to user-space queues Ram Amrani @ 2017-06-13 20:41 ` Doug Ledford 2 siblings, 0 replies; 4+ messages in thread From: Doug Ledford @ 2017-06-13 20:41 UTC (permalink / raw) To: Ram Amrani Cc: Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA, Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA, linux-rdma-u79uwXL29TY76Z2rM5mHXA, Ram Amrani On Mon, 2017-06-05 at 16:32 +0300, Ram Amrani wrote: > From: Ram Amrani <Ram.Amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> > > Changes from v1: > I've rebased the patches agains 4.12-rc and request them to be added > to > the rc and not for-next. > As a result I've used umem->shift instead of the now obsolete umem- > >page_size. > > Michal Kalderon (1): > RDMA/qedr: Initialize byte_len in WC of READ and SEND commands > > Ram Amrani (1): > RDMA/qedr: Add 64KB PAGE_SIZE support to user-space queues > > drivers/infiniband/hw/qedr/qedr.h | 5 ++- > drivers/infiniband/hw/qedr/verbs.c | 68 +++++++++++++++++++++++----- > ---------- > 2 files changed, 45 insertions(+), 28 deletions(-) Series applied, thanks. -- Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> GPG KeyID: B826A3330E572FDD Key fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2017-06-13 20:41 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-06-05 13:32 [PATCH v2 rdma for-rc 0/2] RDMA/qedr: A couple of patches Ram Amrani
[not found] ` <1496669547-22503-1-git-send-email-Ram.Amrani-h88ZbnxC6KDQT0dZR+AlfA@public.gmane.org>
2017-06-05 13:32 ` [PATCH v2 rdma for-rc 1/2] RDMA/qedr: Initialize byte_len in WC of READ and SEND commands Ram Amrani
2017-06-05 13:32 ` [PATCH v2 rdma for-rc 2/2] RDMA/qedr: Add 64KB PAGE_SIZE support to user-space queues Ram Amrani
2017-06-13 20:41 ` [PATCH v2 rdma for-rc 0/2] RDMA/qedr: A couple of patches Doug Ledford
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.