All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH V5 for-next 0/3] Broadcom's driver add global atomics
@ 2021-06-02 15:46 Devesh Sharma
  2021-06-02 15:46 ` [PATCH V5 for-next 1/3] RDMA/bnxt_re: Enable global atomic ops if platform supports Devesh Sharma
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Devesh Sharma @ 2021-06-02 15:46 UTC (permalink / raw)
  To: linux-rdma; +Cc: Devesh Sharma

[-- Attachment #1: Type: text/plain, Size: 1229 bytes --]

Adding automated detection and enablement of global PCI atomic
operation support.
Updated to ABI to pass wqe-mode to user to support rdma-core
pull request:
https://github.com/linux-rdma/rdma-core/pull/1007

v4->v5
 fixed commit msg in patch 0001
 fixed mixing of int with bool
v3->v4
 removed redundant code to enable global atomics
 refactored to honor standard error codes.
v2->v3
 Added additional patch to update ABI. A PR corresponding to this
 is open.

V1->V2
 renamed bnxt_qplib_enable_atomic_ops_to_root to
 bnxt_qplib_determine_atomics

Devesh Sharma (3):
  RDMA/bnxt_re: Enable global atomic ops if platform supports
  bnxt_re: Update maintainers list
  RDMA/bnxt_re: update ABI to pass wqe-mode to user space

 MAINTAINERS                               |  2 --
 drivers/infiniband/hw/bnxt_re/ib_verbs.c  |  7 +++++++
 drivers/infiniband/hw/bnxt_re/main.c      |  3 +++
 drivers/infiniband/hw/bnxt_re/qplib_res.c | 17 +++++++++++++++++
 drivers/infiniband/hw/bnxt_re/qplib_res.h |  1 +
 drivers/infiniband/hw/bnxt_re/qplib_sp.c  | 13 ++++++++++++-
 drivers/infiniband/hw/bnxt_re/qplib_sp.h  |  2 --
 include/uapi/rdma/bnxt_re-abi.h           |  5 ++++-
 8 files changed, 44 insertions(+), 6 deletions(-)

-- 
2.25.1


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4212 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH V5 for-next 1/3] RDMA/bnxt_re: Enable global atomic ops if platform supports
  2021-06-02 15:46 [PATCH V5 for-next 0/3] Broadcom's driver add global atomics Devesh Sharma
@ 2021-06-02 15:46 ` Devesh Sharma
  2021-06-03  5:57   ` Leon Romanovsky
  2021-06-02 15:46 ` [PATCH V5 for-next 2/3] bnxt_re: Update maintainers list Devesh Sharma
  2021-06-02 15:46 ` [PATCH V5 for-next 3/3] RDMA/bnxt_re: update ABI to pass wqe-mode to user space Devesh Sharma
  2 siblings, 1 reply; 6+ messages in thread
From: Devesh Sharma @ 2021-06-02 15:46 UTC (permalink / raw)
  To: linux-rdma; +Cc: Devesh Sharma

[-- Attachment #1: Type: text/plain, Size: 4625 bytes --]

Enabling Atomic operations for Gen P5 devices if the underlying
platform supports global atomic ops.

Fixes: 7ff662b76167 ("Disable atomic capability on bnxt_re adapters")
Signed-off-by: Devesh Sharma <devesh.sharma@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c  |  4 ++++
 drivers/infiniband/hw/bnxt_re/main.c      |  3 +++
 drivers/infiniband/hw/bnxt_re/qplib_res.c | 17 +++++++++++++++++
 drivers/infiniband/hw/bnxt_re/qplib_res.h |  1 +
 drivers/infiniband/hw/bnxt_re/qplib_sp.c  | 13 ++++++++++++-
 drivers/infiniband/hw/bnxt_re/qplib_sp.h  |  2 --
 6 files changed, 37 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 537471ffaa79..a113d8d9e9ed 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -163,6 +163,10 @@ int bnxt_re_query_device(struct ib_device *ibdev,
 	ib_attr->max_qp_init_rd_atom = dev_attr->max_qp_init_rd_atom;
 	ib_attr->atomic_cap = IB_ATOMIC_NONE;
 	ib_attr->masked_atomic_cap = IB_ATOMIC_NONE;
+	if (dev_attr->is_atomic) {
+		ib_attr->atomic_cap = IB_ATOMIC_GLOB;
+		ib_attr->masked_atomic_cap = IB_ATOMIC_GLOB;
+	}
 
 	ib_attr->max_ee_rd_atom = 0;
 	ib_attr->max_res_rd_atom = 0;
diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c
index b090dfa4f4cb..0de4e22f9750 100644
--- a/drivers/infiniband/hw/bnxt_re/main.c
+++ b/drivers/infiniband/hw/bnxt_re/main.c
@@ -128,6 +128,9 @@ static int bnxt_re_setup_chip_ctx(struct bnxt_re_dev *rdev, u8 wqe_mode)
 	rdev->rcfw.res = &rdev->qplib_res;
 
 	bnxt_re_set_drv_mode(rdev, wqe_mode);
+	if (bnxt_qplib_determine_atomics(en_dev->pdev))
+		ibdev_info(&rdev->ibdev,
+			   "platform doesn't support global atomics.");
 	return 0;
 }
 
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_res.c b/drivers/infiniband/hw/bnxt_re/qplib_res.c
index 3ca47004b752..108a591e66ff 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_res.c
+++ b/drivers/infiniband/hw/bnxt_re/qplib_res.c
@@ -959,3 +959,20 @@ int bnxt_qplib_alloc_res(struct bnxt_qplib_res *res, struct pci_dev *pdev,
 	bnxt_qplib_free_res(res);
 	return rc;
 }
+
+int bnxt_qplib_determine_atomics(struct pci_dev *dev)
+{
+	int comp;
+	u16 ctl2;
+
+	comp = pci_enable_atomic_ops_to_root(dev,
+					     PCI_EXP_DEVCAP2_ATOMIC_COMP32);
+	if (comp)
+		return -ENOTSUPP;
+	comp = pci_enable_atomic_ops_to_root(dev,
+					     PCI_EXP_DEVCAP2_ATOMIC_COMP64);
+	if (comp)
+		return -ENOTSUPP;
+	pcie_capability_read_word(dev, PCI_EXP_DEVCTL2, &ctl2);
+	return !(ctl2 & PCI_EXP_DEVCTL2_ATOMIC_REQ);
+}
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_res.h b/drivers/infiniband/hw/bnxt_re/qplib_res.h
index 7a1ab38b95da..d2aea52bd1d8 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_res.h
+++ b/drivers/infiniband/hw/bnxt_re/qplib_res.h
@@ -373,6 +373,7 @@ void bnxt_qplib_free_ctx(struct bnxt_qplib_res *res,
 int bnxt_qplib_alloc_ctx(struct bnxt_qplib_res *res,
 			 struct bnxt_qplib_ctx *ctx,
 			 bool virt_fn, bool is_p5);
+int bnxt_qplib_determine_atomics(struct pci_dev *dev);
 
 static inline void bnxt_qplib_hwq_incr_prod(struct bnxt_qplib_hwq *hwq, u32 cnt)
 {
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_sp.c b/drivers/infiniband/hw/bnxt_re/qplib_sp.c
index 049b3576302b..3d9259632eb3 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_sp.c
+++ b/drivers/infiniband/hw/bnxt_re/qplib_sp.c
@@ -54,6 +54,17 @@ const struct bnxt_qplib_gid bnxt_qplib_gid_zero = {{ 0, 0, 0, 0, 0, 0, 0, 0,
 
 /* Device */
 
+static bool bnxt_qplib_is_atomic_cap(struct bnxt_qplib_rcfw *rcfw)
+{
+	u16 pcie_ctl2 = 0;
+
+	if (!bnxt_qplib_is_chip_gen_p5(rcfw->res->cctx))
+		return false;
+
+	pcie_capability_read_word(rcfw->pdev, PCI_EXP_DEVCTL2, &pcie_ctl2);
+	return (pcie_ctl2 & PCI_EXP_DEVCTL2_ATOMIC_REQ);
+}
+
 static void bnxt_qplib_query_version(struct bnxt_qplib_rcfw *rcfw,
 				     char *fw_ver)
 {
@@ -162,7 +173,7 @@ int bnxt_qplib_get_dev_attr(struct bnxt_qplib_rcfw *rcfw,
 		attr->tqm_alloc_reqs[i * 4 + 3] = *(++tqm_alloc);
 	}
 
-	attr->is_atomic = false;
+	attr->is_atomic = bnxt_qplib_is_atomic_cap(rcfw);
 bail:
 	bnxt_qplib_rcfw_free_sbuf(rcfw, sbuf);
 	return rc;
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_sp.h b/drivers/infiniband/hw/bnxt_re/qplib_sp.h
index bc228340684f..260104783691 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_sp.h
+++ b/drivers/infiniband/hw/bnxt_re/qplib_sp.h
@@ -42,8 +42,6 @@
 
 #define BNXT_QPLIB_RESERVED_QP_WRS	128
 
-#define PCI_EXP_DEVCTL2_ATOMIC_REQ      0x0040
-
 struct bnxt_qplib_dev_attr {
 #define FW_VER_ARR_LEN			4
 	u8				fw_ver[FW_VER_ARR_LEN];
-- 
2.25.1


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4212 bytes --]

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH V5 for-next 2/3] bnxt_re: Update maintainers list
  2021-06-02 15:46 [PATCH V5 for-next 0/3] Broadcom's driver add global atomics Devesh Sharma
  2021-06-02 15:46 ` [PATCH V5 for-next 1/3] RDMA/bnxt_re: Enable global atomic ops if platform supports Devesh Sharma
@ 2021-06-02 15:46 ` Devesh Sharma
  2021-06-02 15:46 ` [PATCH V5 for-next 3/3] RDMA/bnxt_re: update ABI to pass wqe-mode to user space Devesh Sharma
  2 siblings, 0 replies; 6+ messages in thread
From: Devesh Sharma @ 2021-06-02 15:46 UTC (permalink / raw)
  To: linux-rdma; +Cc: Devesh Sharma

[-- Attachment #1: Type: text/plain, Size: 693 bytes --]

Updated the maintainers list and removed
non-active members.

Signed-off-by: Devesh Sharma <devesh.sharma@broadcom.com>
---
 MAINTAINERS | 2 --
 1 file changed, 2 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index bd7aff0c120f..786573a556b9 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3736,8 +3736,6 @@ F:	drivers/gpio/gpio-bcm-kona.c
 BROADCOM NETXTREME-E ROCE DRIVER
 M:	Selvin Xavier <selvin.xavier@broadcom.com>
 M:	Devesh Sharma <devesh.sharma@broadcom.com>
-M:	Somnath Kotur <somnath.kotur@broadcom.com>
-M:	Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
 M:	Naresh Kumar PBS <nareshkumar.pbs@broadcom.com>
 L:	linux-rdma@vger.kernel.org
 S:	Supported
-- 
2.25.1


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4212 bytes --]

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH V5 for-next 3/3] RDMA/bnxt_re: update ABI to pass wqe-mode to user space
  2021-06-02 15:46 [PATCH V5 for-next 0/3] Broadcom's driver add global atomics Devesh Sharma
  2021-06-02 15:46 ` [PATCH V5 for-next 1/3] RDMA/bnxt_re: Enable global atomic ops if platform supports Devesh Sharma
  2021-06-02 15:46 ` [PATCH V5 for-next 2/3] bnxt_re: Update maintainers list Devesh Sharma
@ 2021-06-02 15:46 ` Devesh Sharma
  2 siblings, 0 replies; 6+ messages in thread
From: Devesh Sharma @ 2021-06-02 15:46 UTC (permalink / raw)
  To: linux-rdma; +Cc: Devesh Sharma

[-- Attachment #1: Type: text/plain, Size: 1584 bytes --]

Changing ucontext ABI response structure to pass wqe_mode
to user library.
A flag in comp_mask has been set to indicate presence of
wqe_mode.

Signed-off-by: Devesh Sharma <devesh.sharma@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 3 +++
 include/uapi/rdma/bnxt_re-abi.h          | 5 ++++-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index a113d8d9e9ed..5955713234cb 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -3882,6 +3882,9 @@ int bnxt_re_alloc_ucontext(struct ib_ucontext *ctx, struct ib_udata *udata)
 	resp.max_cqd = dev_attr->max_cq_wqes;
 	resp.rsvd    = 0;
 
+	resp.comp_mask |= BNXT_RE_UCNTX_CMASK_HAVE_MODE;
+	resp.mode = rdev->chip_ctx->modes.wqe_mode;
+
 	rc = ib_copy_to_udata(udata, &resp, min(udata->outlen, sizeof(resp)));
 	if (rc) {
 		ibdev_err(ibdev, "Failed to copy user context");
diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h
index dc52e3cf574c..52205ed2b898 100644
--- a/include/uapi/rdma/bnxt_re-abi.h
+++ b/include/uapi/rdma/bnxt_re-abi.h
@@ -49,7 +49,8 @@
 #define BNXT_RE_CHIP_ID0_CHIP_MET_SFT		0x18
 
 enum {
-	BNXT_RE_UCNTX_CMASK_HAVE_CCTX = 0x1ULL
+	BNXT_RE_UCNTX_CMASK_HAVE_CCTX = 0x1ULL,
+	BNXT_RE_UCNTX_CMASK_HAVE_MODE = 0x02ULL
 };
 
 struct bnxt_re_uctx_resp {
@@ -62,6 +63,8 @@ struct bnxt_re_uctx_resp {
 	__aligned_u64 comp_mask;
 	__u32 chip_id0;
 	__u32 chip_id1;
+	__u32 mode;
+	__u32 rsvd1; /* padding */
 };
 
 /*
-- 
2.25.1


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4212 bytes --]

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH V5 for-next 1/3] RDMA/bnxt_re: Enable global atomic ops if platform supports
  2021-06-02 15:46 ` [PATCH V5 for-next 1/3] RDMA/bnxt_re: Enable global atomic ops if platform supports Devesh Sharma
@ 2021-06-03  5:57   ` Leon Romanovsky
  2021-06-03 10:27     ` Devesh Sharma
  0 siblings, 1 reply; 6+ messages in thread
From: Leon Romanovsky @ 2021-06-03  5:57 UTC (permalink / raw)
  To: Devesh Sharma; +Cc: linux-rdma

On Wed, Jun 02, 2021 at 09:16:16PM +0530, Devesh Sharma wrote:
> Enabling Atomic operations for Gen P5 devices if the underlying
> platform supports global atomic ops.
> 
> Fixes: 7ff662b76167 ("Disable atomic capability on bnxt_re adapters")

First, it looks to me like a feature and not Fix, so I'll be second to
Nicolas's request, please don't add Fixes line to every commit.

Second, even this is a dix, the fixes line is still wrong.
➜  kernel git:(rdma-next) git fixes 7ff662b76167
Fixes: 7ff662b76167 ("RDMA/bnxt_re: Disable atomic capability on bnxt_re adapters")

Thanks

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH V5 for-next 1/3] RDMA/bnxt_re: Enable global atomic ops if platform supports
  2021-06-03  5:57   ` Leon Romanovsky
@ 2021-06-03 10:27     ` Devesh Sharma
  0 siblings, 0 replies; 6+ messages in thread
From: Devesh Sharma @ 2021-06-03 10:27 UTC (permalink / raw)
  To: Leon Romanovsky; +Cc: linux-rdma

[-- Attachment #1: Type: text/plain, Size: 761 bytes --]

On Thu, Jun 3, 2021 at 11:27 AM Leon Romanovsky <leon@kernel.org> wrote:
>
> On Wed, Jun 02, 2021 at 09:16:16PM +0530, Devesh Sharma wrote:
> > Enabling Atomic operations for Gen P5 devices if the underlying
> > platform supports global atomic ops.
> >
> > Fixes: 7ff662b76167 ("Disable atomic capability on bnxt_re adapters")
>
> First, it looks to me like a feature and not Fix, so I'll be second to
> Nicolas's request, please don't add Fixes line to every commit.
>
> Second, even this is a dix, the fixes line is still wrong.
> ➜  kernel git:(rdma-next) git fixes 7ff662b76167
> Fixes: 7ff662b76167 ("RDMA/bnxt_re: Disable atomic capability on bnxt_re adapters")
Okay, I will drop this line then.
>
> Thanks



-- 
-Regards
Devesh

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4212 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-06-03 10:29 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-02 15:46 [PATCH V5 for-next 0/3] Broadcom's driver add global atomics Devesh Sharma
2021-06-02 15:46 ` [PATCH V5 for-next 1/3] RDMA/bnxt_re: Enable global atomic ops if platform supports Devesh Sharma
2021-06-03  5:57   ` Leon Romanovsky
2021-06-03 10:27     ` Devesh Sharma
2021-06-02 15:46 ` [PATCH V5 for-next 2/3] bnxt_re: Update maintainers list Devesh Sharma
2021-06-02 15:46 ` [PATCH V5 for-next 3/3] RDMA/bnxt_re: update ABI to pass wqe-mode to user space Devesh Sharma

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.