[PATCH rdma-rc v2 0/3] RDMA/bnxt_re: Bug fixes

[PATCH rdma-rc v2 0/3] RDMA/bnxt_re: Bug fixes

[Selvin Xavier]

Some simple bug fixes

v1->v2:
 Fix a build issue
 Reported-by: kernel test robot <lkp@intel.com>

Naresh Kumar PBS (1):
  RDMA/bnxt_re: Add missing spin lock initialization

Selvin Xavier (2):
  RDMA/bnxt_re: Disable atomic support on VFs
  RDMA/bnxt_re: Fix query SRQ failure

 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 1 +
 drivers/infiniband/hw/bnxt_re/main.c     | 2 +-
 drivers/infiniband/hw/bnxt_re/qplib_fp.c | 2 ++
 3 files changed, 4 insertions(+), 1 deletion(-)

-- 
2.5.5

[PATCH rdma-rc v2 1/3] RDMA/bnxt_re: Disable atomic support on VFs

[Selvin Xavier]

Atomics is not currently supported for VFs. Enabling only
for PFs.

Fixes: 35f5ace5dea4 ("RDMA/bnxt_re: Enable global atomic ops if platform supports")
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c
index a8688a9..0b870be 100644
--- a/drivers/infiniband/hw/bnxt_re/main.c
+++ b/drivers/infiniband/hw/bnxt_re/main.c
@@ -129,7 +129,7 @@ static int bnxt_re_setup_chip_ctx(struct bnxt_re_dev *rdev, u8 wqe_mode)
 	rdev->rcfw.res = &rdev->qplib_res;
 
 	bnxt_re_set_drv_mode(rdev, wqe_mode);
-	if (bnxt_qplib_determine_atomics(en_dev->pdev))
+	if (!BNXT_VF(bp) && bnxt_qplib_determine_atomics(en_dev->pdev))
 		ibdev_info(&rdev->ibdev,
 			   "platform doesn't support global atomics.");
 	return 0;
-- 
2.5.5

Re: [PATCH rdma-rc v2 1/3] RDMA/bnxt_re: Disable atomic support on VFs

[Jason Gunthorpe]

On Wed, Aug 18, 2021 at 08:25:51PM -0700, Selvin Xavier wrote:
> Atomics is not currently supported for VFs. Enabling only
> for PFs.

Again, what is the user experience here? Does the device crash? QP
blow up?? Silently not work?

Jason

Re: [PATCH rdma-rc v2 1/3] RDMA/bnxt_re: Disable atomic support on VFs

[Selvin Xavier]

[-- Attachment #1: Type: text/plain, Size: 3739 bytes --]

On Thu, Aug 19, 2021 at 6:53 PM Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> On Wed, Aug 18, 2021 at 08:25:51PM -0700, Selvin Xavier wrote:
> > Atomics is not currently supported for VFs. Enabling only
> > for PFs.
>
> Again, what is the user experience here? Does the device crash? QP
> blow up?? Silently not work?
>
This is a host crash when pci_enable_atomic_ops_to_root is called with
a VF pci device (pdev->bus->self is NULL).  So, I want to  avoid
calling it for VF now.
crash> bt
PID: 4481   TASK: ffff89c6941b0000  CPU: 53  COMMAND: "bash"
 #0 [ffff9a94817136d8] machine_kexec at ffffffffb90601a4
 #1 [ffff9a9481713728] __crash_kexec at ffffffffb9190d5d
 #2 [ffff9a94817137f0] crash_kexec at ffffffffb9191c4d
 #3 [ffff9a9481713808] oops_end at ffffffffb9025cd6
 #4 [ffff9a9481713828] page_fault_oops at ffffffffb906e417
 #5 [ffff9a9481713888] exc_page_fault at ffffffffb9a0ad14
 #6 [ffff9a94817138b0] asm_exc_page_fault at ffffffffb9c00ace
    [exception RIP: pcie_capability_read_dword+28]
    RIP: ffffffffb952fd5c  RSP: ffff9a9481713960  RFLAGS: 00010246
    RAX: 0000000000000001  RBX: ffff89c6b1096000  RCX: 0000000000000000
    RDX: ffff9a9481713990  RSI: 0000000000000024  RDI: 0000000000000000
    RBP: 0000000000000080   R8: 0000000000000008   R9: ffff89c64341a2f8
    R10: 0000000000000002  R11: 0000000000000000  R12: ffff89c648bab000
    R13: 0000000000000000  R14: 0000000000000000  R15: ffff89c648bab0c8
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #7 [ffff9a9481713988] pci_enable_atomic_ops_to_root at ffffffffb95359a6
 #8 [ffff9a94817139c0] bnxt_qplib_determine_atomics at
ffffffffc08c1a33 [bnxt_re]
 #9 [ffff9a94817139d0] bnxt_re_dev_init at ffffffffc08ba2d1 [bnxt_re]
#10 [ffff9a9481713a78] bnxt_re_netdev_event at ffffffffc08bab8f [bnxt_re]
#11 [ffff9a9481713aa8] raw_notifier_call_chain at ffffffffb9102cbe
#12 [ffff9a9481713ad0] register_netdevice at ffffffffb9803ff3
#13 [ffff9a9481713b08] register_netdev at ffffffffb980410a
#14 [ffff9a9481713b18] bnxt_init_one at ffffffffc0349572 [bnxt_en]
#15 [ffff9a9481713b70] local_pci_probe at ffffffffb953b92f
#16 [ffff9a9481713ba0] pci_device_probe at ffffffffb953cf8f
#17 [ffff9a9481713be8] really_probe at ffffffffb9659619
#18 [ffff9a9481713c08] __driver_probe_device at ffffffffb96598fb
#19 [ffff9a9481713c28] driver_probe_device at ffffffffb965998f
#20 [ffff9a9481713c48] __device_attach_driver at ffffffffb9659cd2
#21 [ffff9a9481713c70] bus_for_each_drv at ffffffffb9657307
#22 [ffff9a9481713ca8] __device_attach at ffffffffb96593e0
#23 [ffff9a9481713ce8] pci_bus_add_device at ffffffffb9530b7a
#24 [ffff9a9481713d00] pci_iov_add_virtfn at ffffffffb955b1ca
#25 [ffff9a9481713d40] sriov_enable at ffffffffb955b54b
#26 [ffff9a9481713d90] bnxt_sriov_configure at ffffffffc034d913 [bnxt_en]
#27 [ffff9a9481713dd8] sriov_numvfs_store at ffffffffb955acb4
#28 [ffff9a9481713e10] kernfs_fop_write_iter at ffffffffb93f09ad
#29 [ffff9a9481713e48] new_sync_write at ffffffffb933b82c
#30 [ffff9a9481713ed0] vfs_write at ffffffffb933db64
#31 [ffff9a9481713f00] ksys_write at ffffffffb933dd99
#32 [ffff9a9481713f38] do_syscall_64 at ffffffffb9a07897
#33 [ffff9a9481713f50] entry_SYSCALL_64_after_hwframe at ffffffffb9c0007c
    RIP: 00007f450602f648  RSP: 00007ffe880869e8  RFLAGS: 00000246
    RAX: ffffffffffffffda  RBX: 0000000000000002  RCX: 00007f450602f648
    RDX: 0000000000000002  RSI: 0000555c566c4a60  RDI: 0000000000000001
    RBP: 0000555c566c4a60   R8: 000000000000000a   R9: 00007f45060c2580
    R10: 000000000000000a  R11: 0000000000000246  R12: 00007f45063026e0
    R13: 0000000000000002  R14: 00007f45062fd880  R15: 0000000000000002
    ORIG_RAX: 0000000000000001  CS: 0033  SS: 002b

I will update patch with this stack trace information.

> Jason

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4224 bytes --]

[PATCH rdma-rc v2 2/3] RDMA/bnxt_re: Add missing spin lock initialization

[Selvin Xavier]

From: Naresh Kumar PBS <nareshkumar.pbs@broadcom.com>

Add the missing initialization of srq lock.

Fixes: 37cb11acf1f7 ("RDMA/bnxt_re: Add SRQ support for Broadcom adapters")
Signed-off-by: Naresh Kumar PBS <nareshkumar.pbs@broadcom.com>
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 283b6b8..ea0054c 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1681,6 +1681,7 @@ int bnxt_re_create_srq(struct ib_srq *ib_srq,
 	if (nq)
 		nq->budget++;
 	atomic_inc(&rdev->srq_count);
+	spin_lock_init(&srq->lock);
 
 	return 0;
 
-- 
2.5.5

[PATCH rdma-rc v2 3/3] RDMA/bnxt_re: Fix query SRQ failure

[Selvin Xavier]

Fill the missing parameters for the FW command while
querying SRQ.

Fixes: 37cb11acf1f7 ("RDMA/bnxt_re: Add SRQ support for Broadcom adapters")
Signed-off-by: Naresh Kumar PBS <nareshkumar.pbs@broadcom.com>
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/qplib_fp.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/infiniband/hw/bnxt_re/qplib_fp.c b/drivers/infiniband/hw/bnxt_re/qplib_fp.c
index d4d4959..312bf25 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_fp.c
+++ b/drivers/infiniband/hw/bnxt_re/qplib_fp.c
@@ -713,6 +713,8 @@ int bnxt_qplib_query_srq(struct bnxt_qplib_res *res,
 	sbuf = bnxt_qplib_rcfw_alloc_sbuf(rcfw, sizeof(*sb));
 	if (!sbuf)
 		return -ENOMEM;
+	req.resp_size = sizeof(*sb) / BNXT_QPLIB_CMDQE_UNITS;
+	req.srq_cid = cpu_to_le32(srq->id);
 	sb = sbuf->sb;
 	rc = bnxt_qplib_rcfw_send_message(rcfw, (void *)&req, (void *)&resp,
 					  (void *)sbuf, 0);
-- 
2.5.5

Re: [PATCH rdma-rc v2 3/3] RDMA/bnxt_re: Fix query SRQ failure

[Jason Gunthorpe]

On Wed, Aug 18, 2021 at 08:25:53PM -0700, Selvin Xavier wrote:
> Fill the missing parameters for the FW command while
> querying SRQ.
> 
> Fixes: 37cb11acf1f7 ("RDMA/bnxt_re: Add SRQ support for Broadcom adapters")
> Signed-off-by: Naresh Kumar PBS <nareshkumar.pbs@broadcom.com>
> Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
> ---
>  drivers/infiniband/hw/bnxt_re/qplib_fp.c | 2 ++
>  1 file changed, 2 insertions(+)

This commit message is not good enough for -rc, especially with a
fixes line so old. 

What is the user impact, how did this ever work?

Jason

Re: [PATCH rdma-rc v2 3/3] RDMA/bnxt_re: Fix query SRQ failure

[Selvin Xavier]

[-- Attachment #1: Type: text/plain, Size: 851 bytes --]

On Thu, Aug 19, 2021 at 6:48 PM Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> On Wed, Aug 18, 2021 at 08:25:53PM -0700, Selvin Xavier wrote:
> > Fill the missing parameters for the FW command while
> > querying SRQ.
> >
> > Fixes: 37cb11acf1f7 ("RDMA/bnxt_re: Add SRQ support for Broadcom adapters")
> > Signed-off-by: Naresh Kumar PBS <nareshkumar.pbs@broadcom.com>
> > Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
> > ---
> >  drivers/infiniband/hw/bnxt_re/qplib_fp.c | 2 ++
> >  1 file changed, 2 insertions(+)
>
> This commit message is not good enough for -rc, especially with a
> fixes line so old.
>
> What is the user impact, how did this ever work?
>
This never worked. It's an issue with the query_srq only.  So was
missed in the testing of normal srq traffic.
Will post it for rdma-next along with other patches.
Thanks

> Jason

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4224 bytes --]

Re: [PATCH rdma-rc v2 0/3] RDMA/bnxt_re: Bug fixes

[Jason Gunthorpe]

On Wed, Aug 18, 2021 at 08:25:50PM -0700, Selvin Xavier wrote:
> Some simple bug fixes
> 
> v1->v2:
>  Fix a build issue
>  Reported-by: kernel test robot <lkp@intel.com>
> 
> Naresh Kumar PBS (1):
>   RDMA/bnxt_re: Add missing spin lock initialization
> 
> Selvin Xavier (2):
>   RDMA/bnxt_re: Disable atomic support on VFs
>   RDMA/bnxt_re: Fix query SRQ failure

Applied to for-rc, thanks

Jason

Re: [PATCH rdma-rc v2 0/3] RDMA/bnxt_re: Bug fixes

[Jason Gunthorpe]

On Thu, Aug 19, 2021 at 10:25:03AM -0300, Jason Gunthorpe wrote:
> On Wed, Aug 18, 2021 at 08:25:50PM -0700, Selvin Xavier wrote:
> > Some simple bug fixes
> > 
> > v1->v2:
> >  Fix a build issue
> >  Reported-by: kernel test robot <lkp@intel.com>
> > 
> > Naresh Kumar PBS (1):
> > 
> > Selvin Xavier (2):
> >   RDMA/bnxt_re: Disable atomic support on VFs
> >   RDMA/bnxt_re: Fix query SRQ failure
> 
> Applied to for-rc, thanks

Woops, I mean only

   RDMA/bnxt_re: Add missing spin lock initialization

The others need resend with good commit messages.

Jason