* [PATCH] PCI: Do not enable pci atomics on VFs
@ 2021-09-11 10:03 Selvin Xavier
2021-09-13 12:28 ` Andy Gospodarek
2021-09-14 20:16 ` Bjorn Helgaas
0 siblings, 2 replies; 3+ messages in thread
From: Selvin Xavier @ 2021-09-11 10:03 UTC (permalink / raw)
To: linux-pci, bhelgaas
Cc: dledford, jgg, linux-rdma, Felix.Kuehling, Shaoyun.Liu,
Jay.Cornwall, andrew.gospodarek, michael.chan, Selvin Xavier
Host crashes when pci_enable_atomic_ops_to_root is called for VFs
with virtual buses. The virtual buses added to SR-IOV has bus->self
set to NULL and host crashes due to this.
PID: 4481 TASK: ffff89c6941b0000 CPU: 53 COMMAND: "bash"
#0 [ffff9a94817136d8] machine_kexec at ffffffffb90601a4
#1 [ffff9a9481713728] __crash_kexec at ffffffffb9190d5d
#2 [ffff9a94817137f0] crash_kexec at ffffffffb9191c4d
#3 [ffff9a9481713808] oops_end at ffffffffb9025cd6
#4 [ffff9a9481713828] page_fault_oops at ffffffffb906e417
#5 [ffff9a9481713888] exc_page_fault at ffffffffb9a0ad14
#6 [ffff9a94817138b0] asm_exc_page_fault at ffffffffb9c00ace
[exception RIP: pcie_capability_read_dword+28]
RIP: ffffffffb952fd5c RSP: ffff9a9481713960 RFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff89c6b1096000 RCX: 0000000000000000
RDX: ffff9a9481713990 RSI: 0000000000000024 RDI: 0000000000000000
RBP: 0000000000000080 R8: 0000000000000008 R9: ffff89c64341a2f8
R10: 0000000000000002 R11: 0000000000000000 R12: ffff89c648bab000
R13: 0000000000000000 R14: 0000000000000000 R15: ffff89c648bab0c8
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#7 [ffff9a9481713988] pci_enable_atomic_ops_to_root at ffffffffb95359a6
#8 [ffff9a94817139c0] bnxt_qplib_determine_atomics at ffffffffc08c1a33 [bnxt_re]
#9 [ffff9a94817139d0] bnxt_re_dev_init at ffffffffc08ba2d1 [bnxt_re]
RIP: 00007f450602f648 RSP: 00007ffe880869e8 RFLAGS: 00000246
RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f450602f648
RDX: 0000000000000002 RSI: 0000555c566c4a60 RDI: 0000000000000001
RBP: 0000555c566c4a60 R8: 000000000000000a R9: 00007f45060c2580
R10: 000000000000000a R11: 0000000000000246 R12: 00007f45063026e0
R13: 0000000000000002 R14: 00007f45062fd880 R15: 0000000000000002
ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b
AtomicOp Requester Enable bit in the Device Control 2 register
is reserved for VFs and drivers shouldn't enable it for VFs.
Adding a check to return EINVAL if pci_enable_atomic_ops_to_root
is called with VF pci device.
Fixes: 35f5ace5dea4 ("RDMA/bnxt_re: Enable global atomic ops if platform supports")
Fixes: 430a23689dea ("PCI: Add pci_enable_atomic_ops_to_root()")
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
drivers/pci/pci.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index aacf575..d968a36 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3702,6 +3702,14 @@ int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)
struct pci_dev *bridge;
u32 cap, ctl2;
+ /*
+ * As per PCIe r5.0, sec 9.3.5.10, the AtomicOp Requester Enable
+ * bit in the Device Control 2 register is reserved in VFs and the PF
+ * value applies to all associated VFs. Return -EINVAL if called for VFs.
+ */
+ if (dev->is_virtfn)
+ return -EINVAL;
+
if (!pci_is_pcie(dev))
return -EINVAL;
--
2.5.5
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [PATCH] PCI: Do not enable pci atomics on VFs
2021-09-11 10:03 [PATCH] PCI: Do not enable pci atomics on VFs Selvin Xavier
@ 2021-09-13 12:28 ` Andy Gospodarek
2021-09-14 20:16 ` Bjorn Helgaas
1 sibling, 0 replies; 3+ messages in thread
From: Andy Gospodarek @ 2021-09-13 12:28 UTC (permalink / raw)
To: Selvin Xavier
Cc: linux-pci, bhelgaas, dledford, jgg, linux-rdma, Felix.Kuehling,
Shaoyun.Liu, Jay.Cornwall, andrew.gospodarek, michael.chan
[-- Attachment #1: Type: text/plain, Size: 3487 bytes --]
On Sat, Sep 11, 2021 at 03:03:05AM -0700, Selvin Xavier wrote:
> Host crashes when pci_enable_atomic_ops_to_root is called for VFs
> with virtual buses. The virtual buses added to SR-IOV has bus->self
> set to NULL and host crashes due to this.
^^ I _hate_ to say this, but the extra space isn't ideal. Not
sure if the maintainers will want to hold-up the submission for this (or
if they can fix while committing), but something to look out for next
time.
>
> PID: 4481 TASK: ffff89c6941b0000 CPU: 53 COMMAND: "bash"
> #0 [ffff9a94817136d8] machine_kexec at ffffffffb90601a4
> #1 [ffff9a9481713728] __crash_kexec at ffffffffb9190d5d
> #2 [ffff9a94817137f0] crash_kexec at ffffffffb9191c4d
> #3 [ffff9a9481713808] oops_end at ffffffffb9025cd6
> #4 [ffff9a9481713828] page_fault_oops at ffffffffb906e417
> #5 [ffff9a9481713888] exc_page_fault at ffffffffb9a0ad14
> #6 [ffff9a94817138b0] asm_exc_page_fault at ffffffffb9c00ace
> [exception RIP: pcie_capability_read_dword+28]
> RIP: ffffffffb952fd5c RSP: ffff9a9481713960 RFLAGS: 00010246
> RAX: 0000000000000001 RBX: ffff89c6b1096000 RCX: 0000000000000000
> RDX: ffff9a9481713990 RSI: 0000000000000024 RDI: 0000000000000000
> RBP: 0000000000000080 R8: 0000000000000008 R9: ffff89c64341a2f8
> R10: 0000000000000002 R11: 0000000000000000 R12: ffff89c648bab000
> R13: 0000000000000000 R14: 0000000000000000 R15: ffff89c648bab0c8
> ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
> #7 [ffff9a9481713988] pci_enable_atomic_ops_to_root at ffffffffb95359a6
> #8 [ffff9a94817139c0] bnxt_qplib_determine_atomics at ffffffffc08c1a33 [bnxt_re]
> #9 [ffff9a94817139d0] bnxt_re_dev_init at ffffffffc08ba2d1 [bnxt_re]
> RIP: 00007f450602f648 RSP: 00007ffe880869e8 RFLAGS: 00000246
> RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f450602f648
> RDX: 0000000000000002 RSI: 0000555c566c4a60 RDI: 0000000000000001
> RBP: 0000555c566c4a60 R8: 000000000000000a R9: 00007f45060c2580
> R10: 000000000000000a R11: 0000000000000246 R12: 00007f45063026e0
> R13: 0000000000000002 R14: 00007f45062fd880 R15: 0000000000000002
> ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b
>
> AtomicOp Requester Enable bit in the Device Control 2 register
> is reserved for VFs and drivers shouldn't enable it for VFs.
> Adding a check to return EINVAL if pci_enable_atomic_ops_to_root
> is called with VF pci device.
>
> Fixes: 35f5ace5dea4 ("RDMA/bnxt_re: Enable global atomic ops if platform supports")
> Fixes: 430a23689dea ("PCI: Add pci_enable_atomic_ops_to_root()")
> Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Thanks for this, Selvin. Technically this looks like a good fix.
Reviewed-by: Andy Gospodarek <gospo@broadcom.com>
> ---
> drivers/pci/pci.c | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index aacf575..d968a36 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -3702,6 +3702,14 @@ int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)
> struct pci_dev *bridge;
> u32 cap, ctl2;
>
> + /*
> + * As per PCIe r5.0, sec 9.3.5.10, the AtomicOp Requester Enable
> + * bit in the Device Control 2 register is reserved in VFs and the PF
> + * value applies to all associated VFs. Return -EINVAL if called for VFs.
> + */
> + if (dev->is_virtfn)
> + return -EINVAL;
> +
> if (!pci_is_pcie(dev))
> return -EINVAL;
>
> --
> 2.5.5
>
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4222 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] PCI: Do not enable pci atomics on VFs
2021-09-11 10:03 [PATCH] PCI: Do not enable pci atomics on VFs Selvin Xavier
2021-09-13 12:28 ` Andy Gospodarek
@ 2021-09-14 20:16 ` Bjorn Helgaas
1 sibling, 0 replies; 3+ messages in thread
From: Bjorn Helgaas @ 2021-09-14 20:16 UTC (permalink / raw)
To: Selvin Xavier
Cc: linux-pci, bhelgaas, dledford, jgg, linux-rdma, Felix.Kuehling,
Shaoyun.Liu, Jay.Cornwall, andrew.gospodarek, michael.chan
On Sat, Sep 11, 2021 at 03:03:05AM -0700, Selvin Xavier wrote:
> Host crashes when pci_enable_atomic_ops_to_root is called for VFs
> with virtual buses. The virtual buses added to SR-IOV has bus->self
> set to NULL and host crashes due to this.
>
> PID: 4481 TASK: ffff89c6941b0000 CPU: 53 COMMAND: "bash"
> #0 [ffff9a94817136d8] machine_kexec at ffffffffb90601a4
> #1 [ffff9a9481713728] __crash_kexec at ffffffffb9190d5d
> #2 [ffff9a94817137f0] crash_kexec at ffffffffb9191c4d
> #3 [ffff9a9481713808] oops_end at ffffffffb9025cd6
> #4 [ffff9a9481713828] page_fault_oops at ffffffffb906e417
> #5 [ffff9a9481713888] exc_page_fault at ffffffffb9a0ad14
> #6 [ffff9a94817138b0] asm_exc_page_fault at ffffffffb9c00ace
> [exception RIP: pcie_capability_read_dword+28]
> RIP: ffffffffb952fd5c RSP: ffff9a9481713960 RFLAGS: 00010246
> RAX: 0000000000000001 RBX: ffff89c6b1096000 RCX: 0000000000000000
> RDX: ffff9a9481713990 RSI: 0000000000000024 RDI: 0000000000000000
> RBP: 0000000000000080 R8: 0000000000000008 R9: ffff89c64341a2f8
> R10: 0000000000000002 R11: 0000000000000000 R12: ffff89c648bab000
> R13: 0000000000000000 R14: 0000000000000000 R15: ffff89c648bab0c8
> ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
> #7 [ffff9a9481713988] pci_enable_atomic_ops_to_root at ffffffffb95359a6
> #8 [ffff9a94817139c0] bnxt_qplib_determine_atomics at ffffffffc08c1a33 [bnxt_re]
> #9 [ffff9a94817139d0] bnxt_re_dev_init at ffffffffc08ba2d1 [bnxt_re]
> RIP: 00007f450602f648 RSP: 00007ffe880869e8 RFLAGS: 00000246
> RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f450602f648
> RDX: 0000000000000002 RSI: 0000555c566c4a60 RDI: 0000000000000001
> RBP: 0000555c566c4a60 R8: 000000000000000a R9: 00007f45060c2580
> R10: 000000000000000a R11: 0000000000000246 R12: 00007f45063026e0
> R13: 0000000000000002 R14: 00007f45062fd880 R15: 0000000000000002
> ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b
>
> AtomicOp Requester Enable bit in the Device Control 2 register
> is reserved for VFs and drivers shouldn't enable it for VFs.
> Adding a check to return EINVAL if pci_enable_atomic_ops_to_root
> is called with VF pci device.
>
> Fixes: 35f5ace5dea4 ("RDMA/bnxt_re: Enable global atomic ops if platform supports")
> Fixes: 430a23689dea ("PCI: Add pci_enable_atomic_ops_to_root()")
> Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Applied to pci/enumeration for v5.16, thanks!
I fixed the extra space and rewrapped the code comment so it fits in
80 columns.
> ---
> drivers/pci/pci.c | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index aacf575..d968a36 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -3702,6 +3702,14 @@ int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)
> struct pci_dev *bridge;
> u32 cap, ctl2;
>
> + /*
> + * As per PCIe r5.0, sec 9.3.5.10, the AtomicOp Requester Enable
> + * bit in the Device Control 2 register is reserved in VFs and the PF
> + * value applies to all associated VFs. Return -EINVAL if called for VFs.
> + */
> + if (dev->is_virtfn)
> + return -EINVAL;
> +
> if (!pci_is_pcie(dev))
> return -EINVAL;
>
> --
> 2.5.5
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-09-14 20:16 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-11 10:03 [PATCH] PCI: Do not enable pci atomics on VFs Selvin Xavier
2021-09-13 12:28 ` Andy Gospodarek
2021-09-14 20:16 ` Bjorn Helgaas
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).