[PATCH] pci: Add ACS quirk for Broadcom NIC.

[PATCH] pci: Add ACS quirk for Broadcom NIC.

[Michael Chan]

From: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>

Some Broadcom NICs such as the BCM57414 do not advertise PCI-ACS
capability. All functions on such devices are placed under the same
IOMMU group. Attaching a single PF to a userspace application like
OVS-DPDK using VFIO is not possible, since not all functions in the
IOMMU group are bound to VFIO.

Since peer-to-peer transactions are not possible between PFs on these
devices, it is safe to treat them as fully isolated even though the ACS
capability is not advertised. Fix this issue by adding a PCI quirk for
this chip.

Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
 drivers/pci/quirks.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index dcb229de1acb..cb1628e222df 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -4753,6 +4753,8 @@ static const struct pci_dev_acs_enabled {
 	{ PCI_VENDOR_ID_AMPERE, 0xE00A, pci_quirk_xgene_acs },
 	{ PCI_VENDOR_ID_AMPERE, 0xE00B, pci_quirk_xgene_acs },
 	{ PCI_VENDOR_ID_AMPERE, 0xE00C, pci_quirk_xgene_acs },
+	/* Broadcom multi-function device */
+	{ PCI_VENDOR_ID_BROADCOM, 0x16D7, pci_quirk_mf_endpoint_acs },
 	{ PCI_VENDOR_ID_BROADCOM, 0xD714, pci_quirk_brcm_acs },
 	/* Amazon Annapurna Labs */
 	{ PCI_VENDOR_ID_AMAZON_ANNAPURNA_LABS, 0x0031, pci_quirk_al_acs },
-- 
2.18.1

Re: [PATCH] pci: Add ACS quirk for Broadcom NIC.

[Leon Romanovsky]

On Fri, May 21, 2021 at 09:13:17PM -0400, Michael Chan wrote:
> From: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
> 
> Some Broadcom NICs such as the BCM57414 do not advertise PCI-ACS
> capability. All functions on such devices are placed under the same
> IOMMU group. Attaching a single PF to a userspace application like
> OVS-DPDK using VFIO is not possible, since not all functions in the
> IOMMU group are bound to VFIO.
> 
> Since peer-to-peer transactions are not possible between PFs on these
> devices, it is safe to treat them as fully isolated even though the ACS
> capability is not advertised. Fix this issue by adding a PCI quirk for
> this chip.
> 
> Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
> Signed-off-by: Michael Chan <michael.chan@broadcom.com>
> ---
>  drivers/pci/quirks.c | 2 ++
>  1 file changed, 2 insertions(+)

Small nitpick, please don't put dot in the end of git commit subject.
[PATCH] pci: Add ACS quirk for Broadcom NIC.
                                          ^^^^

Thanks

Re: [PATCH] pci: Add ACS quirk for Broadcom NIC.

[Bjorn Helgaas]

On Fri, May 21, 2021 at 09:13:17PM -0400, Michael Chan wrote:
> From: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
> 
> Some Broadcom NICs such as the BCM57414 do not advertise PCI-ACS
> capability. All functions on such devices are placed under the same
> IOMMU group. Attaching a single PF to a userspace application like
> OVS-DPDK using VFIO is not possible, since not all functions in the
> IOMMU group are bound to VFIO.
> 
> Since peer-to-peer transactions are not possible between PFs on these
> devices, it is safe to treat them as fully isolated even though the ACS
> capability is not advertised. Fix this issue by adding a PCI quirk for
> this chip.
> 
> Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
> Signed-off-by: Michael Chan <michael.chan@broadcom.com>

Applied to pci/virtualization for v5.14, thanks!

I adjusted the subject and commit message to follow the typical style:

    PCI: Add ACS quirk for Broadcom BCM57414 NIC

    The Broadcom BCM57414 NIC may be a multi-function device.  While it does
    not advertise an ACS capability, peer-to-peer transactions are not possible
    between the individual functions, so it is safe to treat them as fully
    isolated.

    Add an ACS quirk for this device so the functions can be in independent
    IOMMU groups and attached individually to userspace applications using
    VFIO.

> ---
>  drivers/pci/quirks.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
> index dcb229de1acb..cb1628e222df 100644
> --- a/drivers/pci/quirks.c
> +++ b/drivers/pci/quirks.c
> @@ -4753,6 +4753,8 @@ static const struct pci_dev_acs_enabled {
>  	{ PCI_VENDOR_ID_AMPERE, 0xE00A, pci_quirk_xgene_acs },
>  	{ PCI_VENDOR_ID_AMPERE, 0xE00B, pci_quirk_xgene_acs },
>  	{ PCI_VENDOR_ID_AMPERE, 0xE00C, pci_quirk_xgene_acs },
> +	/* Broadcom multi-function device */
> +	{ PCI_VENDOR_ID_BROADCOM, 0x16D7, pci_quirk_mf_endpoint_acs },
>  	{ PCI_VENDOR_ID_BROADCOM, 0xD714, pci_quirk_brcm_acs },
>  	/* Amazon Annapurna Labs */
>  	{ PCI_VENDOR_ID_AMAZON_ANNAPURNA_LABS, 0x0031, pci_quirk_al_acs },
> -- 
> 2.18.1
>