From: Bjorn Helgaas <helgaas@kernel.org> To: Jayachandran C <jnair@caviumnetworks.com> Cc: linux-pci@vger.kernel.org, Alex Williamson <alex.williamson@redhat.com>, iommu@lists.linux-foundation.org, Jon Masters <jcm@redhat.com>, Robin Murphy <robin.murphy@arm.com>, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH v4 2/2] PCI: quirks: Fix ThunderX2 dma alias handling Date: Mon, 10 Apr 2017 20:28:47 -0500 [thread overview] Message-ID: <20170411012847.GA15291@bhelgaas-glaptop.roam.corp.google.com> (raw) In-Reply-To: <1491225304-3559-3-git-send-email-jnair@caviumnetworks.com> Hi Jayachandran, On Mon, Apr 03, 2017 at 01:15:04PM +0000, Jayachandran C wrote: > The Cavium ThunderX2 arm64 SoCs (called Broadcom Vulcan earlier), the PCI > topology is slightly unusual. For a multi-node system, it looks like: > > [node level PCI bridges - one per node] > [SoC PCI devices with MSI-X but no IOMMU] > [PCI-PCIe "glue" bridges - upto 14, one per real port below] > [PCIe real root ports associated with IOMMU and GICv3 ITS] > [External PCI devices connected to PCIe links] > > The top two levels of bridges should have introduced aliases since they > are PCI and PCI/PCIe bridges, but in the case of ThunderX2 they do not. > In the case of external PCIe devices, the "real" root ports are connected > to the SMMU and the GIC ITS, so PCI-PCIe bridge does not introduce an > alias. The SoC PCI devices are directly connected to the GIC ITS, so the > node level bridges do not introduce an alias either. > > To handle this quirk, we mark the real PCIe root ports and node level > PCI bridges with the flag PCI_DEV_FLAGS_BRIDGE_XLATE_ROOT. With this, > pci_for_each_dma_alias() works correctly for external PCIe devices and > SoC PCI devices. > > For the current revision of Cavium ThunderX2, the VendorID and Device ID > are from Broadcom Vulcan (14e4:90XX). Can you supply some text here about why we want to apply this patch? E.g., does it avoid making unnecessary IOMMU mappings, improve performance, avoid a crash, etc? > Signed-off-by: Jayachandran C <jnair@caviumnetworks.com> > --- > drivers/pci/quirks.c | 14 ++++++++++++++ > 1 file changed, 14 insertions(+) > > diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c > index 6736836..564a84a 100644 > --- a/drivers/pci/quirks.c > +++ b/drivers/pci/quirks.c > @@ -3958,6 +3958,20 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2260, quirk_mic_x200_dma_alias); > DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2264, quirk_mic_x200_dma_alias); > > /* > + * The IOMMU and interrupt controller on Broadcom Vulcan/Cavium ThunderX2 are > + * associated not at the root bus, but at a bridge below. This quirk flag > + * will ensure that the aliases are identified correctly. > + */ > +static void quirk_bridge_cavm_thrx2_pcie_root(struct pci_dev *pdev) > +{ > + pdev->dev_flags |= PCI_DEV_FLAGS_BRIDGE_XLATE_ROOT; > +} > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9000, > + quirk_bridge_cavm_thrx2_pcie_root); > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9084, > + quirk_bridge_cavm_thrx2_pcie_root); > + > +/* > * Intersil/Techwell TW686[4589]-based video capture cards have an empty (zero) > * class code. Fix it. > */ > -- > 2.7.4 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
WARNING: multiple messages have this Message-ID (diff)
From: helgaas@kernel.org (Bjorn Helgaas) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 2/2] PCI: quirks: Fix ThunderX2 dma alias handling Date: Mon, 10 Apr 2017 20:28:47 -0500 [thread overview] Message-ID: <20170411012847.GA15291@bhelgaas-glaptop.roam.corp.google.com> (raw) In-Reply-To: <1491225304-3559-3-git-send-email-jnair@caviumnetworks.com> Hi Jayachandran, On Mon, Apr 03, 2017 at 01:15:04PM +0000, Jayachandran C wrote: > The Cavium ThunderX2 arm64 SoCs (called Broadcom Vulcan earlier), the PCI > topology is slightly unusual. For a multi-node system, it looks like: > > [node level PCI bridges - one per node] > [SoC PCI devices with MSI-X but no IOMMU] > [PCI-PCIe "glue" bridges - upto 14, one per real port below] > [PCIe real root ports associated with IOMMU and GICv3 ITS] > [External PCI devices connected to PCIe links] > > The top two levels of bridges should have introduced aliases since they > are PCI and PCI/PCIe bridges, but in the case of ThunderX2 they do not. > In the case of external PCIe devices, the "real" root ports are connected > to the SMMU and the GIC ITS, so PCI-PCIe bridge does not introduce an > alias. The SoC PCI devices are directly connected to the GIC ITS, so the > node level bridges do not introduce an alias either. > > To handle this quirk, we mark the real PCIe root ports and node level > PCI bridges with the flag PCI_DEV_FLAGS_BRIDGE_XLATE_ROOT. With this, > pci_for_each_dma_alias() works correctly for external PCIe devices and > SoC PCI devices. > > For the current revision of Cavium ThunderX2, the VendorID and Device ID > are from Broadcom Vulcan (14e4:90XX). Can you supply some text here about why we want to apply this patch? E.g., does it avoid making unnecessary IOMMU mappings, improve performance, avoid a crash, etc? > Signed-off-by: Jayachandran C <jnair@caviumnetworks.com> > --- > drivers/pci/quirks.c | 14 ++++++++++++++ > 1 file changed, 14 insertions(+) > > diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c > index 6736836..564a84a 100644 > --- a/drivers/pci/quirks.c > +++ b/drivers/pci/quirks.c > @@ -3958,6 +3958,20 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2260, quirk_mic_x200_dma_alias); > DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2264, quirk_mic_x200_dma_alias); > > /* > + * The IOMMU and interrupt controller on Broadcom Vulcan/Cavium ThunderX2 are > + * associated not at the root bus, but at a bridge below. This quirk flag > + * will ensure that the aliases are identified correctly. > + */ > +static void quirk_bridge_cavm_thrx2_pcie_root(struct pci_dev *pdev) > +{ > + pdev->dev_flags |= PCI_DEV_FLAGS_BRIDGE_XLATE_ROOT; > +} > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9000, > + quirk_bridge_cavm_thrx2_pcie_root); > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9084, > + quirk_bridge_cavm_thrx2_pcie_root); > + > +/* > * Intersil/Techwell TW686[4589]-based video capture cards have an empty (zero) > * class code. Fix it. > */ > -- > 2.7.4 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel at lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2017-04-11 1:28 UTC|newest] Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-04-03 13:15 [PATCH v4 0/2] Handle Cavium ThunderX2 PCI topology quirk Jayachandran C 2017-04-03 13:15 ` Jayachandran C 2017-04-03 13:15 ` Jayachandran C 2017-04-03 13:15 ` [PATCH v4 1/2] PCI: Add device flag PCI_DEV_FLAGS_BRIDGE_XLATE_ROOT Jayachandran C 2017-04-03 13:15 ` Jayachandran C 2017-04-03 13:15 ` Jayachandran C 2017-04-03 14:59 ` Robin Murphy 2017-04-03 14:59 ` Robin Murphy 2017-04-03 14:59 ` Robin Murphy 2017-04-03 13:15 ` [PATCH v4 2/2] PCI: quirks: Fix ThunderX2 dma alias handling Jayachandran C 2017-04-03 13:15 ` Jayachandran C 2017-04-03 13:15 ` Jayachandran C 2017-04-03 15:07 ` Robin Murphy 2017-04-03 15:07 ` Robin Murphy 2017-04-03 15:07 ` Robin Murphy 2017-04-04 11:50 ` Jayachandran C 2017-04-04 11:50 ` Jayachandran C 2017-04-04 11:50 ` Jayachandran C 2017-04-04 14:28 ` Robin Murphy 2017-04-04 14:28 ` Robin Murphy 2017-04-04 14:28 ` Robin Murphy 2017-04-10 11:38 ` Jayachandran C 2017-04-10 11:38 ` Jayachandran C 2017-04-10 11:38 ` Jayachandran C 2017-04-13 6:43 ` Jon Masters 2017-04-13 6:43 ` Jon Masters 2017-04-11 1:28 ` Bjorn Helgaas [this message] 2017-04-11 1:28 ` Bjorn Helgaas 2017-04-11 7:10 ` Jayachandran C 2017-04-11 7:10 ` Jayachandran C 2017-04-11 7:10 ` Jayachandran C 2017-04-11 13:41 ` Bjorn Helgaas 2017-04-11 13:41 ` Bjorn Helgaas 2017-04-11 13:41 ` Bjorn Helgaas 2017-04-11 15:27 ` Jayachandran C 2017-04-11 15:27 ` Jayachandran C 2017-04-11 15:27 ` Jayachandran C 2017-04-11 15:43 ` Jon Masters 2017-04-11 15:43 ` Jon Masters 2017-04-12 16:21 ` Bjorn Helgaas 2017-04-12 16:21 ` Bjorn Helgaas 2017-04-12 16:21 ` Bjorn Helgaas 2017-04-12 18:10 ` Jayachandran C 2017-04-12 18:10 ` Jayachandran C 2017-04-12 18:10 ` Jayachandran C 2017-04-12 19:11 ` Bjorn Helgaas 2017-04-12 19:11 ` Bjorn Helgaas 2017-04-12 19:11 ` Bjorn Helgaas 2017-04-12 20:41 ` Jayachandran C 2017-04-12 20:41 ` Jayachandran C 2017-04-12 20:41 ` Jayachandran C 2017-04-12 23:18 ` Bjorn Helgaas 2017-04-12 23:18 ` Bjorn Helgaas 2017-04-12 23:18 ` Bjorn Helgaas 2017-04-11 15:34 ` Robin Murphy 2017-04-11 15:34 ` Robin Murphy 2017-04-11 15:34 ` Robin Murphy 2017-04-11 13:44 ` [PATCH v4 0/2] Handle Cavium ThunderX2 PCI topology quirk Bjorn Helgaas 2017-04-11 13:44 ` Bjorn Helgaas 2017-04-11 13:44 ` Bjorn Helgaas [not found] ` <CABhMZUXNhKSQALAHP1CBNfWMuw0J0XQ2rzusP4WR_HHH9ox5Yw@mail.gmail.com> [not found] ` <CABhMZUXh=X5k1DQhUcaXD4t9GWfXms80xWV7sAh0ZXD8YK794g@mail.gmail.com> 2017-04-11 14:23 ` Bjorn Helgaas 2017-04-11 14:23 ` Bjorn Helgaas 2017-04-11 16:01 ` David Daney 2017-04-11 16:01 ` David Daney
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20170411012847.GA15291@bhelgaas-glaptop.roam.corp.google.com \ --to=helgaas@kernel.org \ --cc=alex.williamson@redhat.com \ --cc=iommu@lists.linux-foundation.org \ --cc=jcm@redhat.com \ --cc=jnair@caviumnetworks.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-pci@vger.kernel.org \ --cc=robin.murphy@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.