From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Return-Path: Date: Fri, 21 Apr 2017 12:57:05 -0500 From: Bjorn Helgaas To: Jayachandran C Subject: Re: [PATCH v5 2/2] PCI: quirks: Fix ThunderX2 dma alias handling Message-ID: <20170421175705.GB17916@bhelgaas-glaptop.roam.corp.google.com> References: <1492115445-4069-1-git-send-email-jnair@caviumnetworks.com> <1492115445-4069-3-git-send-email-jnair@caviumnetworks.com> <20170414001911.GA13163@bhelgaas-glaptop.roam.corp.google.com> <20170414210646.GA5346@localhost> <20170417174712.GA7575@localhost> <20170421170414.GA2429@localhost> MIME-Version: 1.0 In-Reply-To: <20170421170414.GA2429@localhost> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "linux-pci@vger.kernel.org" , Joerg Roedel , "open list:INTEL IOMMU \(VT-d\)" , Jon Masters , Bjorn Helgaas , Robin Murphy , linux-arm Content-Type: text/plain; charset="us-ascii" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+bjorn=helgaas.com@lists.infradead.org List-ID: On Fri, Apr 21, 2017 at 05:05:41PM +0000, Jayachandran C wrote: > On Fri, Apr 21, 2017 at 10:48:15AM -0500, Bjorn Helgaas wrote: > > On Mon, Apr 17, 2017 at 12:47 PM, Jayachandran C > > wrote: > > > On Fri, Apr 14, 2017 at 09:00:06PM -0500, Bjorn Helgaas wrote: > > >> Could you collect "lspci -vv" output from this system? I'd like to > > >> archive that as background for this IOMMU issue and the ASPM tweaks I > > >> suspect we'll have to do. I *wish* we had more information about that > > >> VIA thing, because I suspect we could get rid of it if we had more > > >> details. > > > > > > The full logs are slightly large, so I have kept them at: > > > https://github.com/jchandra-cavm/thunderx2/blob/master/logs/ > > > The lspci -vv output is lspci-vv.txt and lspci -tvn output is lspci-tvn.txt > > > > > > The output is from 2 socket system, the cards are not on the first slot > > > like the example above, so the bus and device numbers are different. > > > > Can somebody with this system collect the "lspci -xxxx" output as well? > > > > I'm making some lspci changes to handle the PCI-to-PCIe bridge > > correctly, and I can use the "lspci -xxxx" output to create an lspci > > test case. > > [Sorry was AFK for a few days] > > I have updated the above directory with the log. Also tested your next branch > and it works fine on ThunderX2. Thanks! With regard to my lspci changes, they add "Slot-" here: 01:0a.0 PCI bridge: Broadcom Corporation Device 9039 ... - Capabilities: [40] Express (v2) PCI/PCI-X to PCI-Express Bridge, MSI 00 + Capabilities: [40] Express (v2) PCI/PCI-X to PCI-Express Bridge (Slot-), MSI 00 for all your PCI-to-PCIe bridges. I assume the "Slot-" is correct, i.e., the link is not connected to a slot, right? This comes from the "Slot Implemented" bit in the PCIe Capabilities Register. I did notice that all the Root Port devices claim to *not* be connected to slots, which doesn't seem right. For example, 12:00.0 PCI bridge: Broadcom Corporation Device 9084 Bus: primary=12, secondary=13, subordinate=14, sec-latency=0 Capabilities: [ac] Express (v2) Root Port (Slot-), MSI 00 13:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection It seems strange because the 12:00.0 Root Port looks like it probably *does* lead to a slot where the NIC is plugged in. Or is that NIC really soldered down? But I assume there are *some* PCIe slots, so at some of those Root Ports should advertise "Slot+" (which by itself does not imply hotplug support, if that's the concern). Bjorn _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bjorn Helgaas Subject: Re: [PATCH v5 2/2] PCI: quirks: Fix ThunderX2 dma alias handling Date: Fri, 21 Apr 2017 12:57:05 -0500 Message-ID: <20170421175705.GB17916@bhelgaas-glaptop.roam.corp.google.com> References: <1492115445-4069-1-git-send-email-jnair@caviumnetworks.com> <1492115445-4069-3-git-send-email-jnair@caviumnetworks.com> <20170414001911.GA13163@bhelgaas-glaptop.roam.corp.google.com> <20170414210646.GA5346@localhost> <20170417174712.GA7575@localhost> <20170421170414.GA2429@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <20170421170414.GA2429@localhost> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Jayachandran C Cc: "linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "open list:INTEL IOMMU (VT-d)" , Jon Masters , Bjorn Helgaas , linux-arm List-Id: iommu@lists.linux-foundation.org On Fri, Apr 21, 2017 at 05:05:41PM +0000, Jayachandran C wrote: > On Fri, Apr 21, 2017 at 10:48:15AM -0500, Bjorn Helgaas wrote: > > On Mon, Apr 17, 2017 at 12:47 PM, Jayachandran C > > wrote: > > > On Fri, Apr 14, 2017 at 09:00:06PM -0500, Bjorn Helgaas wrote: > > >> Could you collect "lspci -vv" output from this system? I'd like to > > >> archive that as background for this IOMMU issue and the ASPM tweaks I > > >> suspect we'll have to do. I *wish* we had more information about that > > >> VIA thing, because I suspect we could get rid of it if we had more > > >> details. > > > > > > The full logs are slightly large, so I have kept them at: > > > https://github.com/jchandra-cavm/thunderx2/blob/master/logs/ > > > The lspci -vv output is lspci-vv.txt and lspci -tvn output is lspci-tvn.txt > > > > > > The output is from 2 socket system, the cards are not on the first slot > > > like the example above, so the bus and device numbers are different. > > > > Can somebody with this system collect the "lspci -xxxx" output as well? > > > > I'm making some lspci changes to handle the PCI-to-PCIe bridge > > correctly, and I can use the "lspci -xxxx" output to create an lspci > > test case. > > [Sorry was AFK for a few days] > > I have updated the above directory with the log. Also tested your next branch > and it works fine on ThunderX2. Thanks! With regard to my lspci changes, they add "Slot-" here: 01:0a.0 PCI bridge: Broadcom Corporation Device 9039 ... - Capabilities: [40] Express (v2) PCI/PCI-X to PCI-Express Bridge, MSI 00 + Capabilities: [40] Express (v2) PCI/PCI-X to PCI-Express Bridge (Slot-), MSI 00 for all your PCI-to-PCIe bridges. I assume the "Slot-" is correct, i.e., the link is not connected to a slot, right? This comes from the "Slot Implemented" bit in the PCIe Capabilities Register. I did notice that all the Root Port devices claim to *not* be connected to slots, which doesn't seem right. For example, 12:00.0 PCI bridge: Broadcom Corporation Device 9084 Bus: primary=12, secondary=13, subordinate=14, sec-latency=0 Capabilities: [ac] Express (v2) Root Port (Slot-), MSI 00 13:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection It seems strange because the 12:00.0 Root Port looks like it probably *does* lead to a slot where the NIC is plugged in. Or is that NIC really soldered down? But I assume there are *some* PCIe slots, so at some of those Root Ports should advertise "Slot+" (which by itself does not imply hotplug support, if that's the concern). Bjorn From mboxrd@z Thu Jan 1 00:00:00 1970 From: helgaas@kernel.org (Bjorn Helgaas) Date: Fri, 21 Apr 2017 12:57:05 -0500 Subject: [PATCH v5 2/2] PCI: quirks: Fix ThunderX2 dma alias handling In-Reply-To: <20170421170414.GA2429@localhost> References: <1492115445-4069-1-git-send-email-jnair@caviumnetworks.com> <1492115445-4069-3-git-send-email-jnair@caviumnetworks.com> <20170414001911.GA13163@bhelgaas-glaptop.roam.corp.google.com> <20170414210646.GA5346@localhost> <20170417174712.GA7575@localhost> <20170421170414.GA2429@localhost> Message-ID: <20170421175705.GB17916@bhelgaas-glaptop.roam.corp.google.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On Fri, Apr 21, 2017 at 05:05:41PM +0000, Jayachandran C wrote: > On Fri, Apr 21, 2017 at 10:48:15AM -0500, Bjorn Helgaas wrote: > > On Mon, Apr 17, 2017 at 12:47 PM, Jayachandran C > > wrote: > > > On Fri, Apr 14, 2017 at 09:00:06PM -0500, Bjorn Helgaas wrote: > > >> Could you collect "lspci -vv" output from this system? I'd like to > > >> archive that as background for this IOMMU issue and the ASPM tweaks I > > >> suspect we'll have to do. I *wish* we had more information about that > > >> VIA thing, because I suspect we could get rid of it if we had more > > >> details. > > > > > > The full logs are slightly large, so I have kept them at: > > > https://github.com/jchandra-cavm/thunderx2/blob/master/logs/ > > > The lspci -vv output is lspci-vv.txt and lspci -tvn output is lspci-tvn.txt > > > > > > The output is from 2 socket system, the cards are not on the first slot > > > like the example above, so the bus and device numbers are different. > > > > Can somebody with this system collect the "lspci -xxxx" output as well? > > > > I'm making some lspci changes to handle the PCI-to-PCIe bridge > > correctly, and I can use the "lspci -xxxx" output to create an lspci > > test case. > > [Sorry was AFK for a few days] > > I have updated the above directory with the log. Also tested your next branch > and it works fine on ThunderX2. Thanks! With regard to my lspci changes, they add "Slot-" here: 01:0a.0 PCI bridge: Broadcom Corporation Device 9039 ... - Capabilities: [40] Express (v2) PCI/PCI-X to PCI-Express Bridge, MSI 00 + Capabilities: [40] Express (v2) PCI/PCI-X to PCI-Express Bridge (Slot-), MSI 00 for all your PCI-to-PCIe bridges. I assume the "Slot-" is correct, i.e., the link is not connected to a slot, right? This comes from the "Slot Implemented" bit in the PCIe Capabilities Register. I did notice that all the Root Port devices claim to *not* be connected to slots, which doesn't seem right. For example, 12:00.0 PCI bridge: Broadcom Corporation Device 9084 Bus: primary=12, secondary=13, subordinate=14, sec-latency=0 Capabilities: [ac] Express (v2) Root Port (Slot-), MSI 00 13:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection It seems strange because the 12:00.0 Root Port looks like it probably *does* lead to a slot where the NIC is plugged in. Or is that NIC really soldered down? But I assume there are *some* PCIe slots, so at some of those Root Ports should advertise "Slot+" (which by itself does not imply hotplug support, if that's the concern). Bjorn