From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sergei Shtylyov Date: Tue, 24 Jun 2014 21:46:30 +0000 Subject: Re: [PATCH 8/8] ARM: shmobile: henninger: Add dummy PCIe bus clock Message-Id: <53A9F1B6.3060607@cogentembedded.com> List-Id: References: <1401261843-6964-9-git-send-email-phil.edworthy@renesas.com> In-Reply-To: <1401261843-6964-9-git-send-email-phil.edworthy@renesas.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: linux-sh@vger.kernel.org Hello. On 06/23/2014 01:19 PM, Phil Edworthy wrote: [...] >> So I wanted to ask: how did you test PCIe? > I used the following cards: > Intel PRO/1000 PT Server Adapter > Intel Gigabit CT Desktop Adapter > Intel Ethernet Server Adapter I350-T2 > TP-LINK TG-3468 (r8618 based) My boss (helping me with the remote debugging on site) said we had tried all these cards on Henninger and the Intel cards just didn't get detected. The TP-LINK card required a firmware which we're to find yet... > However, I could not find a card that used I/O, all use memory resources. The Intel cards seem to have I/O resources, it's probably the driver that prefers to always use MMIO... >>>> That PCI-PCI bridge has ID 1912:001f which corresponds to Renesas. I >>>> guess >>>> it's a part of the PCIe controller and then I wonder why this device is not >>>> seen when no card is inserted... >>> Simply because the hardware does not support hot plug. So, during probe >> the driver checks for link up, and if so it will call pci_common_init_dev(). >> Not sure how it's connected, could you elaborate? > I'm not sure what you are asking. The Koelsch board has PCIe connected, no mods required (except adding a link to provide the 3.3v aux supply for one card). Actually I meant to ask how the lack of hot plug and the visibility of the built-in bridge is connected. Now I've seen that kernel dies scanning bus with no PCIe link for some (yet unobvious) reason... >> And you don't indicate probe error when there's no link anyway -- which is >> strange. > If there is no link up during probe, it will not be able to get a link up later on, as the R-Car device does not support PCIe hot plug. If link up fails, then the driver will report this, e.g.: > rcar-pcie fe000000.pcie: PCI: PCIe link down That I saw perfectly well. :-) I was not asking about the messages. Well, we're discussing this matter in another thread already... >> As I already said, the PCI-PCI bridge's space doesn't look correct: >> root@10.0.0.111:~# lspci -v >> 00:00.0 PCI bridge: Unknown device 1912:001f (prog-if 00 [Normal decode]) >> Flags: bus master, fast devsel, latency 0 >> Bus: primary, secondary, subordinate, sec-latency=0 >> I/O behind bridge: 00000000-00000fff >> Memory behind bridge: 00000000-000fffff >> Prefetchable memory behind bridge: 00000000-000fffff >> Capabilities: [40] Power Management version 3 >> Capabilities: [50] Message Signalled Interrupts: Mask+ 64bit+ Queue=0/0 >> Enable+ >> Capabilities: [70] Express Root Port (Slot-) IRQ 0 >> Capabilities: [100] Virtual Channel >> Capabilities: [1b0] Device Serial Number 00-00-00-00-00-00-00-00 >> >> 01:00.0 Network controller: Intel Corporation Unknown device 088e (rev 24) >> Subsystem: Intel Corporation Unknown device 4060 >> Flags: bus master, fast devsel, latency 0, IRQ 418 >> Memory at fe200000 (64-bit, non-prefetchable) [size=8K] >> Capabilities: [c8] Power Management version 3 >> Capabilities: [d0] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 >> Enable+ >> Capabilities: [e0] Express Endpoint IRQ 0 >> Capabilities: [100] Advanced Error Reporting >> Capabilities: [140] Device Serial Number cf-9c-3e-ff-ff-87-d9-c4 >> The memory behind bridge should include the 0xfe2000000 address in >> card's >> BAR0 but it doesn't... I/O and prefetchable memory behind bridge also don't >> look right. The strange thing is the kernel reassigns the memory behind >> bridge's range but never writes it back into the PCI-PCI bridge: >> pcieport 0000:00:00.0: BAR 8: assigned [mem 0xfe200000-0xfe2fffff] >> pci 0000:01:00.0: BAR 0: assigned [mem 0xfe200000-0xfe201fff 64bit] > That does look very odd. This is what I get: > root@koelsch:~# lspci -v > 00:00.0 PCI bridge: Renesas Technology Corp. Device 001f (prog-if 00 [Normal decode]) > Flags: bus master, fast devsel, latency 0 > Bus: primary, secondary, subordinate, sec-latency=0 > I/O behind bridge: 00001000-00001fff > Memory behind bridge: fe200000-fe2fffff > Prefetchable memory behind bridge: 38000000-380fffff > Capabilities: [40] Power Management version 3 > Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ > Capabilities: [70] Express Root Port (Slot-), MSI 00 > Capabilities: [100] Virtual Channel > Capabilities: [1b0] Device Serial Number 00-00-00-00-00-00-00-00 > > 01:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection > Subsystem: Intel Corporation Gigabit CT Desktop Adapter > Flags: bus master, fast devsel, latency 0, IRQ 148 > Memory at fe280000 (32-bit, non-prefetchable) [size8K] > Memory at fe200000 (32-bit, non-prefetchable) [sizeQ2K] > I/O ports at 1000 [disabled] [size2] > Memory at fe2a0000 (32-bit, non-prefetchable) [sizeK] > [virtual] Expansion ROM at 38000000 [disabled] [size%6K] > Capabilities: [c8] Power Management version 2 > Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+ > Capabilities: [e0] Express Endpoint, MSI 00 > Capabilities: [a0] MSI-X: Enable+ Count=5 Masked- > Capabilities: [100] Advanced Error Reporting > Capabilities: [140] Device Serial Number 68-05-ca-ff-ff-19-9d-31 > Kernel driver in use: e1000e Yes, the bridge looks to be set up correctly here. I've tried to test PCIe on the Koelsch board (without the 3.3VAUX resistor soldered though) and either a PCIe link didn't get detected or the kernel died horrible death. Trying to bypass the link check in the driver also resulted in a horrible kernel death as described in another thread... > Regards > Phil WBR, Sergei