* PICe hotplug problems
@ 2012-07-10 17:29 Joakim Tjernlund
2012-07-10 18:22 ` Yinghai Lu
0 siblings, 1 reply; 17+ messages in thread
From: Joakim Tjernlund @ 2012-07-10 17:29 UTC (permalink / raw)
To: linux-pci
I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the
device enables its PCIe interface.
The device is connected to a built in root bridge on a P2010(mpc85xx) CPU.
To enable HP in Linux we need to apply some minor hacks.
First, The contoller has MSI but there is no pci_find_capability(dev, PCI_CAP_ID_MSI) so
portdrv_core.c:init_service_irqs() fails this test(dev->pin is false too):
if (!pci_enable_msi(dev) || dev->pin)
Is the code correct? Should our root bridge have PCI_CAP_ID_MSI?
Second problem, the root bride does not have PCI_EXP_FLAGS_SLOT so
portdrv_core.c:get_port_device_capability() fails this test:
if ((cap_mask & PCIE_PORT_SERVICE_HP) && (reg16 & PCI_EXP_FLAGS_SLOT)) {
pci_read_config_dword(dev, pos + PCI_EXP_SLTCAP, ®32);
if (reg32 & PCI_EXP_SLTCAP_HPC) {
services |= PCIE_PORT_SERVICE_HP;
which cause PCIE_PORT_SERVICE_HP to be unset in services.
Is PCI_EXP_FLAGS_SLOT really required for hot plug?
Perhaps there another way?
Jocke
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: PICe hotplug problems 2012-07-10 17:29 PICe hotplug problems Joakim Tjernlund @ 2012-07-10 18:22 ` Yinghai Lu 2012-07-10 21:42 ` Joakim Tjernlund 0 siblings, 1 reply; 17+ messages in thread From: Yinghai Lu @ 2012-07-10 18:22 UTC (permalink / raw) To: Joakim Tjernlund; +Cc: linux-pci On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote: > > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the > device enables its PCIe interface. > > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU. > To enable HP in Linux we need to apply some minor hacks. can you post lspci -vvxxx -s BB:DD:F of the two devices? Yinghai ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: PICe hotplug problems 2012-07-10 18:22 ` Yinghai Lu @ 2012-07-10 21:42 ` Joakim Tjernlund 2012-07-10 22:09 ` Yinghai Lu 0 siblings, 1 reply; 17+ messages in thread From: Joakim Tjernlund @ 2012-07-10 21:42 UTC (permalink / raw) To: Yinghai Lu; +Cc: linux-pci, yhlu.kernel yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09: > > On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund > <joakim.tjernlund@transmode.se> wrote: > > > > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the > > device enables its PCIe interface. > > > > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU. > > To enable HP in Linux we need to apply some minor hacks. > > can you post lspci -vvxxx -s BB:DD:F of the two devices? Not really, this is an embedded device with limited SW. I got busybox and its lspci but that is very limited: # > ./busybox lspci -mk 00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport" 01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540" Does that tell you anything? Jocke ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: PICe hotplug problems 2012-07-10 21:42 ` Joakim Tjernlund @ 2012-07-10 22:09 ` Yinghai Lu 2012-07-10 22:20 ` Bjorn Helgaas 2012-07-11 1:07 ` Joakim Tjernlund 0 siblings, 2 replies; 17+ messages in thread From: Yinghai Lu @ 2012-07-10 22:09 UTC (permalink / raw) To: Joakim Tjernlund; +Cc: linux-pci On Tue, Jul 10, 2012 at 2:42 PM, Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote: > yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09: > >> >> On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund >> <joakim.tjernlund@transmode.se> wrote: >> > >> > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the >> > device enables its PCIe interface. >> > >> > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU. >> > To enable HP in Linux we need to apply some minor hacks. >> >> can you post lspci -vvxxx -s BB:DD:F of the two devices? > > Not really, this is an embedded device with limited SW. I got > busybox and its lspci but that is very limited: > # > ./busybox lspci -mk > 00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport" > 01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540" > > Does that tell you anything? No. Can you compile lspci util as static and run it ? Thanks Yinghai ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: PICe hotplug problems 2012-07-10 22:09 ` Yinghai Lu @ 2012-07-10 22:20 ` Bjorn Helgaas 2012-07-10 22:22 ` Yinghai Lu 2012-07-11 1:07 ` Joakim Tjernlund 1 sibling, 1 reply; 17+ messages in thread From: Bjorn Helgaas @ 2012-07-10 22:20 UTC (permalink / raw) To: Yinghai Lu; +Cc: Joakim Tjernlund, linux-pci On Tue, Jul 10, 2012 at 4:09 PM, Yinghai Lu <yinghai@kernel.org> wrote: > On Tue, Jul 10, 2012 at 2:42 PM, Joakim Tjernlund > <joakim.tjernlund@transmode.se> wrote: >> yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09: >> >>> >>> On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund >>> <joakim.tjernlund@transmode.se> wrote: >>> > >>> > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the >>> > device enables its PCIe interface. >>> > >>> > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU. >>> > To enable HP in Linux we need to apply some minor hacks. >>> >>> can you post lspci -vvxxx -s BB:DD:F of the two devices? >> >> Not really, this is an embedded device with limited SW. I got >> busybox and its lspci but that is very limited: >> # > ./busybox lspci -mk >> 00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport" >> 01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540" >> >> Does that tell you anything? > > No. Can you compile lspci util as static and run it ? You might also be able to get this info as console output by booting with "pci=earlydump". ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: PICe hotplug problems 2012-07-10 22:20 ` Bjorn Helgaas @ 2012-07-10 22:22 ` Yinghai Lu 2012-07-10 22:46 ` Bjorn Helgaas 0 siblings, 1 reply; 17+ messages in thread From: Yinghai Lu @ 2012-07-10 22:22 UTC (permalink / raw) To: Bjorn Helgaas; +Cc: Joakim Tjernlund, linux-pci On Tue, Jul 10, 2012 at 3:20 PM, Bjorn Helgaas <bhelgaas@google.com> wrote: > On Tue, Jul 10, 2012 at 4:09 PM, Yinghai Lu <yinghai@kernel.org> wrote: >> On Tue, Jul 10, 2012 at 2:42 PM, Joakim Tjernlund >> <joakim.tjernlund@transmode.se> wrote: >>> yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09: >>> >>>> >>>> On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund >>>> <joakim.tjernlund@transmode.se> wrote: >>>> > >>>> > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the >>>> > device enables its PCIe interface. >>>> > >>>> > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU. >>>> > To enable HP in Linux we need to apply some minor hacks. >>>> >>>> can you post lspci -vvxxx -s BB:DD:F of the two devices? >>> >>> Not really, this is an embedded device with limited SW. I got >>> busybox and its lspci but that is very limited: >>> # > ./busybox lspci -mk >>> 00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport" >>> 01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540" >>> >>> Does that tell you anything? >> >> No. Can you compile lspci util as static and run it ? > > You might also be able to get this info as console output by booting > with "pci=earlydump". that is only for x86. and his setup is with P2010(mpc85xx) CPU. Thanks Yinghai ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: PICe hotplug problems 2012-07-10 22:22 ` Yinghai Lu @ 2012-07-10 22:46 ` Bjorn Helgaas 2012-07-10 22:49 ` Bjorn Helgaas 0 siblings, 1 reply; 17+ messages in thread From: Bjorn Helgaas @ 2012-07-10 22:46 UTC (permalink / raw) To: Yinghai Lu; +Cc: Joakim Tjernlund, linux-pci On Tue, Jul 10, 2012 at 4:22 PM, Yinghai Lu <yinghai@kernel.org> wrote: > On Tue, Jul 10, 2012 at 3:20 PM, Bjorn Helgaas <bhelgaas@google.com> wrote: >> On Tue, Jul 10, 2012 at 4:09 PM, Yinghai Lu <yinghai@kernel.org> wrote: >>> On Tue, Jul 10, 2012 at 2:42 PM, Joakim Tjernlund >>> <joakim.tjernlund@transmode.se> wrote: >>>> yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09: >>>> >>>>> >>>>> On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund >>>>> <joakim.tjernlund@transmode.se> wrote: >>>>> > >>>>> > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the >>>>> > device enables its PCIe interface. >>>>> > >>>>> > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU. >>>>> > To enable HP in Linux we need to apply some minor hacks. >>>>> >>>>> can you post lspci -vvxxx -s BB:DD:F of the two devices? >>>> >>>> Not really, this is an embedded device with limited SW. I got >>>> busybox and its lspci but that is very limited: >>>> # > ./busybox lspci -mk >>>> 00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport" >>>> 01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540" >>>> >>>> Does that tell you anything? >>> >>> No. Can you compile lspci util as static and run it ? >> >> You might also be able to get this info as console output by booting >> with "pci=earlydump". > > that is only for x86. > > and his setup is with P2010(mpc85xx) CPU. Ah, yes. It's regrettable that this arch-independent functionality is implemented in an arch-specific way. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: PICe hotplug problems 2012-07-10 22:46 ` Bjorn Helgaas @ 2012-07-10 22:49 ` Bjorn Helgaas 0 siblings, 0 replies; 17+ messages in thread From: Bjorn Helgaas @ 2012-07-10 22:49 UTC (permalink / raw) To: Yinghai Lu; +Cc: Joakim Tjernlund, linux-pci On Tue, Jul 10, 2012 at 4:46 PM, Bjorn Helgaas <bhelgaas@google.com> wrote: > On Tue, Jul 10, 2012 at 4:22 PM, Yinghai Lu <yinghai@kernel.org> wrote: >> On Tue, Jul 10, 2012 at 3:20 PM, Bjorn Helgaas <bhelgaas@google.com> wrote: >>> On Tue, Jul 10, 2012 at 4:09 PM, Yinghai Lu <yinghai@kernel.org> wrote: >>>> On Tue, Jul 10, 2012 at 2:42 PM, Joakim Tjernlund >>>> <joakim.tjernlund@transmode.se> wrote: >>>>> yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09: >>>>> >>>>>> >>>>>> On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund >>>>>> <joakim.tjernlund@transmode.se> wrote: >>>>>> > >>>>>> > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the >>>>>> > device enables its PCIe interface. >>>>>> > >>>>>> > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU. >>>>>> > To enable HP in Linux we need to apply some minor hacks. >>>>>> >>>>>> can you post lspci -vvxxx -s BB:DD:F of the two devices? >>>>> >>>>> Not really, this is an embedded device with limited SW. I got >>>>> busybox and its lspci but that is very limited: >>>>> # > ./busybox lspci -mk >>>>> 00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport" >>>>> 01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540" >>>>> >>>>> Does that tell you anything? >>>> >>>> No. Can you compile lspci util as static and run it ? >>> >>> You might also be able to get this info as console output by booting >>> with "pci=earlydump". >> >> that is only for x86. >> >> and his setup is with P2010(mpc85xx) CPU. > > Ah, yes. It's regrettable that this arch-independent functionality is > implemented in an arch-specific way. But I guess the config-space accessors are arch-dependent, and we don't necessarily even *have* them early. So it probably is difficult to do this generically. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: PICe hotplug problems 2012-07-10 22:09 ` Yinghai Lu 2012-07-10 22:20 ` Bjorn Helgaas @ 2012-07-11 1:07 ` Joakim Tjernlund 2012-07-11 1:33 ` Yinghai Lu 1 sibling, 1 reply; 17+ messages in thread From: Joakim Tjernlund @ 2012-07-11 1:07 UTC (permalink / raw) To: Yinghai Lu; +Cc: linux-pci, yhlu.kernel yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00: > > On Tue, Jul 10, 2012 at 2:42 PM, Joakim Tjernlund > <joakim.tjernlund@transmode.se> wrote: > > yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09: > > > >> > >> On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund > >> <joakim.tjernlund@transmode.se> wrote: > >> > > >> > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the > >> > device enables its PCIe interface. > >> > > >> > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU. > >> > To enable HP in Linux we need to apply some minor hacks. > >> > >> can you post lspci -vvxxx -s BB:DD:F of the two devices? > > > > Not really, this is an embedded device with limited SW. I got > > busybox and its lspci but that is very limited: > > # > ./busybox lspci -mk > > 00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport" > > 01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540" > > > > Does that tell you anything? > > No. Can you compile lspci util as static and run it ? That wasn't so hard so here: root@P2020RDB ~ # ./lspci -vvxxx 00:00.0 Class 0604: Device 1957:0079 (rev 21) Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 32 bytes Region 0: Memory at <ignored> (32-bit, non-prefetchable) Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 00000000-00000fff Memory behind bridge: 80000000-9fffffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [44] Power Management version 2 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE- FLReset- DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Kernel driver in use: pcieport 00: 57 19 79 00 06 01 10 00 21 00 20 0b 08 00 01 00 10: 00 00 f0 ff 00 00 00 00 00 01 01 00 00 00 00 00 20: 00 80 f0 9f f1 ff 01 00 00 00 00 00 00 00 00 00 30: 00 00 00 00 44 00 00 00 00 00 00 00 00 00 00 00 40: 00 00 00 00 01 4c 02 fe 00 00 00 00 10 00 41 00 50: 01 00 00 00 1f 28 00 00 41 d4 03 00 08 00 21 00 60: c0 07 00 00 c8 03 40 00 00 00 00 00 00 00 00 00 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01:00.0 Class 0200: Device 14e4:b540 (rev 02) Subsystem: Device 14e4:b540 Physical Slot: 0 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 32 bytes Interrupt: pin A routed to IRQ 16 Region 0: Memory at 80000000 (64-bit, non-prefetchable) [size=256K] Capabilities: [48] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME- Capabilities: [50] Vital Product Data pcilib: sysfs_read_vpd: read failed: Connection timed out Not readable Capabilities: [58] MSI: Enable- Count=1/1 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [ac] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr+ NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend- LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s L1, Latency L0 <4us, L1 <4us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [100 v1] Device Serial Number 00-00-00-00-00-00-00-00 Capabilities: [110 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [150 v1] Power Budgeting <?> Capabilities: [160 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- 00: e4 14 40 b5 06 00 10 00 02 00 00 02 08 00 00 00 10: 04 00 00 80 00 00 00 00 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 e4 14 40 b5 30: 00 00 00 00 48 00 00 00 00 00 00 00 00 01 00 00 40: 00 00 00 00 00 00 00 00 01 50 03 c8 08 20 00 00 50: 03 58 00 00 00 00 00 00 05 ac 80 00 00 00 00 00 60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 a0: 11 00 00 00 00 00 00 00 00 00 00 00 10 00 02 00 b0: 02 80 3c 00 10 2c 10 00 22 6c 01 00 00 00 21 10 c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 d0: 1f 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: PICe hotplug problems 2012-07-11 1:07 ` Joakim Tjernlund @ 2012-07-11 1:33 ` Yinghai Lu 2012-07-11 4:08 ` Joakim Tjernlund ` (3 more replies) 0 siblings, 4 replies; 17+ messages in thread From: Yinghai Lu @ 2012-07-11 1:33 UTC (permalink / raw) To: Joakim Tjernlund; +Cc: linux-pci On Tue, Jul 10, 2012 at 6:07 PM, Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote: > yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00: >> No. Can you compile lspci util as static and run it ? > > That wasn't so hard so here: > > root@P2020RDB ~ # ./lspci -vvxxx > 00:00.0 Class 0604: Device 1957:0079 (rev 21) > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 32 bytes > Region 0: Memory at <ignored> (32-bit, non-prefetchable) > Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 > I/O behind bridge: 00000000-00000fff > Memory behind bridge: 80000000-9fffffff > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- > BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > Capabilities: [44] Power Management version 2 > Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00 > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us > ExtTag- RBE- FLReset- > DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > MaxPayload 128 bytes, MaxReadReq 512 bytes > DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- > LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited > ClockPM- Surprise- LLActRep- BwNot- > LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- > RootCap: CRSVisible- > RootSta: PME ReqID 0000, PMEStatus- PMEPending- There is no slot cap etc, so pciehp will not be loaded. the power of you child device can not be turned off/on. Not sure if can use link off/on make the clock effective. You can turn off and on the pcie link like following: 1. remove the child device echo 1 > /sys/..../0000:01:00.0/remove 2. disable link echo 1 > /sys/..../0000:00.00.0/pcie_link_disable 3. enable link echo 1 > /sys/..../0000:00.00.0/pcie_link_disable 4. rescan the pci bus. echo 1 > /sys/..../0000:00:00.0/rescan_bridge please check link disable patch at git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git for-pci-pcie-link | Subject: [PATCH] PCI: Add link_disable in /sysfs for pcie device | | Found PCIe cards from one vendor, will not respond to scan from bridge, | if we change bus number setting in bridge device. | | Have to do link disable/enable on the pcie root port. | | So try to expose link disable bit of pcie link control register. We can use | echo 1 > /sys/..../link_disable | echo 0 > /sys/..../link_disable | to bring the pcie device back to respond to scan. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: PICe hotplug problems 2012-07-11 1:33 ` Yinghai Lu @ 2012-07-11 4:08 ` Joakim Tjernlund 2012-07-11 6:09 ` Yinghai Lu [not found] ` <OF259177D8.B20ED592-ONC1257A38.00167C85-C1257A38.0016C890@LocalDomain> ` (2 subsequent siblings) 3 siblings, 1 reply; 17+ messages in thread From: Joakim Tjernlund @ 2012-07-11 4:08 UTC (permalink / raw) To: Yinghai Lu; +Cc: linux-pci, yhlu.kernel yhlu.kernel@gmail.com wrote on 2012/07/11 03:33:05: > > On Tue, Jul 10, 2012 at 6:07 PM, Joakim Tjernlund > <joakim.tjernlund@transmode.se> wrote: > > yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00: > > >> No. Can you compile lspci util as static and run it ? > > > > That wasn't so hard so here: > > > > root@P2020RDB ~ # ./lspci -vvxxx > > 00:00.0 Class 0604: Device 1957:0079 (rev 21) > > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > > Latency: 0, Cache Line Size: 32 bytes > > Region 0: Memory at <ignored> (32-bit, non-prefetchable) > > Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 > > I/O behind bridge: 00000000-00000fff > > Memory behind bridge: 80000000-9fffffff > > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- > > BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- > > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > > Capabilities: [44] Power Management version 2 > > Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00 > > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us > > ExtTag- RBE- FLReset- > > DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > > MaxPayload 128 bytes, MaxReadReq 512 bytes > > DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- > > LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited > > ClockPM- Surprise- LLActRep- BwNot- > > LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk- > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > > LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- > > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- > > RootCap: CRSVisible- > > RootSta: PME ReqID 0000, PMEStatus- PMEPending- > > There is no slot cap etc, so pciehp will not be loaded. > the power of you child device can not be turned off/on. > > Not sure if can use link off/on make the clock effective. > > You can turn off and on the pcie link like following: > > 1. remove the child device > echo 1 > /sys/..../0000:01:00.0/remove > 2. disable link > echo 1 > /sys/..../0000:00.00.0/pcie_link_disable > 3. enable link > echo 1 > /sys/..../0000:00.00.0/pcie_link_disable > 4. rescan the pci bus. > echo 1 > /sys/..../0000:00:00.0/rescan_bridge > > please check link disable patch at > git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git > for-pci-pcie-link Thanks, that was really quick. However, the patches does not apply on my 3.4 kernel and it looks non trivial to me fixup. Could you create (even quick and dirty) patches on top of 3.4? Jocke ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: PICe hotplug problems 2012-07-11 4:08 ` Joakim Tjernlund @ 2012-07-11 6:09 ` Yinghai Lu 0 siblings, 0 replies; 17+ messages in thread From: Yinghai Lu @ 2012-07-11 6:09 UTC (permalink / raw) To: Joakim Tjernlund; +Cc: linux-pci On Tue, Jul 10, 2012 at 9:08 PM, Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote: >> please check link disable patch at >> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git >> for-pci-pcie-link > > Thanks, that was really quick. However, the patches does not apply on my > 3.4 kernel and it looks non trivial to me fixup. > > Could you create (even quick and dirty) patches on top of 3.4? you can just use that branch instead. mkdir linux-2.6 || exit -1 cd linux-2.6 git init-db # Add Linus's tree as a remote git remote add linus git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git git remote add yh git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git git remote update git checkout -b yh_2012_07_10 yh/for-pci-pcie-link ^ permalink raw reply [flat|nested] 17+ messages in thread
[parent not found: <OF259177D8.B20ED592-ONC1257A38.00167C85-C1257A38.0016C890@LocalDomain>]
* Re: PICe hotplug problems [not found] ` <OF259177D8.B20ED592-ONC1257A38.00167C85-C1257A38.0016C890@LocalDomain> @ 2012-07-11 5:08 ` Joakim Tjernlund 2012-07-11 14:13 ` Jiang Liu 0 siblings, 1 reply; 17+ messages in thread From: Joakim Tjernlund @ 2012-07-11 5:08 UTC (permalink / raw) Cc: Yinghai Lu, linux-pci, yhlu.kernel Joakim Tjernlund/Transmode wrote on 2012/07/11 06:08:51: > > yhlu.kernel@gmail.com wrote on 2012/07/11 03:33:05: > > > > On Tue, Jul 10, 2012 at 6:07 PM, Joakim Tjernlund > > <joakim.tjernlund@transmode.se> wrote: > > > yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00: > > > > >> No. Can you compile lspci util as static and run it ? > > > > > > That wasn't so hard so here: > > > > > > root@P2020RDB ~ # ./lspci -vvxxx > > > 00:00.0 Class 0604: Device 1957:0079 (rev 21) > > > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- > > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > > > Latency: 0, Cache Line Size: 32 bytes > > > Region 0: Memory at <ignored> (32-bit, non-prefetchable) > > > Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 > > > I/O behind bridge: 00000000-00000fff > > > Memory behind bridge: 80000000-9fffffff > > > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- > > > BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- > > > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > > > Capabilities: [44] Power Management version 2 > > > Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) > > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > > Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00 > > > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us > > > ExtTag- RBE- FLReset- > > > DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ > > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > > > MaxPayload 128 bytes, MaxReadReq 512 bytes > > > DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- > > > LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited > > > ClockPM- Surprise- LLActRep- BwNot- > > > LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk- > > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > > > LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- > > > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- > > > RootCap: CRSVisible- > > > RootSta: PME ReqID 0000, PMEStatus- PMEPending- > > > > There is no slot cap etc, so pciehp will not be loaded. > > the power of you child device can not be turned off/on. > > > > Not sure if can use link off/on make the clock effective. > > > > You can turn off and on the pcie link like following: > > > > 1. remove the child device > > echo 1 > /sys/..../0000:01:00.0/remove > > 2. disable link > > echo 1 > /sys/..../0000:00.00.0/pcie_link_disable > > 3. enable link > > echo 1 > /sys/..../0000:00.00.0/pcie_link_disable > > 4. rescan the pci bus. > > echo 1 > /sys/..../0000:00:00.0/rescan_bridge > > > > please check link disable patch at > > git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git > > for-pci-pcie-link > Thanks, that was really quick. However, the patches does not apply on my > 3.4 kernel and it looks non trivial to me fixup. > > Could you create (even quick and dirty) patches on top of 3.4? I noted that the msi_bus for 0000:01:00.0 return nothing (cat msi_bus) If I cat it into a file and check the size it is zero. Any idea what this is about: cat vpd cat: read error: Connection timed out hmm, just saw this in dmesg: pci 0000:01:00.0: vpd r/w failed. This is likely a firmware bug on this device. Contact the card vendor for a firmware update. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: PICe hotplug problems 2012-07-11 5:08 ` Joakim Tjernlund @ 2012-07-11 14:13 ` Jiang Liu 0 siblings, 0 replies; 17+ messages in thread From: Jiang Liu @ 2012-07-11 14:13 UTC (permalink / raw) To: Joakim Tjernlund >> Thanks, that was really quick. However, the patches does not apply on my >> 3.4 kernel and it looks non trivial to me fixup. >> >> Could you create (even quick and dirty) patches on top of 3.4? > > I noted that the msi_bus for 0000:01:00.0 return nothing (cat msi_bus) > If I cat it into a file and check the size it is zero. > > Any idea what this is about: > cat vpd > cat: read error: Connection timed out > > hmm, just saw this in dmesg: > pci 0000:01:00.0: vpd r/w failed. This is likely a firmware bug on this device. Contact the card vendor for a firmware update. VPD stands for Vital Product Data, vendor may store product specific information in VPD, such as "serial number" etc. It may depend on some micro-controller on the PCI card to access those data. And VPD is optional, doesn't affect normal PCI functionalities. > > -- > To unsubscribe from this list: send the line "unsubscribe linux-pci" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: PICe hotplug problems 2012-07-11 1:33 ` Yinghai Lu 2012-07-11 4:08 ` Joakim Tjernlund [not found] ` <OF259177D8.B20ED592-ONC1257A38.00167C85-C1257A38.0016C890@LocalDomain> @ 2012-07-18 13:07 ` Joakim Tjernlund [not found] ` <OF85CC1E00.6E804B96-ONC1257A3F.00477FD2-C1257A3F.0048111D@LocalDomain> 3 siblings, 0 replies; 17+ messages in thread From: Joakim Tjernlund @ 2012-07-18 13:07 UTC (permalink / raw) To: Yinghai Lu; +Cc: linux-pci, yhlu.kernel yhlu.kernel@gmail.com wrote on 2012/07/11 03:33:05: > > On Tue, Jul 10, 2012 at 6:07 PM, Joakim Tjernlund > <joakim.tjernlund@transmode.se> wrote: > > yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00: > > >> No. Can you compile lspci util as static and run it ? > > > > That wasn't so hard so here: > > > > root@P2020RDB ~ # ./lspci -vvxxx > > 00:00.0 Class 0604: Device 1957:0079 (rev 21) > > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > > Latency: 0, Cache Line Size: 32 bytes > > Region 0: Memory at <ignored> (32-bit, non-prefetchable) > > Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 > > I/O behind bridge: 00000000-00000fff > > Memory behind bridge: 80000000-9fffffff > > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- > > BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- > > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > > Capabilities: [44] Power Management version 2 > > Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00 > > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us > > ExtTag- RBE- FLReset- > > DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > > MaxPayload 128 bytes, MaxReadReq 512 bytes > > DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- > > LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited > > ClockPM- Surprise- LLActRep- BwNot- > > LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk- > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > > LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- > > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- > > RootCap: CRSVisible- > > RootSta: PME ReqID 0000, PMEStatus- PMEPending- > > There is no slot cap etc, so pciehp will not be loaded. > the power of you child device can not be turned off/on. > > Not sure if can use link off/on make the clock effective. > > You can turn off and on the pcie link like following: > > 1. remove the child device > echo 1 > /sys/..../0000:01:00.0/remove > 2. disable link > echo 1 > /sys/..../0000:00.00.0/pcie_link_disable > 3. enable link > echo 1 > /sys/..../0000:00.00.0/pcie_link_disable > 4. rescan the pci bus. > echo 1 > /sys/..../0000:00:00.0/rescan_bridge > > please check link disable patch at > git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git > for-pci-pcie-link > > | Subject: [PATCH] PCI: Add link_disable in /sysfs for pcie device > | > | Found PCIe cards from one vendor, will not respond to scan from bridge, > | if we change bus number setting in bridge device. > | > | Have to do link disable/enable on the pcie root port. > | > | So try to expose link disable bit of pcie link control register. We can use > | echo 1 > /sys/..../link_disable > | echo 0 > /sys/..../link_disable > | to bring the pcie device back to respond to scan. Sorry for the delay, got pulled into some high prio stuff. Anyhow, I have backported your patch to 3.4 as we cannot upgrade easily and I cannot make it work. I do # > echo 1 > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable # > echo 0 > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable # > echo 1 > /sys/devices/pci0000:00/0000:00:00.0/rescan # > lspci 00:00.0 Class 0604: Device 1957:0079 (rev 21) #> dmesg pcieport 0000:00:00.0: pcie_link_disable_set: lnk_ctrl = 18 pcieport 0000:00:00.0: pcie_link_disable_set: lnk_ctrl = 8 find /sys -name pcie_link_disable find /sys -name remove find /sys -name rescan shows /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable /sys/devices/pci0000:00/0000:00:00.0/remove /sys/bus/pci/rescan /sys/devices/pci0000:00/0000:00:00.0/rescan /sys/devices/pci0000:00/0000:00:00.0/pci_bus/0000:01/rescan /sys/devices/pci0000:00/pci_bus/0000:00/rescan so something is missing, but what? Jocke ^ permalink raw reply [flat|nested] 17+ messages in thread
[parent not found: <OF85CC1E00.6E804B96-ONC1257A3F.00477FD2-C1257A3F.0048111D@LocalDomain>]
* Re: PICe hotplug problems [not found] ` <OF85CC1E00.6E804B96-ONC1257A3F.00477FD2-C1257A3F.0048111D@LocalDomain> @ 2012-07-18 23:34 ` Joakim Tjernlund [not found] ` <OF57E68083.D4D2A234-ONC1257A3F.008141AA-C1257A3F.00817A68@LocalDomain> 1 sibling, 0 replies; 17+ messages in thread From: Joakim Tjernlund @ 2012-07-18 23:34 UTC (permalink / raw) Cc: Yinghai Lu, linux-pci, yhlu.kernel Joakim Tjernlund/Transmode wrote on 2012/07/18 15:07:09: > > yhlu.kernel@gmail.com wrote on 2012/07/11 03:33:05: > > > > On Tue, Jul 10, 2012 at 6:07 PM, Joakim Tjernlund > > <joakim.tjernlund@transmode.se> wrote: > > > yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00: > > > > >> No. Can you compile lspci util as static and run it ? > > > > > > That wasn't so hard so here: > > > > > > root@P2020RDB ~ # ./lspci -vvxxx > > > 00:00.0 Class 0604: Device 1957:0079 (rev 21) > > > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- > > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > > > Latency: 0, Cache Line Size: 32 bytes > > > Region 0: Memory at <ignored> (32-bit, non-prefetchable) > > > Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 > > > I/O behind bridge: 00000000-00000fff > > > Memory behind bridge: 80000000-9fffffff > > > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- > > > BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- > > > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > > > Capabilities: [44] Power Management version 2 > > > Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) > > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > > Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00 > > > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us > > > ExtTag- RBE- FLReset- > > > DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ > > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > > > MaxPayload 128 bytes, MaxReadReq 512 bytes > > > DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- > > > LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited > > > ClockPM- Surprise- LLActRep- BwNot- > > > LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk- > > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > > > LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- > > > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- > > > RootCap: CRSVisible- > > > RootSta: PME ReqID 0000, PMEStatus- PMEPending- > > > > There is no slot cap etc, so pciehp will not be loaded. > > the power of you child device can not be turned off/on. > > > > Not sure if can use link off/on make the clock effective. > > > > You can turn off and on the pcie link like following: > > > > 1. remove the child device > > echo 1 > /sys/..../0000:01:00.0/remove > > 2. disable link > > echo 1 > /sys/..../0000:00.00.0/pcie_link_disable > > 3. enable link > > echo 1 > /sys/..../0000:00.00.0/pcie_link_disable > > 4. rescan the pci bus. > > echo 1 > /sys/..../0000:00:00.0/rescan_bridge > > > > please check link disable patch at > > git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git > > for-pci-pcie-link > > > > | Subject: [PATCH] PCI: Add link_disable in /sysfs for pcie device > > | > > | Found PCIe cards from one vendor, will not respond to scan from bridge, > > | if we change bus number setting in bridge device. > > | > > | Have to do link disable/enable on the pcie root port. > > | > > | So try to expose link disable bit of pcie link control register. We can use > > | echo 1 > /sys/..../link_disable > > | echo 0 > /sys/..../link_disable > > | to bring the pcie device back to respond to scan. > > Sorry for the delay, got pulled into some high prio stuff. > > Anyhow, I have backported your patch to 3.4 as we cannot upgrade easily and > I cannot make it work. I do > > # > echo 1 > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable > # > echo 0 > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable > # > echo 1 > /sys/devices/pci0000:00/0000:00:00.0/rescan > # > lspci > 00:00.0 Class 0604: Device 1957:0079 (rev 21) > > #> dmesg > pcieport 0000:00:00.0: pcie_link_disable_set: lnk_ctrl = 18 > pcieport 0000:00:00.0: pcie_link_disable_set: lnk_ctrl = 8 > > find /sys -name pcie_link_disable > find /sys -name remove > find /sys -name rescan > > shows > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable > /sys/devices/pci0000:00/0000:00:00.0/remove > /sys/bus/pci/rescan > /sys/devices/pci0000:00/0000:00:00.0/rescan > /sys/devices/pci0000:00/0000:00:00.0/pci_bus/0000:01/rescan > /sys/devices/pci0000:00/pci_bus/0000:00/rescan > > so something is missing, but what? Tried with FAKE hotplug too but no luck. What is best pcie_hotplug or fake hotplug? Would be nice if I could eliminate one of them. Jocke ^ permalink raw reply [flat|nested] 17+ messages in thread
[parent not found: <OF57E68083.D4D2A234-ONC1257A3F.008141AA-C1257A3F.00817A68@LocalDomain>]
* Re: PICe hotplug problems [not found] ` <OF57E68083.D4D2A234-ONC1257A3F.008141AA-C1257A3F.00817A68@LocalDomain> @ 2012-07-19 9:17 ` Joakim Tjernlund 0 siblings, 0 replies; 17+ messages in thread From: Joakim Tjernlund @ 2012-07-19 9:17 UTC (permalink / raw) Cc: Yinghai Lu, linux-pci, yhlu.kernel Joakim Tjernlund/Transmode wrote on 2012/07/19 01:34:14: > > Joakim Tjernlund/Transmode wrote on 2012/07/18 15:07:09: > > > > yhlu.kernel@gmail.com wrote on 2012/07/11 03:33:05: > > > > > > On Tue, Jul 10, 2012 at 6:07 PM, Joakim Tjernlund > > > <joakim.tjernlund@transmode.se> wrote: > > > > yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00: > > > > > > >> No. Can you compile lspci util as static and run it ? > > > > > > > > That wasn't so hard so here: > > > > > > > > root@P2020RDB ~ # ./lspci -vvxxx > > > > 00:00.0 Class 0604: Device 1957:0079 (rev 21) > > > > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- > > > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > > > > Latency: 0, Cache Line Size: 32 bytes > > > > Region 0: Memory at <ignored> (32-bit, non-prefetchable) > > > > Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 > > > > I/O behind bridge: 00000000-00000fff > > > > Memory behind bridge: 80000000-9fffffff > > > > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- > > > > BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- > > > > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > > > > Capabilities: [44] Power Management version 2 > > > > Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) > > > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > > > Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00 > > > > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us > > > > ExtTag- RBE- FLReset- > > > > DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ > > > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > > > > MaxPayload 128 bytes, MaxReadReq 512 bytes > > > > DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- > > > > LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited > > > > ClockPM- Surprise- LLActRep- BwNot- > > > > LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk- > > > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > > > > LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- > > > > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- > > > > RootCap: CRSVisible- > > > > RootSta: PME ReqID 0000, PMEStatus- PMEPending- > > > > > > There is no slot cap etc, so pciehp will not be loaded. > > > the power of you child device can not be turned off/on. > > > > > > Not sure if can use link off/on make the clock effective. > > > > > > You can turn off and on the pcie link like following: > > > > > > 1. remove the child device > > > echo 1 > /sys/..../0000:01:00.0/remove > > > 2. disable link > > > echo 1 > /sys/..../0000:00.00.0/pcie_link_disable > > > 3. enable link > > > echo 1 > /sys/..../0000:00.00.0/pcie_link_disable > > > 4. rescan the pci bus. > > > echo 1 > /sys/..../0000:00:00.0/rescan_bridge > > > > > > please check link disable patch at > > > git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git > > > for-pci-pcie-link > > > > > > | Subject: [PATCH] PCI: Add link_disable in /sysfs for pcie device > > > | > > > | Found PCIe cards from one vendor, will not respond to scan from bridge, > > > | if we change bus number setting in bridge device. > > > | > > > | Have to do link disable/enable on the pcie root port. > > > | > > > | So try to expose link disable bit of pcie link control register. We can use > > > | echo 1 > /sys/..../link_disable > > > | echo 0 > /sys/..../link_disable > > > | to bring the pcie device back to respond to scan. > > > > Sorry for the delay, got pulled into some high prio stuff. > > > > Anyhow, I have backported your patch to 3.4 as we cannot upgrade easily and > > I cannot make it work. I do > > > > # > echo 1 > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable > > # > echo 0 > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable > > # > echo 1 > /sys/devices/pci0000:00/0000:00:00.0/rescan > > # > lspci > > 00:00.0 Class 0604: Device 1957:0079 (rev 21) > > > > #> dmesg > > pcieport 0000:00:00.0: pcie_link_disable_set: lnk_ctrl = 18 > > pcieport 0000:00:00.0: pcie_link_disable_set: lnk_ctrl = 8 > > > > find /sys -name pcie_link_disable > > find /sys -name remove > > find /sys -name rescan > > > > shows > > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable > > /sys/devices/pci0000:00/0000:00:00.0/remove > > /sys/bus/pci/rescan > > /sys/devices/pci0000:00/0000:00:00.0/rescan > > /sys/devices/pci0000:00/0000:00:00.0/pci_bus/0000:01/rescan > > /sys/devices/pci0000:00/pci_bus/0000:00/rescan > > > > so something is missing, but what? > > Tried with FAKE hotplug too but no luck. > > What is best pcie_hotplug or fake hotplug? > Would be nice if I could eliminate one of them. I found a register to turn on Slot and SlotClk so now I get: root@P2020RDB ~ # lspci -vvvvxx 00:00.0 Class 0604: Device 1957:0079 (rev 21) Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Region 0: Memory at <ignored> (32-bit, non-prefetchable) Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 00000000-00000fff Memory behind bridge: 90000000-900fffff Prefetchable memory behind bridge: 0000000080000000-000000008dffffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [44] Power Management version 2 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [4c] Express (v1) Root Port (Slot+), MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise- Slot #0, PowerLimit 15.000W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Off, PwrInd Off, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock- Changed: MRL- PresDet- LinkState- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- 00: 57 19 79 00 06 01 10 00 21 00 20 0b 00 00 01 00 10: 00 00 f0 ff 00 00 00 00 00 01 01 00 00 00 00 00 20: 00 90 00 90 01 80 f1 8d 00 00 00 00 00 00 00 00 30: 00 00 00 00 44 00 00 00 00 00 00 00 00 00 00 00 However I still don't get it to work echo 1 > /sys/..../rescan does not expose the device behind the RC nor does toggling pcie_link_disable Jocke ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2012-07-19 9:17 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-07-10 17:29 PICe hotplug problems Joakim Tjernlund
2012-07-10 18:22 ` Yinghai Lu
2012-07-10 21:42 ` Joakim Tjernlund
2012-07-10 22:09 ` Yinghai Lu
2012-07-10 22:20 ` Bjorn Helgaas
2012-07-10 22:22 ` Yinghai Lu
2012-07-10 22:46 ` Bjorn Helgaas
2012-07-10 22:49 ` Bjorn Helgaas
2012-07-11 1:07 ` Joakim Tjernlund
2012-07-11 1:33 ` Yinghai Lu
2012-07-11 4:08 ` Joakim Tjernlund
2012-07-11 6:09 ` Yinghai Lu
[not found] ` <OF259177D8.B20ED592-ONC1257A38.00167C85-C1257A38.0016C890@LocalDomain>
2012-07-11 5:08 ` Joakim Tjernlund
2012-07-11 14:13 ` Jiang Liu
2012-07-18 13:07 ` Joakim Tjernlund
[not found] ` <OF85CC1E00.6E804B96-ONC1257A3F.00477FD2-C1257A3F.0048111D@LocalDomain>
2012-07-18 23:34 ` Joakim Tjernlund
[not found] ` <OF57E68083.D4D2A234-ONC1257A3F.008141AA-C1257A3F.00817A68@LocalDomain>
2012-07-19 9:17 ` Joakim Tjernlund
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.