All of lore.kernel.org
 help / color / mirror / Atom feed
* PICe hotplug problems
@ 2012-07-10 17:29 Joakim Tjernlund
  2012-07-10 18:22 ` Yinghai Lu
  0 siblings, 1 reply; 17+ messages in thread
From: Joakim Tjernlund @ 2012-07-10 17:29 UTC (permalink / raw)
  To: linux-pci


I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the
device enables its PCIe interface.

The device is connected to a built in root bridge on a P2010(mpc85xx) CPU.
To enable HP in Linux we need to apply some minor hacks.

First, The contoller has MSI but there is no pci_find_capability(dev, PCI_CAP_ID_MSI) so
portdrv_core.c:init_service_irqs() fails this test(dev->pin is false too):
 	if (!pci_enable_msi(dev) || dev->pin)

Is the code correct? Should our root bridge have PCI_CAP_ID_MSI?

Second problem, the root bride does not have PCI_EXP_FLAGS_SLOT so
portdrv_core.c:get_port_device_capability() fails this test:
  if ((cap_mask & PCIE_PORT_SERVICE_HP) && (reg16 & PCI_EXP_FLAGS_SLOT)) {
		pci_read_config_dword(dev, pos + PCI_EXP_SLTCAP, &reg32);
		if (reg32 & PCI_EXP_SLTCAP_HPC) {
			services |= PCIE_PORT_SERVICE_HP;

which cause PCIE_PORT_SERVICE_HP to be unset in services.

Is PCI_EXP_FLAGS_SLOT really required for hot plug?

Perhaps there another way?

 Jocke


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
  2012-07-10 17:29 PICe hotplug problems Joakim Tjernlund
@ 2012-07-10 18:22 ` Yinghai Lu
  2012-07-10 21:42   ` Joakim Tjernlund
  0 siblings, 1 reply; 17+ messages in thread
From: Yinghai Lu @ 2012-07-10 18:22 UTC (permalink / raw)
  To: Joakim Tjernlund; +Cc: linux-pci

On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund
<joakim.tjernlund@transmode.se> wrote:
>
> I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the
> device enables its PCIe interface.
>
> The device is connected to a built in root bridge on a P2010(mpc85xx) CPU.
> To enable HP in Linux we need to apply some minor hacks.

can you post lspci -vvxxx -s BB:DD:F of the two devices?

Yinghai

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
  2012-07-10 18:22 ` Yinghai Lu
@ 2012-07-10 21:42   ` Joakim Tjernlund
  2012-07-10 22:09     ` Yinghai Lu
  0 siblings, 1 reply; 17+ messages in thread
From: Joakim Tjernlund @ 2012-07-10 21:42 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: linux-pci, yhlu.kernel

yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09:

>
> On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund
> <joakim.tjernlund@transmode.se> wrote:
> >
> > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the
> > device enables its PCIe interface.
> >
> > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU.
> > To enable HP in Linux we need to apply some minor hacks.
>
> can you post lspci -vvxxx -s BB:DD:F of the two devices?

Not really, this is an embedded device with limited SW. I got
busybox and its lspci but that is very limited:
# > ./busybox lspci -mk
00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport"
01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540"

Does that tell you anything?

 Jocke


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
  2012-07-10 21:42   ` Joakim Tjernlund
@ 2012-07-10 22:09     ` Yinghai Lu
  2012-07-10 22:20       ` Bjorn Helgaas
  2012-07-11  1:07       ` Joakim Tjernlund
  0 siblings, 2 replies; 17+ messages in thread
From: Yinghai Lu @ 2012-07-10 22:09 UTC (permalink / raw)
  To: Joakim Tjernlund; +Cc: linux-pci

On Tue, Jul 10, 2012 at 2:42 PM, Joakim Tjernlund
<joakim.tjernlund@transmode.se> wrote:
> yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09:
>
>>
>> On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund
>> <joakim.tjernlund@transmode.se> wrote:
>> >
>> > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the
>> > device enables its PCIe interface.
>> >
>> > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU.
>> > To enable HP in Linux we need to apply some minor hacks.
>>
>> can you post lspci -vvxxx -s BB:DD:F of the two devices?
>
> Not really, this is an embedded device with limited SW. I got
> busybox and its lspci but that is very limited:
> # > ./busybox lspci -mk
> 00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport"
> 01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540"
>
> Does that tell you anything?

No. Can you compile lspci util as static and run it ?

Thanks

Yinghai

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
  2012-07-10 22:09     ` Yinghai Lu
@ 2012-07-10 22:20       ` Bjorn Helgaas
  2012-07-10 22:22         ` Yinghai Lu
  2012-07-11  1:07       ` Joakim Tjernlund
  1 sibling, 1 reply; 17+ messages in thread
From: Bjorn Helgaas @ 2012-07-10 22:20 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: Joakim Tjernlund, linux-pci

On Tue, Jul 10, 2012 at 4:09 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> On Tue, Jul 10, 2012 at 2:42 PM, Joakim Tjernlund
> <joakim.tjernlund@transmode.se> wrote:
>> yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09:
>>
>>>
>>> On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund
>>> <joakim.tjernlund@transmode.se> wrote:
>>> >
>>> > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the
>>> > device enables its PCIe interface.
>>> >
>>> > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU.
>>> > To enable HP in Linux we need to apply some minor hacks.
>>>
>>> can you post lspci -vvxxx -s BB:DD:F of the two devices?
>>
>> Not really, this is an embedded device with limited SW. I got
>> busybox and its lspci but that is very limited:
>> # > ./busybox lspci -mk
>> 00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport"
>> 01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540"
>>
>> Does that tell you anything?
>
> No. Can you compile lspci util as static and run it ?

You might also be able to get this info as console output by booting
with "pci=earlydump".

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
  2012-07-10 22:20       ` Bjorn Helgaas
@ 2012-07-10 22:22         ` Yinghai Lu
  2012-07-10 22:46           ` Bjorn Helgaas
  0 siblings, 1 reply; 17+ messages in thread
From: Yinghai Lu @ 2012-07-10 22:22 UTC (permalink / raw)
  To: Bjorn Helgaas; +Cc: Joakim Tjernlund, linux-pci

On Tue, Jul 10, 2012 at 3:20 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Tue, Jul 10, 2012 at 4:09 PM, Yinghai Lu <yinghai@kernel.org> wrote:
>> On Tue, Jul 10, 2012 at 2:42 PM, Joakim Tjernlund
>> <joakim.tjernlund@transmode.se> wrote:
>>> yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09:
>>>
>>>>
>>>> On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund
>>>> <joakim.tjernlund@transmode.se> wrote:
>>>> >
>>>> > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the
>>>> > device enables its PCIe interface.
>>>> >
>>>> > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU.
>>>> > To enable HP in Linux we need to apply some minor hacks.
>>>>
>>>> can you post lspci -vvxxx -s BB:DD:F of the two devices?
>>>
>>> Not really, this is an embedded device with limited SW. I got
>>> busybox and its lspci but that is very limited:
>>> # > ./busybox lspci -mk
>>> 00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport"
>>> 01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540"
>>>
>>> Does that tell you anything?
>>
>> No. Can you compile lspci util as static and run it ?
>
> You might also be able to get this info as console output by booting
> with "pci=earlydump".

that is only for x86.

and his setup is with P2010(mpc85xx) CPU.

Thanks

Yinghai

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
  2012-07-10 22:22         ` Yinghai Lu
@ 2012-07-10 22:46           ` Bjorn Helgaas
  2012-07-10 22:49             ` Bjorn Helgaas
  0 siblings, 1 reply; 17+ messages in thread
From: Bjorn Helgaas @ 2012-07-10 22:46 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: Joakim Tjernlund, linux-pci

On Tue, Jul 10, 2012 at 4:22 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> On Tue, Jul 10, 2012 at 3:20 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
>> On Tue, Jul 10, 2012 at 4:09 PM, Yinghai Lu <yinghai@kernel.org> wrote:
>>> On Tue, Jul 10, 2012 at 2:42 PM, Joakim Tjernlund
>>> <joakim.tjernlund@transmode.se> wrote:
>>>> yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09:
>>>>
>>>>>
>>>>> On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund
>>>>> <joakim.tjernlund@transmode.se> wrote:
>>>>> >
>>>>> > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the
>>>>> > device enables its PCIe interface.
>>>>> >
>>>>> > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU.
>>>>> > To enable HP in Linux we need to apply some minor hacks.
>>>>>
>>>>> can you post lspci -vvxxx -s BB:DD:F of the two devices?
>>>>
>>>> Not really, this is an embedded device with limited SW. I got
>>>> busybox and its lspci but that is very limited:
>>>> # > ./busybox lspci -mk
>>>> 00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport"
>>>> 01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540"
>>>>
>>>> Does that tell you anything?
>>>
>>> No. Can you compile lspci util as static and run it ?
>>
>> You might also be able to get this info as console output by booting
>> with "pci=earlydump".
>
> that is only for x86.
>
> and his setup is with P2010(mpc85xx) CPU.

Ah, yes.  It's regrettable that this arch-independent functionality is
implemented in an arch-specific way.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
  2012-07-10 22:46           ` Bjorn Helgaas
@ 2012-07-10 22:49             ` Bjorn Helgaas
  0 siblings, 0 replies; 17+ messages in thread
From: Bjorn Helgaas @ 2012-07-10 22:49 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: Joakim Tjernlund, linux-pci

On Tue, Jul 10, 2012 at 4:46 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Tue, Jul 10, 2012 at 4:22 PM, Yinghai Lu <yinghai@kernel.org> wrote:
>> On Tue, Jul 10, 2012 at 3:20 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
>>> On Tue, Jul 10, 2012 at 4:09 PM, Yinghai Lu <yinghai@kernel.org> wrote:
>>>> On Tue, Jul 10, 2012 at 2:42 PM, Joakim Tjernlund
>>>> <joakim.tjernlund@transmode.se> wrote:
>>>>> yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09:
>>>>>
>>>>>>
>>>>>> On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund
>>>>>> <joakim.tjernlund@transmode.se> wrote:
>>>>>> >
>>>>>> > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the
>>>>>> > device enables its PCIe interface.
>>>>>> >
>>>>>> > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU.
>>>>>> > To enable HP in Linux we need to apply some minor hacks.
>>>>>>
>>>>>> can you post lspci -vvxxx -s BB:DD:F of the two devices?
>>>>>
>>>>> Not really, this is an embedded device with limited SW. I got
>>>>> busybox and its lspci but that is very limited:
>>>>> # > ./busybox lspci -mk
>>>>> 00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport"
>>>>> 01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540"
>>>>>
>>>>> Does that tell you anything?
>>>>
>>>> No. Can you compile lspci util as static and run it ?
>>>
>>> You might also be able to get this info as console output by booting
>>> with "pci=earlydump".
>>
>> that is only for x86.
>>
>> and his setup is with P2010(mpc85xx) CPU.
>
> Ah, yes.  It's regrettable that this arch-independent functionality is
> implemented in an arch-specific way.

But I guess the config-space accessors are arch-dependent, and we
don't necessarily even *have* them early.  So it probably is difficult
to do this generically.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
  2012-07-10 22:09     ` Yinghai Lu
  2012-07-10 22:20       ` Bjorn Helgaas
@ 2012-07-11  1:07       ` Joakim Tjernlund
  2012-07-11  1:33         ` Yinghai Lu
  1 sibling, 1 reply; 17+ messages in thread
From: Joakim Tjernlund @ 2012-07-11  1:07 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: linux-pci, yhlu.kernel

yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00:
>
> On Tue, Jul 10, 2012 at 2:42 PM, Joakim Tjernlund
> <joakim.tjernlund@transmode.se> wrote:
> > yhlu.kernel@gmail.com wrote on 2012/07/10 20:22:09:
> >
> >>
> >> On Tue, Jul 10, 2012 at 10:29 AM, Joakim Tjernlund
> >> <joakim.tjernlund@transmode.se> wrote:
> >> >
> >> > I got an PCIe device that is woken up by user space, clocks needs to configured in various ways before the
> >> > device enables its PCIe interface.
> >> >
> >> > The device is connected to a built in root bridge on a P2010(mpc85xx) CPU.
> >> > To enable HP in Linux we need to apply some minor hacks.
> >>
> >> can you post lspci -vvxxx -s BB:DD:F of the two devices?
> >
> > Not really, this is an embedded device with limited SW. I got
> > busybox and its lspci but that is very limited:
> > # > ./busybox lspci -mk
> > 00:00.0 "Class 0604" "1957" "0079" "0000" "0000" "pcieport"
> > 01:00.0 "Class 0200" "14e4" "b540" "14e4" "b540"
> >
> > Does that tell you anything?
>
> No. Can you compile lspci util as static and run it ?

That wasn't so hard so here:

root@P2020RDB ~ # ./lspci  -vvxxx
00:00.0 Class 0604: Device 1957:0079 (rev 21)
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 32 bytes
	Region 0: Memory at <ignored> (32-bit, non-prefetchable)
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 00000000-00000fff
	Memory behind bridge: 80000000-9fffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [44] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE- FLReset-
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Kernel driver in use: pcieport
00: 57 19 79 00 06 01 10 00 21 00 20 0b 08 00 01 00
10: 00 00 f0 ff 00 00 00 00 00 01 01 00 00 00 00 00
20: 00 80 f0 9f f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 44 00 00 00 00 00 00 00 00 00 00 00
40: 00 00 00 00 01 4c 02 fe 00 00 00 00 10 00 41 00
50: 01 00 00 00 1f 28 00 00 41 d4 03 00 08 00 21 00
60: c0 07 00 00 c8 03 40 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

01:00.0 Class 0200: Device 14e4:b540 (rev 02)
	Subsystem: Device 14e4:b540
	Physical Slot: 0
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 32 bytes
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at 80000000 (64-bit, non-prefetchable) [size=256K]
	Capabilities: [48] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] Vital Product Data
pcilib: sysfs_read_vpd: read failed: Connection timed out
		Not readable
	Capabilities: [58] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [ac] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr+ NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x2, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Device Serial Number 00-00-00-00-00-00-00-00
	Capabilities: [110 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [150 v1] Power Budgeting <?>
	Capabilities: [160 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
00: e4 14 40 b5 06 00 10 00 02 00 00 02 08 00 00 00
10: 04 00 00 80 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 e4 14 40 b5
30: 00 00 00 00 48 00 00 00 00 00 00 00 00 01 00 00
40: 00 00 00 00 00 00 00 00 01 50 03 c8 08 20 00 00
50: 03 58 00 00 00 00 00 00 05 ac 80 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 11 00 00 00 00 00 00 00 00 00 00 00 10 00 02 00
b0: 02 80 3c 00 10 2c 10 00 22 6c 01 00 00 00 21 10
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 1f 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
  2012-07-11  1:07       ` Joakim Tjernlund
@ 2012-07-11  1:33         ` Yinghai Lu
  2012-07-11  4:08           ` Joakim Tjernlund
                             ` (3 more replies)
  0 siblings, 4 replies; 17+ messages in thread
From: Yinghai Lu @ 2012-07-11  1:33 UTC (permalink / raw)
  To: Joakim Tjernlund; +Cc: linux-pci

On Tue, Jul 10, 2012 at 6:07 PM, Joakim Tjernlund
<joakim.tjernlund@transmode.se> wrote:
> yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00:

>> No. Can you compile lspci util as static and run it ?
>
> That wasn't so hard so here:
>
> root@P2020RDB ~ # ./lspci  -vvxxx
> 00:00.0 Class 0604: Device 1957:0079 (rev 21)
>         Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
>         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>         Latency: 0, Cache Line Size: 32 bytes
>         Region 0: Memory at <ignored> (32-bit, non-prefetchable)
>         Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
>         I/O behind bridge: 00000000-00000fff
>         Memory behind bridge: 80000000-9fffffff
>         Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
>         BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
>                 PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
>         Capabilities: [44] Power Management version 2
>                 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
>                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
>         Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00
>                 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
>                         ExtTag- RBE- FLReset-
>                 DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
>                         RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
>                         MaxPayload 128 bytes, MaxReadReq 512 bytes
>                 DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
>                 LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited
>                         ClockPM- Surprise- LLActRep- BwNot-
>                 LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
>                         ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
>                 LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
>                 RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
>                 RootCap: CRSVisible-
>                 RootSta: PME ReqID 0000, PMEStatus- PMEPending-

There is no slot cap etc, so pciehp will not be loaded.
the power of you child device can not be turned off/on.

Not sure if can use link off/on make the clock effective.

You can turn off and on the pcie link like following:

1. remove the child device
     echo 1 > /sys/..../0000:01:00.0/remove
2. disable link
     echo 1 > /sys/..../0000:00.00.0/pcie_link_disable
3. enable link
     echo 1 > /sys/..../0000:00.00.0/pcie_link_disable
4. rescan the pci bus.
     echo 1 > /sys/..../0000:00:00.0/rescan_bridge

please check link disable patch at
git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git
for-pci-pcie-link

| Subject: [PATCH] PCI: Add link_disable in /sysfs for pcie device
|
| Found PCIe cards from one vendor, will not respond to scan from bridge,
| if we change bus number setting in bridge device.
|
| Have to do link disable/enable on the pcie root port.
|
|  So try to expose link disable bit of pcie link control register. We can use
|  echo 1 > /sys/..../link_disable
|  echo 0 > /sys/..../link_disable
| to bring the pcie device back to respond to scan.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
  2012-07-11  1:33         ` Yinghai Lu
@ 2012-07-11  4:08           ` Joakim Tjernlund
  2012-07-11  6:09             ` Yinghai Lu
       [not found]           ` <OF259177D8.B20ED592-ONC1257A38.00167C85-C1257A38.0016C890@LocalDomain>
                             ` (2 subsequent siblings)
  3 siblings, 1 reply; 17+ messages in thread
From: Joakim Tjernlund @ 2012-07-11  4:08 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: linux-pci, yhlu.kernel



yhlu.kernel@gmail.com wrote on 2012/07/11 03:33:05:
>
> On Tue, Jul 10, 2012 at 6:07 PM, Joakim Tjernlund
> <joakim.tjernlund@transmode.se> wrote:
> > yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00:
>
> >> No. Can you compile lspci util as static and run it ?
> >
> > That wasn't so hard so here:
> >
> > root@P2020RDB ~ # ./lspci  -vvxxx
> > 00:00.0 Class 0604: Device 1957:0079 (rev 21)
> >         Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> >         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >         Latency: 0, Cache Line Size: 32 bytes
> >         Region 0: Memory at <ignored> (32-bit, non-prefetchable)
> >         Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
> >         I/O behind bridge: 00000000-00000fff
> >         Memory behind bridge: 80000000-9fffffff
> >         Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> >         BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
> >                 PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> >         Capabilities: [44] Power Management version 2
> >                 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
> >                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> >         Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00
> >                 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
> >                         ExtTag- RBE- FLReset-
> >                 DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
> >                         RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
> >                         MaxPayload 128 bytes, MaxReadReq 512 bytes
> >                 DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
> >                 LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited
> >                         ClockPM- Surprise- LLActRep- BwNot-
> >                 LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
> >                         ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> >                 LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
> >                 RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
> >                 RootCap: CRSVisible-
> >                 RootSta: PME ReqID 0000, PMEStatus- PMEPending-
>
> There is no slot cap etc, so pciehp will not be loaded.
> the power of you child device can not be turned off/on.
>
> Not sure if can use link off/on make the clock effective.
>
> You can turn off and on the pcie link like following:
>
> 1. remove the child device
>      echo 1 > /sys/..../0000:01:00.0/remove
> 2. disable link
>      echo 1 > /sys/..../0000:00.00.0/pcie_link_disable
> 3. enable link
>      echo 1 > /sys/..../0000:00.00.0/pcie_link_disable
> 4. rescan the pci bus.
>      echo 1 > /sys/..../0000:00:00.0/rescan_bridge
>
> please check link disable patch at
> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git
> for-pci-pcie-link

Thanks, that was really quick. However, the patches does not apply on my
3.4 kernel and it looks non trivial to me fixup.

Could you create (even quick and dirty) patches on top of 3.4?

 Jocke


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
       [not found]           ` <OF259177D8.B20ED592-ONC1257A38.00167C85-C1257A38.0016C890@LocalDomain>
@ 2012-07-11  5:08             ` Joakim Tjernlund
  2012-07-11 14:13               ` Jiang Liu
  0 siblings, 1 reply; 17+ messages in thread
From: Joakim Tjernlund @ 2012-07-11  5:08 UTC (permalink / raw)
  Cc: Yinghai Lu, linux-pci, yhlu.kernel

Joakim Tjernlund/Transmode wrote on 2012/07/11 06:08:51:
>
> yhlu.kernel@gmail.com wrote on 2012/07/11 03:33:05:
> >
> > On Tue, Jul 10, 2012 at 6:07 PM, Joakim Tjernlund
> > <joakim.tjernlund@transmode.se> wrote:
> > > yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00:
> >
> > >> No. Can you compile lspci util as static and run it ?
> > >
> > > That wasn't so hard so here:
> > >
> > > root@P2020RDB ~ # ./lspci  -vvxxx
> > > 00:00.0 Class 0604: Device 1957:0079 (rev 21)
> > >         Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> > >         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> > >         Latency: 0, Cache Line Size: 32 bytes
> > >         Region 0: Memory at <ignored> (32-bit, non-prefetchable)
> > >         Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
> > >         I/O behind bridge: 00000000-00000fff
> > >         Memory behind bridge: 80000000-9fffffff
> > >         Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> > >         BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
> > >                 PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> > >         Capabilities: [44] Power Management version 2
> > >                 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
> > >                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> > >         Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00
> > >                 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
> > >                         ExtTag- RBE- FLReset-
> > >                 DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
> > >                         RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
> > >                         MaxPayload 128 bytes, MaxReadReq 512 bytes
> > >                 DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
> > >                 LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited
> > >                         ClockPM- Surprise- LLActRep- BwNot-
> > >                 LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
> > >                         ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> > >                 LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
> > >                 RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
> > >                 RootCap: CRSVisible-
> > >                 RootSta: PME ReqID 0000, PMEStatus- PMEPending-
> >
> > There is no slot cap etc, so pciehp will not be loaded.
> > the power of you child device can not be turned off/on.
> >
> > Not sure if can use link off/on make the clock effective.
> >
> > You can turn off and on the pcie link like following:
> >
> > 1. remove the child device
> >      echo 1 > /sys/..../0000:01:00.0/remove
> > 2. disable link
> >      echo 1 > /sys/..../0000:00.00.0/pcie_link_disable
> > 3. enable link
> >      echo 1 > /sys/..../0000:00.00.0/pcie_link_disable
> > 4. rescan the pci bus.
> >      echo 1 > /sys/..../0000:00:00.0/rescan_bridge
> >
> > please check link disable patch at
> > git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git
> > for-pci-pcie-link

> Thanks, that was really quick. However, the patches does not apply on my
> 3.4 kernel and it looks non trivial to me fixup.
>
> Could you create (even quick and dirty) patches on top of 3.4?

I noted that the msi_bus for 0000:01:00.0 return nothing (cat msi_bus)
If I cat it into a file and check the size it is zero.

Any idea what this is about:
  cat vpd
  cat: read error: Connection timed out

hmm, just saw this in dmesg:
   pci 0000:01:00.0: vpd r/w failed.  This is likely a firmware bug on this device.  Contact the card vendor for a firmware update.


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
  2012-07-11  4:08           ` Joakim Tjernlund
@ 2012-07-11  6:09             ` Yinghai Lu
  0 siblings, 0 replies; 17+ messages in thread
From: Yinghai Lu @ 2012-07-11  6:09 UTC (permalink / raw)
  To: Joakim Tjernlund; +Cc: linux-pci

On Tue, Jul 10, 2012 at 9:08 PM, Joakim Tjernlund
<joakim.tjernlund@transmode.se> wrote:
>> please check link disable patch at
>> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git
>> for-pci-pcie-link
>
> Thanks, that was really quick. However, the patches does not apply on my
> 3.4 kernel and it looks non trivial to me fixup.
>
> Could you create (even quick and dirty) patches on top of 3.4?

you can just use that branch instead.

mkdir linux-2.6 || exit -1
cd linux-2.6

git init-db
# Add Linus's tree as a remote
git remote add linus
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git

git remote add yh
git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git

git remote update

git checkout -b yh_2012_07_10 yh/for-pci-pcie-link

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
  2012-07-11  5:08             ` Joakim Tjernlund
@ 2012-07-11 14:13               ` Jiang Liu
  0 siblings, 0 replies; 17+ messages in thread
From: Jiang Liu @ 2012-07-11 14:13 UTC (permalink / raw)
  To: Joakim Tjernlund


>> Thanks, that was really quick. However, the patches does not apply on my
>> 3.4 kernel and it looks non trivial to me fixup.
>>
>> Could you create (even quick and dirty) patches on top of 3.4?
> 
> I noted that the msi_bus for 0000:01:00.0 return nothing (cat msi_bus)
> If I cat it into a file and check the size it is zero.
> 
> Any idea what this is about:
>   cat vpd
>   cat: read error: Connection timed out
> 
> hmm, just saw this in dmesg:
>    pci 0000:01:00.0: vpd r/w failed.  This is likely a firmware bug on this device.  Contact the card vendor for a firmware update.
VPD stands for Vital Product Data, vendor may store product specific information
in VPD, such as "serial number" etc. It may depend on some micro-controller on
the PCI card to access those data. And VPD is optional, doesn't affect normal
PCI functionalities.

> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 



^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
  2012-07-11  1:33         ` Yinghai Lu
  2012-07-11  4:08           ` Joakim Tjernlund
       [not found]           ` <OF259177D8.B20ED592-ONC1257A38.00167C85-C1257A38.0016C890@LocalDomain>
@ 2012-07-18 13:07           ` Joakim Tjernlund
       [not found]           ` <OF85CC1E00.6E804B96-ONC1257A3F.00477FD2-C1257A3F.0048111D@LocalDomain>
  3 siblings, 0 replies; 17+ messages in thread
From: Joakim Tjernlund @ 2012-07-18 13:07 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: linux-pci, yhlu.kernel

yhlu.kernel@gmail.com wrote on 2012/07/11 03:33:05:
>
> On Tue, Jul 10, 2012 at 6:07 PM, Joakim Tjernlund
> <joakim.tjernlund@transmode.se> wrote:
> > yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00:
>
> >> No. Can you compile lspci util as static and run it ?
> >
> > That wasn't so hard so here:
> >
> > root@P2020RDB ~ # ./lspci  -vvxxx
> > 00:00.0 Class 0604: Device 1957:0079 (rev 21)
> >         Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> >         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >         Latency: 0, Cache Line Size: 32 bytes
> >         Region 0: Memory at <ignored> (32-bit, non-prefetchable)
> >         Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
> >         I/O behind bridge: 00000000-00000fff
> >         Memory behind bridge: 80000000-9fffffff
> >         Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> >         BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
> >                 PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> >         Capabilities: [44] Power Management version 2
> >                 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
> >                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> >         Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00
> >                 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
> >                         ExtTag- RBE- FLReset-
> >                 DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
> >                         RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
> >                         MaxPayload 128 bytes, MaxReadReq 512 bytes
> >                 DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
> >                 LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited
> >                         ClockPM- Surprise- LLActRep- BwNot-
> >                 LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
> >                         ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> >                 LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
> >                 RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
> >                 RootCap: CRSVisible-
> >                 RootSta: PME ReqID 0000, PMEStatus- PMEPending-
>
> There is no slot cap etc, so pciehp will not be loaded.
> the power of you child device can not be turned off/on.
>
> Not sure if can use link off/on make the clock effective.
>
> You can turn off and on the pcie link like following:
>
> 1. remove the child device
>      echo 1 > /sys/..../0000:01:00.0/remove
> 2. disable link
>      echo 1 > /sys/..../0000:00.00.0/pcie_link_disable
> 3. enable link
>      echo 1 > /sys/..../0000:00.00.0/pcie_link_disable
> 4. rescan the pci bus.
>      echo 1 > /sys/..../0000:00:00.0/rescan_bridge
>
> please check link disable patch at
> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git
> for-pci-pcie-link
>
> | Subject: [PATCH] PCI: Add link_disable in /sysfs for pcie device
> |
> | Found PCIe cards from one vendor, will not respond to scan from bridge,
> | if we change bus number setting in bridge device.
> |
> | Have to do link disable/enable on the pcie root port.
> |
> |  So try to expose link disable bit of pcie link control register. We can use
> |  echo 1 > /sys/..../link_disable
> |  echo 0 > /sys/..../link_disable
> | to bring the pcie device back to respond to scan.

Sorry for the delay, got pulled into some high prio stuff.

Anyhow, I have backported your patch to 3.4 as we cannot upgrade easily and
I cannot make it work. I do

# > echo 1 > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable
# > echo 0 > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable
# > echo 1 > /sys/devices/pci0000:00/0000:00:00.0/rescan

# > lspci
00:00.0 Class 0604: Device 1957:0079 (rev 21)

#> dmesg
pcieport 0000:00:00.0: pcie_link_disable_set: lnk_ctrl = 18
pcieport 0000:00:00.0: pcie_link_disable_set: lnk_ctrl = 8

find /sys -name pcie_link_disable
find /sys -name remove
find /sys -name rescan

shows
/sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable
/sys/devices/pci0000:00/0000:00:00.0/remove
/sys/bus/pci/rescan
/sys/devices/pci0000:00/0000:00:00.0/rescan
/sys/devices/pci0000:00/0000:00:00.0/pci_bus/0000:01/rescan
/sys/devices/pci0000:00/pci_bus/0000:00/rescan

so something is missing, but what?

 Jocke


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
       [not found]           ` <OF85CC1E00.6E804B96-ONC1257A3F.00477FD2-C1257A3F.0048111D@LocalDomain>
@ 2012-07-18 23:34             ` Joakim Tjernlund
       [not found]             ` <OF57E68083.D4D2A234-ONC1257A3F.008141AA-C1257A3F.00817A68@LocalDomain>
  1 sibling, 0 replies; 17+ messages in thread
From: Joakim Tjernlund @ 2012-07-18 23:34 UTC (permalink / raw)
  Cc: Yinghai Lu, linux-pci, yhlu.kernel

Joakim Tjernlund/Transmode wrote on 2012/07/18 15:07:09:
>
> yhlu.kernel@gmail.com wrote on 2012/07/11 03:33:05:
> >
> > On Tue, Jul 10, 2012 at 6:07 PM, Joakim Tjernlund
> > <joakim.tjernlund@transmode.se> wrote:
> > > yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00:
> >
> > >> No. Can you compile lspci util as static and run it ?
> > >
> > > That wasn't so hard so here:
> > >
> > > root@P2020RDB ~ # ./lspci  -vvxxx
> > > 00:00.0 Class 0604: Device 1957:0079 (rev 21)
> > >         Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> > >         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> > >         Latency: 0, Cache Line Size: 32 bytes
> > >         Region 0: Memory at <ignored> (32-bit, non-prefetchable)
> > >         Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
> > >         I/O behind bridge: 00000000-00000fff
> > >         Memory behind bridge: 80000000-9fffffff
> > >         Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> > >         BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
> > >                 PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> > >         Capabilities: [44] Power Management version 2
> > >                 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
> > >                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> > >         Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00
> > >                 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
> > >                         ExtTag- RBE- FLReset-
> > >                 DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
> > >                         RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
> > >                         MaxPayload 128 bytes, MaxReadReq 512 bytes
> > >                 DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
> > >                 LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited
> > >                         ClockPM- Surprise- LLActRep- BwNot-
> > >                 LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
> > >                         ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> > >                 LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
> > >                 RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
> > >                 RootCap: CRSVisible-
> > >                 RootSta: PME ReqID 0000, PMEStatus- PMEPending-
> >
> > There is no slot cap etc, so pciehp will not be loaded.
> > the power of you child device can not be turned off/on.
> >
> > Not sure if can use link off/on make the clock effective.
> >
> > You can turn off and on the pcie link like following:
> >
> > 1. remove the child device
> >      echo 1 > /sys/..../0000:01:00.0/remove
> > 2. disable link
> >      echo 1 > /sys/..../0000:00.00.0/pcie_link_disable
> > 3. enable link
> >      echo 1 > /sys/..../0000:00.00.0/pcie_link_disable
> > 4. rescan the pci bus.
> >      echo 1 > /sys/..../0000:00:00.0/rescan_bridge
> >
> > please check link disable patch at
> > git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git
> > for-pci-pcie-link
> >
> > | Subject: [PATCH] PCI: Add link_disable in /sysfs for pcie device
> > |
> > | Found PCIe cards from one vendor, will not respond to scan from bridge,
> > | if we change bus number setting in bridge device.
> > |
> > | Have to do link disable/enable on the pcie root port.
> > |
> > |  So try to expose link disable bit of pcie link control register. We can use
> > |  echo 1 > /sys/..../link_disable
> > |  echo 0 > /sys/..../link_disable
> > | to bring the pcie device back to respond to scan.
>
> Sorry for the delay, got pulled into some high prio stuff.
>
> Anyhow, I have backported your patch to 3.4 as we cannot upgrade easily and
> I cannot make it work. I do
>
> # > echo 1 > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable
> # > echo 0 > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable
> # > echo 1 > /sys/devices/pci0000:00/0000:00:00.0/rescan

> # > lspci
> 00:00.0 Class 0604: Device 1957:0079 (rev 21)
>
> #> dmesg
> pcieport 0000:00:00.0: pcie_link_disable_set: lnk_ctrl = 18
> pcieport 0000:00:00.0: pcie_link_disable_set: lnk_ctrl = 8
>
> find /sys -name pcie_link_disable
> find /sys -name remove
> find /sys -name rescan
>
> shows
> /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable
> /sys/devices/pci0000:00/0000:00:00.0/remove
> /sys/bus/pci/rescan
> /sys/devices/pci0000:00/0000:00:00.0/rescan
> /sys/devices/pci0000:00/0000:00:00.0/pci_bus/0000:01/rescan
> /sys/devices/pci0000:00/pci_bus/0000:00/rescan
>
> so something is missing, but what?

Tried with FAKE hotplug too but no luck.

What is best pcie_hotplug or fake hotplug?
Would be nice if I could eliminate one of them.

 Jocke


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: PICe hotplug problems
       [not found]             ` <OF57E68083.D4D2A234-ONC1257A3F.008141AA-C1257A3F.00817A68@LocalDomain>
@ 2012-07-19  9:17               ` Joakim Tjernlund
  0 siblings, 0 replies; 17+ messages in thread
From: Joakim Tjernlund @ 2012-07-19  9:17 UTC (permalink / raw)
  Cc: Yinghai Lu, linux-pci, yhlu.kernel

Joakim Tjernlund/Transmode wrote on 2012/07/19 01:34:14:
>
> Joakim Tjernlund/Transmode wrote on 2012/07/18 15:07:09:
> >
> > yhlu.kernel@gmail.com wrote on 2012/07/11 03:33:05:
> > >
> > > On Tue, Jul 10, 2012 at 6:07 PM, Joakim Tjernlund
> > > <joakim.tjernlund@transmode.se> wrote:
> > > > yhlu.kernel@gmail.com wrote on 2012/07/11 00:09:00:
> > >
> > > >> No. Can you compile lspci util as static and run it ?
> > > >
> > > > That wasn't so hard so here:
> > > >
> > > > root@P2020RDB ~ # ./lspci  -vvxxx
> > > > 00:00.0 Class 0604: Device 1957:0079 (rev 21)
> > > >         Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> > > >         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> > > >         Latency: 0, Cache Line Size: 32 bytes
> > > >         Region 0: Memory at <ignored> (32-bit, non-prefetchable)
> > > >         Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
> > > >         I/O behind bridge: 00000000-00000fff
> > > >         Memory behind bridge: 80000000-9fffffff
> > > >         Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> > > >         BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
> > > >                 PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> > > >         Capabilities: [44] Power Management version 2
> > > >                 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
> > > >                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> > > >         Capabilities: [4c] Express (v1) Root Port (Slot-), MSI 00
> > > >                 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
> > > >                         ExtTag- RBE- FLReset-
> > > >                 DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
> > > >                         RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
> > > >                         MaxPayload 128 bytes, MaxReadReq 512 bytes
> > > >                 DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
> > > >                 LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited
> > > >                         ClockPM- Surprise- LLActRep- BwNot-
> > > >                 LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
> > > >                         ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> > > >                 LnkSta: Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
> > > >                 RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
> > > >                 RootCap: CRSVisible-
> > > >                 RootSta: PME ReqID 0000, PMEStatus- PMEPending-
> > >
> > > There is no slot cap etc, so pciehp will not be loaded.
> > > the power of you child device can not be turned off/on.
> > >
> > > Not sure if can use link off/on make the clock effective.
> > >
> > > You can turn off and on the pcie link like following:
> > >
> > > 1. remove the child device
> > >      echo 1 > /sys/..../0000:01:00.0/remove
> > > 2. disable link
> > >      echo 1 > /sys/..../0000:00.00.0/pcie_link_disable
> > > 3. enable link
> > >      echo 1 > /sys/..../0000:00.00.0/pcie_link_disable
> > > 4. rescan the pci bus.
> > >      echo 1 > /sys/..../0000:00:00.0/rescan_bridge
> > >
> > > please check link disable patch at
> > > git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git
> > > for-pci-pcie-link
> > >
> > > | Subject: [PATCH] PCI: Add link_disable in /sysfs for pcie device
> > > |
> > > | Found PCIe cards from one vendor, will not respond to scan from bridge,
> > > | if we change bus number setting in bridge device.
> > > |
> > > | Have to do link disable/enable on the pcie root port.
> > > |
> > > |  So try to expose link disable bit of pcie link control register. We can use
> > > |  echo 1 > /sys/..../link_disable
> > > |  echo 0 > /sys/..../link_disable
> > > | to bring the pcie device back to respond to scan.
> >
> > Sorry for the delay, got pulled into some high prio stuff.
> >
> > Anyhow, I have backported your patch to 3.4 as we cannot upgrade easily and
> > I cannot make it work. I do
> >
> > # > echo 1 > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable
> > # > echo 0 > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable
> > # > echo 1 > /sys/devices/pci0000:00/0000:00:00.0/rescan

> > # > lspci
> > 00:00.0 Class 0604: Device 1957:0079 (rev 21)
> >
> > #> dmesg
> > pcieport 0000:00:00.0: pcie_link_disable_set: lnk_ctrl = 18
> > pcieport 0000:00:00.0: pcie_link_disable_set: lnk_ctrl = 8
> >
> > find /sys -name pcie_link_disable
> > find /sys -name remove
> > find /sys -name rescan
> >
> > shows
> > /sys/devices/pci0000:00/0000:00:00.0/pcie_link_disable
> > /sys/devices/pci0000:00/0000:00:00.0/remove
> > /sys/bus/pci/rescan
> > /sys/devices/pci0000:00/0000:00:00.0/rescan
> > /sys/devices/pci0000:00/0000:00:00.0/pci_bus/0000:01/rescan
> > /sys/devices/pci0000:00/pci_bus/0000:00/rescan
> >
> > so something is missing, but what?
>
> Tried with FAKE hotplug too but no luck.
>
> What is best pcie_hotplug or fake hotplug?
> Would be nice if I could eliminate one of them.

I found a register to turn on Slot and SlotClk so now I get:
root@P2020RDB ~ # lspci -vvvvxx
00:00.0 Class 0604: Device 1957:0079 (rev 21)
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Region 0: Memory at <ignored> (32-bit, non-prefetchable)
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 00000000-00000fff
	Memory behind bridge: 90000000-900fffff
	Prefetchable memory behind bridge: 0000000080000000-000000008dffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [44] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [4c] Express (v1) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE- FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x2, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise-
			Slot #0, PowerLimit 15.000W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Off, PwrInd Off, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
			Changed: MRL- PresDet- LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
00: 57 19 79 00 06 01 10 00 21 00 20 0b 00 00 01 00
10: 00 00 f0 ff 00 00 00 00 00 01 01 00 00 00 00 00
20: 00 90 00 90 01 80 f1 8d 00 00 00 00 00 00 00 00
30: 00 00 00 00 44 00 00 00 00 00 00 00 00 00 00 00

However I still don't get it to work
echo 1 > /sys/..../rescan
does not expose the device behind the RC nor does toggling pcie_link_disable

 Jocke


^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2012-07-19  9:17 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-07-10 17:29 PICe hotplug problems Joakim Tjernlund
2012-07-10 18:22 ` Yinghai Lu
2012-07-10 21:42   ` Joakim Tjernlund
2012-07-10 22:09     ` Yinghai Lu
2012-07-10 22:20       ` Bjorn Helgaas
2012-07-10 22:22         ` Yinghai Lu
2012-07-10 22:46           ` Bjorn Helgaas
2012-07-10 22:49             ` Bjorn Helgaas
2012-07-11  1:07       ` Joakim Tjernlund
2012-07-11  1:33         ` Yinghai Lu
2012-07-11  4:08           ` Joakim Tjernlund
2012-07-11  6:09             ` Yinghai Lu
     [not found]           ` <OF259177D8.B20ED592-ONC1257A38.00167C85-C1257A38.0016C890@LocalDomain>
2012-07-11  5:08             ` Joakim Tjernlund
2012-07-11 14:13               ` Jiang Liu
2012-07-18 13:07           ` Joakim Tjernlund
     [not found]           ` <OF85CC1E00.6E804B96-ONC1257A3F.00477FD2-C1257A3F.0048111D@LocalDomain>
2012-07-18 23:34             ` Joakim Tjernlund
     [not found]             ` <OF57E68083.D4D2A234-ONC1257A3F.008141AA-C1257A3F.00817A68@LocalDomain>
2012-07-19  9:17               ` Joakim Tjernlund

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.