All of lore.kernel.org
 help / color / mirror / Atom feed
* bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
       [not found] ` <87r4s0opck.fsf@nemi.mork.no>
@ 2012-07-25  4:08   ` Bjørn Mork
  2012-07-25  4:34     ` Huang Ying
  0 siblings, 1 reply; 89+ messages in thread
From: Bjørn Mork @ 2012-07-25  4:08 UTC (permalink / raw)
  To: Huang Ying
  Cc: Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas, linux-pci,
	linux-usb, Alan Stern

[-- Attachment #1: Type: text/plain, Size: 1052 bytes --]

Enabling autosuspend for USB causes hotplug failure in the current
linux-next. Newly plugged devices are not detected at all until the
port/controller is manually powered on by writing "on" to power/control.
Testing is pretty simple:

  1) for f in /sys/bus/usb/devices/*/power/control; do echo auto > $f; done
  2) wait for the controllers to suspend
  3) plugin a new USB device


I've bisected the regression down to this commit:


commit 448bd857d48e69b33ef323739dc6d8ca20d4cda7
Author: Huang Ying <ying.huang@intel.com>
Date:   Sat Jun 23 10:23:51 2012 +0800

    PCI/PM: add PCIe runtime D3cold support
    

Looks like this somehow powers down my USB host controllers in such a
way that they are unable to detect connections to their ports.  The
system is a pretty standard 4 year old intel based laptop.  Full lspci
listing is attached.

Please let me know if you need further information.  And please fix this
before rc1.  It effectively makes USB non-functional if autosuspend is
enabled.



Thanks,
Bjørn


[-- Attachment #2: lspci.txt --]
[-- Type: text/plain, Size: 22272 bytes --]

00:00.0 Host bridge [0600]: Intel Corporation Mobile 4 Series Chipset Memory Controller Hub [8086:2a40] (rev 07)
	Subsystem: Lenovo Device [17aa:20e0]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
	Latency: 0
	Capabilities: [e0] Vendor Specific Information: Len=0a <?>
	Kernel driver in use: agpgart-intel

00:02.0 VGA compatible controller [0300]: Intel Corporation Mobile 4 Series Chipset Integrated Graphics Controller [8086:2a42] (rev 07) (prog-if 00 [VGA controller])
	Subsystem: Lenovo Device [17aa:20e4]
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 45
	Region 0: Memory at f0000000 (64-bit, non-prefetchable) [size=4M]
	Region 2: Memory at d0000000 (64-bit, prefetchable) [size=256M]
	Region 4: I/O ports at 1800 [size=8]
	Expansion ROM at <unassigned> [disabled]
	Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee0300c  Data: 41a1
	Capabilities: [d0] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: i915

00:02.1 Display controller [0380]: Intel Corporation Mobile 4 Series Chipset Integrated Graphics Controller [8086:2a43] (rev 07)
	Subsystem: Lenovo Device [17aa:20e4]
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Region 0: Memory at f0400000 (64-bit, non-prefetchable) [size=1M]
	Capabilities: [d0] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

00:03.0 Communication controller [0780]: Intel Corporation Mobile 4 Series Chipset MEI Controller [8086:2a44] (rev 07)
	Subsystem: Lenovo Device [17aa:20e6]
	Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx+
	Interrupt: pin A routed to IRQ 44
	Region 0: Memory at f0826800 (64-bit, non-prefetchable) [size=16]
	Capabilities: [50] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [8c] MSI: Enable+ Count=1/1 Maskable- 64bit+
		Address: 00000000fee0300c  Data: 4199

00:19.0 Ethernet controller [0200]: Intel Corporation 82567LM Gigabit Network Connection [8086:10f5] (rev 03)
	Subsystem: Lenovo Device [17aa:20ee]
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 20
	Region 0: Memory at f0600000 (32-bit, non-prefetchable) [disabled] [size=128K]
	Region 1: Memory at f0625000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Region 2: I/O ports at 1840 [disabled] [size=32]
	Capabilities: [c8] Power Management version 2
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D3 NoSoftRst- PME-Enable+ DSel=0 DScale=1 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 00000000fee0300c  Data: 4122
	Capabilities: [e0] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: e1000e

00:1a.0 USB controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 [8086:2937] (rev 03) (prog-if 00 [UHCI])
	Subsystem: Lenovo Device [17aa:20f0]
	Control: I/O+ Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 20
	Region 4: I/O ports at 1860 [size=32]
	Capabilities: [50] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: uhci_hcd

00:1a.1 USB controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 [8086:2938] (rev 03) (prog-if 00 [UHCI])
	Subsystem: Lenovo Device [17aa:20f0]
	Control: I/O+ Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin B routed to IRQ 21
	Region 4: I/O ports at 1880 [size=32]
	Capabilities: [50] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: uhci_hcd

00:1a.2 USB controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6 [8086:2939] (rev 03) (prog-if 00 [UHCI])
	Subsystem: Lenovo Device [17aa:20f0]
	Control: I/O+ Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin C routed to IRQ 22
	Region 4: I/O ports at 18a0 [size=32]
	Capabilities: [50] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: uhci_hcd

00:1a.7 USB controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 [8086:293c] (rev 03) (prog-if 20 [EHCI])
	Subsystem: Lenovo Device [17aa:20f1]
	Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin D routed to IRQ 23
	Region 0: Memory at f0826c00 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D3 NoSoftRst- PME-Enable+ DSel=0 DScale=0 PME-
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Capabilities: [98] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: ehci_hcd

00:1b.0 Audio device [0403]: Intel Corporation 82801I (ICH9 Family) HD Audio Controller [8086:293e] (rev 03)
	Subsystem: Lenovo Device [17aa:20f2]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 47
	Region 0: Memory at f0620000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=55mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [60] MSI: Enable+ Count=1/1 Maskable- 64bit+
		Address: 00000000fee0300c  Data: 41b1
	Capabilities: [70] Express (v1) Root Complex Integrated Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE- FLReset+
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
		VC1:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable- ID=0 ArbSelect=Fixed TC/VC=00
			Status:	NegoPending- InProgress-
	Capabilities: [130 v1] Root Complex Link
		Desc:	PortNumber=0f ComponentID=02 EltType=Config
		Link0:	Desc:	TargetPort=00 TargetComponent=02 AssocRCRB- LinkType=MemMapped LinkValid+
			Addr:	00000000fed1c000
	Kernel driver in use: snd_hda_intel

00:1c.0 PCI bridge [0604]: Intel Corporation 82801I (ICH9 Family) PCI Express Port 1 [8086:2940] (rev 03) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
	I/O behind bridge: 00002000-00002fff
	Memory behind bridge: c0000000-c01fffff
	Prefetchable memory behind bridge: 00000000c0200000-00000000c03fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v1) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #1, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <1us, L1 <4us
			ClockPM- Surprise- LLActRep+ BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+
			Slot #0, PowerLimit 6.500W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet+ CmdCplt+ HPIrq+ LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
			Changed: MRL- PresDet- LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee0300c  Data: 4179
	Capabilities: [90] Subsystem: Lenovo Device [17aa:20f3]
	Capabilities: [a0] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D3 NoSoftRst- PME-Enable+ DSel=0 DScale=0 PME-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed+ WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [180 v1] Root Complex Link
		Desc:	PortNumber=01 ComponentID=02 EltType=Config
		Link0:	Desc:	TargetPort=00 TargetComponent=02 AssocRCRB- LinkType=MemMapped LinkValid+
			Addr:	00000000fed1c000
	Kernel driver in use: pcieport

00:1c.1 PCI bridge [0604]: Intel Corporation 82801I (ICH9 Family) PCI Express Port 2 [8086:2942] (rev 03) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
	I/O behind bridge: 00003000-00003fff
	Memory behind bridge: f0500000-f05fffff
	Prefetchable memory behind bridge: 00000000c0400000-00000000c05fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v1) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #2, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <256ns, L1 <4us
			ClockPM- Surprise- LLActRep+ BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+
			Slot #1, PowerLimit 6.500W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet+ CmdCplt+ HPIrq+ LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState+
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee0300c  Data: 4181
	Capabilities: [90] Subsystem: Lenovo Device [17aa:20f3]
	Capabilities: [a0] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed+ WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=01
			Status:	NegoPending- InProgress-
	Capabilities: [180 v1] Root Complex Link
		Desc:	PortNumber=02 ComponentID=02 EltType=Config
		Link0:	Desc:	TargetPort=00 TargetComponent=02 AssocRCRB- LinkType=MemMapped LinkValid+
			Addr:	00000000fed1c000
	Kernel driver in use: pcieport

00:1d.0 USB controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 [8086:2934] (rev 03) (prog-if 00 [UHCI])
	Subsystem: Lenovo Device [17aa:20f0]
	Control: I/O+ Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 16
	Region 4: I/O ports at 18c0 [size=32]
	Capabilities: [50] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: uhci_hcd

00:1d.1 USB controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 [8086:2935] (rev 03) (prog-if 00 [UHCI])
	Subsystem: Lenovo Device [17aa:20f0]
	Control: I/O+ Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin B routed to IRQ 17
	Region 4: I/O ports at 18e0 [size=32]
	Capabilities: [50] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: uhci_hcd

00:1d.2 USB controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 [8086:2936] (rev 03) (prog-if 00 [UHCI])
	Subsystem: Lenovo Device [17aa:20f0]
	Control: I/O+ Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin C routed to IRQ 18
	Region 4: I/O ports at 1c00 [size=32]
	Capabilities: [50] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: uhci_hcd

00:1d.7 USB controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 [8086:293a] (rev 03) (prog-if 20 [EHCI])
	Subsystem: Lenovo Device [17aa:20f1]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin D routed to IRQ 19
	Region 0: Memory at f0827000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Capabilities: [98] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: ehci_hcd

00:1e.0 PCI bridge [0604]: Intel Corporation 82801 Mobile PCI Bridge [8086:2448] (rev 93) (prog-if 01 [Subtractive decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Bus: primary=00, secondary=15, subordinate=15, sec-latency=32
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: fff00000-000fffff
	Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
	Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [50] Subsystem: Lenovo Device [17aa:20f4]

00:1f.0 ISA bridge [0601]: Intel Corporation ICH9M-E LPC Interface Controller [8086:2917] (rev 03)
	Subsystem: Lenovo Device [17aa:20f5]
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Capabilities: [e0] Vendor Specific Information: Len=0c <?>

00:1f.2 SATA controller [0106]: Intel Corporation 82801IBM/IEM (ICH9M/ICH9M-E) 4 port SATA Controller [AHCI mode] [8086:2929] (rev 03) (prog-if 01 [AHCI 1.0])
	Subsystem: Lenovo Device [17aa:20f8]
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin B routed to IRQ 43
	Region 0: I/O ports at 1c48 [size=8]
	Region 1: I/O ports at 183c [size=4]
	Region 2: I/O ports at 1c40 [size=8]
	Region 3: I/O ports at 1838 [size=4]
	Region 4: I/O ports at 1c20 [size=32]
	Region 5: Memory at f0826000 (32-bit, non-prefetchable) [size=2K]
	Capabilities: [80] MSI: Enable+ Count=1/16 Maskable- 64bit-
		Address: fee0300c  Data: 4191
	Capabilities: [70] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [a8] SATA HBA v1.0 BAR4 Offset=00000004
	Capabilities: [b0] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: ahci

00:1f.3 SMBus [0c05]: Intel Corporation 82801I (ICH9 Family) SMBus Controller [8086:2930] (rev 03)
	Subsystem: Lenovo Device [17aa:20f9]
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 23
	Region 0: Memory at f0827400 (64-bit, non-prefetchable) [size=256]
	Region 4: I/O ports at 1c60 [size=32]
	Kernel driver in use: i801_smbus

03:00.0 Network controller [0280]: Intel Corporation Ultimate N WiFi Link 5300 [8086:4236]
	Subsystem: Intel Corporation Device [8086:1011]
	Physical Slot: 1
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 46
	Region 0: Memory at f0500000 (64-bit, non-prefetchable) [size=8K]
	Capabilities: [c8] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
		Address: 00000000fee0300c  Data: 41a9
	Capabilities: [e0] Express (v1) Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L1 unlimited
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <128ns, L1 <32us
			ClockPM+ Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Device Serial Number 00-16-ea-ff-ff-b3-07-88
	Kernel driver in use: iwlwifi


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-25  4:08   ` bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure Bjørn Mork
@ 2012-07-25  4:34     ` Huang Ying
  2012-07-25  9:58       ` Bjørn Mork
  0 siblings, 1 reply; 89+ messages in thread
From: Huang Ying @ 2012-07-25  4:34 UTC (permalink / raw)
  To: Bjørn Mork
  Cc: Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas, linux-pci,
	linux-usb, Alan Stern

Hi, Bjorn,

On Wed, 2012-07-25 at 06:08 +0200, Bjørn Mork wrote:
> Enabling autosuspend for USB causes hotplug failure in the current
> linux-next. Newly plugged devices are not detected at all until the
> port/controller is manually powered on by writing "on" to power/control.
> Testing is pretty simple:
> 
>   1) for f in /sys/bus/usb/devices/*/power/control; do echo auto > $f; done

Have you done:

for f in /sys/bus/pci/devices/*/power/confol; do echo auto > $f; done

?

If not, the pci device will not be suspended at all.

>   2) wait for the controllers to suspend
>   3) plugin a new USB device

After plugin the new USB device, is there anything in dmesg?

> 
> I've bisected the regression down to this commit:
> 
> 
> commit 448bd857d48e69b33ef323739dc6d8ca20d4cda7
> Author: Huang Ying <ying.huang@intel.com>
> Date:   Sat Jun 23 10:23:51 2012 +0800
> 
>     PCI/PM: add PCIe runtime D3cold support
>     
> 
> Looks like this somehow powers down my USB host controllers in such a
> way that they are unable to detect connections to their ports.  The
> system is a pretty standard 4 year old intel based laptop.  Full lspci
> listing is attached.
>
> Please let me know if you need further information.  And please fix this
> before rc1.  It effectively makes USB non-functional if autosuspend is
> enabled.

Can you provide the output of the following command lines?

grep . /sys/bus/pci/devices/*/power/*
grep . /sys/bus/usb/devices/*/power/*
lsusb

After the controllers suspended.

Best Reards,
Huang Ying



^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-25  4:34     ` Huang Ying
@ 2012-07-25  9:58       ` Bjørn Mork
  2012-07-25 13:30         ` huang ying
  0 siblings, 1 reply; 89+ messages in thread
From: Bjørn Mork @ 2012-07-25  9:58 UTC (permalink / raw)
  To: Huang Ying
  Cc: Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas, linux-pci,
	linux-usb, Alan Stern

[-- Attachment #1: Type: text/plain, Size: 5738 bytes --]

Huang Ying <ying.huang@intel.com> writes:
> On Wed, 2012-07-25 at 06:08 +0200, Bjørn Mork wrote:
>> Enabling autosuspend for USB causes hotplug failure in the current
>> linux-next. Newly plugged devices are not detected at all until the
>> port/controller is manually powered on by writing "on" to power/control.
>> Testing is pretty simple:
>> 
>>   1) for f in /sys/bus/usb/devices/*/power/control; do echo auto > $f; done
>
> Have you done:
>
> for f in /sys/bus/pci/devices/*/power/confol; do echo auto > $f; done
>
> ?
>
> If not, the pci device will not be suspended at all.

Yes, sorry for missing that.  I had it automatically enabled.  Yes,
autosuspend for the PCI device and all child devices must be enabled for
the device to be suspended at all, of course.

>>   2) wait for the controllers to suspend
>>   3) plugin a new USB device
>
> After plugin the new USB device, is there anything in dmesg?

No. Absolutely nothing, so the USB devices is not enumerated.  Another
indication of the same:  Plugging a device like an Android phone, which
normally detects being connected to a host and presents a device type
menu to the user, results in the charging LED lighting up but no menu.


Trying to show the sequence of events:

1)  the controllers are suspended:

Jul 25 11:27:12 nemi kernel: [   38.962792] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D2
Jul 25 11:27:12 nemi kernel: [   39.006718] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D2
Jul 25 11:27:15 nemi kernel: [   41.808471] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D2
Jul 25 11:27:15 nemi kernel: [   41.824123] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D2
Jul 25 11:27:15 nemi kernel: [   41.824194] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D2

2) I connect a new USB device here

3) I read all power/files triggering this warning (as expected),
  included for timing info - this happens a while after connecting the
  USB device:

Jul 25 11:28:34 nemi kernel: [  121.440558] WARNING! power/level is deprecated; use power/control instead



4) I wake up the usb2 bus I connected the device to, and it is
immediately found (no info about power state change here?):

nemi:/home/bjorn# echo on > /sys/bus/usb/devices/usb2/power/control 


Jul 25 11:29:32 nemi kernel: [  179.121418] ehci_hcd 0000:00:1d.7: setting latency timer to 64
Jul 25 11:29:32 nemi kernel: [  179.364088] usb 2-1: new high-speed USB device number 3 using ehci_hcd
Jul 25 11:29:33 nemi kernel: [  179.500042] usb 2-1: New USB device found, idVendor=0fce, idProduct=2138
Jul 25 11:29:33 nemi kernel: [  179.500055] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Jul 25 11:29:33 nemi kernel: [  179.500063] usb 2-1: Product: SEMC HSUSB Device
Jul 25 11:29:33 nemi kernel: [  179.500069] usb 2-1: Manufacturer: SEMC
Jul 25 11:29:33 nemi kernel: [  179.500075] usb 2-1: SerialNumber: 43423531314D42573144




Doing the same with commit 448bd857d reverted:


1)  the controllers are suspended (to state D3? instead of D2?):

Jul 25 11:34:01 nemi kernel: [   37.064955] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D3
Jul 25 11:34:01 nemi kernel: [   37.106586] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D3
Jul 25 11:34:04 nemi kernel: [   39.808329] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D3
Jul 25 11:34:04 nemi kernel: [   39.840054] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D3
Jul 25 11:34:04 nemi kernel: [   39.840068] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D3

2) plugging a USB device causes immediate wakeup (and logs changes state
   to D0):

Jul 25 11:34:43 nemi kernel: [   78.575357] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D0
Jul 25 11:34:43 nemi kernel: [   78.588162] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D0
Jul 25 11:34:43 nemi kernel: [   78.588292] ehci_hcd 0000:00:1d.7: setting latency timer to 64
Jul 25 11:34:43 nemi kernel: [   78.832337] usb 2-1: new high-speed USB device number 3 using ehci_hcd
Jul 25 11:34:43 nemi kernel: [   78.968286] usb 2-1: New USB device found, idVendor=0fce, idProduct=2138
Jul 25 11:34:43 nemi kernel: [   78.968300] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Jul 25 11:34:43 nemi kernel: [   78.968307] usb 2-1: Product: SEMC HSUSB Device
Jul 25 11:34:43 nemi kernel: [   78.968314] usb 2-1: Manufacturer: SEMC




>> 
>> I've bisected the regression down to this commit:
>> 
>> 
>> commit 448bd857d48e69b33ef323739dc6d8ca20d4cda7
>> Author: Huang Ying <ying.huang@intel.com>
>> Date:   Sat Jun 23 10:23:51 2012 +0800
>> 
>>     PCI/PM: add PCIe runtime D3cold support
>>     
>> 
>> Looks like this somehow powers down my USB host controllers in such a
>> way that they are unable to detect connections to their ports.  The
>> system is a pretty standard 4 year old intel based laptop.  Full lspci
>> listing is attached.
>>
>> Please let me know if you need further information.  And please fix this
>> before rc1.  It effectively makes USB non-functional if autosuspend is
>> enabled.
>
> Can you provide the output of the following command lines?
>
> grep . /sys/bus/pci/devices/*/power/*
> grep . /sys/bus/usb/devices/*/power/*
> lsusb
>
> After the controllers suspended.

I've attached two copies of this output:  

 - powerstate-withbug.txt, from a plain next-20120724 kernel
 - powerstate-reverted.txt, from a next-20120724 kernel with commit
   448bd857d reverted

I cannot really see any interesting differences in these.  But the D3/D2
difference in the kernel log might be of interest?



Bjørn


[-- Attachment #2: powerstate-withbug.txt --]
[-- Type: text/plain, Size: 30392 bytes --]

/sys/bus/usb/devices/1-0:1.0/power/async:enabled
/sys/bus/usb/devices/1-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/1-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/1-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/1-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/1-6:1.0/power/async:enabled
/sys/bus/usb/devices/1-6:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/1-6:1.0/power/runtime_enabled:disabled
/sys/bus/usb/devices/1-6:1.0/power/runtime_status:unsupported
/sys/bus/usb/devices/1-6:1.0/power/runtime_usage:0
/sys/bus/usb/devices/1-6:1.1/power/async:enabled
/sys/bus/usb/devices/1-6:1.1/power/runtime_active_kids:0
/sys/bus/usb/devices/1-6:1.1/power/runtime_enabled:disabled
/sys/bus/usb/devices/1-6:1.1/power/runtime_status:unsupported
/sys/bus/usb/devices/1-6:1.1/power/runtime_usage:0
/sys/bus/usb/devices/1-6/power/active_duration:37124
/sys/bus/usb/devices/1-6/power/async:enabled
/sys/bus/usb/devices/1-6/power/autosuspend:2
/sys/bus/usb/devices/1-6/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/1-6/power/connected_duration:119252
/sys/bus/usb/devices/1-6/power/control:auto
/sys/bus/usb/devices/1-6/power/level:auto
/sys/bus/usb/devices/1-6/power/persist:1
/sys/bus/usb/devices/1-6/power/runtime_active_kids:0
/sys/bus/usb/devices/1-6/power/runtime_active_time:36896
/sys/bus/usb/devices/1-6/power/runtime_enabled:enabled
/sys/bus/usb/devices/1-6/power/runtime_status:suspended
/sys/bus/usb/devices/1-6/power/runtime_suspended_time:82112
/sys/bus/usb/devices/1-6/power/runtime_usage:0
/sys/bus/usb/devices/2-0:1.0/power/async:enabled
/sys/bus/usb/devices/2-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/2-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/2-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/2-4:1.0/power/async:enabled
/sys/bus/usb/devices/2-4:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-4:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/2-4:1.0/power/runtime_usage:0
/sys/bus/usb/devices/2-4:1.19/power/async:enabled
/sys/bus/usb/devices/2-4:1.19/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4:1.19/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-4:1.19/power/runtime_status:suspended
/sys/bus/usb/devices/2-4:1.19/power/runtime_usage:0
/sys/bus/usb/devices/2-4:1.20/power/async:enabled
/sys/bus/usb/devices/2-4:1.20/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4:1.20/power/runtime_enabled:disabled
/sys/bus/usb/devices/2-4:1.20/power/runtime_status:unsupported
/sys/bus/usb/devices/2-4:1.20/power/runtime_usage:0
/sys/bus/usb/devices/2-4:1.2/power/async:enabled
/sys/bus/usb/devices/2-4:1.2/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4:1.2/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-4:1.2/power/runtime_status:suspended
/sys/bus/usb/devices/2-4:1.2/power/runtime_usage:0
/sys/bus/usb/devices/2-4:1.3/power/async:enabled
/sys/bus/usb/devices/2-4:1.3/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4:1.3/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-4:1.3/power/runtime_status:suspended
/sys/bus/usb/devices/2-4:1.3/power/runtime_usage:0
/sys/bus/usb/devices/2-4:1.8/power/async:enabled
/sys/bus/usb/devices/2-4:1.8/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4:1.8/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-4:1.8/power/runtime_status:suspended
/sys/bus/usb/devices/2-4:1.8/power/runtime_usage:0
/sys/bus/usb/devices/2-4/power/active_duration:36976
/sys/bus/usb/devices/2-4/power/async:enabled
/sys/bus/usb/devices/2-4/power/autosuspend:2
/sys/bus/usb/devices/2-4/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/2-4/power/connected_duration:119000
/sys/bus/usb/devices/2-4/power/control:auto
/sys/bus/usb/devices/2-4/power/level:auto
/sys/bus/usb/devices/2-4/power/persist:1
/sys/bus/usb/devices/2-4/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4/power/runtime_active_time:36748
/sys/bus/usb/devices/2-4/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-4/power/runtime_status:suspended
/sys/bus/usb/devices/2-4/power/runtime_suspended_time:82008
/sys/bus/usb/devices/2-4/power/runtime_usage:0
/sys/bus/usb/devices/2-4/power/wakeup:disabled
/sys/bus/usb/devices/3-0:1.0/power/async:enabled
/sys/bus/usb/devices/3-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/3-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/3-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/3-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/3-2:1.0/power/async:enabled
/sys/bus/usb/devices/3-2:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/3-2:1.0/power/runtime_enabled:disabled
/sys/bus/usb/devices/3-2:1.0/power/runtime_status:unsupported
/sys/bus/usb/devices/3-2:1.0/power/runtime_usage:0
/sys/bus/usb/devices/3-2/power/active_duration:36736
/sys/bus/usb/devices/3-2/power/async:enabled
/sys/bus/usb/devices/3-2/power/autosuspend:2
/sys/bus/usb/devices/3-2/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/3-2/power/connected_duration:118616
/sys/bus/usb/devices/3-2/power/control:auto
/sys/bus/usb/devices/3-2/power/level:auto
/sys/bus/usb/devices/3-2/power/persist:1
/sys/bus/usb/devices/3-2/power/runtime_active_kids:0
/sys/bus/usb/devices/3-2/power/runtime_active_time:36496
/sys/bus/usb/devices/3-2/power/runtime_enabled:enabled
/sys/bus/usb/devices/3-2/power/runtime_status:suspended
/sys/bus/usb/devices/3-2/power/runtime_suspended_time:81864
/sys/bus/usb/devices/3-2/power/runtime_usage:0
/sys/bus/usb/devices/3-2/power/wakeup:disabled
/sys/bus/usb/devices/4-0:1.0/power/async:enabled
/sys/bus/usb/devices/4-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/4-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/4-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/4-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/4-2:1.0/power/async:enabled
/sys/bus/usb/devices/4-2:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/4-2:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/4-2:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/4-2:1.0/power/runtime_usage:0
/sys/bus/usb/devices/4-2:1.1/power/async:enabled
/sys/bus/usb/devices/4-2:1.1/power/runtime_active_kids:0
/sys/bus/usb/devices/4-2:1.1/power/runtime_enabled:enabled
/sys/bus/usb/devices/4-2:1.1/power/runtime_status:suspended
/sys/bus/usb/devices/4-2:1.1/power/runtime_usage:0
/sys/bus/usb/devices/4-2:1.2/power/async:enabled
/sys/bus/usb/devices/4-2:1.2/power/runtime_active_kids:0
/sys/bus/usb/devices/4-2:1.2/power/runtime_enabled:disabled
/sys/bus/usb/devices/4-2:1.2/power/runtime_status:unsupported
/sys/bus/usb/devices/4-2:1.2/power/runtime_usage:0
/sys/bus/usb/devices/4-2:1.3/power/async:enabled
/sys/bus/usb/devices/4-2:1.3/power/runtime_active_kids:0
/sys/bus/usb/devices/4-2:1.3/power/runtime_enabled:disabled
/sys/bus/usb/devices/4-2:1.3/power/runtime_status:unsupported
/sys/bus/usb/devices/4-2:1.3/power/runtime_usage:0
/sys/bus/usb/devices/4-2/power/active_duration:26552
/sys/bus/usb/devices/4-2/power/async:enabled
/sys/bus/usb/devices/4-2/power/autosuspend:2
/sys/bus/usb/devices/4-2/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/4-2/power/connected_duration:118200
/sys/bus/usb/devices/4-2/power/control:auto
/sys/bus/usb/devices/4-2/power/level:auto
/sys/bus/usb/devices/4-2/power/persist:1
/sys/bus/usb/devices/4-2/power/runtime_active_kids:0
/sys/bus/usb/devices/4-2/power/runtime_active_time:26376
/sys/bus/usb/devices/4-2/power/runtime_enabled:enabled
/sys/bus/usb/devices/4-2/power/runtime_status:suspended
/sys/bus/usb/devices/4-2/power/runtime_suspended_time:91568
/sys/bus/usb/devices/4-2/power/runtime_usage:0
/sys/bus/usb/devices/4-2/power/wakeup:disabled
/sys/bus/usb/devices/5-0:1.0/power/async:enabled
/sys/bus/usb/devices/5-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/5-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/5-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/5-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/6-0:1.0/power/async:enabled
/sys/bus/usb/devices/6-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/6-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/6-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/6-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/7-0:1.0/power/async:enabled
/sys/bus/usb/devices/7-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/7-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/7-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/7-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/8-0:1.0/power/async:enabled
/sys/bus/usb/devices/8-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/8-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/8-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/8-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/usb1/power/active_duration:40060
/sys/bus/usb/devices/usb1/power/async:enabled
/sys/bus/usb/devices/usb1/power/autosuspend:2
/sys/bus/usb/devices/usb1/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb1/power/connected_duration:119692
/sys/bus/usb/devices/usb1/power/control:auto
/sys/bus/usb/devices/usb1/power/level:auto
/sys/bus/usb/devices/usb1/power/runtime_active_kids:0
/sys/bus/usb/devices/usb1/power/runtime_active_time:40044
/sys/bus/usb/devices/usb1/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb1/power/runtime_status:suspended
/sys/bus/usb/devices/usb1/power/runtime_suspended_time:79632
/sys/bus/usb/devices/usb1/power/runtime_usage:0
/sys/bus/usb/devices/usb1/power/wakeup:disabled
/sys/bus/usb/devices/usb2/power/active_duration:40044
/sys/bus/usb/devices/usb2/power/async:enabled
/sys/bus/usb/devices/usb2/power/autosuspend:2
/sys/bus/usb/devices/usb2/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb2/power/connected_duration:119680
/sys/bus/usb/devices/usb2/power/control:auto
/sys/bus/usb/devices/usb2/power/level:auto
/sys/bus/usb/devices/usb2/power/runtime_active_kids:0
/sys/bus/usb/devices/usb2/power/runtime_active_time:40028
/sys/bus/usb/devices/usb2/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb2/power/runtime_status:suspended
/sys/bus/usb/devices/usb2/power/runtime_suspended_time:79636
/sys/bus/usb/devices/usb2/power/runtime_usage:0
/sys/bus/usb/devices/usb2/power/wakeup:disabled
/sys/bus/usb/devices/usb3/power/active_duration:40024
/sys/bus/usb/devices/usb3/power/async:enabled
/sys/bus/usb/devices/usb3/power/autosuspend:2
/sys/bus/usb/devices/usb3/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb3/power/connected_duration:119660
/sys/bus/usb/devices/usb3/power/control:auto
/sys/bus/usb/devices/usb3/power/level:auto
/sys/bus/usb/devices/usb3/power/runtime_active_kids:0
/sys/bus/usb/devices/usb3/power/runtime_active_time:40024
/sys/bus/usb/devices/usb3/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb3/power/runtime_status:suspended
/sys/bus/usb/devices/usb3/power/runtime_suspended_time:79636
/sys/bus/usb/devices/usb3/power/runtime_usage:0
/sys/bus/usb/devices/usb3/power/wakeup:disabled
/sys/bus/usb/devices/usb4/power/active_duration:31056
/sys/bus/usb/devices/usb4/power/async:enabled
/sys/bus/usb/devices/usb4/power/autosuspend:2
/sys/bus/usb/devices/usb4/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb4/power/connected_duration:119656
/sys/bus/usb/devices/usb4/power/control:auto
/sys/bus/usb/devices/usb4/power/level:auto
/sys/bus/usb/devices/usb4/power/runtime_active_kids:0
/sys/bus/usb/devices/usb4/power/runtime_active_time:31056
/sys/bus/usb/devices/usb4/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb4/power/runtime_status:suspended
/sys/bus/usb/devices/usb4/power/runtime_suspended_time:88600
/sys/bus/usb/devices/usb4/power/runtime_usage:0
/sys/bus/usb/devices/usb4/power/wakeup:disabled
/sys/bus/usb/devices/usb5/power/active_duration:4020
/sys/bus/usb/devices/usb5/power/async:enabled
/sys/bus/usb/devices/usb5/power/autosuspend:2
/sys/bus/usb/devices/usb5/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb5/power/connected_duration:119656
/sys/bus/usb/devices/usb5/power/control:auto
/sys/bus/usb/devices/usb5/power/level:auto
/sys/bus/usb/devices/usb5/power/runtime_active_kids:0
/sys/bus/usb/devices/usb5/power/runtime_active_time:4020
/sys/bus/usb/devices/usb5/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb5/power/runtime_status:suspended
/sys/bus/usb/devices/usb5/power/runtime_suspended_time:115636
/sys/bus/usb/devices/usb5/power/runtime_usage:0
/sys/bus/usb/devices/usb5/power/wakeup:disabled
/sys/bus/usb/devices/usb6/power/active_duration:2104
/sys/bus/usb/devices/usb6/power/async:enabled
/sys/bus/usb/devices/usb6/power/autosuspend:2
/sys/bus/usb/devices/usb6/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb6/power/connected_duration:119652
/sys/bus/usb/devices/usb6/power/control:auto
/sys/bus/usb/devices/usb6/power/level:auto
/sys/bus/usb/devices/usb6/power/runtime_active_kids:0
/sys/bus/usb/devices/usb6/power/runtime_active_time:2104
/sys/bus/usb/devices/usb6/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb6/power/runtime_status:suspended
/sys/bus/usb/devices/usb6/power/runtime_suspended_time:117548
/sys/bus/usb/devices/usb6/power/runtime_usage:0
/sys/bus/usb/devices/usb6/power/wakeup:disabled
/sys/bus/usb/devices/usb7/power/active_duration:4012
/sys/bus/usb/devices/usb7/power/async:enabled
/sys/bus/usb/devices/usb7/power/autosuspend:2
/sys/bus/usb/devices/usb7/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb7/power/connected_duration:119648
/sys/bus/usb/devices/usb7/power/control:auto
/sys/bus/usb/devices/usb7/power/level:auto
/sys/bus/usb/devices/usb7/power/runtime_active_kids:0
/sys/bus/usb/devices/usb7/power/runtime_active_time:4012
/sys/bus/usb/devices/usb7/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb7/power/runtime_status:suspended
/sys/bus/usb/devices/usb7/power/runtime_suspended_time:115636
/sys/bus/usb/devices/usb7/power/runtime_usage:0
/sys/bus/usb/devices/usb7/power/wakeup:disabled
/sys/bus/usb/devices/usb8/power/active_duration:2104
/sys/bus/usb/devices/usb8/power/async:enabled
/sys/bus/usb/devices/usb8/power/autosuspend:2
/sys/bus/usb/devices/usb8/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb8/power/connected_duration:119644
/sys/bus/usb/devices/usb8/power/control:auto
/sys/bus/usb/devices/usb8/power/level:auto
/sys/bus/usb/devices/usb8/power/runtime_active_kids:0
/sys/bus/usb/devices/usb8/power/runtime_active_time:2104
/sys/bus/usb/devices/usb8/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb8/power/runtime_status:suspended
/sys/bus/usb/devices/usb8/power/runtime_suspended_time:117540
/sys/bus/usb/devices/usb8/power/runtime_usage:0
/sys/bus/usb/devices/usb8/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:00.0/power/async:enabled
/sys/bus/pci/devices/0000:00:00.0/power/control:auto
/sys/bus/pci/devices/0000:00:00.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:00.0/power/runtime_active_time:120464
/sys/bus/pci/devices/0000:00:00.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:00.0/power/runtime_status:active
/sys/bus/pci/devices/0000:00:00.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:00.0/power/runtime_usage:1
/sys/bus/pci/devices/0000:00:02.0/power/async:enabled
/sys/bus/pci/devices/0000:00:02.0/power/control:auto
/sys/bus/pci/devices/0000:00:02.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:02.0/power/runtime_active_time:104204
/sys/bus/pci/devices/0000:00:02.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:02.0/power/runtime_status:active
/sys/bus/pci/devices/0000:00:02.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:02.0/power/runtime_usage:1
/sys/bus/pci/devices/0000:00:02.1/power/async:enabled
/sys/bus/pci/devices/0000:00:02.1/power/control:auto
/sys/bus/pci/devices/0000:00:02.1/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:02.1/power/runtime_active_time:0
/sys/bus/pci/devices/0000:00:02.1/power/runtime_enabled:disabled
/sys/bus/pci/devices/0000:00:02.1/power/runtime_status:unsupported
/sys/bus/pci/devices/0000:00:02.1/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:02.1/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:03.0/power/async:enabled
/sys/bus/pci/devices/0000:00:03.0/power/control:auto
/sys/bus/pci/devices/0000:00:03.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:03.0/power/runtime_active_time:105364
/sys/bus/pci/devices/0000:00:03.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:03.0/power/runtime_status:active
/sys/bus/pci/devices/0000:00:03.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:03.0/power/runtime_usage:1
/sys/bus/pci/devices/0000:00:03.0/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:19.0/power/async:enabled
/sys/bus/pci/devices/0000:00:19.0/power/control:auto
/sys/bus/pci/devices/0000:00:19.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:19.0/power/runtime_active_time:27628
/sys/bus/pci/devices/0000:00:19.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:19.0/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:19.0/power/runtime_suspended_time:92316
/sys/bus/pci/devices/0000:00:19.0/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:19.0/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:1a.0/power/async:enabled
/sys/bus/pci/devices/0000:00:1a.0/power/control:auto
/sys/bus/pci/devices/0000:00:1a.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1a.0/power/runtime_active_time:40024
/sys/bus/pci/devices/0000:00:1a.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1a.0/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1a.0/power/runtime_suspended_time:79640
/sys/bus/pci/devices/0000:00:1a.0/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_last_time_ms:1788
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1a.1/power/async:enabled
/sys/bus/pci/devices/0000:00:1a.1/power/control:auto
/sys/bus/pci/devices/0000:00:1a.1/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1a.1/power/runtime_active_time:37172
/sys/bus/pci/devices/0000:00:1a.1/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1a.1/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1a.1/power/runtime_suspended_time:82488
/sys/bus/pci/devices/0000:00:1a.1/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_last_time_ms:1789
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1a.2/power/async:enabled
/sys/bus/pci/devices/0000:00:1a.2/power/control:auto
/sys/bus/pci/devices/0000:00:1a.2/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1a.2/power/runtime_active_time:37172
/sys/bus/pci/devices/0000:00:1a.2/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1a.2/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1a.2/power/runtime_suspended_time:82488
/sys/bus/pci/devices/0000:00:1a.2/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_last_time_ms:1794
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1a.7/power/async:enabled
/sys/bus/pci/devices/0000:00:1a.7/power/control:auto
/sys/bus/pci/devices/0000:00:1a.7/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1a.7/power/runtime_active_time:40076
/sys/bus/pci/devices/0000:00:1a.7/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1a.7/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1a.7/power/runtime_suspended_time:79624
/sys/bus/pci/devices/0000:00:1a.7/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_last_time_ms:1765
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1b.0/power/async:enabled
/sys/bus/pci/devices/0000:00:1b.0/power/control:auto
/sys/bus/pci/devices/0000:00:1b.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1b.0/power/runtime_active_time:101776
/sys/bus/pci/devices/0000:00:1b.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1b.0/power/runtime_status:active
/sys/bus/pci/devices/0000:00:1b.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:1b.0/power/runtime_usage:1
/sys/bus/pci/devices/0000:00:1b.0/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:1c.0/power/async:enabled
/sys/bus/pci/devices/0000:00:1c.0/power/control:auto
/sys/bus/pci/devices/0000:00:1c.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1c.0/power/runtime_active_time:38056
/sys/bus/pci/devices/0000:00:1c.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1c.0/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1c.0/power/runtime_suspended_time:82464
/sys/bus/pci/devices/0000:00:1c.0/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1c.0/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:1c.1/power/async:enabled
/sys/bus/pci/devices/0000:00:1c.1/power/control:auto
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_active_kids:1
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_active_time:120520
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_status:active
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1c.1/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:1d.0/power/async:enabled
/sys/bus/pci/devices/0000:00:1d.0/power/control:auto
/sys/bus/pci/devices/0000:00:1d.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1d.0/power/runtime_active_time:37212
/sys/bus/pci/devices/0000:00:1d.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1d.0/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1d.0/power/runtime_suspended_time:82444
/sys/bus/pci/devices/0000:00:1d.0/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_last_time_ms:1797
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1d.1/power/async:enabled
/sys/bus/pci/devices/0000:00:1d.1/power/control:auto
/sys/bus/pci/devices/0000:00:1d.1/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1d.1/power/runtime_active_time:37220
/sys/bus/pci/devices/0000:00:1d.1/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1d.1/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1d.1/power/runtime_suspended_time:82436
/sys/bus/pci/devices/0000:00:1d.1/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_last_time_ms:1802
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1d.2/power/async:enabled
/sys/bus/pci/devices/0000:00:1d.2/power/control:auto
/sys/bus/pci/devices/0000:00:1d.2/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1d.2/power/runtime_active_time:37220
/sys/bus/pci/devices/0000:00:1d.2/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1d.2/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1d.2/power/runtime_suspended_time:82432
/sys/bus/pci/devices/0000:00:1d.2/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_last_time_ms:1805
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1d.7/power/async:enabled
/sys/bus/pci/devices/0000:00:1d.7/power/control:auto
/sys/bus/pci/devices/0000:00:1d.7/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1d.7/power/runtime_active_time:40060
/sys/bus/pci/devices/0000:00:1d.7/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1d.7/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1d.7/power/runtime_suspended_time:79628
/sys/bus/pci/devices/0000:00:1d.7/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_last_time_ms:1781
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1e.0/power/async:enabled
/sys/bus/pci/devices/0000:00:1e.0/power/control:auto
/sys/bus/pci/devices/0000:00:1e.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1e.0/power/runtime_active_time:0
/sys/bus/pci/devices/0000:00:1e.0/power/runtime_enabled:disabled
/sys/bus/pci/devices/0000:00:1e.0/power/runtime_status:unsupported
/sys/bus/pci/devices/0000:00:1e.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:1e.0/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1e.0/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:1f.0/power/async:enabled
/sys/bus/pci/devices/0000:00:1f.0/power/control:auto
/sys/bus/pci/devices/0000:00:1f.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1f.0/power/runtime_active_time:0
/sys/bus/pci/devices/0000:00:1f.0/power/runtime_enabled:disabled
/sys/bus/pci/devices/0000:00:1f.0/power/runtime_status:unsupported
/sys/bus/pci/devices/0000:00:1f.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:1f.0/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1f.2/power/async:enabled
/sys/bus/pci/devices/0000:00:1f.2/power/control:auto
/sys/bus/pci/devices/0000:00:1f.2/power/runtime_active_kids:4
/sys/bus/pci/devices/0000:00:1f.2/power/runtime_active_time:119672
/sys/bus/pci/devices/0000:00:1f.2/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1f.2/power/runtime_status:active
/sys/bus/pci/devices/0000:00:1f.2/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:1f.2/power/runtime_usage:1
/sys/bus/pci/devices/0000:00:1f.2/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:1f.3/power/async:enabled
/sys/bus/pci/devices/0000:00:1f.3/power/control:auto
/sys/bus/pci/devices/0000:00:1f.3/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1f.3/power/runtime_active_time:105156
/sys/bus/pci/devices/0000:00:1f.3/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1f.3/power/runtime_status:active
/sys/bus/pci/devices/0000:00:1f.3/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:1f.3/power/runtime_usage:1
/sys/bus/pci/devices/0000:03:00.0/power/async:enabled
/sys/bus/pci/devices/0000:03:00.0/power/control:auto
/sys/bus/pci/devices/0000:03:00.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:03:00.0/power/runtime_active_time:104580
/sys/bus/pci/devices/0000:03:00.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:03:00.0/power/runtime_status:active
/sys/bus/pci/devices/0000:03:00.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:03:00.0/power/runtime_usage:1
/sys/bus/pci/devices/0000:03:00.0/power/wakeup:disabled
Bus 001 Device 004: ID 17ef:4807 Lenovo UVC Camera
Bus 002 Device 002: ID 1199:68a2 Sierra Wireless, Inc. 
Bus 003 Device 002: ID 08ff:2810 AuthenTec, Inc. AES2810
Bus 004 Device 002: ID 0a5c:2145 Broadcom Corp. Bluetooth with Enhanced Data Rate II
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

[-- Attachment #3: powerstate-reverted.txt --]
[-- Type: text/plain, Size: 31221 bytes --]

/sys/bus/usb/devices/1-0:1.0/power/async:enabled
/sys/bus/usb/devices/1-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/1-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/1-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/1-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/1-6:1.0/power/async:enabled
/sys/bus/usb/devices/1-6:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/1-6:1.0/power/runtime_enabled:disabled
/sys/bus/usb/devices/1-6:1.0/power/runtime_status:unsupported
/sys/bus/usb/devices/1-6:1.0/power/runtime_usage:0
/sys/bus/usb/devices/1-6:1.1/power/async:enabled
/sys/bus/usb/devices/1-6:1.1/power/runtime_active_kids:0
/sys/bus/usb/devices/1-6:1.1/power/runtime_enabled:disabled
/sys/bus/usb/devices/1-6:1.1/power/runtime_status:unsupported
/sys/bus/usb/devices/1-6:1.1/power/runtime_usage:0
/sys/bus/usb/devices/1-6/power/active_duration:35204
/sys/bus/usb/devices/1-6/power/async:enabled
/sys/bus/usb/devices/1-6/power/autosuspend:2
/sys/bus/usb/devices/1-6/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/1-6/power/connected_duration:70960
/sys/bus/usb/devices/1-6/power/control:auto
/sys/bus/usb/devices/1-6/power/level:auto
/sys/bus/usb/devices/1-6/power/persist:1
/sys/bus/usb/devices/1-6/power/runtime_active_kids:0
/sys/bus/usb/devices/1-6/power/runtime_active_time:34976
/sys/bus/usb/devices/1-6/power/runtime_enabled:enabled
/sys/bus/usb/devices/1-6/power/runtime_status:suspended
/sys/bus/usb/devices/1-6/power/runtime_suspended_time:35740
/sys/bus/usb/devices/1-6/power/runtime_usage:0
/sys/bus/usb/devices/2-0:1.0/power/async:enabled
/sys/bus/usb/devices/2-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/2-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/2-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/2-4:1.0/power/async:enabled
/sys/bus/usb/devices/2-4:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-4:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/2-4:1.0/power/runtime_usage:0
/sys/bus/usb/devices/2-4:1.19/power/async:enabled
/sys/bus/usb/devices/2-4:1.19/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4:1.19/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-4:1.19/power/runtime_status:suspended
/sys/bus/usb/devices/2-4:1.19/power/runtime_usage:0
/sys/bus/usb/devices/2-4:1.20/power/async:enabled
/sys/bus/usb/devices/2-4:1.20/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4:1.20/power/runtime_enabled:disabled
/sys/bus/usb/devices/2-4:1.20/power/runtime_status:unsupported
/sys/bus/usb/devices/2-4:1.20/power/runtime_usage:0
/sys/bus/usb/devices/2-4:1.2/power/async:enabled
/sys/bus/usb/devices/2-4:1.2/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4:1.2/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-4:1.2/power/runtime_status:suspended
/sys/bus/usb/devices/2-4:1.2/power/runtime_usage:0
/sys/bus/usb/devices/2-4:1.3/power/async:enabled
/sys/bus/usb/devices/2-4:1.3/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4:1.3/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-4:1.3/power/runtime_status:suspended
/sys/bus/usb/devices/2-4:1.3/power/runtime_usage:0
/sys/bus/usb/devices/2-4:1.8/power/async:enabled
/sys/bus/usb/devices/2-4:1.8/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4:1.8/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-4:1.8/power/runtime_status:suspended
/sys/bus/usb/devices/2-4:1.8/power/runtime_usage:0
/sys/bus/usb/devices/2-4/power/active_duration:35056
/sys/bus/usb/devices/2-4/power/async:enabled
/sys/bus/usb/devices/2-4/power/autosuspend:2
/sys/bus/usb/devices/2-4/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/2-4/power/connected_duration:70708
/sys/bus/usb/devices/2-4/power/control:auto
/sys/bus/usb/devices/2-4/power/level:auto
/sys/bus/usb/devices/2-4/power/persist:1
/sys/bus/usb/devices/2-4/power/runtime_active_kids:0
/sys/bus/usb/devices/2-4/power/runtime_active_time:34828
/sys/bus/usb/devices/2-4/power/runtime_enabled:enabled
/sys/bus/usb/devices/2-4/power/runtime_status:suspended
/sys/bus/usb/devices/2-4/power/runtime_suspended_time:35636
/sys/bus/usb/devices/2-4/power/runtime_usage:0
/sys/bus/usb/devices/2-4/power/wakeup:disabled
/sys/bus/usb/devices/3-0:1.0/power/async:enabled
/sys/bus/usb/devices/3-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/3-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/3-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/3-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/3-2:1.0/power/async:enabled
/sys/bus/usb/devices/3-2:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/3-2:1.0/power/runtime_enabled:disabled
/sys/bus/usb/devices/3-2:1.0/power/runtime_status:unsupported
/sys/bus/usb/devices/3-2:1.0/power/runtime_usage:0
/sys/bus/usb/devices/3-2/power/active_duration:34820
/sys/bus/usb/devices/3-2/power/async:enabled
/sys/bus/usb/devices/3-2/power/autosuspend:2
/sys/bus/usb/devices/3-2/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/3-2/power/connected_duration:70324
/sys/bus/usb/devices/3-2/power/control:auto
/sys/bus/usb/devices/3-2/power/level:auto
/sys/bus/usb/devices/3-2/power/persist:1
/sys/bus/usb/devices/3-2/power/runtime_active_kids:0
/sys/bus/usb/devices/3-2/power/runtime_active_time:34580
/sys/bus/usb/devices/3-2/power/runtime_enabled:enabled
/sys/bus/usb/devices/3-2/power/runtime_status:suspended
/sys/bus/usb/devices/3-2/power/runtime_suspended_time:35488
/sys/bus/usb/devices/3-2/power/runtime_usage:0
/sys/bus/usb/devices/3-2/power/wakeup:disabled
/sys/bus/usb/devices/4-0:1.0/power/async:enabled
/sys/bus/usb/devices/4-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/4-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/4-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/4-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/4-2:1.0/power/async:enabled
/sys/bus/usb/devices/4-2:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/4-2:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/4-2:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/4-2:1.0/power/runtime_usage:0
/sys/bus/usb/devices/4-2:1.1/power/async:enabled
/sys/bus/usb/devices/4-2:1.1/power/runtime_active_kids:0
/sys/bus/usb/devices/4-2:1.1/power/runtime_enabled:enabled
/sys/bus/usb/devices/4-2:1.1/power/runtime_status:suspended
/sys/bus/usb/devices/4-2:1.1/power/runtime_usage:0
/sys/bus/usb/devices/4-2:1.2/power/async:enabled
/sys/bus/usb/devices/4-2:1.2/power/runtime_active_kids:0
/sys/bus/usb/devices/4-2:1.2/power/runtime_enabled:disabled
/sys/bus/usb/devices/4-2:1.2/power/runtime_status:unsupported
/sys/bus/usb/devices/4-2:1.2/power/runtime_usage:0
/sys/bus/usb/devices/4-2:1.3/power/async:enabled
/sys/bus/usb/devices/4-2:1.3/power/runtime_active_kids:0
/sys/bus/usb/devices/4-2:1.3/power/runtime_enabled:disabled
/sys/bus/usb/devices/4-2:1.3/power/runtime_status:unsupported
/sys/bus/usb/devices/4-2:1.3/power/runtime_usage:0
/sys/bus/usb/devices/4-2/power/active_duration:25788
/sys/bus/usb/devices/4-2/power/async:enabled
/sys/bus/usb/devices/4-2/power/autosuspend:2
/sys/bus/usb/devices/4-2/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/4-2/power/connected_duration:69908
/sys/bus/usb/devices/4-2/power/control:auto
/sys/bus/usb/devices/4-2/power/level:auto
/sys/bus/usb/devices/4-2/power/persist:1
/sys/bus/usb/devices/4-2/power/runtime_active_kids:0
/sys/bus/usb/devices/4-2/power/runtime_active_time:25612
/sys/bus/usb/devices/4-2/power/runtime_enabled:enabled
/sys/bus/usb/devices/4-2/power/runtime_status:suspended
/sys/bus/usb/devices/4-2/power/runtime_suspended_time:44040
/sys/bus/usb/devices/4-2/power/runtime_usage:0
/sys/bus/usb/devices/4-2/power/wakeup:disabled
/sys/bus/usb/devices/5-0:1.0/power/async:enabled
/sys/bus/usb/devices/5-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/5-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/5-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/5-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/6-0:1.0/power/async:enabled
/sys/bus/usb/devices/6-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/6-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/6-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/6-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/7-0:1.0/power/async:enabled
/sys/bus/usb/devices/7-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/7-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/7-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/7-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/8-0:1.0/power/async:enabled
/sys/bus/usb/devices/8-0:1.0/power/runtime_active_kids:0
/sys/bus/usb/devices/8-0:1.0/power/runtime_enabled:enabled
/sys/bus/usb/devices/8-0:1.0/power/runtime_status:suspended
/sys/bus/usb/devices/8-0:1.0/power/runtime_usage:0
/sys/bus/usb/devices/usb1/power/active_duration:38188
/sys/bus/usb/devices/usb1/power/async:enabled
/sys/bus/usb/devices/usb1/power/autosuspend:2
/sys/bus/usb/devices/usb1/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb1/power/connected_duration:71404
/sys/bus/usb/devices/usb1/power/control:auto
/sys/bus/usb/devices/usb1/power/level:auto
/sys/bus/usb/devices/usb1/power/runtime_active_kids:0
/sys/bus/usb/devices/usb1/power/runtime_active_time:38168
/sys/bus/usb/devices/usb1/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb1/power/runtime_status:suspended
/sys/bus/usb/devices/usb1/power/runtime_suspended_time:33216
/sys/bus/usb/devices/usb1/power/runtime_usage:0
/sys/bus/usb/devices/usb1/power/wakeup:disabled
/sys/bus/usb/devices/usb2/power/active_duration:38168
/sys/bus/usb/devices/usb2/power/async:enabled
/sys/bus/usb/devices/usb2/power/autosuspend:2
/sys/bus/usb/devices/usb2/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb2/power/connected_duration:71384
/sys/bus/usb/devices/usb2/power/control:auto
/sys/bus/usb/devices/usb2/power/level:auto
/sys/bus/usb/devices/usb2/power/runtime_active_kids:0
/sys/bus/usb/devices/usb2/power/runtime_active_time:38152
/sys/bus/usb/devices/usb2/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb2/power/runtime_status:suspended
/sys/bus/usb/devices/usb2/power/runtime_suspended_time:33216
/sys/bus/usb/devices/usb2/power/runtime_usage:0
/sys/bus/usb/devices/usb2/power/wakeup:disabled
/sys/bus/usb/devices/usb3/power/active_duration:38152
/sys/bus/usb/devices/usb3/power/async:enabled
/sys/bus/usb/devices/usb3/power/autosuspend:2
/sys/bus/usb/devices/usb3/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb3/power/connected_duration:71372
/sys/bus/usb/devices/usb3/power/control:auto
/sys/bus/usb/devices/usb3/power/level:auto
/sys/bus/usb/devices/usb3/power/runtime_active_kids:0
/sys/bus/usb/devices/usb3/power/runtime_active_time:38152
/sys/bus/usb/devices/usb3/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb3/power/runtime_status:suspended
/sys/bus/usb/devices/usb3/power/runtime_suspended_time:33220
/sys/bus/usb/devices/usb3/power/runtime_usage:0
/sys/bus/usb/devices/usb3/power/wakeup:disabled
/sys/bus/usb/devices/usb4/power/active_duration:30168
/sys/bus/usb/devices/usb4/power/async:enabled
/sys/bus/usb/devices/usb4/power/autosuspend:2
/sys/bus/usb/devices/usb4/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb4/power/connected_duration:71372
/sys/bus/usb/devices/usb4/power/control:auto
/sys/bus/usb/devices/usb4/power/level:auto
/sys/bus/usb/devices/usb4/power/runtime_active_kids:0
/sys/bus/usb/devices/usb4/power/runtime_active_time:30168
/sys/bus/usb/devices/usb4/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb4/power/runtime_status:suspended
/sys/bus/usb/devices/usb4/power/runtime_suspended_time:41204
/sys/bus/usb/devices/usb4/power/runtime_usage:0
/sys/bus/usb/devices/usb4/power/wakeup:disabled
/sys/bus/usb/devices/usb5/power/active_duration:4156
/sys/bus/usb/devices/usb5/power/async:enabled
/sys/bus/usb/devices/usb5/power/autosuspend:2
/sys/bus/usb/devices/usb5/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb5/power/connected_duration:71360
/sys/bus/usb/devices/usb5/power/control:auto
/sys/bus/usb/devices/usb5/power/level:auto
/sys/bus/usb/devices/usb5/power/runtime_active_kids:0
/sys/bus/usb/devices/usb5/power/runtime_active_time:4156
/sys/bus/usb/devices/usb5/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb5/power/runtime_status:suspended
/sys/bus/usb/devices/usb5/power/runtime_suspended_time:67204
/sys/bus/usb/devices/usb5/power/runtime_usage:0
/sys/bus/usb/devices/usb5/power/wakeup:disabled
/sys/bus/usb/devices/usb6/power/active_duration:2136
/sys/bus/usb/devices/usb6/power/async:enabled
/sys/bus/usb/devices/usb6/power/autosuspend:2
/sys/bus/usb/devices/usb6/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb6/power/connected_duration:71356
/sys/bus/usb/devices/usb6/power/control:auto
/sys/bus/usb/devices/usb6/power/level:auto
/sys/bus/usb/devices/usb6/power/runtime_active_kids:0
/sys/bus/usb/devices/usb6/power/runtime_active_time:2136
/sys/bus/usb/devices/usb6/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb6/power/runtime_status:suspended
/sys/bus/usb/devices/usb6/power/runtime_suspended_time:69224
/sys/bus/usb/devices/usb6/power/runtime_usage:0
/sys/bus/usb/devices/usb6/power/wakeup:disabled
/sys/bus/usb/devices/usb7/power/active_duration:4148
/sys/bus/usb/devices/usb7/power/async:enabled
/sys/bus/usb/devices/usb7/power/autosuspend:2
/sys/bus/usb/devices/usb7/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb7/power/connected_duration:71356
/sys/bus/usb/devices/usb7/power/control:auto
/sys/bus/usb/devices/usb7/power/level:auto
/sys/bus/usb/devices/usb7/power/runtime_active_kids:0
/sys/bus/usb/devices/usb7/power/runtime_active_time:4148
/sys/bus/usb/devices/usb7/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb7/power/runtime_status:suspended
/sys/bus/usb/devices/usb7/power/runtime_suspended_time:67208
/sys/bus/usb/devices/usb7/power/runtime_usage:0
/sys/bus/usb/devices/usb7/power/wakeup:disabled
/sys/bus/usb/devices/usb8/power/active_duration:2128
/sys/bus/usb/devices/usb8/power/async:enabled
/sys/bus/usb/devices/usb8/power/autosuspend:2
/sys/bus/usb/devices/usb8/power/autosuspend_delay_ms:2000
/sys/bus/usb/devices/usb8/power/connected_duration:71352
/sys/bus/usb/devices/usb8/power/control:auto
/sys/bus/usb/devices/usb8/power/level:auto
/sys/bus/usb/devices/usb8/power/runtime_active_kids:0
/sys/bus/usb/devices/usb8/power/runtime_active_time:2128
/sys/bus/usb/devices/usb8/power/runtime_enabled:enabled
/sys/bus/usb/devices/usb8/power/runtime_status:suspended
/sys/bus/usb/devices/usb8/power/runtime_suspended_time:69224
/sys/bus/usb/devices/usb8/power/runtime_usage:0
/sys/bus/usb/devices/usb8/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:00.0/power/async:enabled
/sys/bus/pci/devices/0000:00:00.0/power/control:auto
/sys/bus/pci/devices/0000:00:00.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:00.0/power/runtime_active_time:72052
/sys/bus/pci/devices/0000:00:00.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:00.0/power/runtime_status:active
/sys/bus/pci/devices/0000:00:00.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:00.0/power/runtime_usage:1
/sys/bus/pci/devices/0000:00:02.0/power/async:enabled
/sys/bus/pci/devices/0000:00:02.0/power/control:auto
/sys/bus/pci/devices/0000:00:02.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:02.0/power/runtime_active_time:57832
/sys/bus/pci/devices/0000:00:02.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:02.0/power/runtime_status:active
/sys/bus/pci/devices/0000:00:02.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:02.0/power/runtime_usage:1
/sys/bus/pci/devices/0000:00:02.1/power/async:enabled
/sys/bus/pci/devices/0000:00:02.1/power/control:auto
/sys/bus/pci/devices/0000:00:02.1/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:02.1/power/runtime_active_time:0
/sys/bus/pci/devices/0000:00:02.1/power/runtime_enabled:disabled
/sys/bus/pci/devices/0000:00:02.1/power/runtime_status:unsupported
/sys/bus/pci/devices/0000:00:02.1/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:02.1/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:03.0/power/async:enabled
/sys/bus/pci/devices/0000:00:03.0/power/control:auto
/sys/bus/pci/devices/0000:00:03.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:03.0/power/runtime_active_time:58044
/sys/bus/pci/devices/0000:00:03.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:03.0/power/runtime_status:active
/sys/bus/pci/devices/0000:00:03.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:03.0/power/runtime_usage:1
/sys/bus/pci/devices/0000:00:03.0/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:19.0/power/async:enabled
/sys/bus/pci/devices/0000:00:19.0/power/control:auto
/sys/bus/pci/devices/0000:00:19.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:19.0/power/runtime_active_time:26732
/sys/bus/pci/devices/0000:00:19.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:19.0/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:19.0/power/runtime_suspended_time:44896
/sys/bus/pci/devices/0000:00:19.0/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:19.0/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:1a.0/power/async:enabled
/sys/bus/pci/devices/0000:00:1a.0/power/control:auto
/sys/bus/pci/devices/0000:00:1a.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1a.0/power/runtime_active_time:38152
/sys/bus/pci/devices/0000:00:1a.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1a.0/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1a.0/power/runtime_suspended_time:33224
/sys/bus/pci/devices/0000:00:1a.0/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_last_time_ms:1659
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1a.0/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1a.1/power/async:enabled
/sys/bus/pci/devices/0000:00:1a.1/power/control:auto
/sys/bus/pci/devices/0000:00:1a.1/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1a.1/power/runtime_active_time:35404
/sys/bus/pci/devices/0000:00:1a.1/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1a.1/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1a.1/power/runtime_suspended_time:35972
/sys/bus/pci/devices/0000:00:1a.1/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_last_time_ms:1660
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1a.1/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1a.2/power/async:enabled
/sys/bus/pci/devices/0000:00:1a.2/power/control:auto
/sys/bus/pci/devices/0000:00:1a.2/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1a.2/power/runtime_active_time:35396
/sys/bus/pci/devices/0000:00:1a.2/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1a.2/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1a.2/power/runtime_suspended_time:35968
/sys/bus/pci/devices/0000:00:1a.2/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_last_time_ms:1672
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1a.2/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1a.7/power/async:enabled
/sys/bus/pci/devices/0000:00:1a.7/power/control:auto
/sys/bus/pci/devices/0000:00:1a.7/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1a.7/power/runtime_active_time:38220
/sys/bus/pci/devices/0000:00:1a.7/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1a.7/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1a.7/power/runtime_suspended_time:33196
/sys/bus/pci/devices/0000:00:1a.7/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_last_time_ms:1641
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1a.7/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1b.0/power/async:enabled
/sys/bus/pci/devices/0000:00:1b.0/power/control:auto
/sys/bus/pci/devices/0000:00:1b.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1b.0/power/runtime_active_time:54808
/sys/bus/pci/devices/0000:00:1b.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1b.0/power/runtime_status:active
/sys/bus/pci/devices/0000:00:1b.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:1b.0/power/runtime_usage:1
/sys/bus/pci/devices/0000:00:1b.0/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:1c.0/power/async:enabled
/sys/bus/pci/devices/0000:00:1c.0/power/control:auto
/sys/bus/pci/devices/0000:00:1c.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1c.0/power/runtime_active_time:36172
/sys/bus/pci/devices/0000:00:1c.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1c.0/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1c.0/power/runtime_suspended_time:35940
/sys/bus/pci/devices/0000:00:1c.0/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1c.0/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:1c.1/power/async:enabled
/sys/bus/pci/devices/0000:00:1c.1/power/control:auto
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_active_kids:1
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_active_time:72108
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_status:active
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1c.1/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:1d.0/power/async:enabled
/sys/bus/pci/devices/0000:00:1d.0/power/control:auto
/sys/bus/pci/devices/0000:00:1d.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1d.0/power/runtime_active_time:35432
/sys/bus/pci/devices/0000:00:1d.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1d.0/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1d.0/power/runtime_suspended_time:35932
/sys/bus/pci/devices/0000:00:1d.0/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_last_time_ms:1674
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1d.0/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1d.1/power/async:enabled
/sys/bus/pci/devices/0000:00:1d.1/power/control:auto
/sys/bus/pci/devices/0000:00:1d.1/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1d.1/power/runtime_active_time:35440
/sys/bus/pci/devices/0000:00:1d.1/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1d.1/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1d.1/power/runtime_suspended_time:35924
/sys/bus/pci/devices/0000:00:1d.1/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_last_time_ms:1680
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1d.1/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1d.2/power/async:enabled
/sys/bus/pci/devices/0000:00:1d.2/power/control:auto
/sys/bus/pci/devices/0000:00:1d.2/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1d.2/power/runtime_active_time:35440
/sys/bus/pci/devices/0000:00:1d.2/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1d.2/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1d.2/power/runtime_suspended_time:35916
/sys/bus/pci/devices/0000:00:1d.2/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_last_time_ms:1682
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1d.2/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1d.7/power/async:enabled
/sys/bus/pci/devices/0000:00:1d.7/power/control:auto
/sys/bus/pci/devices/0000:00:1d.7/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1d.7/power/runtime_active_time:38200
/sys/bus/pci/devices/0000:00:1d.7/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1d.7/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1d.7/power/runtime_suspended_time:33196
/sys/bus/pci/devices/0000:00:1d.7/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup:enabled
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_abort_count:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_active:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_active_count:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_count:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_expire_count:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_last_time_ms:1657
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_max_time_ms:0
/sys/bus/pci/devices/0000:00:1d.7/power/wakeup_total_time_ms:0
/sys/bus/pci/devices/0000:00:1e.0/power/async:enabled
/sys/bus/pci/devices/0000:00:1e.0/power/control:auto
/sys/bus/pci/devices/0000:00:1e.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1e.0/power/runtime_active_time:0
/sys/bus/pci/devices/0000:00:1e.0/power/runtime_enabled:disabled
/sys/bus/pci/devices/0000:00:1e.0/power/runtime_status:unsupported
/sys/bus/pci/devices/0000:00:1e.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:1e.0/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1e.0/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:1f.0/power/async:enabled
/sys/bus/pci/devices/0000:00:1f.0/power/control:auto
/sys/bus/pci/devices/0000:00:1f.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1f.0/power/runtime_active_time:0
/sys/bus/pci/devices/0000:00:1f.0/power/runtime_enabled:disabled
/sys/bus/pci/devices/0000:00:1f.0/power/runtime_status:unsupported
/sys/bus/pci/devices/0000:00:1f.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:1f.0/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1f.2/power/async:enabled
/sys/bus/pci/devices/0000:00:1f.2/power/control:auto
/sys/bus/pci/devices/0000:00:1f.2/power/runtime_active_kids:4
/sys/bus/pci/devices/0000:00:1f.2/power/runtime_active_time:71380
/sys/bus/pci/devices/0000:00:1f.2/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1f.2/power/runtime_status:active
/sys/bus/pci/devices/0000:00:1f.2/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:1f.2/power/runtime_usage:1
/sys/bus/pci/devices/0000:00:1f.2/power/wakeup:disabled
/sys/bus/pci/devices/0000:00:1f.3/power/async:enabled
/sys/bus/pci/devices/0000:00:1f.3/power/control:auto
/sys/bus/pci/devices/0000:00:1f.3/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1f.3/power/runtime_active_time:54952
/sys/bus/pci/devices/0000:00:1f.3/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1f.3/power/runtime_status:active
/sys/bus/pci/devices/0000:00:1f.3/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:00:1f.3/power/runtime_usage:1
/sys/bus/pci/devices/0000:03:00.0/power/async:enabled
/sys/bus/pci/devices/0000:03:00.0/power/control:auto
/sys/bus/pci/devices/0000:03:00.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:03:00.0/power/runtime_active_time:56836
/sys/bus/pci/devices/0000:03:00.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:03:00.0/power/runtime_status:active
/sys/bus/pci/devices/0000:03:00.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:03:00.0/power/runtime_usage:1
/sys/bus/pci/devices/0000:03:00.0/power/wakeup:disabled
Bus 001 Device 004: ID 17ef:4807 Lenovo UVC Camera
Bus 002 Device 002: ID 1199:68a2 Sierra Wireless, Inc. 
Bus 003 Device 002: ID 08ff:2810 AuthenTec, Inc. AES2810
Bus 004 Device 002: ID 0a5c:2145 Broadcom Corp. Bluetooth with Enhanced Data Rate II
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 004: ID 17ef:4807 Lenovo UVC Camera
Bus 002 Device 003: ID 0fce:2138 Sony Ericsson Mobile Communications AB Xperia X10 mini pro (Debug)
Bus 002 Device 002: ID 1199:68a2 Sierra Wireless, Inc. 
Bus 003 Device 002: ID 08ff:2810 AuthenTec, Inc. AES2810
Bus 004 Device 002: ID 0a5c:2145 Broadcom Corp. Bluetooth with Enhanced Data Rate II
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-25  9:58       ` Bjørn Mork
@ 2012-07-25 13:30         ` huang ying
  2012-07-25 13:58           ` Bjørn Mork
  2012-07-25 19:51           ` [PATCH] PCI / PM: Fix messages printed by acpi_pci_set_power_state() Rafael J. Wysocki
  0 siblings, 2 replies; 89+ messages in thread
From: huang ying @ 2012-07-25 13:30 UTC (permalink / raw)
  To: Bjørn Mork
  Cc: Huang Ying, Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

Hi, Bjorn,

Thank you very much for your detailed information.

On Wed, Jul 25, 2012 at 5:58 PM, Bjørn Mork <bjorn@mork.no> wrote:
> Huang Ying <ying.huang@intel.com> writes:
>> On Wed, 2012-07-25 at 06:08 +0200, Bjørn Mork wrote:
>>> Enabling autosuspend for USB causes hotplug failure in the current
>>> linux-next. Newly plugged devices are not detected at all until the
>>> port/controller is manually powered on by writing "on" to power/control.
>>> Testing is pretty simple:
>>>
>>>   1) for f in /sys/bus/usb/devices/*/power/control; do echo auto > $f; done
>>
>> Have you done:
>>
>> for f in /sys/bus/pci/devices/*/power/confol; do echo auto > $f; done
>>
>> ?
>>
>> If not, the pci device will not be suspended at all.
>
> Yes, sorry for missing that.  I had it automatically enabled.  Yes,
> autosuspend for the PCI device and all child devices must be enabled for
> the device to be suspended at all, of course.
>
>>>   2) wait for the controllers to suspend
>>>   3) plugin a new USB device
>>
>> After plugin the new USB device, is there anything in dmesg?
>
> No. Absolutely nothing, so the USB devices is not enumerated.  Another
> indication of the same:  Plugging a device like an Android phone, which
> normally detects being connected to a host and presents a device type
> menu to the user, results in the charging LED lighting up but no menu.
>
>
> Trying to show the sequence of events:
>
> 1)  the controllers are suspended:
>
> Jul 25 11:27:12 nemi kernel: [   38.962792] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D2
> Jul 25 11:27:12 nemi kernel: [   39.006718] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D2
> Jul 25 11:27:15 nemi kernel: [   41.808471] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D2
> Jul 25 11:27:15 nemi kernel: [   41.824123] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D2
> Jul 25 11:27:15 nemi kernel: [   41.824194] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D2

Here uhci controller is put into D2

[snip]
>
> Doing the same with commit 448bd857d reverted:
>
>
> 1)  the controllers are suspended (to state D3? instead of D2?):
>
> Jul 25 11:34:01 nemi kernel: [   37.064955] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D3
> Jul 25 11:34:01 nemi kernel: [   37.106586] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D3
> Jul 25 11:34:04 nemi kernel: [   39.808329] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D3
> Jul 25 11:34:04 nemi kernel: [   39.840054] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D3
> Jul 25 11:34:04 nemi kernel: [   39.840068] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D3

With commit reverted, the uhci_controller is put into D3 (ACPI D3cold).

And the uhci controller on your system may not work properly under D2
state, while OK in D3 state, and the commit will make uhci controller
choose D2 instead of D3.

Please try following command line before testing.

for f in /sys/bus/pci/devices/*/d3cold_allowed; do echo 1 > $f; done

And please provide the output of the following command line.

acpidump

Best Regards,
Huang Ying

[snip]

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-25 13:30         ` huang ying
@ 2012-07-25 13:58           ` Bjørn Mork
  2012-07-25 18:56             ` Rafael J. Wysocki
                               ` (2 more replies)
  2012-07-25 19:51           ` [PATCH] PCI / PM: Fix messages printed by acpi_pci_set_power_state() Rafael J. Wysocki
  1 sibling, 3 replies; 89+ messages in thread
From: Bjørn Mork @ 2012-07-25 13:58 UTC (permalink / raw)
  To: huang ying
  Cc: Huang Ying, Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

huang ying <huang.ying.caritas@gmail.com> writes:

> Hi, Bjorn,
>
> Thank you very much for your detailed information.
>
> On Wed, Jul 25, 2012 at 5:58 PM, Bjørn Mork <bjorn@mork.no> wrote:
>> Huang Ying <ying.huang@intel.com> writes:
>>> On Wed, 2012-07-25 at 06:08 +0200, Bjørn Mork wrote:
>>>> Enabling autosuspend for USB causes hotplug failure in the current
>>>> linux-next. Newly plugged devices are not detected at all until the
>>>> port/controller is manually powered on by writing "on" to power/control.
>>>> Testing is pretty simple:
>>>>
>>>>   1) for f in /sys/bus/usb/devices/*/power/control; do echo auto > $f; done
>>>
>>> Have you done:
>>>
>>> for f in /sys/bus/pci/devices/*/power/confol; do echo auto > $f; done
>>>
>>> ?
>>>
>>> If not, the pci device will not be suspended at all.
>>
>> Yes, sorry for missing that.  I had it automatically enabled.  Yes,
>> autosuspend for the PCI device and all child devices must be enabled for
>> the device to be suspended at all, of course.
>>
>>>>   2) wait for the controllers to suspend
>>>>   3) plugin a new USB device
>>>
>>> After plugin the new USB device, is there anything in dmesg?
>>
>> No. Absolutely nothing, so the USB devices is not enumerated.  Another
>> indication of the same:  Plugging a device like an Android phone, which
>> normally detects being connected to a host and presents a device type
>> menu to the user, results in the charging LED lighting up but no menu.
>>
>>
>> Trying to show the sequence of events:
>>
>> 1)  the controllers are suspended:
>>
>> Jul 25 11:27:12 nemi kernel: [   38.962792] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D2
>> Jul 25 11:27:12 nemi kernel: [   39.006718] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D2
>> Jul 25 11:27:15 nemi kernel: [   41.808471] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D2
>> Jul 25 11:27:15 nemi kernel: [   41.824123] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D2
>> Jul 25 11:27:15 nemi kernel: [   41.824194] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D2
>
> Here uhci controller is put into D2
>
> [snip]
>>
>> Doing the same with commit 448bd857d reverted:
>>
>>
>> 1)  the controllers are suspended (to state D3? instead of D2?):
>>
>> Jul 25 11:34:01 nemi kernel: [   37.064955] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D3
>> Jul 25 11:34:01 nemi kernel: [   37.106586] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D3
>> Jul 25 11:34:04 nemi kernel: [   39.808329] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D3
>> Jul 25 11:34:04 nemi kernel: [   39.840054] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D3
>> Jul 25 11:34:04 nemi kernel: [   39.840068] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D3
>
> With commit reverted, the uhci_controller is put into D3 (ACPI D3cold).
>
> And the uhci controller on your system may not work properly under D2
> state, while OK in D3 state, and the commit will make uhci controller
> choose D2 instead of D3.
>
> Please try following command line before testing.
>
> for f in /sys/bus/pci/devices/*/d3cold_allowed; do echo 1 > $f; done

That made it work.  The USB controllers ended up in D4 though:

Jul 25 15:52:33 nemi kernel: [  152.753280] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D0
Jul 25 15:52:33 nemi kernel: [  152.753453] uhci_hcd 0000:00:1a.0: setting latency timer to 64
Jul 25 15:52:33 nemi kernel: [  152.753619] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D4
Jul 25 15:52:33 nemi kernel: [  152.753875] uhci_hcd 0000:00:1a.1: setting latency timer to 64
Jul 25 15:52:33 nemi kernel: [  152.754153] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D0
Jul 25 15:52:33 nemi kernel: [  152.754279] uhci_hcd 0000:00:1a.2: setting latency timer to 64
Jul 25 15:52:33 nemi kernel: [  152.754432] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D4
Jul 25 15:52:33 nemi kernel: [  152.754668] ehci_hcd 0000:00:1a.7: setting latency timer to 64
Jul 25 15:52:33 nemi kernel: [  152.768089] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D4
Jul 25 15:52:33 nemi kernel: [  152.768573] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D0
Jul 25 15:52:33 nemi kernel: [  152.768726] uhci_hcd 0000:00:1d.0: setting latency timer to 64
Jul 25 15:52:33 nemi kernel: [  152.768891] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D4
Jul 25 15:52:33 nemi kernel: [  152.769144] uhci_hcd 0000:00:1d.1: setting latency timer to 64
Jul 25 15:52:33 nemi kernel: [  152.769530] uhci_hcd 0000:00:1d.2: setting latency timer to 64
Jul 25 15:52:33 nemi kernel: [  152.769902] ehci_hcd 0000:00:1d.7: setting latency timer to 64
Jul 25 15:52:33 nemi kernel: [  152.784189] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D4


was that expected?  Anyway, waking up the controller from this state by
plugging a USB device works, so it's a perfectly OK workaround.  Is this
something that could/should be implemented as a system specific quirk,
or is the problem more generic?  Even if it is a ACPI implementation
issue I would expect it to be common to a number of Lenovo laptops of
the same generation as mine (~2008).

> And please provide the output of the following command line.
>
> acpidump

Attached.  Thanks a lot for all your help debugging this issue.


Bjørn


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-25 13:58           ` Bjørn Mork
@ 2012-07-25 18:56             ` Rafael J. Wysocki
  2012-07-25 20:02             ` Rafael J. Wysocki
  2012-07-26  8:54             ` Huang Ying
  2 siblings, 0 replies; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-07-25 18:56 UTC (permalink / raw)
  To: Bjørn Mork
  Cc: huang ying, Huang Ying, Zheng Yan, Bjorn Helgaas, linux-pci,
	linux-usb, Alan Stern

On Wednesday, July 25, 2012, Bjørn Mork wrote:
> huang ying <huang.ying.caritas@gmail.com> writes:
> 
> > Hi, Bjorn,
> >
> > Thank you very much for your detailed information.
> >
> > On Wed, Jul 25, 2012 at 5:58 PM, Bjørn Mork <bjorn@mork.no> wrote:
> >> Huang Ying <ying.huang@intel.com> writes:
> >>> On Wed, 2012-07-25 at 06:08 +0200, Bjørn Mork wrote:
> >>>> Enabling autosuspend for USB causes hotplug failure in the current
> >>>> linux-next. Newly plugged devices are not detected at all until the
> >>>> port/controller is manually powered on by writing "on" to power/control.
> >>>> Testing is pretty simple:
> >>>>
> >>>>   1) for f in /sys/bus/usb/devices/*/power/control; do echo auto > $f; done
> >>>
> >>> Have you done:
> >>>
> >>> for f in /sys/bus/pci/devices/*/power/confol; do echo auto > $f; done
> >>>
> >>> ?
> >>>
> >>> If not, the pci device will not be suspended at all.
> >>
> >> Yes, sorry for missing that.  I had it automatically enabled.  Yes,
> >> autosuspend for the PCI device and all child devices must be enabled for
> >> the device to be suspended at all, of course.
> >>
> >>>>   2) wait for the controllers to suspend
> >>>>   3) plugin a new USB device
> >>>
> >>> After plugin the new USB device, is there anything in dmesg?
> >>
> >> No. Absolutely nothing, so the USB devices is not enumerated.  Another
> >> indication of the same:  Plugging a device like an Android phone, which
> >> normally detects being connected to a host and presents a device type
> >> menu to the user, results in the charging LED lighting up but no menu.
> >>
> >>
> >> Trying to show the sequence of events:
> >>
> >> 1)  the controllers are suspended:
> >>
> >> Jul 25 11:27:12 nemi kernel: [   38.962792] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D2
> >> Jul 25 11:27:12 nemi kernel: [   39.006718] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D2
> >> Jul 25 11:27:15 nemi kernel: [   41.808471] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D2
> >> Jul 25 11:27:15 nemi kernel: [   41.824123] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D2
> >> Jul 25 11:27:15 nemi kernel: [   41.824194] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D2
> >
> > Here uhci controller is put into D2
> >
> > [snip]
> >>
> >> Doing the same with commit 448bd857d reverted:
> >>
> >>
> >> 1)  the controllers are suspended (to state D3? instead of D2?):
> >>
> >> Jul 25 11:34:01 nemi kernel: [   37.064955] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D3
> >> Jul 25 11:34:01 nemi kernel: [   37.106586] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D3
> >> Jul 25 11:34:04 nemi kernel: [   39.808329] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D3
> >> Jul 25 11:34:04 nemi kernel: [   39.840054] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D3
> >> Jul 25 11:34:04 nemi kernel: [   39.840068] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D3
> >
> > With commit reverted, the uhci_controller is put into D3 (ACPI D3cold).
> >
> > And the uhci controller on your system may not work properly under D2
> > state, while OK in D3 state, and the commit will make uhci controller
> > choose D2 instead of D3.
> >
> > Please try following command line before testing.
> >
> > for f in /sys/bus/pci/devices/*/d3cold_allowed; do echo 1 > $f; done
> 
> That made it work.  The USB controllers ended up in D4 though:

The D4 really is D3_cold.

I'll send a patch to fix the state labeling.

Thanks,
Rafael


> Jul 25 15:52:33 nemi kernel: [  152.753280] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D0
> Jul 25 15:52:33 nemi kernel: [  152.753453] uhci_hcd 0000:00:1a.0: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.753619] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D4
> Jul 25 15:52:33 nemi kernel: [  152.753875] uhci_hcd 0000:00:1a.1: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.754153] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D0
> Jul 25 15:52:33 nemi kernel: [  152.754279] uhci_hcd 0000:00:1a.2: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.754432] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D4
> Jul 25 15:52:33 nemi kernel: [  152.754668] ehci_hcd 0000:00:1a.7: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.768089] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D4
> Jul 25 15:52:33 nemi kernel: [  152.768573] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D0
> Jul 25 15:52:33 nemi kernel: [  152.768726] uhci_hcd 0000:00:1d.0: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.768891] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D4
> Jul 25 15:52:33 nemi kernel: [  152.769144] uhci_hcd 0000:00:1d.1: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.769530] uhci_hcd 0000:00:1d.2: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.769902] ehci_hcd 0000:00:1d.7: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.784189] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D4
> 
> 
> was that expected?  Anyway, waking up the controller from this state by
> plugging a USB device works, so it's a perfectly OK workaround.  Is this
> something that could/should be implemented as a system specific quirk,
> or is the problem more generic?  Even if it is a ACPI implementation
> issue I would expect it to be common to a number of Lenovo laptops of
> the same generation as mine (~2008).
> 
> > And please provide the output of the following command line.
> >
> > acpidump
> 
> Attached.  Thanks a lot for all your help debugging this issue.
> 
> 
> Bjørn
> 
> 
> 


^ permalink raw reply	[flat|nested] 89+ messages in thread

* [PATCH] PCI / PM: Fix messages printed by acpi_pci_set_power_state()
  2012-07-25 13:30         ` huang ying
  2012-07-25 13:58           ` Bjørn Mork
@ 2012-07-25 19:51           ` Rafael J. Wysocki
  2012-07-25 20:02             ` Alan Stern
  1 sibling, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-07-25 19:51 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: huang ying, Bjørn Mork, Huang Ying, Zheng Yan, linux-pci,
	linux-usb, Alan Stern, Linux PM list


If a PCI device is put into D3_cold by acpi_bus_set_power(),
the message printed by acpi_pci_set_power_state() says that its
power state has been changed to D4, which doesn't make sense.
In turn, if the device is put into D3_hot, the message says just
"D3" without specifying which variant of D3 it is.

Fix that by using an array of state names corresponding to the
PCI device power states instead of building the state name from
the numeric value corresponding to the given state directly.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
---
 drivers/pci/pci-acpi.c |   14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

Index: linux/drivers/pci/pci-acpi.c
===================================================================
--- linux.orig/drivers/pci/pci-acpi.c
+++ linux/drivers/pci/pci-acpi.c
@@ -265,9 +265,17 @@ static int acpi_pci_set_power_state(stru
 		error = acpi_bus_set_power(handle, state_conv[state]);
 	}
 
-	if (!error)
-		dev_printk(KERN_INFO, &dev->dev,
-				"power state changed by ACPI to D%d\n", state);
+	if (!error) {
+		static const char *state_name[] = {
+			[PCI_D0] = "D0",
+			[PCI_D1] = "D1",
+			[PCI_D2] = "D2",
+			[PCI_D3hot] = "D3hot",
+			[PCI_D3cold] = "D3cold"
+		};
+		dev_info(&dev->dev, "power state changed by ACPI to %s\n",
+			 state_name[state]);
+	}
 
 	return error;
 }

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-25 13:58           ` Bjørn Mork
  2012-07-25 18:56             ` Rafael J. Wysocki
@ 2012-07-25 20:02             ` Rafael J. Wysocki
  2012-07-25 22:36               ` Bjørn Mork
  2012-07-26  2:38               ` Huang Ying
  2012-07-26  8:54             ` Huang Ying
  2 siblings, 2 replies; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-07-25 20:02 UTC (permalink / raw)
  To: Bjørn Mork, Huang Ying
  Cc: huang ying, Zheng Yan, Bjorn Helgaas, linux-pci, linux-usb, Alan Stern

On Wednesday, July 25, 2012, Bjørn Mork wrote:
> huang ying <huang.ying.caritas@gmail.com> writes:
> 
> > Hi, Bjorn,
> >
> > Thank you very much for your detailed information.
> >
> > On Wed, Jul 25, 2012 at 5:58 PM, Bjørn Mork <bjorn@mork.no> wrote:
> >> Huang Ying <ying.huang@intel.com> writes:
> >>> On Wed, 2012-07-25 at 06:08 +0200, Bjørn Mork wrote:
> >>>> Enabling autosuspend for USB causes hotplug failure in the current
> >>>> linux-next. Newly plugged devices are not detected at all until the
> >>>> port/controller is manually powered on by writing "on" to power/control.
> >>>> Testing is pretty simple:
> >>>>
> >>>>   1) for f in /sys/bus/usb/devices/*/power/control; do echo auto > $f; done
> >>>
> >>> Have you done:
> >>>
> >>> for f in /sys/bus/pci/devices/*/power/confol; do echo auto > $f; done
> >>>
> >>> ?
> >>>
> >>> If not, the pci device will not be suspended at all.
> >>
> >> Yes, sorry for missing that.  I had it automatically enabled.  Yes,
> >> autosuspend for the PCI device and all child devices must be enabled for
> >> the device to be suspended at all, of course.
> >>
> >>>>   2) wait for the controllers to suspend
> >>>>   3) plugin a new USB device
> >>>
> >>> After plugin the new USB device, is there anything in dmesg?
> >>
> >> No. Absolutely nothing, so the USB devices is not enumerated.  Another
> >> indication of the same:  Plugging a device like an Android phone, which
> >> normally detects being connected to a host and presents a device type
> >> menu to the user, results in the charging LED lighting up but no menu.
> >>
> >>
> >> Trying to show the sequence of events:
> >>
> >> 1)  the controllers are suspended:
> >>
> >> Jul 25 11:27:12 nemi kernel: [   38.962792] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D2
> >> Jul 25 11:27:12 nemi kernel: [   39.006718] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D2
> >> Jul 25 11:27:15 nemi kernel: [   41.808471] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D2
> >> Jul 25 11:27:15 nemi kernel: [   41.824123] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D2
> >> Jul 25 11:27:15 nemi kernel: [   41.824194] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D2
> >
> > Here uhci controller is put into D2
> >
> > [snip]
> >>
> >> Doing the same with commit 448bd857d reverted:
> >>
> >>
> >> 1)  the controllers are suspended (to state D3? instead of D2?):
> >>
> >> Jul 25 11:34:01 nemi kernel: [   37.064955] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D3
> >> Jul 25 11:34:01 nemi kernel: [   37.106586] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D3
> >> Jul 25 11:34:04 nemi kernel: [   39.808329] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D3
> >> Jul 25 11:34:04 nemi kernel: [   39.840054] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D3
> >> Jul 25 11:34:04 nemi kernel: [   39.840068] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D3
> >
> > With commit reverted, the uhci_controller is put into D3 (ACPI D3cold).
> >
> > And the uhci controller on your system may not work properly under D2
> > state, while OK in D3 state, and the commit will make uhci controller
> > choose D2 instead of D3.
> >
> > Please try following command line before testing.
> >
> > for f in /sys/bus/pci/devices/*/d3cold_allowed; do echo 1 > $f; done
> 
> That made it work.  The USB controllers ended up in D4 though:
> 
> Jul 25 15:52:33 nemi kernel: [  152.753280] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D0
> Jul 25 15:52:33 nemi kernel: [  152.753453] uhci_hcd 0000:00:1a.0: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.753619] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D4
> Jul 25 15:52:33 nemi kernel: [  152.753875] uhci_hcd 0000:00:1a.1: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.754153] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D0
> Jul 25 15:52:33 nemi kernel: [  152.754279] uhci_hcd 0000:00:1a.2: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.754432] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D4
> Jul 25 15:52:33 nemi kernel: [  152.754668] ehci_hcd 0000:00:1a.7: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.768089] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D4
> Jul 25 15:52:33 nemi kernel: [  152.768573] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D0
> Jul 25 15:52:33 nemi kernel: [  152.768726] uhci_hcd 0000:00:1d.0: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.768891] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D4
> Jul 25 15:52:33 nemi kernel: [  152.769144] uhci_hcd 0000:00:1d.1: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.769530] uhci_hcd 0000:00:1d.2: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.769902] ehci_hcd 0000:00:1d.7: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.784189] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D4
> 
> 
> was that expected?  Anyway, waking up the controller from this state by
> plugging a USB device works, so it's a perfectly OK workaround.  Is this
> something that could/should be implemented as a system specific quirk,
> or is the problem more generic?  Even if it is a ACPI implementation
> issue I would expect it to be common to a number of Lenovo laptops of
> the same generation as mine (~2008).
> 
> > And please provide the output of the following command line.
> >
> > acpidump
> 
> Attached.  Thanks a lot for all your help debugging this issue.

This is not really necessary.  The problem is the change of behavior I was
worried about some time ago.

Namely, the device in question apparently doesn't support ACPI D3_hot,
so acpi_pm_device_sleep_state() returns D2 as the target state, where it
would return D3(cold) before the commit in question.

To fix the regression, we'd need to reverse the current initial value of
the d3cold_allowed flag so that it is set to "true" by default and only
set to "false" for devices where we suspect there may be problems with
D3_cold (like PCI Express ports).

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: [PATCH] PCI / PM: Fix messages printed by acpi_pci_set_power_state()
  2012-07-25 19:51           ` [PATCH] PCI / PM: Fix messages printed by acpi_pci_set_power_state() Rafael J. Wysocki
@ 2012-07-25 20:02             ` Alan Stern
  2012-07-25 20:48               ` [PATCH][update] " Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-07-25 20:02 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Bjorn Helgaas, huang ying, Bjørn Mork, Huang Ying,
	Zheng Yan, linux-pci, linux-usb, Linux PM list

On Wed, 25 Jul 2012, Rafael J. Wysocki wrote:

> 
> If a PCI device is put into D3_cold by acpi_bus_set_power(),
> the message printed by acpi_pci_set_power_state() says that its
> power state has been changed to D4, which doesn't make sense.
> In turn, if the device is put into D3_hot, the message says just
> "D3" without specifying which variant of D3 it is.
> 
> Fix that by using an array of state names corresponding to the
> PCI device power states instead of building the state name from
> the numeric value corresponding to the given state directly.
> 
> Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
> ---
>  drivers/pci/pci-acpi.c |   14 +++++++++++---
>  1 file changed, 11 insertions(+), 3 deletions(-)
> 
> Index: linux/drivers/pci/pci-acpi.c
> ===================================================================
> --- linux.orig/drivers/pci/pci-acpi.c
> +++ linux/drivers/pci/pci-acpi.c
> @@ -265,9 +265,17 @@ static int acpi_pci_set_power_state(stru
>  		error = acpi_bus_set_power(handle, state_conv[state]);
>  	}
>  
> -	if (!error)
> -		dev_printk(KERN_INFO, &dev->dev,
> -				"power state changed by ACPI to D%d\n", state);
> +	if (!error) {
> +		static const char *state_name[] = {
> +			[PCI_D0] = "D0",
> +			[PCI_D1] = "D1",
> +			[PCI_D2] = "D2",
> +			[PCI_D3hot] = "D3hot",
> +			[PCI_D3cold] = "D3cold"
> +		};
> +		dev_info(&dev->dev, "power state changed by ACPI to %s\n",
> +			 state_name[state]);
> +	}

How about using the pci_power_name macro defined in
include/linux/pci.h instead?

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* [PATCH][update] PCI / PM: Fix messages printed by acpi_pci_set_power_state()
  2012-07-25 20:02             ` Alan Stern
@ 2012-07-25 20:48               ` Rafael J. Wysocki
  0 siblings, 0 replies; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-07-25 20:48 UTC (permalink / raw)
  To: Alan Stern, Bjorn Helgaas
  Cc: huang ying, Bjørn Mork, Huang Ying, Zheng Yan, linux-pci,
	linux-usb, Linux PM list

On Wednesday, July 25, 2012, Alan Stern wrote:
> On Wed, 25 Jul 2012, Rafael J. Wysocki wrote:
> 
> > 
> > If a PCI device is put into D3_cold by acpi_bus_set_power(),
> > the message printed by acpi_pci_set_power_state() says that its
> > power state has been changed to D4, which doesn't make sense.
> > In turn, if the device is put into D3_hot, the message says just
> > "D3" without specifying which variant of D3 it is.
> > 
> > Fix that by using an array of state names corresponding to the
> > PCI device power states instead of building the state name from
> > the numeric value corresponding to the given state directly.
> > 
> > Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
> > ---
> >  drivers/pci/pci-acpi.c |   14 +++++++++++---
> >  1 file changed, 11 insertions(+), 3 deletions(-)
> > 
> > Index: linux/drivers/pci/pci-acpi.c
> > ===================================================================
> > --- linux.orig/drivers/pci/pci-acpi.c
> > +++ linux/drivers/pci/pci-acpi.c
> > @@ -265,9 +265,17 @@ static int acpi_pci_set_power_state(stru
> >  		error = acpi_bus_set_power(handle, state_conv[state]);
> >  	}
> >  
> > -	if (!error)
> > -		dev_printk(KERN_INFO, &dev->dev,
> > -				"power state changed by ACPI to D%d\n", state);
> > +	if (!error) {
> > +		static const char *state_name[] = {
> > +			[PCI_D0] = "D0",
> > +			[PCI_D1] = "D1",
> > +			[PCI_D2] = "D2",
> > +			[PCI_D3hot] = "D3hot",
> > +			[PCI_D3cold] = "D3cold"
> > +		};
> > +		dev_info(&dev->dev, "power state changed by ACPI to %s\n",
> > +			 state_name[state]);
> > +	}
> 
> How about using the pci_power_name macro defined in
> include/linux/pci.h instead?

That's even better, I forgot that the macro was there. :-)

Updated patch follows.

Thanks,
Rafael

---
From: Rafael J. Wysocki <rjw@sisk.pl>
Subject: PCI / PM: Fix messages printed by acpi_pci_set_power_state()

If a PCI device is put into D3_cold by acpi_bus_set_power(),
the message printed by acpi_pci_set_power_state() says that its
power state has been changed to D4, which doesn't make sense.
In turn, if the device is put into D3_hot, the message simply
says "D3" without specifying the variant of the D3 state.

Fix this by using the pci_power_name() macro for printing the state
name instead of building it from the numeric value corresponding to
the given state directly.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
---
 drivers/pci/pci-acpi.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Index: linux/drivers/pci/pci-acpi.c
===================================================================
--- linux.orig/drivers/pci/pci-acpi.c
+++ linux/drivers/pci/pci-acpi.c
@@ -266,8 +266,8 @@ static int acpi_pci_set_power_state(stru
 	}
 
 	if (!error)
-		dev_printk(KERN_INFO, &dev->dev,
-				"power state changed by ACPI to D%d\n", state);
+		dev_info(&dev->dev, "power state changed by ACPI to %s\n",
+			 pci_power_name(state));
 
 	return error;
 }

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-25 20:02             ` Rafael J. Wysocki
@ 2012-07-25 22:36               ` Bjørn Mork
  2012-07-26  2:38                 ` Huang Ying
  2012-07-26  2:38               ` Huang Ying
  1 sibling, 1 reply; 89+ messages in thread
From: Bjørn Mork @ 2012-07-25 22:36 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Huang Ying, huang ying, Zheng Yan, Bjorn Helgaas, linux-pci,
	linux-usb, Alan Stern

"Rafael J. Wysocki" <rjw@sisk.pl> writes:

> This is not really necessary.  The problem is the change of behavior I was
> worried about some time ago.
>
> Namely, the device in question apparently doesn't support ACPI D3_hot,
> so acpi_pm_device_sleep_state() returns D2 as the target state, where it
> would return D3(cold) before the commit in question.
>
> To fix the regression, we'd need to reverse the current initial value of
> the d3cold_allowed flag so that it is set to "true" by default and only
> set to "false" for devices where we suspect there may be problems with
> D3_cold (like PCI Express ports).

Good.  May I also request a few lines describing d3cold_allowed in e.g
Documentation/ABI/testing/sysfs-bus-pci?  I must admit that I thought
that was the policy for any new sysfs file, but it seems that is only
in the USB subsystem?

Anyway, I believe it ought to be a requirement.  If we are meant to
tweak some knob, then we must known how and when to do so.  If not, then
why add it in the first place?


Bjørn

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-25 20:02             ` Rafael J. Wysocki
  2012-07-25 22:36               ` Bjørn Mork
@ 2012-07-26  2:38               ` Huang Ying
  1 sibling, 0 replies; 89+ messages in thread
From: Huang Ying @ 2012-07-26  2:38 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Bjørn Mork, huang ying, Zheng Yan, Bjorn Helgaas, linux-pci,
	linux-usb, Alan Stern

Hi, Rafael,

On Wed, 2012-07-25 at 22:02 +0200, Rafael J. Wysocki wrote:
> On Wednesday, July 25, 2012, Bjørn Mork wrote:
> > huang ying <huang.ying.caritas@gmail.com> writes:
> > 
> > > Hi, Bjorn,
> > >
> > > Thank you very much for your detailed information.
> > >
> > > On Wed, Jul 25, 2012 at 5:58 PM, Bjørn Mork <bjorn@mork.no> wrote:
> > >> Huang Ying <ying.huang@intel.com> writes:
> > >>> On Wed, 2012-07-25 at 06:08 +0200, Bjørn Mork wrote:
> > >>>> Enabling autosuspend for USB causes hotplug failure in the current
> > >>>> linux-next. Newly plugged devices are not detected at all until the
> > >>>> port/controller is manually powered on by writing "on" to power/control.
> > >>>> Testing is pretty simple:
> > >>>>
> > >>>>   1) for f in /sys/bus/usb/devices/*/power/control; do echo auto > $f; done
> > >>>
> > >>> Have you done:
> > >>>
> > >>> for f in /sys/bus/pci/devices/*/power/confol; do echo auto > $f; done
> > >>>
> > >>> ?
> > >>>
> > >>> If not, the pci device will not be suspended at all.
> > >>
> > >> Yes, sorry for missing that.  I had it automatically enabled.  Yes,
> > >> autosuspend for the PCI device and all child devices must be enabled for
> > >> the device to be suspended at all, of course.
> > >>
> > >>>>   2) wait for the controllers to suspend
> > >>>>   3) plugin a new USB device
> > >>>
> > >>> After plugin the new USB device, is there anything in dmesg?
> > >>
> > >> No. Absolutely nothing, so the USB devices is not enumerated.  Another
> > >> indication of the same:  Plugging a device like an Android phone, which
> > >> normally detects being connected to a host and presents a device type
> > >> menu to the user, results in the charging LED lighting up but no menu.
> > >>
> > >>
> > >> Trying to show the sequence of events:
> > >>
> > >> 1)  the controllers are suspended:
> > >>
> > >> Jul 25 11:27:12 nemi kernel: [   38.962792] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D2
> > >> Jul 25 11:27:12 nemi kernel: [   39.006718] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D2
> > >> Jul 25 11:27:15 nemi kernel: [   41.808471] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D2
> > >> Jul 25 11:27:15 nemi kernel: [   41.824123] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D2
> > >> Jul 25 11:27:15 nemi kernel: [   41.824194] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D2
> > >
> > > Here uhci controller is put into D2
> > >
> > > [snip]
> > >>
> > >> Doing the same with commit 448bd857d reverted:
> > >>
> > >>
> > >> 1)  the controllers are suspended (to state D3? instead of D2?):
> > >>
> > >> Jul 25 11:34:01 nemi kernel: [   37.064955] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D3
> > >> Jul 25 11:34:01 nemi kernel: [   37.106586] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D3
> > >> Jul 25 11:34:04 nemi kernel: [   39.808329] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D3
> > >> Jul 25 11:34:04 nemi kernel: [   39.840054] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D3
> > >> Jul 25 11:34:04 nemi kernel: [   39.840068] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D3
> > >
> > > With commit reverted, the uhci_controller is put into D3 (ACPI D3cold).
> > >
> > > And the uhci controller on your system may not work properly under D2
> > > state, while OK in D3 state, and the commit will make uhci controller
> > > choose D2 instead of D3.
> > >
> > > Please try following command line before testing.
> > >
> > > for f in /sys/bus/pci/devices/*/d3cold_allowed; do echo 1 > $f; done
> > 
> > That made it work.  The USB controllers ended up in D4 though:
> > 
> > Jul 25 15:52:33 nemi kernel: [  152.753280] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D0
> > Jul 25 15:52:33 nemi kernel: [  152.753453] uhci_hcd 0000:00:1a.0: setting latency timer to 64
> > Jul 25 15:52:33 nemi kernel: [  152.753619] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D4
> > Jul 25 15:52:33 nemi kernel: [  152.753875] uhci_hcd 0000:00:1a.1: setting latency timer to 64
> > Jul 25 15:52:33 nemi kernel: [  152.754153] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D0
> > Jul 25 15:52:33 nemi kernel: [  152.754279] uhci_hcd 0000:00:1a.2: setting latency timer to 64
> > Jul 25 15:52:33 nemi kernel: [  152.754432] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D4
> > Jul 25 15:52:33 nemi kernel: [  152.754668] ehci_hcd 0000:00:1a.7: setting latency timer to 64
> > Jul 25 15:52:33 nemi kernel: [  152.768089] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D4
> > Jul 25 15:52:33 nemi kernel: [  152.768573] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D0
> > Jul 25 15:52:33 nemi kernel: [  152.768726] uhci_hcd 0000:00:1d.0: setting latency timer to 64
> > Jul 25 15:52:33 nemi kernel: [  152.768891] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D4
> > Jul 25 15:52:33 nemi kernel: [  152.769144] uhci_hcd 0000:00:1d.1: setting latency timer to 64
> > Jul 25 15:52:33 nemi kernel: [  152.769530] uhci_hcd 0000:00:1d.2: setting latency timer to 64
> > Jul 25 15:52:33 nemi kernel: [  152.769902] ehci_hcd 0000:00:1d.7: setting latency timer to 64
> > Jul 25 15:52:33 nemi kernel: [  152.784189] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D4
> > 
> > 
> > was that expected?  Anyway, waking up the controller from this state by
> > plugging a USB device works, so it's a perfectly OK workaround.  Is this
> > something that could/should be implemented as a system specific quirk,
> > or is the problem more generic?  Even if it is a ACPI implementation
> > issue I would expect it to be common to a number of Lenovo laptops of
> > the same generation as mine (~2008).
> > 
> > > And please provide the output of the following command line.
> > >
> > > acpidump
> > 
> > Attached.  Thanks a lot for all your help debugging this issue.
> 
> This is not really necessary.  The problem is the change of behavior I was
> worried about some time ago.
> 
> Namely, the device in question apparently doesn't support ACPI D3_hot,
> so acpi_pm_device_sleep_state() returns D2 as the target state, where it
> would return D3(cold) before the commit in question.
> 
> To fix the regression, we'd need to reverse the current initial value of
> the d3cold_allowed flag so that it is set to "true" by default and only
> set to "false" for devices where we suspect there may be problems with
> D3_cold (like PCI Express ports).

Yes.  This is a good way to fix this issue and keep original behavior
for most devices.  I will work out a fix.

Best Regards,
Huang Ying



^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-25 22:36               ` Bjørn Mork
@ 2012-07-26  2:38                 ` Huang Ying
  0 siblings, 0 replies; 89+ messages in thread
From: Huang Ying @ 2012-07-26  2:38 UTC (permalink / raw)
  To: Bjørn Mork
  Cc: Rafael J. Wysocki, huang ying, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

On Thu, 2012-07-26 at 00:36 +0200, Bjørn Mork wrote:
> "Rafael J. Wysocki" <rjw@sisk.pl> writes:
> 
> > This is not really necessary.  The problem is the change of behavior I was
> > worried about some time ago.
> >
> > Namely, the device in question apparently doesn't support ACPI D3_hot,
> > so acpi_pm_device_sleep_state() returns D2 as the target state, where it
> > would return D3(cold) before the commit in question.
> >
> > To fix the regression, we'd need to reverse the current initial value of
> > the d3cold_allowed flag so that it is set to "true" by default and only
> > set to "false" for devices where we suspect there may be problems with
> > D3_cold (like PCI Express ports).
> 
> Good.  May I also request a few lines describing d3cold_allowed in e.g
> Documentation/ABI/testing/sysfs-bus-pci?  I must admit that I thought
> that was the policy for any new sysfs file, but it seems that is only
> in the USB subsystem?
> 
> Anyway, I believe it ought to be a requirement.  If we are meant to
> tweak some knob, then we must known how and when to do so.  If not, then
> why add it in the first place?

Yes.  I should have added the ABI document.  Will write a patch for
that.

Best Regards,
Huang Ying



^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-25 13:58           ` Bjørn Mork
  2012-07-25 18:56             ` Rafael J. Wysocki
  2012-07-25 20:02             ` Rafael J. Wysocki
@ 2012-07-26  8:54             ` Huang Ying
  2012-07-26 10:35               ` Bjørn Mork
  2 siblings, 1 reply; 89+ messages in thread
From: Huang Ying @ 2012-07-26  8:54 UTC (permalink / raw)
  To: Bjørn Mork
  Cc: huang ying, Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

On Wed, 2012-07-25 at 15:58 +0200, Bjørn Mork wrote:
> huang ying <huang.ying.caritas@gmail.com> writes:
> 
> > Hi, Bjorn,
> >
> > Thank you very much for your detailed information.
> >
> > On Wed, Jul 25, 2012 at 5:58 PM, Bjørn Mork <bjorn@mork.no> wrote:
> >> Huang Ying <ying.huang@intel.com> writes:
> >>> On Wed, 2012-07-25 at 06:08 +0200, Bjørn Mork wrote:
> >>>> Enabling autosuspend for USB causes hotplug failure in the current
> >>>> linux-next. Newly plugged devices are not detected at all until the
> >>>> port/controller is manually powered on by writing "on" to power/control.
> >>>> Testing is pretty simple:
> >>>>
> >>>>   1) for f in /sys/bus/usb/devices/*/power/control; do echo auto > $f; done
> >>>
> >>> Have you done:
> >>>
> >>> for f in /sys/bus/pci/devices/*/power/confol; do echo auto > $f; done
> >>>
> >>> ?
> >>>
> >>> If not, the pci device will not be suspended at all.
> >>
> >> Yes, sorry for missing that.  I had it automatically enabled.  Yes,
> >> autosuspend for the PCI device and all child devices must be enabled for
> >> the device to be suspended at all, of course.
> >>
> >>>>   2) wait for the controllers to suspend
> >>>>   3) plugin a new USB device
> >>>
> >>> After plugin the new USB device, is there anything in dmesg?
> >>
> >> No. Absolutely nothing, so the USB devices is not enumerated.  Another
> >> indication of the same:  Plugging a device like an Android phone, which
> >> normally detects being connected to a host and presents a device type
> >> menu to the user, results in the charging LED lighting up but no menu.
> >>
> >>
> >> Trying to show the sequence of events:
> >>
> >> 1)  the controllers are suspended:
> >>
> >> Jul 25 11:27:12 nemi kernel: [   38.962792] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D2
> >> Jul 25 11:27:12 nemi kernel: [   39.006718] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D2
> >> Jul 25 11:27:15 nemi kernel: [   41.808471] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D2
> >> Jul 25 11:27:15 nemi kernel: [   41.824123] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D2
> >> Jul 25 11:27:15 nemi kernel: [   41.824194] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D2
> >
> > Here uhci controller is put into D2
> >
> > [snip]
> >>
> >> Doing the same with commit 448bd857d reverted:
> >>
> >>
> >> 1)  the controllers are suspended (to state D3? instead of D2?):
> >>
> >> Jul 25 11:34:01 nemi kernel: [   37.064955] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D3
> >> Jul 25 11:34:01 nemi kernel: [   37.106586] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D3
> >> Jul 25 11:34:04 nemi kernel: [   39.808329] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D3
> >> Jul 25 11:34:04 nemi kernel: [   39.840054] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D3
> >> Jul 25 11:34:04 nemi kernel: [   39.840068] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D3
> >
> > With commit reverted, the uhci_controller is put into D3 (ACPI D3cold).
> >
> > And the uhci controller on your system may not work properly under D2
> > state, while OK in D3 state, and the commit will make uhci controller
> > choose D2 instead of D3.
> >
> > Please try following command line before testing.
> >
> > for f in /sys/bus/pci/devices/*/d3cold_allowed; do echo 1 > $f; done
> 
> That made it work.  The USB controllers ended up in D4 though:
> 
> Jul 25 15:52:33 nemi kernel: [  152.753280] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D0
> Jul 25 15:52:33 nemi kernel: [  152.753453] uhci_hcd 0000:00:1a.0: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.753619] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D4
> Jul 25 15:52:33 nemi kernel: [  152.753875] uhci_hcd 0000:00:1a.1: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.754153] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D0
> Jul 25 15:52:33 nemi kernel: [  152.754279] uhci_hcd 0000:00:1a.2: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.754432] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D4
> Jul 25 15:52:33 nemi kernel: [  152.754668] ehci_hcd 0000:00:1a.7: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.768089] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D4
> Jul 25 15:52:33 nemi kernel: [  152.768573] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D0
> Jul 25 15:52:33 nemi kernel: [  152.768726] uhci_hcd 0000:00:1d.0: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.768891] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D4
> Jul 25 15:52:33 nemi kernel: [  152.769144] uhci_hcd 0000:00:1d.1: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.769530] uhci_hcd 0000:00:1d.2: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.769902] ehci_hcd 0000:00:1d.7: setting latency timer to 64
> Jul 25 15:52:33 nemi kernel: [  152.784189] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D4
> 
> 
> was that expected?  Anyway, waking up the controller from this state by
> plugging a USB device works, so it's a perfectly OK workaround.  Is this
> something that could/should be implemented as a system specific quirk,
> or is the problem more generic?  Even if it is a ACPI implementation
> issue I would expect it to be common to a number of Lenovo laptops of
> the same generation as mine (~2008).
> 
> > And please provide the output of the following command line.
> >
> > acpidump
> 
> Attached.  Thanks a lot for all your help debugging this issue.

Do you have time to try the below patch?

Best Regards,
Huang Ying

Subject: [BUGFIX] PCI/PM: enable D3/D3cold by default for most devices

This patch fixes the following bug:

http://marc.info/?l=linux-usb&m=134318961120825&w=2

Originally, device lower power states include D1, D2, D3.  After that,
D3 is further divided into D3hot and D3cold.  To support both scenario
safely, original D3 is mapped to D3cold.

When adding D3cold support, because worry about some device may have
broken D3cold support, D3cold is disabled by default.  This disable D3
on original platform too.  But some original platform may only have
working D3, but no working D1, D2.  The root cause of the above bug is
it too.

To deal with this, this patch enables D3/D3cold by default for most
devices.  This restores the original behavior.  For some devices that
suspected to have broken D3cold support, such as PCIe port, D3cold is
disabled by default.

Signed-off-by: Huang Ying <ying.huang@intel.com>
---
 drivers/pci/pci.c              |    1 +
 drivers/pci/pcie/portdrv_pci.c |    5 +++++
 2 files changed, 6 insertions(+)

--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -1941,6 +1941,7 @@ void pci_pm_init(struct pci_dev *dev)
 	dev->pm_cap = pm;
 	dev->d3_delay = PCI_PM_D3_WAIT;
 	dev->d3cold_delay = PCI_PM_D3COLD_WAIT;
+	dev->d3cold_allowed = true;
 
 	dev->d1_support = false;
 	dev->d2_support = false;
--- a/drivers/pci/pcie/portdrv_pci.c
+++ b/drivers/pci/pcie/portdrv_pci.c
@@ -200,6 +200,11 @@ static int __devinit pcie_portdrv_probe(
 		return status;
 
 	pci_save_state(dev);
+	/*
+	 * D3cold may not work properly on some PCIe port, so disable
+	 * it by default.
+	 */
+	dev->d3cold_allowed = false;
 	if (!pci_match_id(port_runtime_pm_black_list, dev))
 		pm_runtime_put_noidle(&dev->dev);
 



^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-26  8:54             ` Huang Ying
@ 2012-07-26 10:35               ` Bjørn Mork
  2012-07-26 11:02                 ` Bjørn Mork
                                   ` (2 more replies)
  0 siblings, 3 replies; 89+ messages in thread
From: Bjørn Mork @ 2012-07-26 10:35 UTC (permalink / raw)
  To: Huang Ying
  Cc: huang ying, Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

Huang Ying <ying.huang@intel.com> writes:

> Do you have time to try the below patch?

Sure.  Looks OK wrt the USB problems, but may cause problems with the
PCIe WiFi card.  Unless those are related to other changes in -next.

Anyway, for I applied your patch on top of next-20120724 for
consistency (still without Rafael print fix, so we get the D4 below).  
This results in different stats for the uhci_hcd and ehci_hcd:


Jul 26 12:13:42 nemi kernel: [   72.820305] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D2
Jul 26 12:13:42 nemi kernel: [   72.835101] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D2
Jul 26 12:13:44 nemi kernel: [   74.808770] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D2
Jul 26 12:13:44 nemi kernel: [   74.840293] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D4
Jul 26 12:13:44 nemi kernel: [   74.840326] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D4

I assume that is expected, based on the lspci output I posted earlier.
Overall I get a nice mix of allowed/disallowed:



nemi:/home/bjorn# grep . /sys/bus/pci/devices/*/d3cold_allowed
/sys/bus/pci/devices/0000:00:00.0/d3cold_allowed:0
/sys/bus/pci/devices/0000:00:02.0/d3cold_allowed:1
/sys/bus/pci/devices/0000:00:02.1/d3cold_allowed:1
/sys/bus/pci/devices/0000:00:03.0/d3cold_allowed:1
/sys/bus/pci/devices/0000:00:19.0/d3cold_allowed:1
/sys/bus/pci/devices/0000:00:1a.0/d3cold_allowed:0
/sys/bus/pci/devices/0000:00:1a.1/d3cold_allowed:0
/sys/bus/pci/devices/0000:00:1a.2/d3cold_allowed:0
/sys/bus/pci/devices/0000:00:1a.7/d3cold_allowed:1
/sys/bus/pci/devices/0000:00:1b.0/d3cold_allowed:1
/sys/bus/pci/devices/0000:00:1c.0/d3cold_allowed:0
/sys/bus/pci/devices/0000:00:1c.1/d3cold_allowed:0
/sys/bus/pci/devices/0000:00:1d.0/d3cold_allowed:0
/sys/bus/pci/devices/0000:00:1d.1/d3cold_allowed:0
/sys/bus/pci/devices/0000:00:1d.2/d3cold_allowed:0
/sys/bus/pci/devices/0000:00:1d.7/d3cold_allowed:1
/sys/bus/pci/devices/0000:00:1e.0/d3cold_allowed:0
/sys/bus/pci/devices/0000:00:1f.0/d3cold_allowed:0
/sys/bus/pci/devices/0000:00:1f.2/d3cold_allowed:1
/sys/bus/pci/devices/0000:00:1f.3/d3cold_allowed:0
/sys/bus/pci/devices/0000:03:00.0/d3cold_allowed:1


USB hotplugging seems to work fine with this.  But attempting to
unload/reload the wifi drivers resulted in this:




Jul 26 12:20:43 nemi kernel: [  493.812266] iwlwifi 0000:03:00.0: Refused to change power state, currently in D3
Jul 26 12:20:43 nemi kernel: [  493.812331] iwlwifi 0000:03:00.0: pci_resource_len = 0x00002000
Jul 26 12:20:43 nemi kernel: [  493.812335] iwlwifi 0000:03:00.0: pci_resource_base = ffffc900055a4000
Jul 26 12:20:43 nemi kernel: [  493.812339] iwlwifi 0000:03:00.0: HW Revision ID = 0x0
Jul 26 12:20:43 nemi kernel: [  493.812350] iwlwifi 0000:03:00.0: pci_enable_msi failed(0Xffffffea)
Jul 26 12:20:43 nemi kernel: [  493.812377] driver: '0000:03:00.0': driver_bound: bound to device 'iwlwifi'
Jul 26 12:20:43 nemi kernel: [  493.812385] bus: 'pci': really_probe: bound device 0000:03:00.0 to driver iwlwifi
Jul 26 12:20:43 nemi kernel: [  493.813634] iwldvm: Intel(R) Wireless WiFi Link AGN driver for Linux, in-tree:
Jul 26 12:20:43 nemi kernel: [  493.813637] iwldvm: Copyright(c) 2003-2012 Intel Corporation
Jul 26 12:20:43 nemi kernel: [  493.813912] device: '0000:03:00.0': device_add
Jul 26 12:20:43 nemi kernel: [  493.813939] PM: Adding info for No Bus:0000:03:00.0
Jul 26 12:20:43 nemi kernel: [  493.813947] firmware 0000:03:00.0: firmware: requesting iwlwifi-5000-5.ucode
Jul 26 12:20:43 nemi kernel: [  493.827551] PM: Removing info for No Bus:0000:03:00.0
Jul 26 12:20:43 nemi kernel: [  493.827615] iwlwifi 0000:03:00.0: loaded firmware version 8.83.5.1 build 33692
Jul 26 12:20:43 nemi kernel: [  493.828170] iwlwifi 0000:03:00.0: CONFIG_IWLWIFI_DEBUG disabled
Jul 26 12:20:43 nemi kernel: [  493.828175] iwlwifi 0000:03:00.0: CONFIG_IWLWIFI_DEBUGFS disabled
Jul 26 12:20:43 nemi kernel: [  493.828178] iwlwifi 0000:03:00.0: CONFIG_IWLWIFI_DEVICE_TRACING disabled
Jul 26 12:20:43 nemi kernel: [  493.828182] iwlwifi 0000:03:00.0: CONFIG_IWLWIFI_DEVICE_TESTMODE disabled
Jul 26 12:20:43 nemi kernel: [  493.828185] iwlwifi 0000:03:00.0: CONFIG_IWLWIFI_P2P disabled
Jul 26 12:20:43 nemi kernel: [  493.828190] iwlwifi 0000:03:00.0: Detected Intel(R) Ultimate N WiFi Link 5300 AGN, REV=0xFFFFFFFF
Jul 26 12:20:43 nemi kernel: [  493.828218] iwlwifi 0000:03:00.0: L1 Enabled; Disabling L0S
Jul 26 12:20:43 nemi kernel: [  493.832013] ------------[ cut here ]------------
Jul 26 12:20:43 nemi kernel: [  493.832013] WARNING: at drivers/net/wireless/iwlwifi/iwl-io.c:150 iwl_grab_nic_access+0x47/0x54 [iwlwifi]()
Jul 26 12:20:43 nemi kernel: [  493.832013] Hardware name: 2776LEG
Jul 26 12:20:43 nemi kernel: [  493.832013] Timeout waiting for hardware access (CSR_GP_CNTRL 0xffffffff)
Jul 26 12:20:43 nemi kernel: [  493.832013] Modules linked in: iwldvm iwlwifi mac80211 cfg80211 cpufreq_userspace cpufreq_stats cpufreq_conservative cpufreq_powersave xt_multiport xt_hl ip6table_filter iptable_filter ip_tables ip6_tables x_tables rfcomm bnep binfmt_misc fuse nfsd nfs lockd fscache auth_rpcgss nfs_acl sunrpc 8021q garp stp llc tun ext2 loop snd_hda_codec_conexant snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm thinkpad_acpi snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device arc4 snd i915 coretemp kvm_intel qcserial drm_kms_helper usb_wwan acpi_cpufreq drm psmouse lpc_ich i2c_algo_bit soundcore usbserial qmi_wwan usbnet mii cdc_wdm kvm microcode btusb evdev bluetooth i2c_i801 serio_raw mfd_core rfkill mei snd_page_alloc crc16 i2c_core battery nvram ac video wmi mperf processor button ext3 mbcache jbd sha256_generic ablk_helper cryptd aes_x86_64 aes_generic cbc dm_crypt dm_mod nbd sd_mod crc_t10dif sr_mod cdrom uhci_hcd ahci libahci libata ehci_hc
Jul 26 12:20:43 nemi kernel: d scsi_mod thermal thermal_sys usbcore usb_common e1000e [last unloaded: cfg80211]
Jul 26 12:20:43 nemi kernel: [  493.832013] Pid: 200, comm: kworker/0:2 Not tainted 3.5.0-rc2-next-20120724+ #23
Jul 26 12:20:43 nemi kernel: [  493.832013] Call Trace:
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff8103d0fd>] ? warn_slowpath_common+0x78/0x8c
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff8103d1af>] ? warn_slowpath_fmt+0x45/0x4a
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffffa03822be>] ? iwl_grab_nic_access+0x47/0x54 [iwlwifi]
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffffa0382590>] ? iwl_write_prph+0x29/0x56 [iwlwifi]
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffffa03882cb>] ? iwl_apm_init+0x13a/0x16b [iwlwifi]
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffffa03883fd>] ? iwl_trans_pcie_start_hw+0x101/0x15b [iwlwifi]
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffffa026aa5b>] ? iwl_op_mode_dvm_start+0x246/0x96a [iwldvm]
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffffa03835d1>] ? iwl_ucode_callback+0x9e5/0xad8 [iwlwifi]
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff812793c2>] ? _request_firmware_prepare+0x1e2/0x1e2
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff81279471>] ? request_firmware_work_func+0xaf/0xe4
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff81054dea>] ? process_one_work+0x1ff/0x311
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff810550f7>] ? worker_thread+0x1fb/0x2fb
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff81054efc>] ? process_one_work+0x311/0x311
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff81054efc>] ? process_one_work+0x311/0x311
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff810588fd>] ? kthread+0x81/0x89
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff813703c4>] ? kernel_thread_helper+0x4/0x10
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff8105887c>] ? kthread_freezable_should_stop+0x53/0x53
Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff813703c0>] ? gs_change+0x13/0x13
Jul 26 12:20:43 nemi kernel: [  493.832013] ---[ end trace fcaaf916dd43b7ca ]---
Jul 26 12:20:43 nemi kernel: [  493.864785] iwlwifi 0000:03:00.0: bad EEPROM/OTP signature, type=OTP, EEPROM_GP=0x00000007
Jul 26 12:20:43 nemi kernel: [  493.864791] iwlwifi 0000:03:00.0: EEPROM not found, EEPROM_GP=0xffffffff
Jul 26 12:20:43 nemi kernel: [  493.864795] iwlwifi 0000:03:00.0: Unable to init EEPROM


That does not look good...

The bridge and WiFi devices power status at this point is:

bjorn@nemi:~$  grep . /sys/bus/pci/devices/0000:00:1c.1/power/*
/sys/bus/pci/devices/0000:00:1c.1/power/async:enabled
grep: /sys/bus/pci/devices/0000:00:1c.1/power/autosuspend_delay_ms: Input/output error
/sys/bus/pci/devices/0000:00:1c.1/power/control:auto
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_active_time:390576
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_status:suspended
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_suspended_time:697004
/sys/bus/pci/devices/0000:00:1c.1/power/runtime_usage:0
/sys/bus/pci/devices/0000:00:1c.1/power/wakeup:disabled

bjorn@nemi:~$ grep . /sys/bus/pci/devices/0000:03:00.0/power/*
/sys/bus/pci/devices/0000:03:00.0/power/async:enabled
grep: /sys/bus/pci/devices/0000:03:00.0/power/autosuspend_delay_ms: Input/output error
/sys/bus/pci/devices/0000:03:00.0/power/control:auto
/sys/bus/pci/devices/0000:03:00.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:03:00.0/power/runtime_active_time:0
/sys/bus/pci/devices/0000:03:00.0/power/runtime_enabled:disabled
/sys/bus/pci/devices/0000:03:00.0/power/runtime_status:unsupported
/sys/bus/pci/devices/0000:03:00.0/power/runtime_suspended_time:70220
/sys/bus/pci/devices/0000:03:00.0/power/runtime_usage:0
/sys/bus/pci/devices/0000:03:00.0/power/wakeup:disabled



I don't really understand the last one.  How can suspended_time > 0 when
status is unsupported and autosuspend is disabled?



Bjørn

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-26 10:35               ` Bjørn Mork
@ 2012-07-26 11:02                 ` Bjørn Mork
  2012-07-26 12:04                   ` Bjørn Mork
  2012-07-26 15:03                 ` Alan Stern
  2012-07-27  5:35                 ` Huang Ying
  2 siblings, 1 reply; 89+ messages in thread
From: Bjørn Mork @ 2012-07-26 11:02 UTC (permalink / raw)
  To: Huang Ying
  Cc: huang ying, Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

Bjørn Mork <bjorn@mork.no> writes:

>
> Jul 26 12:20:43 nemi kernel: [  493.812266] iwlwifi 0000:03:00.0: Refused to change power state, currently in D3
> Jul 26 12:20:43 nemi kernel: [  493.812331] iwlwifi 0000:03:00.0: pci_resource_len = 0x00002000
> Jul 26 12:20:43 nemi kernel: [  493.812335] iwlwifi 0000:03:00.0: pci_resource_base = ffffc900055a4000
> Jul 26 12:20:43 nemi kernel: [  493.812339] iwlwifi 0000:03:00.0: HW Revision ID = 0x0
> Jul 26 12:20:43 nemi kernel: [  493.812350] iwlwifi 0000:03:00.0: pci_enable_msi failed(0Xffffffea)
[..]
> Jul 26 12:20:43 nemi kernel: [  493.832013] ---[ end trace fcaaf916dd43b7ca ]---
> Jul 26 12:20:43 nemi kernel: [  493.864785] iwlwifi 0000:03:00.0: bad EEPROM/OTP signature, type=OTP, EEPROM_GP=0x00000007
> Jul 26 12:20:43 nemi kernel: [  493.864791] iwlwifi 0000:03:00.0: EEPROM not found, EEPROM_GP=0xffffffff
> Jul 26 12:20:43 nemi kernel: [  493.864795] iwlwifi 0000:03:00.0: Unable to init EEPROM


Doing 

 echo on >/sys/bus/pci/devices/0000:00:1c.1/power/control

to force the brigde on and then unload/reload driver "fixes" this issue.
But I believe it is safe to say that this does not work as it should.  I
assume the bridge should have been automatically woken up first here?

And why the odd autosuspend output?  The Wifi device and bridge PM caps
look compatible to my untrained eye, and should make both D3hot and
D3cold supported ?:


        Capabilities: [c8] Power Management version 3
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

        Capabilities: [a0] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-




Bjørn

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-26 11:02                 ` Bjørn Mork
@ 2012-07-26 12:04                   ` Bjørn Mork
  0 siblings, 0 replies; 89+ messages in thread
From: Bjørn Mork @ 2012-07-26 12:04 UTC (permalink / raw)
  To: Huang Ying
  Cc: huang ying, Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

Bjørn Mork <bjorn@mork.no> writes:

>         Capabilities: [c8] Power Management version 3
>                 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
>                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
>
>         Capabilities: [a0] Power Management version 2
>                 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
>                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
>

That was from the previous captured lspci output.  But I assume you
might be interested in the state when the driver fails. The default with
your newest patch is to allow d3cold for the WiFi device but disallow it
for the bridge (PCIe port).  Which I guess is the intention based on the
patch description.

This seems to work.  The PCIe port ends up in D3:


nemi:/home/bjorn# lspci -vvnns 00:1c.1
00:1c.1 PCI bridge [0604]: Intel Corporation 82801I (ICH9 Family) PCI Express Port 2 [8086:2942] (rev 03) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
        I/O behind bridge: 00003000-00003fff
        Memory behind bridge: f0500000-f05fffff
        Prefetchable memory behind bridge: 00000000c0400000-00000000c05fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Express (v1) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #2, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <256ns, L1 <4us
                        ClockPM- Surprise- LLActRep+ BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+
                        Slot #1, PowerLimit 6.500W; Interlock- NoCompl-
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet+ CmdCplt+ HPIrq+ LinkChg-
                        Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
                        Changed: MRL- PresDet- LinkState+
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
        Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: fee0300c  Data: 4143
        Capabilities: [90] Subsystem: Lenovo Device [17aa:20f3]
        Capabilities: [a0] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D3 NoSoftRst- PME-Enable+ DSel=0 DScale=0 PME-
        Capabilities: [100 v1] Virtual Channel
                Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                Arb:    Fixed+ WRR32- WRR64- WRR128-
                Ctrl:   ArbSelect=Fixed
                Status: InProgress-
                VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=01
                        Status: NegoPending- InProgress-
        Capabilities: [180 v1] Root Complex Link
                Desc:   PortNumber=02 ComponentID=02 EltType=Config
                Link0:  Desc:   TargetPort=00 TargetComponent=02 AssocRCRB- LinkType=MemMapped LinkValid+
                        Addr:   00000000fed1c000
        Kernel driver in use: pcieport


Attempting to read the WiFi device config at this point is futile:

nemi:/home/bjorn# lspci -vvnns 03:00.0
03:00.0 Network controller [0280]: Intel Corporation Ultimate N WiFi Link 5300 [8086:4236] (rev ff) (prog-if ff)
        !!! Unknown header type 7f



Does that work as expected BTW?  Would be nice if any attempt to read
config would wake the brigde to allow it, would it not?  I have
absolutely no idea whether that's is achievable..

But any attempt to load a driver for the WiFi device should most
definitely wake the bridge.  But it does not:

nemi:/home/bjorn# modprobe iwldvm; lspci -vvnns 00:1c.1
00:1c.1 PCI bridge [0604]: Intel Corporation 82801I (ICH9 Family) PCI Express Port 2 [8086:2942] (rev 03) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
        I/O behind bridge: 00003000-00003fff
        Memory behind bridge: f0500000-f05fffff
        Prefetchable memory behind bridge: 00000000c0400000-00000000c05fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Express (v1) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #2, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <256ns, L1 <4us
                        ClockPM- Surprise- LLActRep+ BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+
                        Slot #1, PowerLimit 6.500W; Interlock- NoCompl-
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet+ CmdCplt+ HPIrq+ LinkChg-
                        Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
                        Changed: MRL- PresDet- LinkState+
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
        Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: fee0300c  Data: 4143
        Capabilities: [90] Subsystem: Lenovo Device [17aa:20f3]
        Capabilities: [a0] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D3 NoSoftRst- PME-Enable+ DSel=0 DScale=0 PME-
        Capabilities: [100 v1] Virtual Channel
                Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                Arb:    Fixed+ WRR32- WRR64- WRR128-
                Ctrl:   ArbSelect=Fixed
                Status: InProgress-
                VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=01
                        Status: NegoPending- InProgress-
        Capabilities: [180 v1] Root Complex Link
                Desc:   PortNumber=02 ComponentID=02 EltType=Config
                Link0:  Desc:   TargetPort=00 TargetComponent=02 AssocRCRB- LinkType=MemMapped LinkValid+
                        Addr:   00000000fed1c000
        Kernel driver in use: pcieport


I have tried all 4 combinations of d3cold_allowed for these 2 devices,
but none of them make any difference.  The default with your patches is
to disallow it for the PCIe port.  One strange issue is that the PCIe
port goes into the same stat even if I set d3cold_allowed to 1:

        Capabilities: [a0] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D3 NoSoftRst- PME-Enable+ DSel=0 DScale=0 PME-


Should't the status then be "D4" or "D3cold" or whatever lspci will call
it?  At least different?  The d3cold_allowed setting does not seem to
change anything for this port.



Bjørn

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-26 10:35               ` Bjørn Mork
  2012-07-26 11:02                 ` Bjørn Mork
@ 2012-07-26 15:03                 ` Alan Stern
  2012-07-26 16:24                   ` Bjørn Mork
  2012-07-27  5:35                 ` Huang Ying
  2 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-07-26 15:03 UTC (permalink / raw)
  To: Bjørn Mork
  Cc: Huang Ying, huang ying, Rafael J. Wysocki, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Thu, 26 Jul 2012, Bjørn Mork wrote:

> Huang Ying <ying.huang@intel.com> writes:
> 
> > Do you have time to try the below patch?
> 
> Sure.  Looks OK wrt the USB problems, but may cause problems with the
> PCIe WiFi card.  Unless those are related to other changes in -next.
> 
> Anyway, for I applied your patch on top of next-20120724 for
> consistency (still without Rafael print fix, so we get the D4 below).  
> This results in different stats for the uhci_hcd and ehci_hcd:
> 
> 
> Jul 26 12:13:42 nemi kernel: [   72.820305] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D2
> Jul 26 12:13:42 nemi kernel: [   72.835101] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D2
> Jul 26 12:13:44 nemi kernel: [   74.808770] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D2
> Jul 26 12:13:44 nemi kernel: [   74.840293] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D4
> Jul 26 12:13:44 nemi kernel: [   74.840326] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D4
> 
> I assume that is expected, based on the lspci output I posted earlier.
> Overall I get a nice mix of allowed/disallowed:

...

> USB hotplugging seems to work fine with this.

Don't be too sure.  Have you tested to see if it still works after 
doing "rmmod ehci-hcd"?

So far you have tested the EHCI controllers, but have you tested the 
UHCI controllers?  Unloading ehci-hcd will force them to be used.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-26 15:03                 ` Alan Stern
@ 2012-07-26 16:24                   ` Bjørn Mork
  0 siblings, 0 replies; 89+ messages in thread
From: Bjørn Mork @ 2012-07-26 16:24 UTC (permalink / raw)
  To: Alan Stern
  Cc: Huang Ying, huang ying, Rafael J. Wysocki, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

Alan Stern <stern@rowland.harvard.edu> writes:

>> USB hotplugging seems to work fine with this.
>
> Don't be too sure.  Have you tested to see if it still works after 
> doing "rmmod ehci-hcd"?
>
> So far you have tested the EHCI controllers, but have you tested the 
> UHCI controllers?  Unloading ehci-hcd will force them to be used.

Good point.  Now tested and seems to work.

But I'd appreciate it if someone else were able to doublecheck my
results.  I am juggling with a number of settings and could easily mess
up something, invalidating the tests.  I believe the issue should be
present in lots of systems, isn't that correct?  Any GM45 based laptop
should do as a test bench, and maybe others too.

The WiFi issues I had also indicates that these changes should be tested
on as many different systems as possible. 



Bjørn

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-26 10:35               ` Bjørn Mork
  2012-07-26 11:02                 ` Bjørn Mork
  2012-07-26 15:03                 ` Alan Stern
@ 2012-07-27  5:35                 ` Huang Ying
  2012-07-27  9:11                   ` Bjørn Mork
  2012-07-27 15:03                   ` Alan Stern
  2 siblings, 2 replies; 89+ messages in thread
From: Huang Ying @ 2012-07-27  5:35 UTC (permalink / raw)
  To: Bjørn Mork
  Cc: huang ying, Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

On Thu, 2012-07-26 at 12:35 +0200, Bjørn Mork wrote:
> Huang Ying <ying.huang@intel.com> writes:
> 
> > Do you have time to try the below patch?
> 
> Sure.  Looks OK wrt the USB problems, but may cause problems with the
> PCIe WiFi card.  Unless those are related to other changes in -next.
> 
> Anyway, for I applied your patch on top of next-20120724 for
> consistency (still without Rafael print fix, so we get the D4 below).  
> This results in different stats for the uhci_hcd and ehci_hcd:
> 
> 
> Jul 26 12:13:42 nemi kernel: [   72.820305] uhci_hcd 0000:00:1a.2: power state changed by ACPI to D2
> Jul 26 12:13:42 nemi kernel: [   72.835101] uhci_hcd 0000:00:1d.0: power state changed by ACPI to D2
> Jul 26 12:13:44 nemi kernel: [   74.808770] uhci_hcd 0000:00:1a.0: power state changed by ACPI to D2
> Jul 26 12:13:44 nemi kernel: [   74.840293] ehci_hcd 0000:00:1d.7: power state changed by ACPI to D4
> Jul 26 12:13:44 nemi kernel: [   74.840326] ehci_hcd 0000:00:1a.7: power state changed by ACPI to D4
> 
> I assume that is expected, based on the lspci output I posted earlier.
> Overall I get a nice mix of allowed/disallowed:
> 
> 
> 
> nemi:/home/bjorn# grep . /sys/bus/pci/devices/*/d3cold_allowed
> /sys/bus/pci/devices/0000:00:00.0/d3cold_allowed:0
> /sys/bus/pci/devices/0000:00:02.0/d3cold_allowed:1
> /sys/bus/pci/devices/0000:00:02.1/d3cold_allowed:1
> /sys/bus/pci/devices/0000:00:03.0/d3cold_allowed:1
> /sys/bus/pci/devices/0000:00:19.0/d3cold_allowed:1
> /sys/bus/pci/devices/0000:00:1a.0/d3cold_allowed:0
> /sys/bus/pci/devices/0000:00:1a.1/d3cold_allowed:0
> /sys/bus/pci/devices/0000:00:1a.2/d3cold_allowed:0
> /sys/bus/pci/devices/0000:00:1a.7/d3cold_allowed:1
> /sys/bus/pci/devices/0000:00:1b.0/d3cold_allowed:1
> /sys/bus/pci/devices/0000:00:1c.0/d3cold_allowed:0
> /sys/bus/pci/devices/0000:00:1c.1/d3cold_allowed:0
> /sys/bus/pci/devices/0000:00:1d.0/d3cold_allowed:0
> /sys/bus/pci/devices/0000:00:1d.1/d3cold_allowed:0
> /sys/bus/pci/devices/0000:00:1d.2/d3cold_allowed:0
> /sys/bus/pci/devices/0000:00:1d.7/d3cold_allowed:1
> /sys/bus/pci/devices/0000:00:1e.0/d3cold_allowed:0
> /sys/bus/pci/devices/0000:00:1f.0/d3cold_allowed:0
> /sys/bus/pci/devices/0000:00:1f.2/d3cold_allowed:1
> /sys/bus/pci/devices/0000:00:1f.3/d3cold_allowed:0
> /sys/bus/pci/devices/0000:03:00.0/d3cold_allowed:1
> 
> 
> USB hotplugging seems to work fine with this.  But attempting to
> unload/reload the wifi drivers resulted in this:
> 
> 
> 
> 
> Jul 26 12:20:43 nemi kernel: [  493.812266] iwlwifi 0000:03:00.0: Refused to change power state, currently in D3

I think this is because the parent bridge (PCIe port) is in lower power
state.  So write to PM registers of child device takes no effect.

> Jul 26 12:20:43 nemi kernel: [  493.812331] iwlwifi 0000:03:00.0: pci_resource_len = 0x00002000
> Jul 26 12:20:43 nemi kernel: [  493.812335] iwlwifi 0000:03:00.0: pci_resource_base = ffffc900055a4000
> Jul 26 12:20:43 nemi kernel: [  493.812339] iwlwifi 0000:03:00.0: HW Revision ID = 0x0
> Jul 26 12:20:43 nemi kernel: [  493.812350] iwlwifi 0000:03:00.0: pci_enable_msi failed(0Xffffffea)
> Jul 26 12:20:43 nemi kernel: [  493.812377] driver: '0000:03:00.0': driver_bound: bound to device 'iwlwifi'
> Jul 26 12:20:43 nemi kernel: [  493.812385] bus: 'pci': really_probe: bound device 0000:03:00.0 to driver iwlwifi
> Jul 26 12:20:43 nemi kernel: [  493.813634] iwldvm: Intel(R) Wireless WiFi Link AGN driver for Linux, in-tree:
> Jul 26 12:20:43 nemi kernel: [  493.813637] iwldvm: Copyright(c) 2003-2012 Intel Corporation
> Jul 26 12:20:43 nemi kernel: [  493.813912] device: '0000:03:00.0': device_add
> Jul 26 12:20:43 nemi kernel: [  493.813939] PM: Adding info for No Bus:0000:03:00.0
> Jul 26 12:20:43 nemi kernel: [  493.813947] firmware 0000:03:00.0: firmware: requesting iwlwifi-5000-5.ucode
> Jul 26 12:20:43 nemi kernel: [  493.827551] PM: Removing info for No Bus:0000:03:00.0
> Jul 26 12:20:43 nemi kernel: [  493.827615] iwlwifi 0000:03:00.0: loaded firmware version 8.83.5.1 build 33692
> Jul 26 12:20:43 nemi kernel: [  493.828170] iwlwifi 0000:03:00.0: CONFIG_IWLWIFI_DEBUG disabled
> Jul 26 12:20:43 nemi kernel: [  493.828175] iwlwifi 0000:03:00.0: CONFIG_IWLWIFI_DEBUGFS disabled
> Jul 26 12:20:43 nemi kernel: [  493.828178] iwlwifi 0000:03:00.0: CONFIG_IWLWIFI_DEVICE_TRACING disabled
> Jul 26 12:20:43 nemi kernel: [  493.828182] iwlwifi 0000:03:00.0: CONFIG_IWLWIFI_DEVICE_TESTMODE disabled
> Jul 26 12:20:43 nemi kernel: [  493.828185] iwlwifi 0000:03:00.0: CONFIG_IWLWIFI_P2P disabled
> Jul 26 12:20:43 nemi kernel: [  493.828190] iwlwifi 0000:03:00.0: Detected Intel(R) Ultimate N WiFi Link 5300 AGN, REV=0xFFFFFFFF
> Jul 26 12:20:43 nemi kernel: [  493.828218] iwlwifi 0000:03:00.0: L1 Enabled; Disabling L0S
> Jul 26 12:20:43 nemi kernel: [  493.832013] ------------[ cut here ]------------
> Jul 26 12:20:43 nemi kernel: [  493.832013] WARNING: at drivers/net/wireless/iwlwifi/iwl-io.c:150 iwl_grab_nic_access+0x47/0x54 [iwlwifi]()
> Jul 26 12:20:43 nemi kernel: [  493.832013] Hardware name: 2776LEG
> Jul 26 12:20:43 nemi kernel: [  493.832013] Timeout waiting for hardware access (CSR_GP_CNTRL 0xffffffff)
> Jul 26 12:20:43 nemi kernel: [  493.832013] Modules linked in: iwldvm iwlwifi mac80211 cfg80211 cpufreq_userspace cpufreq_stats cpufreq_conservative cpufreq_powersave xt_multiport xt_hl ip6table_filter iptable_filter ip_tables ip6_tables x_tables rfcomm bnep binfmt_misc fuse nfsd nfs lockd fscache auth_rpcgss nfs_acl sunrpc 8021q garp stp llc tun ext2 loop snd_hda_codec_conexant snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm thinkpad_acpi snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device arc4 snd i915 coretemp kvm_intel qcserial drm_kms_helper usb_wwan acpi_cpufreq drm psmouse lpc_ich i2c_algo_bit soundcore usbserial qmi_wwan usbnet mii cdc_wdm kvm microcode btusb evdev bluetooth i2c_i801 serio_raw mfd_core rfkill mei snd_page_alloc crc16 i2c_core battery nvram ac video wmi mperf processor button ext3 mbcache jbd sha256_generic ablk_helper cryptd aes_x86_64 aes_generic cbc dm_crypt dm_mod nbd sd_mod crc_t10dif sr_mod cdrom uhci_hcd ahci libahci libata ehci_hc
> Jul 26 12:20:43 nemi kernel: d scsi_mod thermal thermal_sys usbcore usb_common e1000e [last unloaded: cfg80211]
> Jul 26 12:20:43 nemi kernel: [  493.832013] Pid: 200, comm: kworker/0:2 Not tainted 3.5.0-rc2-next-20120724+ #23
> Jul 26 12:20:43 nemi kernel: [  493.832013] Call Trace:
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff8103d0fd>] ? warn_slowpath_common+0x78/0x8c
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff8103d1af>] ? warn_slowpath_fmt+0x45/0x4a
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffffa03822be>] ? iwl_grab_nic_access+0x47/0x54 [iwlwifi]
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffffa0382590>] ? iwl_write_prph+0x29/0x56 [iwlwifi]
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffffa03882cb>] ? iwl_apm_init+0x13a/0x16b [iwlwifi]
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffffa03883fd>] ? iwl_trans_pcie_start_hw+0x101/0x15b [iwlwifi]
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffffa026aa5b>] ? iwl_op_mode_dvm_start+0x246/0x96a [iwldvm]
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffffa03835d1>] ? iwl_ucode_callback+0x9e5/0xad8 [iwlwifi]
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff812793c2>] ? _request_firmware_prepare+0x1e2/0x1e2
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff81279471>] ? request_firmware_work_func+0xaf/0xe4
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff81054dea>] ? process_one_work+0x1ff/0x311
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff810550f7>] ? worker_thread+0x1fb/0x2fb
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff81054efc>] ? process_one_work+0x311/0x311
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff81054efc>] ? process_one_work+0x311/0x311
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff810588fd>] ? kthread+0x81/0x89
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff813703c4>] ? kernel_thread_helper+0x4/0x10
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff8105887c>] ? kthread_freezable_should_stop+0x53/0x53
> Jul 26 12:20:43 nemi kernel: [  493.832013]  [<ffffffff813703c0>] ? gs_change+0x13/0x13
> Jul 26 12:20:43 nemi kernel: [  493.832013] ---[ end trace fcaaf916dd43b7ca ]---
> Jul 26 12:20:43 nemi kernel: [  493.864785] iwlwifi 0000:03:00.0: bad EEPROM/OTP signature, type=OTP, EEPROM_GP=0x00000007
> Jul 26 12:20:43 nemi kernel: [  493.864791] iwlwifi 0000:03:00.0: EEPROM not found, EEPROM_GP=0xffffffff
> Jul 26 12:20:43 nemi kernel: [  493.864795] iwlwifi 0000:03:00.0: Unable to init EEPROM
> 
> 
> That does not look good...
> 
> The bridge and WiFi devices power status at this point is:
> 
> bjorn@nemi:~$  grep . /sys/bus/pci/devices/0000:00:1c.1/power/*
> /sys/bus/pci/devices/0000:00:1c.1/power/async:enabled
> grep: /sys/bus/pci/devices/0000:00:1c.1/power/autosuspend_delay_ms: Input/output error
> /sys/bus/pci/devices/0000:00:1c.1/power/control:auto
> /sys/bus/pci/devices/0000:00:1c.1/power/runtime_active_kids:0
> /sys/bus/pci/devices/0000:00:1c.1/power/runtime_active_time:390576
> /sys/bus/pci/devices/0000:00:1c.1/power/runtime_enabled:enabled
> /sys/bus/pci/devices/0000:00:1c.1/power/runtime_status:suspended
> /sys/bus/pci/devices/0000:00:1c.1/power/runtime_suspended_time:697004
> /sys/bus/pci/devices/0000:00:1c.1/power/runtime_usage:0
> /sys/bus/pci/devices/0000:00:1c.1/power/wakeup:disabled
> 
> bjorn@nemi:~$ grep . /sys/bus/pci/devices/0000:03:00.0/power/*
> /sys/bus/pci/devices/0000:03:00.0/power/async:enabled
> grep: /sys/bus/pci/devices/0000:03:00.0/power/autosuspend_delay_ms: Input/output error
> /sys/bus/pci/devices/0000:03:00.0/power/control:auto
> /sys/bus/pci/devices/0000:03:00.0/power/runtime_active_kids:0
> /sys/bus/pci/devices/0000:03:00.0/power/runtime_active_time:0
> /sys/bus/pci/devices/0000:03:00.0/power/runtime_enabled:disabled
> /sys/bus/pci/devices/0000:03:00.0/power/runtime_status:unsupported
> /sys/bus/pci/devices/0000:03:00.0/power/runtime_suspended_time:70220
> /sys/bus/pci/devices/0000:03:00.0/power/runtime_usage:0
> /sys/bus/pci/devices/0000:03:00.0/power/wakeup:disabled
> 
> 
> 
> I don't really understand the last one.  How can suspended_time > 0 when
> status is unsupported and autosuspend is disabled?

Do you have time to try the following patch?

Best Regards,
Huang Ying

---
 drivers/pci/pci-driver.c |    6 ++++++
 1 file changed, 6 insertions(+)

--- a/drivers/pci/pci-driver.c
+++ b/drivers/pci/pci-driver.c
@@ -280,8 +280,12 @@ static long local_pci_probe(void *_ddi)
 {
 	struct drv_dev_and_id *ddi = _ddi;
 	struct device *dev = &ddi->dev->dev;
+	struct device *parent = dev->parent;
 	int rc;
 
+	/* The parent bridge must be in active state when probing */
+	if (parent)
+		pm_runtime_get_sync(parent);
 	/* Unbound PCI devices are always set to disabled and suspended.
 	 * During probe, the device is set to enabled and active and the
 	 * usage count is incremented.  If the driver supports runtime PM,
@@ -298,6 +302,8 @@ static long local_pci_probe(void *_ddi)
 		pm_runtime_set_suspended(dev);
 		pm_runtime_put_noidle(dev);
 	}
+	if (parent)
+		pm_runtime_put(parent);
 	return rc;
 }
 



^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-27  5:35                 ` Huang Ying
@ 2012-07-27  9:11                   ` Bjørn Mork
  2012-07-30  3:15                     ` Huang Ying
  2012-07-27 15:03                   ` Alan Stern
  1 sibling, 1 reply; 89+ messages in thread
From: Bjørn Mork @ 2012-07-27  9:11 UTC (permalink / raw)
  To: Huang Ying
  Cc: huang ying, Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

Huang Ying <ying.huang@intel.com> writes:

> Do you have time to try the following patch?
>
> Best Regards,
> Huang Ying
>
> ---
>  drivers/pci/pci-driver.c |    6 ++++++
>  1 file changed, 6 insertions(+)
>
> --- a/drivers/pci/pci-driver.c
> +++ b/drivers/pci/pci-driver.c
> @@ -280,8 +280,12 @@ static long local_pci_probe(void *_ddi)
>  {
>  	struct drv_dev_and_id *ddi = _ddi;
>  	struct device *dev = &ddi->dev->dev;
> +	struct device *parent = dev->parent;
>  	int rc;
>  
> +	/* The parent bridge must be in active state when probing */
> +	if (parent)
> +		pm_runtime_get_sync(parent);
>  	/* Unbound PCI devices are always set to disabled and suspended.
>  	 * During probe, the device is set to enabled and active and the
>  	 * usage count is incremented.  If the driver supports runtime PM,
> @@ -298,6 +302,8 @@ static long local_pci_probe(void *_ddi)
>  		pm_runtime_set_suspended(dev);
>  		pm_runtime_put_noidle(dev);
>  	}
> +	if (parent)
> +		pm_runtime_put(parent);
>  	return rc;
>  }
>  


Yup, that worked in the quick test I just did.

 lspci reading the device config will still not wake the bridge, but I
assume that is intentional?  But loading the device driver now wakes
both the bridge and the device, so that works.


Bjørn

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-27  5:35                 ` Huang Ying
  2012-07-27  9:11                   ` Bjørn Mork
@ 2012-07-27 15:03                   ` Alan Stern
  2012-07-27 19:11                     ` Rafael J. Wysocki
  1 sibling, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-07-27 15:03 UTC (permalink / raw)
  To: Huang Ying
  Cc: Bjørn Mork, huang ying, Rafael J. Wysocki, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Fri, 27 Jul 2012, Huang Ying wrote:

> --- a/drivers/pci/pci-driver.c
> +++ b/drivers/pci/pci-driver.c
> @@ -280,8 +280,12 @@ static long local_pci_probe(void *_ddi)
>  {
>  	struct drv_dev_and_id *ddi = _ddi;
>  	struct device *dev = &ddi->dev->dev;
> +	struct device *parent = dev->parent;
>  	int rc;
>  
> +	/* The parent bridge must be in active state when probing */
> +	if (parent)
> +		pm_runtime_get_sync(parent);

Ooh, this is a very good point.  I completely missed it.

>  	/* Unbound PCI devices are always set to disabled and suspended.
>  	 * During probe, the device is set to enabled and active and the
>  	 * usage count is incremented.  If the driver supports runtime PM,
> @@ -298,6 +302,8 @@ static long local_pci_probe(void *_ddi)
>  		pm_runtime_set_suspended(dev);
>  		pm_runtime_put_noidle(dev);
>  	}
> +	if (parent)
> +		pm_runtime_put(parent);

You should use pm_runtime_put_sync(), not pm_runtime_put().

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-27 15:03                   ` Alan Stern
@ 2012-07-27 19:11                     ` Rafael J. Wysocki
  2012-07-27 19:39                       ` Alan Stern
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-07-27 19:11 UTC (permalink / raw)
  To: Alan Stern
  Cc: Huang Ying, Bjørn Mork, huang ying, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Friday, July 27, 2012, Alan Stern wrote:
> On Fri, 27 Jul 2012, Huang Ying wrote:
> 
> > --- a/drivers/pci/pci-driver.c
> > +++ b/drivers/pci/pci-driver.c
> > @@ -280,8 +280,12 @@ static long local_pci_probe(void *_ddi)
> >  {
> >  	struct drv_dev_and_id *ddi = _ddi;
> >  	struct device *dev = &ddi->dev->dev;
> > +	struct device *parent = dev->parent;
> >  	int rc;
> >  
> > +	/* The parent bridge must be in active state when probing */
> > +	if (parent)
> > +		pm_runtime_get_sync(parent);
> 
> Ooh, this is a very good point.  I completely missed it.
> 
> >  	/* Unbound PCI devices are always set to disabled and suspended.
> >  	 * During probe, the device is set to enabled and active and the
> >  	 * usage count is incremented.  If the driver supports runtime PM,
> > @@ -298,6 +302,8 @@ static long local_pci_probe(void *_ddi)
> >  		pm_runtime_set_suspended(dev);
> >  		pm_runtime_put_noidle(dev);
> >  	}
> > +	if (parent)
> > +		pm_runtime_put(parent);
> 
> You should use pm_runtime_put_sync(), not pm_runtime_put().

Hmm, why exactly?

Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-27 19:11                     ` Rafael J. Wysocki
@ 2012-07-27 19:39                       ` Alan Stern
  2012-07-27 19:54                         ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-07-27 19:39 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Huang Ying, Bjørn Mork, huang ying, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Fri, 27 Jul 2012, Rafael J. Wysocki wrote:

> On Friday, July 27, 2012, Alan Stern wrote:
> > On Fri, 27 Jul 2012, Huang Ying wrote:
> > 
> > > --- a/drivers/pci/pci-driver.c
> > > +++ b/drivers/pci/pci-driver.c
> > > @@ -280,8 +280,12 @@ static long local_pci_probe(void *_ddi)
> > >  {
> > >  	struct drv_dev_and_id *ddi = _ddi;
> > >  	struct device *dev = &ddi->dev->dev;
> > > +	struct device *parent = dev->parent;
> > >  	int rc;
> > >  
> > > +	/* The parent bridge must be in active state when probing */
> > > +	if (parent)
> > > +		pm_runtime_get_sync(parent);
> > 
> > Ooh, this is a very good point.  I completely missed it.
> > 
> > >  	/* Unbound PCI devices are always set to disabled and suspended.
> > >  	 * During probe, the device is set to enabled and active and the
> > >  	 * usage count is incremented.  If the driver supports runtime PM,
> > > @@ -298,6 +302,8 @@ static long local_pci_probe(void *_ddi)
> > >  		pm_runtime_set_suspended(dev);
> > >  		pm_runtime_put_noidle(dev);
> > >  	}
> > > +	if (parent)
> > > +		pm_runtime_put(parent);
> > 
> > You should use pm_runtime_put_sync(), not pm_runtime_put().
> 
> Hmm, why exactly?

Because it's more efficient to do something directly than to run it in 
a workqueue.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-27 19:39                       ` Alan Stern
@ 2012-07-27 19:54                         ` Rafael J. Wysocki
  2012-07-28 16:12                           ` Alan Stern
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-07-27 19:54 UTC (permalink / raw)
  To: Alan Stern
  Cc: Huang Ying, Bjørn Mork, huang ying, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Friday, July 27, 2012, Alan Stern wrote:
> On Fri, 27 Jul 2012, Rafael J. Wysocki wrote:
> 
> > On Friday, July 27, 2012, Alan Stern wrote:
> > > On Fri, 27 Jul 2012, Huang Ying wrote:
> > > 
> > > > --- a/drivers/pci/pci-driver.c
> > > > +++ b/drivers/pci/pci-driver.c
> > > > @@ -280,8 +280,12 @@ static long local_pci_probe(void *_ddi)
> > > >  {
> > > >  	struct drv_dev_and_id *ddi = _ddi;
> > > >  	struct device *dev = &ddi->dev->dev;
> > > > +	struct device *parent = dev->parent;
> > > >  	int rc;
> > > >  
> > > > +	/* The parent bridge must be in active state when probing */
> > > > +	if (parent)
> > > > +		pm_runtime_get_sync(parent);
> > > 
> > > Ooh, this is a very good point.  I completely missed it.
> > > 
> > > >  	/* Unbound PCI devices are always set to disabled and suspended.
> > > >  	 * During probe, the device is set to enabled and active and the
> > > >  	 * usage count is incremented.  If the driver supports runtime PM,
> > > > @@ -298,6 +302,8 @@ static long local_pci_probe(void *_ddi)
> > > >  		pm_runtime_set_suspended(dev);
> > > >  		pm_runtime_put_noidle(dev);
> > > >  	}
> > > > +	if (parent)
> > > > +		pm_runtime_put(parent);
> > > 
> > > You should use pm_runtime_put_sync(), not pm_runtime_put().
> > 
> > Hmm, why exactly?
> 
> Because it's more efficient to do something directly than to run it in 
> a workqueue.

Well, depends.  If that results in a power off (the parent goes into
D3 for example), we may wait as long as 10 ms for that to complete.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-27 19:54                         ` Rafael J. Wysocki
@ 2012-07-28 16:12                           ` Alan Stern
  2012-07-28 20:26                             ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-07-28 16:12 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Huang Ying, Bjørn Mork, huang ying, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Fri, 27 Jul 2012, Rafael J. Wysocki wrote:

> > > > > +	if (parent)
> > > > > +		pm_runtime_put(parent);
> > > > 
> > > > You should use pm_runtime_put_sync(), not pm_runtime_put().
> > > 
> > > Hmm, why exactly?
> > 
> > Because it's more efficient to do something directly than to run it in 
> > a workqueue.
> 
> Well, depends.  If that results in a power off (the parent goes into
> D3 for example), we may wait as long as 10 ms for that to complete.

True, but so what?  You'd also have to wait 10 ms for the workqueue
item to complete if pm_runtime_put() was used, plus the additional
overhead of scheduling the workqueue thread.

Now, if the length of time required for the probe to run were an issue 
then yes, the synchronous routine could make things take longer.  But 
probing is generally done asynchronously anyway, right?  And usually 
what matters is the time before the device becomes available, not the 
time for the probe to run.

Alan Stern



^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-28 16:12                           ` Alan Stern
@ 2012-07-28 20:26                             ` Rafael J. Wysocki
  2012-07-28 21:12                               ` Alan Stern
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-07-28 20:26 UTC (permalink / raw)
  To: Alan Stern
  Cc: Huang Ying, Bjørn Mork, huang ying, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Saturday, July 28, 2012, Alan Stern wrote:
> On Fri, 27 Jul 2012, Rafael J. Wysocki wrote:
> 
> > > > > > +	if (parent)
> > > > > > +		pm_runtime_put(parent);
> > > > > 
> > > > > You should use pm_runtime_put_sync(), not pm_runtime_put().
> > > > 
> > > > Hmm, why exactly?
> > > 
> > > Because it's more efficient to do something directly than to run it in 
> > > a workqueue.
> > 
> > Well, depends.  If that results in a power off (the parent goes into
> > D3 for example), we may wait as long as 10 ms for that to complete.
> 
> True, but so what?  You'd also have to wait 10 ms for the workqueue
> item to complete if pm_runtime_put() was used,

Are you sure?  pm_runtime_put() leads to rpm_idle() with the RPM_ASYNC
flag set, which causes it to queue up the work item and return immediately
without waiting.  Why exactly do you think I'd need to wait, then?

> plus the additional overhead of scheduling the workqueue thread.
> 
> Now, if the length of time required for the probe to run were an issue 
> then yes, the synchronous routine could make things take longer.  But 
> probing is generally done asynchronously anyway, right?  And usually 
> what matters is the time before the device becomes available, not the 
> time for the probe to run.

Still, I see no reason to wait synchronously for the _parent_ to suspend.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-28 20:26                             ` Rafael J. Wysocki
@ 2012-07-28 21:12                               ` Alan Stern
  2012-07-29 13:55                                 ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-07-28 21:12 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Huang Ying, Bjørn Mork, huang ying, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Sat, 28 Jul 2012, Rafael J. Wysocki wrote:

> On Saturday, July 28, 2012, Alan Stern wrote:
> > On Fri, 27 Jul 2012, Rafael J. Wysocki wrote:
> > 
> > > > > > > +	if (parent)
> > > > > > > +		pm_runtime_put(parent);
> > > > > > 
> > > > > > You should use pm_runtime_put_sync(), not pm_runtime_put().
> > > > > 
> > > > > Hmm, why exactly?
> > > > 
> > > > Because it's more efficient to do something directly than to run it in 
> > > > a workqueue.
> > > 
> > > Well, depends.  If that results in a power off (the parent goes into
> > > D3 for example), we may wait as long as 10 ms for that to complete.
> > 
> > True, but so what?  You'd also have to wait 10 ms for the workqueue
> > item to complete if pm_runtime_put() was used,
> 
> Are you sure?  pm_runtime_put() leads to rpm_idle() with the RPM_ASYNC
> flag set, which causes it to queue up the work item and return immediately
> without waiting.  Why exactly do you think I'd need to wait, then?

What I mean is, it takes 10 ms for the parent to suspend regardless of
whether or not you use the _sync form of the call.  If you're waiting
for the parent to suspend, you would therefore have to wait 10 ms in
either case.

Likewise, if you're waiting for the PCI device to become usable then
you don't have to wait for 10 ms, because you can use it as soon as the
probe routine returns.

> > plus the additional overhead of scheduling the workqueue thread.
> > 
> > Now, if the length of time required for the probe to run were an issue 
> > then yes, the synchronous routine could make things take longer.  But 
> > probing is generally done asynchronously anyway, right?  And usually 
> > what matters is the time before the device becomes available, not the 
> > time for the probe to run.
> 
> Still, I see no reason to wait synchronously for the _parent_ to suspend.

Why not?  There's no penalty, particularly since most device probing 
happens asynchronously these days anyway.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-28 21:12                               ` Alan Stern
@ 2012-07-29 13:55                                 ` Rafael J. Wysocki
  2012-07-29 14:55                                   ` Alan Stern
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-07-29 13:55 UTC (permalink / raw)
  To: Alan Stern
  Cc: Huang Ying, Bjørn Mork, huang ying, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Saturday, July 28, 2012, Alan Stern wrote:
> On Sat, 28 Jul 2012, Rafael J. Wysocki wrote:
> 
> > On Saturday, July 28, 2012, Alan Stern wrote:
> > > On Fri, 27 Jul 2012, Rafael J. Wysocki wrote:
> > > 
> > > > > > > > +	if (parent)
> > > > > > > > +		pm_runtime_put(parent);
> > > > > > > 
> > > > > > > You should use pm_runtime_put_sync(), not pm_runtime_put().
> > > > > > 
> > > > > > Hmm, why exactly?
> > > > > 
> > > > > Because it's more efficient to do something directly than to run it in 
> > > > > a workqueue.
> > > > 
> > > > Well, depends.  If that results in a power off (the parent goes into
> > > > D3 for example), we may wait as long as 10 ms for that to complete.
> > > 
> > > True, but so what?  You'd also have to wait 10 ms for the workqueue
> > > item to complete if pm_runtime_put() was used,
> > 
> > Are you sure?  pm_runtime_put() leads to rpm_idle() with the RPM_ASYNC
> > flag set, which causes it to queue up the work item and return immediately
> > without waiting.  Why exactly do you think I'd need to wait, then?
> 
> What I mean is, it takes 10 ms for the parent to suspend regardless of
> whether or not you use the _sync form of the call.  If you're waiting
> for the parent to suspend, you would therefore have to wait 10 ms in
> either case.
> 
> Likewise, if you're waiting for the PCI device to become usable then
> you don't have to wait for 10 ms, because you can use it as soon as the
> probe routine returns.

The difference is, if you use _put_sync(), you need to wait the extra 10 ms
for local_pci_probe() to return (if the parent is actually suspended),
although you might not need to wait for it if you used _put(), right?

Which, to me, means that using _put_sync() may not be always better.
It probably doesn't matter a lot, but then the workqueue overhead shouldn't
matter a lot either.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-29 13:55                                 ` Rafael J. Wysocki
@ 2012-07-29 14:55                                   ` Alan Stern
  2012-07-29 19:18                                     ` Rafael J. Wysocki
  2012-07-29 20:12                                     ` bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure Jassi Brar
  0 siblings, 2 replies; 89+ messages in thread
From: Alan Stern @ 2012-07-29 14:55 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Huang Ying, Bjørn Mork, huang ying, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Sun, 29 Jul 2012, Rafael J. Wysocki wrote:

> The difference is, if you use _put_sync(), you need to wait the extra 10 ms
> for local_pci_probe() to return (if the parent is actually suspended),
> although you might not need to wait for it if you used _put(), right?

Yes, that's the difference.  But who waits for local_pci_probe() to 
return?  :-)

> Which, to me, means that using _put_sync() may not be always better.
> It probably doesn't matter a lot, but then the workqueue overhead shouldn't
> matter a lot either.

It's that in the end, the extra overhead is pretty small.  For me
there's also an issue of style: If you do a synchronous get then it
looks odd not to do a synchronous put.  My feeling has always been that
the async routines are for use in non-process contexts, where the sync
routines can't be used.  Using them just to return a little more
quickly is a foreign idea.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-29 14:55                                   ` Alan Stern
@ 2012-07-29 19:18                                     ` Rafael J. Wysocki
  2012-07-31 20:31                                       ` Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...) Rafael J. Wysocki
  2012-07-29 20:12                                     ` bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure Jassi Brar
  1 sibling, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-07-29 19:18 UTC (permalink / raw)
  To: Alan Stern
  Cc: Huang Ying, Bjørn Mork, huang ying, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Sunday, July 29, 2012, Alan Stern wrote:
> On Sun, 29 Jul 2012, Rafael J. Wysocki wrote:
> 
> > The difference is, if you use _put_sync(), you need to wait the extra 10 ms
> > for local_pci_probe() to return (if the parent is actually suspended),
> > although you might not need to wait for it if you used _put(), right?
> 
> Yes, that's the difference.  But who waits for local_pci_probe() to 
> return?  :-)

pci_register_driver() might, but that's not a big deal.  Hot-plug might
as well, though.

> > Which, to me, means that using _put_sync() may not be always better.
> > It probably doesn't matter a lot, but then the workqueue overhead shouldn't
> > matter a lot either.
> 
> It's that in the end, the extra overhead is pretty small.  For me
> there's also an issue of style: If you do a synchronous get then it
> looks odd not to do a synchronous put.  My feeling has always been that
> the async routines are for use in non-process contexts, where the sync
> routines can't be used.  Using them just to return a little more
> quickly is a foreign idea.

I see. :-)

The reason for using sync get is quite obvious: we want to be able to access
the device later on.  For sync put that's not so clear, though. There are cases
when we definitely want to do it, like the failing .probe() in which we want to
disable runtime PM next to the put, but usually it doesn't hurt (too much) to
defer it IMO.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-29 14:55                                   ` Alan Stern
  2012-07-29 19:18                                     ` Rafael J. Wysocki
@ 2012-07-29 20:12                                     ` Jassi Brar
  2012-07-29 21:44                                       ` Alan Stern
  1 sibling, 1 reply; 89+ messages in thread
From: Jassi Brar @ 2012-07-29 20:12 UTC (permalink / raw)
  To: Alan Stern
  Cc: Rafael J. Wysocki, Huang Ying, Bjørn Mork, huang ying,
	Zheng Yan, Bjorn Helgaas, linux-pci, linux-usb

On Sun, Jul 29, 2012 at 8:25 PM, Alan Stern <stern@rowland.harvard.edu> wrote:
>
> For me there's also an issue of style: If you do a synchronous get then it
> looks odd not to do a synchronous put.  My feeling has always been that
> the async routines are for use in non-process contexts, where the sync
> routines can't be used.  Using them just to return a little more
> quickly is a foreign idea.
>
Another way of looking at it is - I need h/w to be active in order to
proceed so I call get_sync but I don't care if the h/w is not put down
immediately after I am done using it. So the put could be
relaxed/async - At best, we could avoid an unnecessary suspend-resume
cycle which could be expensive power and time wise. At worst, we
return a bit quicker. Or so do I think.

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-29 20:12                                     ` bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure Jassi Brar
@ 2012-07-29 21:44                                       ` Alan Stern
  0 siblings, 0 replies; 89+ messages in thread
From: Alan Stern @ 2012-07-29 21:44 UTC (permalink / raw)
  To: Jassi Brar
  Cc: Rafael J. Wysocki, Huang Ying, Bjørn Mork, huang ying,
	Zheng Yan, Bjorn Helgaas, linux-pci, linux-usb

On Mon, 30 Jul 2012, Jassi Brar wrote:

> On Sun, Jul 29, 2012 at 8:25 PM, Alan Stern <stern@rowland.harvard.edu> wrote:
> >
> > For me there's also an issue of style: If you do a synchronous get then it
> > looks odd not to do a synchronous put.  My feeling has always been that
> > the async routines are for use in non-process contexts, where the sync
> > routines can't be used.  Using them just to return a little more
> > quickly is a foreign idea.
> >
> Another way of looking at it is - I need h/w to be active in order to
> proceed so I call get_sync but I don't care if the h/w is not put down
> immediately after I am done using it. So the put could be
> relaxed/async - At best, we could avoid an unnecessary suspend-resume
> cycle which could be expensive power and time wise. At worst, we
> return a bit quicker. Or so do I think.

If the reason for the choice is to opportunistically delay suspending,
there are better ways of doing it: pm_schedule_suspend,
pm_runtime_put_autosuspend.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-27  9:11                   ` Bjørn Mork
@ 2012-07-30  3:15                     ` Huang Ying
  2012-07-30  8:08                       ` Bjørn Mork
  2012-07-30 14:19                       ` Alan Stern
  0 siblings, 2 replies; 89+ messages in thread
From: Huang Ying @ 2012-07-30  3:15 UTC (permalink / raw)
  To: Bjørn Mork
  Cc: huang ying, Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

On Fri, 2012-07-27 at 11:11 +0200, Bjørn Mork wrote:
> Huang Ying <ying.huang@intel.com> writes:
> 
> > Do you have time to try the following patch?
> >
> > Best Regards,
> > Huang Ying
> >
> > ---
> >  drivers/pci/pci-driver.c |    6 ++++++
> >  1 file changed, 6 insertions(+)
> >
> > --- a/drivers/pci/pci-driver.c
> > +++ b/drivers/pci/pci-driver.c
> > @@ -280,8 +280,12 @@ static long local_pci_probe(void *_ddi)
> >  {
> >  	struct drv_dev_and_id *ddi = _ddi;
> >  	struct device *dev = &ddi->dev->dev;
> > +	struct device *parent = dev->parent;
> >  	int rc;
> >  
> > +	/* The parent bridge must be in active state when probing */
> > +	if (parent)
> > +		pm_runtime_get_sync(parent);
> >  	/* Unbound PCI devices are always set to disabled and suspended.
> >  	 * During probe, the device is set to enabled and active and the
> >  	 * usage count is incremented.  If the driver supports runtime PM,
> > @@ -298,6 +302,8 @@ static long local_pci_probe(void *_ddi)
> >  		pm_runtime_set_suspended(dev);
> >  		pm_runtime_put_noidle(dev);
> >  	}
> > +	if (parent)
> > +		pm_runtime_put(parent);
> >  	return rc;
> >  }
> >  
> 
> 
> Yup, that worked in the quick test I just did.
> 
>  lspci reading the device config will still not wake the bridge, but I
> assume that is intentional?  But loading the device driver now wakes
> both the bridge and the device, so that works.

Do you have time to test the following patch to fix the lspci issue?

Subject: [BUGFIX] PCI/PM: Keep parent bridge active when read/write config reg

This patch fixes the following bug:

http://marc.info/?l=linux-pci&m=134338059022620&w=2

Where lspci does not work properly if a device and the corresponding
parent bridge (such as PCIe port) is suspended.  This is because the
device configuration space registers will be not accessible if the
corresponding parent bridge is suspended.

To solve the issue, the bridge/PCIe port connected to the device is
put into active state before read/write configuration space registers.

To avoid resume/suspend PCIe port for each configuration register
read/write, a small delay is added before the PCIe port to go
suspended.

Reported-by: Bjorn Mork <bjorn@mork.no>
Signed-off-by: Huang Ying <ying.huang@intel.com>
---
 drivers/pci/pci-sysfs.c        |   14 ++++++++++++++
 drivers/pci/pcie/portdrv_pci.c |    9 +++++++++
 2 files changed, 23 insertions(+)

--- a/drivers/pci/pci-sysfs.c
+++ b/drivers/pci/pci-sysfs.c
@@ -464,6 +464,7 @@ pci_read_config(struct file *filp, struc
 		char *buf, loff_t off, size_t count)
 {
 	struct pci_dev *dev = to_pci_dev(container_of(kobj,struct device,kobj));
+	struct device *parent = dev->dev.parent;
 	unsigned int size = 64;
 	loff_t init_off = off;
 	u8 *data = (u8*) buf;
@@ -484,6 +485,9 @@ pci_read_config(struct file *filp, struc
 		size = count;
 	}
 
+	if (parent)
+		pm_runtime_get_sync(parent);
+
 	if ((off & 1) && size) {
 		u8 val;
 		pci_user_read_config_byte(dev, off, &val);
@@ -529,6 +533,9 @@ pci_read_config(struct file *filp, struc
 		--size;
 	}
 
+	if (parent)
+		pm_runtime_put(parent);
+
 	return count;
 }
 
@@ -538,6 +545,7 @@ pci_write_config(struct file* filp, stru
 		 char *buf, loff_t off, size_t count)
 {
 	struct pci_dev *dev = to_pci_dev(container_of(kobj,struct device,kobj));
+	struct device *parent = dev->dev.parent;
 	unsigned int size = count;
 	loff_t init_off = off;
 	u8 *data = (u8*) buf;
@@ -549,6 +557,9 @@ pci_write_config(struct file* filp, stru
 		count = size;
 	}
 	
+	if (parent)
+		pm_runtime_get_sync(parent);
+
 	if ((off & 1) && size) {
 		pci_user_write_config_byte(dev, off, data[off - init_off]);
 		off++;
@@ -587,6 +598,9 @@ pci_write_config(struct file* filp, stru
 		--size;
 	}
 
+	if (parent)
+		pm_runtime_put(parent);
+
 	return count;
 }
 
--- a/drivers/pci/pcie/portdrv_pci.c
+++ b/drivers/pci/pcie/portdrv_pci.c
@@ -140,9 +140,17 @@ static int pcie_port_runtime_resume(stru
 {
 	return 0;
 }
+
+static int pcie_port_runtime_idle(struct device *dev)
+{
+	/* Delay for a short while to prevent too frequent suspend/resume */
+	pm_schedule_suspend(dev, 10);
+	return -EBUSY;
+}
 #else
 #define pcie_port_runtime_suspend	NULL
 #define pcie_port_runtime_resume	NULL
+#define pcie_port_runtime_idle		NULL
 #endif
 
 static const struct dev_pm_ops pcie_portdrv_pm_ops = {
@@ -155,6 +163,7 @@ static const struct dev_pm_ops pcie_port
 	.resume_noirq	= pcie_port_resume_noirq,
 	.runtime_suspend = pcie_port_runtime_suspend,
 	.runtime_resume = pcie_port_runtime_resume,
+	.runtime_idle	= pcie_port_runtime_idle,
 };
 
 #define PCIE_PORTDRV_PM_OPS	(&pcie_portdrv_pm_ops)



^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-30  3:15                     ` Huang Ying
@ 2012-07-30  8:08                       ` Bjørn Mork
  2012-07-30 13:31                         ` huang ying
  2012-07-30 14:19                       ` Alan Stern
  1 sibling, 1 reply; 89+ messages in thread
From: Bjørn Mork @ 2012-07-30  8:08 UTC (permalink / raw)
  To: Huang Ying
  Cc: huang ying, Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

Huang Ying <ying.huang@intel.com> writes:

> Do you have time to test the following patch to fix the lspci issue?
>
> Subject: [BUGFIX] PCI/PM: Keep parent bridge active when read/write config reg


Sure.  But keep this going and I will file a request for modular build
of the PCI subsystem :-)

The patch works fine for me:


nemi:/home/bjorn# lspci -t
-[0000:00]-+-00.0
           +-02.0
           +-02.1
           +-03.0
           +-19.0
           +-1a.0
           +-1a.1
           +-1a.2
           +-1a.7
           +-1b.0
           +-1c.0-[02]--
           +-1c.1-[03]----00.0
           +-1d.0
           +-1d.1
           +-1d.2
           +-1d.7
           +-1e.0-[15]--
           +-1f.0
           +-1f.2
           \-1f.3

nemi:/home/bjorn# lspci -vvnns 1c.1; lspci -vvnns 3:0
00:1c.1 PCI bridge [0604]: Intel Corporation 82801I (ICH9 Family) PCI Express Port 2 [8086:2942] (rev 03) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
        I/O behind bridge: 00003000-00003fff
        Memory behind bridge: f0500000-f05fffff
        Prefetchable memory behind bridge: 00000000c0400000-00000000c05fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Express (v1) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #2, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <256ns, L1 <4us
                        ClockPM- Surprise- LLActRep+ BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+
                        Slot #1, PowerLimit 6.500W; Interlock- NoCompl-
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                        Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
                        Changed: MRL- PresDet- LinkState+
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
        Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: fee0300c  Data: 4143
        Capabilities: [90] Subsystem: Lenovo Device [17aa:20f3]
        Capabilities: [a0] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D3 NoSoftRst- PME-Enable+ DSel=0 DScale=0 PME-
        Capabilities: [100 v1] Virtual Channel
                Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                Arb:    Fixed+ WRR32- WRR64- WRR128-
                Ctrl:   ArbSelect=Fixed
                Status: InProgress-
                VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=01
                        Status: NegoPending- InProgress-
        Capabilities: [180 v1] Root Complex Link
                Desc:   PortNumber=02 ComponentID=02 EltType=Config
                Link0:  Desc:   TargetPort=00 TargetComponent=02 AssocRCRB- LinkType=MemMapped LinkValid+
                        Addr:   00000000fed1c000
        Kernel driver in use: pcieport

03:00.0 Network controller [0280]: Intel Corporation Ultimate N WiFi Link 5300 [8086:4236]
        Subsystem: Intel Corporation Device [8086:1011]
        Physical Slot: 1
        Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 17
        Region 0: Memory at f0500000 (64-bit, non-prefetchable) [size=8K]
        Capabilities: [c8] Power Management version 3
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 00000000fee0300c  Data: 4183
        Capabilities: [e0] Express (v1) Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L1 unlimited
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <128ns, L1 <32us
                        ClockPM+ Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
        Capabilities: [140 v1] Device Serial Number 00-16-ea-ff-ff-b3-07-88





Bjørn

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-30  8:08                       ` Bjørn Mork
@ 2012-07-30 13:31                         ` huang ying
  2012-07-30 16:57                           ` Bjørn Mork
  0 siblings, 1 reply; 89+ messages in thread
From: huang ying @ 2012-07-30 13:31 UTC (permalink / raw)
  To: Bjørn Mork
  Cc: Huang Ying, Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

On Mon, Jul 30, 2012 at 4:08 PM, Bjørn Mork <bjorn@mork.no> wrote:
> Huang Ying <ying.huang@intel.com> writes:
>
>> Do you have time to test the following patch to fix the lspci issue?
>>
>> Subject: [BUGFIX] PCI/PM: Keep parent bridge active when read/write config reg
>
>
> Sure.  But keep this going and I will file a request for modular build
> of the PCI subsystem :-)

Why?  All not so old PC hardware has PCI/PCIe devices.

Best Regards,
Huang Ying

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-30  3:15                     ` Huang Ying
  2012-07-30  8:08                       ` Bjørn Mork
@ 2012-07-30 14:19                       ` Alan Stern
  2012-07-31  0:24                         ` Huang Ying
  2012-07-31  3:18                         ` Huang Ying
  1 sibling, 2 replies; 89+ messages in thread
From: Alan Stern @ 2012-07-30 14:19 UTC (permalink / raw)
  To: Huang Ying
  Cc: Bjørn Mork, huang ying, Rafael J. Wysocki, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Mon, 30 Jul 2012, Huang Ying wrote:

> > Yup, that worked in the quick test I just did.
> > 
> >  lspci reading the device config will still not wake the bridge, but I
> > assume that is intentional?  But loading the device driver now wakes
> > both the bridge and the device, so that works.
> 
> Do you have time to test the following patch to fix the lspci issue?
> 
> Subject: [BUGFIX] PCI/PM: Keep parent bridge active when read/write config reg
> 
> This patch fixes the following bug:
> 
> http://marc.info/?l=linux-pci&m=134338059022620&w=2
> 
> Where lspci does not work properly if a device and the corresponding
> parent bridge (such as PCIe port) is suspended.  This is because the
> device configuration space registers will be not accessible if the
> corresponding parent bridge is suspended.
> 
> To solve the issue, the bridge/PCIe port connected to the device is
> put into active state before read/write configuration space registers.

What happens when you run lspci and the device is in D3cold?  Then even 
if the parent bridge is active, lspci will still fail.

It seems that in this case you need to resume the device itself, not 
just its parent.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-30 13:31                         ` huang ying
@ 2012-07-30 16:57                           ` Bjørn Mork
  2012-07-31  0:22                             ` Huang Ying
  0 siblings, 1 reply; 89+ messages in thread
From: Bjørn Mork @ 2012-07-30 16:57 UTC (permalink / raw)
  To: huang ying
  Cc: Huang Ying, Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

huang ying <huang.ying.caritas@gmail.com> writes:

> On Mon, Jul 30, 2012 at 4:08 PM, Bjørn Mork <bjorn@mork.no> wrote:
>> Huang Ying <ying.huang@intel.com> writes:
>>
>>> Do you have time to test the following patch to fix the lspci issue?
>>>
>>> Subject: [BUGFIX] PCI/PM: Keep parent bridge active when read/write config reg
>>
>>
>> Sure.  But keep this going and I will file a request for modular build
>> of the PCI subsystem :-)
>
> Why?  All not so old PC hardware has PCI/PCIe devices.

So that I didn't have to reboot all the time to test your new patches...

It was a (bad) joke.  I don't really think my laptop would work all that
well without PCI.


Bjørn

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-30 16:57                           ` Bjørn Mork
@ 2012-07-31  0:22                             ` Huang Ying
  0 siblings, 0 replies; 89+ messages in thread
From: Huang Ying @ 2012-07-31  0:22 UTC (permalink / raw)
  To: Bjørn Mork
  Cc: huang ying, Rafael J. Wysocki, Zheng Yan, Bjorn Helgaas,
	linux-pci, linux-usb, Alan Stern

On Mon, 2012-07-30 at 18:57 +0200, Bjørn Mork wrote:
> huang ying <huang.ying.caritas@gmail.com> writes:
> 
> > On Mon, Jul 30, 2012 at 4:08 PM, Bjørn Mork <bjorn@mork.no> wrote:
> >> Huang Ying <ying.huang@intel.com> writes:
> >>
> >>> Do you have time to test the following patch to fix the lspci issue?
> >>>
> >>> Subject: [BUGFIX] PCI/PM: Keep parent bridge active when read/write config reg
> >>
> >>
> >> Sure.  But keep this going and I will file a request for modular build
> >> of the PCI subsystem :-)
> >
> > Why?  All not so old PC hardware has PCI/PCIe devices.
> 
> So that I didn't have to reboot all the time to test your new patches...
> 
> It was a (bad) joke.  I don't really think my laptop would work all that
> well without PCI.

Haha, understood now.

Best Regards,
Huang Ying


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-30 14:19                       ` Alan Stern
@ 2012-07-31  0:24                         ` Huang Ying
  2012-07-31  3:18                         ` Huang Ying
  1 sibling, 0 replies; 89+ messages in thread
From: Huang Ying @ 2012-07-31  0:24 UTC (permalink / raw)
  To: Alan Stern
  Cc: Bjørn Mork, huang ying, Rafael J. Wysocki, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Mon, 2012-07-30 at 10:19 -0400, Alan Stern wrote:
> On Mon, 30 Jul 2012, Huang Ying wrote:
> 
> > > Yup, that worked in the quick test I just did.
> > > 
> > >  lspci reading the device config will still not wake the bridge, but I
> > > assume that is intentional?  But loading the device driver now wakes
> > > both the bridge and the device, so that works.
> > 
> > Do you have time to test the following patch to fix the lspci issue?
> > 
> > Subject: [BUGFIX] PCI/PM: Keep parent bridge active when read/write config reg
> > 
> > This patch fixes the following bug:
> > 
> > http://marc.info/?l=linux-pci&m=134338059022620&w=2
> > 
> > Where lspci does not work properly if a device and the corresponding
> > parent bridge (such as PCIe port) is suspended.  This is because the
> > device configuration space registers will be not accessible if the
> > corresponding parent bridge is suspended.
> > 
> > To solve the issue, the bridge/PCIe port connected to the device is
> > put into active state before read/write configuration space registers.
> 
> What happens when you run lspci and the device is in D3cold?  Then even 
> if the parent bridge is active, lspci will still fail.
> 
> It seems that in this case you need to resume the device itself, not 
> just its parent.

Yes.  Will do that.

Best Regards,
Huang Ying



^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-30 14:19                       ` Alan Stern
  2012-07-31  0:24                         ` Huang Ying
@ 2012-07-31  3:18                         ` Huang Ying
  2012-07-31 17:07                           ` Alan Stern
  1 sibling, 1 reply; 89+ messages in thread
From: Huang Ying @ 2012-07-31  3:18 UTC (permalink / raw)
  To: Alan Stern
  Cc: Bjørn Mork, huang ying, Rafael J. Wysocki, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Mon, 2012-07-30 at 10:19 -0400, Alan Stern wrote:
> On Mon, 30 Jul 2012, Huang Ying wrote:
> 
> > > Yup, that worked in the quick test I just did.
> > > 
> > >  lspci reading the device config will still not wake the bridge, but I
> > > assume that is intentional?  But loading the device driver now wakes
> > > both the bridge and the device, so that works.
> > 
> > Do you have time to test the following patch to fix the lspci issue?
> > 
> > Subject: [BUGFIX] PCI/PM: Keep parent bridge active when read/write config reg
> > 
> > This patch fixes the following bug:
> > 
> > http://marc.info/?l=linux-pci&m=134338059022620&w=2
> > 
> > Where lspci does not work properly if a device and the corresponding
> > parent bridge (such as PCIe port) is suspended.  This is because the
> > device configuration space registers will be not accessible if the
> > corresponding parent bridge is suspended.
> > 
> > To solve the issue, the bridge/PCIe port connected to the device is
> > put into active state before read/write configuration space registers.
> 
> What happens when you run lspci and the device is in D3cold?  Then even 
> if the parent bridge is active, lspci will still fail.
> 
> It seems that in this case you need to resume the device itself, not 
> just its parent.

How about the following patch?

Subject: [BUGFIX] PCI/PM: Keep parent bridge active when read/write config reg

This patch fixes the following bug:

http://marc.info/?l=linux-pci&m=134338059022620&w=2

Where lspci does not work properly if a device and the corresponding
parent bridge (such as PCIe port) is suspended.  This is because the
device configuration space registers will be not accessible if the
corresponding parent bridge is suspended or the device is put into
D3cold state.

To solve the issue, the bridge/PCIe port connected to the device is
put into active state before read/write configuration space registers.
If the device is in D3cold state, it will be put into active state
too.

To avoid resume/suspend PCIe port for each configuration register
read/write, a small delay is added before the PCIe port to go
suspended.

Reported-by: Bjorn Mork <bjorn@mork.no>
Signed-off-by: Huang Ying <ying.huang@intel.com>
---
 drivers/pci/pci-sysfs.c        |   68 ++++++++++++++++++++++++++++++-----------
 drivers/pci/pcie/portdrv_pci.c |    9 +++++
 2 files changed, 60 insertions(+), 17 deletions(-)

--- a/drivers/pci/pci-sysfs.c
+++ b/drivers/pci/pci-sysfs.c
@@ -463,15 +463,17 @@ pci_read_config(struct file *filp, struc
 		struct bin_attribute *bin_attr,
 		char *buf, loff_t off, size_t count)
 {
-	struct pci_dev *dev = to_pci_dev(container_of(kobj,struct device,kobj));
+	struct device *dev = container_of(kobj,struct device,kobj);
+	struct pci_dev *pdev = to_pci_dev(dev);
+	struct device *parent = dev->parent;
 	unsigned int size = 64;
 	loff_t init_off = off;
 	u8 *data = (u8*) buf;
 
 	/* Several chips lock up trying to read undefined config space */
 	if (security_capable(filp->f_cred, &init_user_ns, CAP_SYS_ADMIN) == 0) {
-		size = dev->cfg_size;
-	} else if (dev->hdr_type == PCI_HEADER_TYPE_CARDBUS) {
+		size = pdev->cfg_size;
+	} else if (pdev->hdr_type == PCI_HEADER_TYPE_CARDBUS) {
 		size = 128;
 	}
 
@@ -484,9 +486,20 @@ pci_read_config(struct file *filp, struc
 		size = count;
 	}
 
+	if (parent)
+		pm_runtime_get_sync(parent);
+	pm_runtime_get_noresume(dev);
+	/*
+	 * pdev->current_state is set to PCI_D3cold during suspending,
+	 * so wait until suspending completes
+	 */
+	pm_runtime_barrier(dev);
+	if (pdev->current_state == PCI_D3cold)
+		pm_runtime_resume(dev);
+
 	if ((off & 1) && size) {
 		u8 val;
-		pci_user_read_config_byte(dev, off, &val);
+		pci_user_read_config_byte(pdev, off, &val);
 		data[off - init_off] = val;
 		off++;
 		size--;
@@ -494,7 +507,7 @@ pci_read_config(struct file *filp, struc
 
 	if ((off & 3) && size > 2) {
 		u16 val;
-		pci_user_read_config_word(dev, off, &val);
+		pci_user_read_config_word(pdev, off, &val);
 		data[off - init_off] = val & 0xff;
 		data[off - init_off + 1] = (val >> 8) & 0xff;
 		off += 2;
@@ -503,7 +516,7 @@ pci_read_config(struct file *filp, struc
 
 	while (size > 3) {
 		u32 val;
-		pci_user_read_config_dword(dev, off, &val);
+		pci_user_read_config_dword(pdev, off, &val);
 		data[off - init_off] = val & 0xff;
 		data[off - init_off + 1] = (val >> 8) & 0xff;
 		data[off - init_off + 2] = (val >> 16) & 0xff;
@@ -514,7 +527,7 @@ pci_read_config(struct file *filp, struc
 
 	if (size >= 2) {
 		u16 val;
-		pci_user_read_config_word(dev, off, &val);
+		pci_user_read_config_word(pdev, off, &val);
 		data[off - init_off] = val & 0xff;
 		data[off - init_off + 1] = (val >> 8) & 0xff;
 		off += 2;
@@ -523,12 +536,16 @@ pci_read_config(struct file *filp, struc
 
 	if (size > 0) {
 		u8 val;
-		pci_user_read_config_byte(dev, off, &val);
+		pci_user_read_config_byte(pdev, off, &val);
 		data[off - init_off] = val;
 		off++;
 		--size;
 	}
 
+	pm_runtime_put(dev);
+	if (parent)
+		pm_runtime_put(parent);
+
 	return count;
 }
 
@@ -537,20 +554,33 @@ pci_write_config(struct file* filp, stru
 		 struct bin_attribute *bin_attr,
 		 char *buf, loff_t off, size_t count)
 {
-	struct pci_dev *dev = to_pci_dev(container_of(kobj,struct device,kobj));
+	struct device *dev = container_of(kobj,struct device,kobj);
+	struct pci_dev *pdev = to_pci_dev(dev);
+	struct device *parent = dev->parent;
 	unsigned int size = count;
 	loff_t init_off = off;
 	u8 *data = (u8*) buf;
 
-	if (off > dev->cfg_size)
+	if (off > pdev->cfg_size)
 		return 0;
-	if (off + count > dev->cfg_size) {
-		size = dev->cfg_size - off;
+	if (off + count > pdev->cfg_size) {
+		size = pdev->cfg_size - off;
 		count = size;
 	}
 	
+	if (parent)
+		pm_runtime_get_sync(parent);
+	pm_runtime_get_noresume(dev);
+	/*
+	 * pdev->current_state is set to PCI_D3cold during suspending,
+	 * so wait until suspending completes
+	 */
+	pm_runtime_barrier(dev);
+	if (pdev->current_state == PCI_D3cold)
+		pm_runtime_resume(dev);
+
 	if ((off & 1) && size) {
-		pci_user_write_config_byte(dev, off, data[off - init_off]);
+		pci_user_write_config_byte(pdev, off, data[off - init_off]);
 		off++;
 		size--;
 	}
@@ -558,7 +588,7 @@ pci_write_config(struct file* filp, stru
 	if ((off & 3) && size > 2) {
 		u16 val = data[off - init_off];
 		val |= (u16) data[off - init_off + 1] << 8;
-                pci_user_write_config_word(dev, off, val);
+                pci_user_write_config_word(pdev, off, val);
                 off += 2;
                 size -= 2;
         }
@@ -568,7 +598,7 @@ pci_write_config(struct file* filp, stru
 		val |= (u32) data[off - init_off + 1] << 8;
 		val |= (u32) data[off - init_off + 2] << 16;
 		val |= (u32) data[off - init_off + 3] << 24;
-		pci_user_write_config_dword(dev, off, val);
+		pci_user_write_config_dword(pdev, off, val);
 		off += 4;
 		size -= 4;
 	}
@@ -576,17 +606,21 @@ pci_write_config(struct file* filp, stru
 	if (size >= 2) {
 		u16 val = data[off - init_off];
 		val |= (u16) data[off - init_off + 1] << 8;
-		pci_user_write_config_word(dev, off, val);
+		pci_user_write_config_word(pdev, off, val);
 		off += 2;
 		size -= 2;
 	}
 
 	if (size) {
-		pci_user_write_config_byte(dev, off, data[off - init_off]);
+		pci_user_write_config_byte(pdev, off, data[off - init_off]);
 		off++;
 		--size;
 	}
 
+	pm_runtime_put(dev);
+	if (parent)
+		pm_runtime_put(parent);
+
 	return count;
 }
 
--- a/drivers/pci/pcie/portdrv_pci.c
+++ b/drivers/pci/pcie/portdrv_pci.c
@@ -140,9 +140,17 @@ static int pcie_port_runtime_resume(stru
 {
 	return 0;
 }
+
+static int pcie_port_runtime_idle(struct device *dev)
+{
+	/* Delay for a short while to prevent too frequent suspend/resume */
+	pm_schedule_suspend(dev, 10);
+	return -EBUSY;
+}
 #else
 #define pcie_port_runtime_suspend	NULL
 #define pcie_port_runtime_resume	NULL
+#define pcie_port_runtime_idle		NULL
 #endif
 
 static const struct dev_pm_ops pcie_portdrv_pm_ops = {
@@ -155,6 +163,7 @@ static const struct dev_pm_ops pcie_port
 	.resume_noirq	= pcie_port_resume_noirq,
 	.runtime_suspend = pcie_port_runtime_suspend,
 	.runtime_resume = pcie_port_runtime_resume,
+	.runtime_idle	= pcie_port_runtime_idle,
 };
 
 #define PCIE_PORTDRV_PM_OPS	(&pcie_portdrv_pm_ops)



^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure
  2012-07-31  3:18                         ` Huang Ying
@ 2012-07-31 17:07                           ` Alan Stern
  0 siblings, 0 replies; 89+ messages in thread
From: Alan Stern @ 2012-07-31 17:07 UTC (permalink / raw)
  To: Huang Ying
  Cc: Bjørn Mork, huang ying, Rafael J. Wysocki, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb

On Tue, 31 Jul 2012, Huang Ying wrote:

> > What happens when you run lspci and the device is in D3cold?  Then even 
> > if the parent bridge is active, lspci will still fail.
> > 
> > It seems that in this case you need to resume the device itself, not 
> > just its parent.
> 
> How about the following patch?
> 
> Subject: [BUGFIX] PCI/PM: Keep parent bridge active when read/write config reg
> 
> This patch fixes the following bug:
> 
> http://marc.info/?l=linux-pci&m=134338059022620&w=2
> 
> Where lspci does not work properly if a device and the corresponding
> parent bridge (such as PCIe port) is suspended.  This is because the
> device configuration space registers will be not accessible if the
> corresponding parent bridge is suspended or the device is put into
> D3cold state.
> 
> To solve the issue, the bridge/PCIe port connected to the device is
> put into active state before read/write configuration space registers.
> If the device is in D3cold state, it will be put into active state
> too.
> 
> To avoid resume/suspend PCIe port for each configuration register
> read/write, a small delay is added before the PCIe port to go
> suspended.

The patch looks reasonable to me, but I haven't tested it.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-07-29 19:18                                     ` Rafael J. Wysocki
@ 2012-07-31 20:31                                       ` Rafael J. Wysocki
  2012-07-31 21:05                                         ` Alan Stern
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-07-31 20:31 UTC (permalink / raw)
  To: Alan Stern
  Cc: Huang Ying, Bjørn Mork, huang ying, Zheng Yan,
	Bjorn Helgaas, linux-pci, linux-usb, Linux PM list

On Sunday, July 29, 2012, Rafael J. Wysocki wrote:
> On Sunday, July 29, 2012, Alan Stern wrote:
> > On Sun, 29 Jul 2012, Rafael J. Wysocki wrote:
> > 
> > > The difference is, if you use _put_sync(), you need to wait the extra 10 ms
> > > for local_pci_probe() to return (if the parent is actually suspended),
> > > although you might not need to wait for it if you used _put(), right?
> > 
> > Yes, that's the difference.  But who waits for local_pci_probe() to 
> > return?  :-)
> 
> pci_register_driver() might, but that's not a big deal.  Hot-plug might
> as well, though.
> 
> > > Which, to me, means that using _put_sync() may not be always better.
> > > It probably doesn't matter a lot, but then the workqueue overhead shouldn't
> > > matter a lot either.
> > 
> > It's that in the end, the extra overhead is pretty small.  For me
> > there's also an issue of style: If you do a synchronous get then it
> > looks odd not to do a synchronous put.  My feeling has always been that
> > the async routines are for use in non-process contexts, where the sync
> > routines can't be used.  Using them just to return a little more
> > quickly is a foreign idea.
> 
> I see. :-)
> 
> The reason for using sync get is quite obvious: we want to be able to access
> the device later on.  For sync put that's not so clear, though. There are cases
> when we definitely want to do it, like the failing .probe() in which we want to
> disable runtime PM next to the put, but usually it doesn't hurt (too much) to
> defer it IMO.

Now it occured to me that perhaps we don't need the current asynchronous
pm_runtime_get() at all.  The usefulness of it is quite questionable, because
either we want to resume the device immediately, for which pm_runtime_get_sync()
should be used, or we just want to bump up the usage counter, in which cases
pm_runtime_get_noresume() should always be sufficient.  I fail to see any
particularly compelling use case for pm_runtime_get() doing an asynchronous
resume at the moment, but perhaps that's just me.

However, I receive reports of people using pm_runtime_get() where they really
should use pm_runtime_get_sync(), so I wonder if we can simply rename
pm_runtime_get_sync() as pm_runtime_get() and drop the asynchronous version
altogether?

Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-07-31 20:31                                       ` Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...) Rafael J. Wysocki
@ 2012-07-31 21:05                                         ` Alan Stern
  2012-07-31 21:34                                           ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-07-31 21:05 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: linux-pci, USB list, Linux PM list

[CC: list trimmed]

On Tue, 31 Jul 2012, Rafael J. Wysocki wrote:

> Now it occured to me that perhaps we don't need the current asynchronous
> pm_runtime_get() at all.  The usefulness of it is quite questionable, because
> either we want to resume the device immediately, for which pm_runtime_get_sync()
> should be used, or we just want to bump up the usage counter, in which cases
> pm_runtime_get_noresume() should always be sufficient.  I fail to see any
> particularly compelling use case for pm_runtime_get() doing an asynchronous
> resume at the moment, but perhaps that's just me.

There are indeed valid uses for pm_runtime_get().  We are forced to use
it in non-sleepable contexts when we want to resume the device as
quickly as possible.  Example: a driver receives an I/O request from an
interrupt handler.

> However, I receive reports of people using pm_runtime_get() where they really
> should use pm_runtime_get_sync(), so I wonder if we can simply rename
> pm_runtime_get_sync() as pm_runtime_get() and drop the asynchronous version
> altogether?

Well, IMO the naming should have been the other way around from the
start.  That is, we should have made pm_runtime_get be the synchronous
routine and pm_runtime_get_async be the asynchronous one.  But it's too
late to change now.

And no, we can't get rid of the async version.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-07-31 21:05                                         ` Alan Stern
@ 2012-07-31 21:34                                           ` Rafael J. Wysocki
  2012-07-31 21:49                                             ` Rafael J. Wysocki
  2012-08-01 14:36                                             ` Alan Stern
  0 siblings, 2 replies; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-07-31 21:34 UTC (permalink / raw)
  To: Alan Stern; +Cc: linux-pci, USB list, Linux PM list

On Tuesday, July 31, 2012, Alan Stern wrote:
> [CC: list trimmed]
> 
> On Tue, 31 Jul 2012, Rafael J. Wysocki wrote:
> 
> > Now it occured to me that perhaps we don't need the current asynchronous
> > pm_runtime_get() at all.  The usefulness of it is quite questionable, because
> > either we want to resume the device immediately, for which pm_runtime_get_sync()
> > should be used, or we just want to bump up the usage counter, in which cases
> > pm_runtime_get_noresume() should always be sufficient.  I fail to see any
> > particularly compelling use case for pm_runtime_get() doing an asynchronous
> > resume at the moment, but perhaps that's just me.
> 
> There are indeed valid uses for pm_runtime_get().  We are forced to use
> it in non-sleepable contexts when we want to resume the device as
> quickly as possible.  Example: a driver receives an I/O request from an
> interrupt handler.

Is it actually suitable to be used in such contexts?  It doesn't give any
guarantee that the device will be active when it returns, so the caller can't
really rely on it.  The caller always has to check if the device has been
resumed before accessing it anyway, so it may as well do a _get_noresume(),
do the check and do pm_request_resume() if needed directly.

Now, as far as interrupt handlers are concerned, there are two cases: either
the device has to be active to generate an interrupt (like PCI), or the
interrupt is generated by something else on behalf of it.  We don't seem to
handle the first case very well right now, because in that case the interrupt
handler will always know that the device is active, so it should do a
_get_noresume() and then change the device's status to "active" without
calling any kind of "resume", but we don't provide a helper for that.  In the
second case calling pm_runtime_get() doesn't really help either.  I think it's
better to do _get_noresume(), check the status and if "suspended", set a flag
for the resume callback to do something specific before returning successfully,
then call pm_request_resume() and return.

> > However, I receive reports of people using pm_runtime_get() where they really
> > should use pm_runtime_get_sync(), so I wonder if we can simply rename
> > pm_runtime_get_sync() as pm_runtime_get() and drop the asynchronous version
> > altogether?
> 
> Well, IMO the naming should have been the other way around from the
> start.  That is, we should have made pm_runtime_get be the synchronous
> routine and pm_runtime_get_async be the asynchronous one.  But it's too
> late to change now.

I'm not sure it is too late.  If we first change all the instances of
pm_runtime_get() to pm_runtime_get_async() and then all of the instances of
pm_runtime_get_sync() to pm_runtime_get(), it should be technically possible.
Of course, it would be confusing, but that's a different matter. :-)

> And no, we can't get rid of the async version.

I'm still not sure of that.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-07-31 21:34                                           ` Rafael J. Wysocki
@ 2012-07-31 21:49                                             ` Rafael J. Wysocki
  2012-08-01 14:36                                             ` Alan Stern
  1 sibling, 0 replies; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-07-31 21:49 UTC (permalink / raw)
  To: Alan Stern; +Cc: linux-pci, USB list, Linux PM list

On Tuesday, July 31, 2012, Rafael J. Wysocki wrote:
> On Tuesday, July 31, 2012, Alan Stern wrote:
> > [CC: list trimmed]
> > 
> > On Tue, 31 Jul 2012, Rafael J. Wysocki wrote:
> > 
> > > Now it occured to me that perhaps we don't need the current asynchronous
> > > pm_runtime_get() at all.  The usefulness of it is quite questionable, because
> > > either we want to resume the device immediately, for which pm_runtime_get_sync()
> > > should be used, or we just want to bump up the usage counter, in which cases
> > > pm_runtime_get_noresume() should always be sufficient.  I fail to see any
> > > particularly compelling use case for pm_runtime_get() doing an asynchronous
> > > resume at the moment, but perhaps that's just me.
> > 
> > There are indeed valid uses for pm_runtime_get().  We are forced to use
> > it in non-sleepable contexts when we want to resume the device as
> > quickly as possible.  Example: a driver receives an I/O request from an
> > interrupt handler.
> 
> Is it actually suitable to be used in such contexts?  It doesn't give any
> guarantee that the device will be active when it returns, so the caller can't
> really rely on it.  The caller always has to check if the device has been
> resumed before accessing it anyway, so it may as well do a _get_noresume(),
> do the check and do pm_request_resume() if needed directly.
> 
> Now, as far as interrupt handlers are concerned, there are two cases: either
> the device has to be active to generate an interrupt (like PCI), or the
> interrupt is generated by something else on behalf of it.  We don't seem to
> handle the first case very well right now, because in that case the interrupt
> handler will always know that the device is active, so it should do a
> _get_noresume() and then change the device's status to "active" without
> calling any kind of "resume", but we don't provide a helper for that.

Unless, of course, this is a shared interrupt, in which case the handler may
be invoked, because _another_ device has generated an interrupt, so the
"active" check will have to be done anyway.

> In the second case calling pm_runtime_get() doesn't really help either.
> I think it's better to do _get_noresume(), check the status and if
> "suspended", set a flag for the resume callback to do something specific
> before returning successfully, then call pm_request_resume() and return.

I realize that this may be somewhat racy, so perhaps we need something like
pm_request_resume_and_call(dev, func) that will execute the given function once
the device has been resumed (or just happens to be "active" when the resume
work item is run for it).

> > > However, I receive reports of people using pm_runtime_get() where they really
> > > should use pm_runtime_get_sync(), so I wonder if we can simply rename
> > > pm_runtime_get_sync() as pm_runtime_get() and drop the asynchronous version
> > > altogether?
> > 
> > Well, IMO the naming should have been the other way around from the
> > start.  That is, we should have made pm_runtime_get be the synchronous
> > routine and pm_runtime_get_async be the asynchronous one.  But it's too
> > late to change now.
> 
> I'm not sure it is too late.  If we first change all the instances of
> pm_runtime_get() to pm_runtime_get_async() and then all of the instances of
> pm_runtime_get_sync() to pm_runtime_get(), it should be technically possible.
> Of course, it would be confusing, but that's a different matter. :-)
> 
> > And no, we can't get rid of the async version.
> 
> I'm still not sure of that.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-07-31 21:34                                           ` Rafael J. Wysocki
  2012-07-31 21:49                                             ` Rafael J. Wysocki
@ 2012-08-01 14:36                                             ` Alan Stern
  2012-08-01 21:24                                               ` Rafael J. Wysocki
  1 sibling, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-08-01 14:36 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: linux-pci, USB list, Linux PM list

On Tue, 31 Jul 2012, Rafael J. Wysocki wrote:

> On Tuesday, July 31, 2012, Alan Stern wrote:
> > [CC: list trimmed]
> > 
> > On Tue, 31 Jul 2012, Rafael J. Wysocki wrote:
> > 
> > > Now it occured to me that perhaps we don't need the current asynchronous
> > > pm_runtime_get() at all.  The usefulness of it is quite questionable, because
> > > either we want to resume the device immediately, for which pm_runtime_get_sync()
> > > should be used, or we just want to bump up the usage counter, in which cases
> > > pm_runtime_get_noresume() should always be sufficient.  I fail to see any
> > > particularly compelling use case for pm_runtime_get() doing an asynchronous
> > > resume at the moment, but perhaps that's just me.
> > 
> > There are indeed valid uses for pm_runtime_get().  We are forced to use
> > it in non-sleepable contexts when we want to resume the device as
> > quickly as possible.  Example: a driver receives an I/O request from an
> > interrupt handler.
> 
> Is it actually suitable to be used in such contexts?  It doesn't give any
> guarantee that the device will be active when it returns, so the caller can't
> really rely on it.

Of course not.  When you're in interrupt context, you can't wait around
to see if the device will actually resume.  (Unless you're using 
irq-safe runtime PM, but we can ignore that case.)

>  The caller always has to check if the device has been
> resumed before accessing it anyway, so it may as well do a _get_noresume(),
> do the check and do pm_request_resume() if needed directly.

This is exactly equivalent to doing pm_runtime_get() followed by a 
check, except that it's longer.

Yes, I agree that pm_runtime_get(dev) is nothing more than a shorthand
way of doing pm_runtime_get_noresume(dev) followed by 
pm_request_resume(dev).  That doesn't mean it should be eliminated.  
Shorthands are convenient.

As another example, pm_runtime_get_sync(dev) is nothing more than a 
shorthand for pm_runtime_get_noresume(dev) followed by 
pm_runtime_resume(dev).  IMO, having these joint functions that do a 
get/put combined with a suspend/resume/idle is very useful.

> Now, as far as interrupt handlers are concerned, there are two cases: either
> the device has to be active to generate an interrupt (like PCI), or the
> interrupt is generated by something else on behalf of it.  We don't seem to
> handle the first case very well right now, because in that case the interrupt
> handler will always know that the device is active, so it should do a
> _get_noresume() and then change the device's status to "active" without
> calling any kind of "resume", but we don't provide a helper for that.

I disagree.  Even if the device has to be active to generate an IRQ,
it's possible for the interrupt handler to be delayed until after the
device is suspended (unless the suspend routine explicitly calls
synchronize_irq(), which would be pretty foolish).  Hence the handler 
can't make any assumptions about the device's state.

>  In the
> second case calling pm_runtime_get() doesn't really help either.  I think it's
> better to do _get_noresume(), check the status and if "suspended", set a flag
> for the resume callback to do something specific before returning successfully,
> then call pm_request_resume() and return.

This is exactly equivalent to: pm_runtime_get(), check the status, if 
"suspended" then set a flag, otherwise do the work.  Except that again, 
it's longer.

Check out the sample driver code in section 9 of 
Documentation/power/runtime_pm.txt, especially the foo_read_or_write() 
routine.

> > Well, IMO the naming should have been the other way around from the
> > start.  That is, we should have made pm_runtime_get be the synchronous
> > routine and pm_runtime_get_async be the asynchronous one.  But it's too
> > late to change now.
> 
> I'm not sure it is too late.  If we first change all the instances of
> pm_runtime_get() to pm_runtime_get_async() and then all of the instances of
> pm_runtime_get_sync() to pm_runtime_get(), it should be technically possible.
> Of course, it would be confusing, but that's a different matter. :-)

If you're willing to risk a certain amount of confusion, count me in.  :-)

While you're changing names around, consider also adding a "_runtime"  
somewhere to:

	pm_children_suspended,
	pm_schedule_suspend,
	pm_request_idle,
	pm_request_resume,
	pm_request_autosuspend.

For example, we could have pm_runtime_idle_async instead of 
pm_request_idle.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-01 14:36                                             ` Alan Stern
@ 2012-08-01 21:24                                               ` Rafael J. Wysocki
  2012-08-02 20:16                                                 ` Alan Stern
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-01 21:24 UTC (permalink / raw)
  To: Alan Stern; +Cc: linux-pci, USB list, Linux PM list

On Wednesday, August 01, 2012, Alan Stern wrote:
> On Tue, 31 Jul 2012, Rafael J. Wysocki wrote:
> 
> > On Tuesday, July 31, 2012, Alan Stern wrote:
> > > [CC: list trimmed]
> > > 
> > > On Tue, 31 Jul 2012, Rafael J. Wysocki wrote:
> > > 
> > > > Now it occured to me that perhaps we don't need the current asynchronous
> > > > pm_runtime_get() at all.  The usefulness of it is quite questionable, because
> > > > either we want to resume the device immediately, for which pm_runtime_get_sync()
> > > > should be used, or we just want to bump up the usage counter, in which cases
> > > > pm_runtime_get_noresume() should always be sufficient.  I fail to see any
> > > > particularly compelling use case for pm_runtime_get() doing an asynchronous
> > > > resume at the moment, but perhaps that's just me.
> > > 
> > > There are indeed valid uses for pm_runtime_get().  We are forced to use
> > > it in non-sleepable contexts when we want to resume the device as
> > > quickly as possible.  Example: a driver receives an I/O request from an
> > > interrupt handler.
> > 
> > Is it actually suitable to be used in such contexts?  It doesn't give any
> > guarantee that the device will be active when it returns, so the caller can't
> > really rely on it.
> 
> Of course not.  When you're in interrupt context, you can't wait around
> to see if the device will actually resume.  (Unless you're using 
> irq-safe runtime PM, but we can ignore that case.)
> 
> >  The caller always has to check if the device has been
> > resumed before accessing it anyway, so it may as well do a _get_noresume(),
> > do the check and do pm_request_resume() if needed directly.
> 
> This is exactly equivalent to doing pm_runtime_get() followed by a 
> check, except that it's longer.

Except that I meant something different from what I wrote, sorry about that
(must be too much heat).

What I really thought about was to do _get_noresume(), then check if the
device is active and if not, queue up a work item (or another delayed
execution in process context) that will do pm_runtime_resume() and
then access the hardware.

Why do I think it's better than plain pm_runtime_get()?  Because the code
path calling pm_runtime_resume() will know that it is safe to access the
hardware after it has returned (unless it returns an error code, but
that's exceptional).

In contrast, with pm_runtime_get() there is no way to predict when the
device is going to be resumed, so if the device happens to be suspended
when pm_runtime_get() returns, the driver kind of has to poll it until
it becomes active, or use a wait queue woken up from the resume callback,
or do all of the processing in the resume callback itself (like in the
example you mentioned).  I'm not sure if the expectation that all driver
writers will be able to implement any of these options correctly is a realistic
one.

> Yes, I agree that pm_runtime_get(dev) is nothing more than a shorthand
> way of doing pm_runtime_get_noresume(dev) followed by 
> pm_request_resume(dev).  That doesn't mean it should be eliminated.  
> Shorthands are convenient.
> 
> As another example, pm_runtime_get_sync(dev) is nothing more than a 
> shorthand for pm_runtime_get_noresume(dev) followed by 
> pm_runtime_resume(dev).  IMO, having these joint functions that do a 
> get/put combined with a suspend/resume/idle is very useful.
> 
> > Now, as far as interrupt handlers are concerned, there are two cases: either
> > the device has to be active to generate an interrupt (like PCI), or the
> > interrupt is generated by something else on behalf of it.  We don't seem to
> > handle the first case very well right now, because in that case the interrupt
> > handler will always know that the device is active, so it should do a
> > _get_noresume() and then change the device's status to "active" without
> > calling any kind of "resume", but we don't provide a helper for that.
> 
> I disagree.  Even if the device has to be active to generate an IRQ,
> it's possible for the interrupt handler to be delayed until after the
> device is suspended (unless the suspend routine explicitly calls
> synchronize_irq(), which would be pretty foolish).  Hence the handler 
> can't make any assumptions about the device's state.
> 
> >  In the
> > second case calling pm_runtime_get() doesn't really help either.  I think it's
> > better to do _get_noresume(), check the status and if "suspended", set a flag
> > for the resume callback to do something specific before returning successfully,
> > then call pm_request_resume() and return.
> 
> This is exactly equivalent to: pm_runtime_get(), check the status, if 
> "suspended" then set a flag, otherwise do the work.  Except that again, 
> it's longer.
> 
> Check out the sample driver code in section 9 of 
> Documentation/power/runtime_pm.txt, especially the foo_read_or_write() 
> routine.

Well, that shouldn't need the is_suspended flag at all, methinks, and the
reason it does need it is because it uses pm_runtime_get().  Moreover,
processing requests in the resume callback is not something I'd recommend
to anyone, because that's going to happen when the status of the device
is RPM_RESUMING, we're holding a reference on the parent (if there's one) etc.

So, it looks like I don't really agree with the example. :-)

> > > Well, IMO the naming should have been the other way around from the
> > > start.  That is, we should have made pm_runtime_get be the synchronous
> > > routine and pm_runtime_get_async be the asynchronous one.  But it's too
> > > late to change now.
> > 
> > I'm not sure it is too late.  If we first change all the instances of
> > pm_runtime_get() to pm_runtime_get_async() and then all of the instances of
> > pm_runtime_get_sync() to pm_runtime_get(), it should be technically possible.
> > Of course, it would be confusing, but that's a different matter. :-)
> 
> If you're willing to risk a certain amount of confusion, count me in.  :-)
> 
> While you're changing names around, consider also adding a "_runtime"  
> somewhere to:
> 
> 	pm_children_suspended,
> 	pm_schedule_suspend,
> 	pm_request_idle,
> 	pm_request_resume,
> 	pm_request_autosuspend.
> 
> For example, we could have pm_runtime_idle_async instead of 
> pm_request_idle.

Well, these are not as misleading as pm_runtime_get(), at least in principle.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-01 21:24                                               ` Rafael J. Wysocki
@ 2012-08-02 20:16                                                 ` Alan Stern
  2012-08-02 21:26                                                   ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-08-02 20:16 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: linux-pci, USB list, Linux PM list

On Wed, 1 Aug 2012, Rafael J. Wysocki wrote:

> What I really thought about was to do _get_noresume(), then check if the
> device is active and if not, queue up a work item (or another delayed
> execution in process context) that will do pm_runtime_resume() and
> then access the hardware.
> 
> Why do I think it's better than plain pm_runtime_get()?  Because the code
> path calling pm_runtime_resume() will know that it is safe to access the
> hardware after it has returned (unless it returns an error code, but
> that's exceptional).
> 
> In contrast, with pm_runtime_get() there is no way to predict when the
> device is going to be resumed, so if the device happens to be suspended
> when pm_runtime_get() returns, the driver kind of has to poll it until
> it becomes active, or use a wait queue woken up from the resume callback,
> or do all of the processing in the resume callback itself (like in the
> example you mentioned).  I'm not sure if the expectation that all driver
> writers will be able to implement any of these options correctly is a realistic
> one.

I don't know about that -- the logic involved in doing the processing 
within the resume callback isn't terribly complicated.  At least, not 
much more complicated than the logic involved in setting up a custom 
work routine as you suggest.

Anyway with the existing code, driver writers are free to choose 
whichever approach they prefer.

> > Check out the sample driver code in section 9 of 
> > Documentation/power/runtime_pm.txt, especially the foo_read_or_write() 
> > routine.
> 
> Well, that shouldn't need the is_suspended flag at all, methinks, and the
> reason it does need it is because it uses pm_runtime_get().

Not so.  Consider your scheme.  When starting an I/O request, you call
pm_runtime_get_noresume() and then check to see if the device is
active, say by calling pm_runtime_suspended().  Suppose at that moment
the suspend callback has just finished and has released the private
spinlock.  The device's status is still RPM_SUSPENDING, so
pm_runtime_suspended() returns 0 and you try to carry out the I/O.

To fix this problem you have to synchronize the status checking with
the suspend/resume operations.  This means the status changes have to
occur under the protection of the private lock, which means a private
flag is needed.

>  Moreover,
> processing requests in the resume callback is not something I'd recommend
> to anyone, because that's going to happen when the status of the device
> is RPM_RESUMING, we're holding a reference on the parent (if there's one) etc.

I don't see any problem with that.  The parent's child_count will be 
incremented while the requests are being processed regardless.  And if 
you've been waiting for the device to resume in order to carry out some 
processing, within the resume callback is the logical place to do the 
work.  It avoids the overhead of a second context switch.

> So, it looks like I don't really agree with the example. :-)

Feel free to add your scheme as a second example in the document.  :-)
But please don't remove the first example, unless you can find 
something actually wrong with it.

> > While you're changing names around, consider also adding a "_runtime"  
> > somewhere to:
> > 
> > 	pm_children_suspended,
> > 	pm_schedule_suspend,
> > 	pm_request_idle,
> > 	pm_request_resume,
> > 	pm_request_autosuspend.
> > 
> > For example, we could have pm_runtime_idle_async instead of 
> > pm_request_idle.
> 
> Well, these are not as misleading as pm_runtime_get(), at least in principle.

No, but it would be good to be more consistent about our naming.  
Making sure all the function names contain "_runtime" would help.

I suppose we could keep pm_runtime_get_sync as is, and just change
pm_runtime_get to pm_runtime_get_async (and likewise for _put).  That
could reduce the confusion during the changeover.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-02 20:16                                                 ` Alan Stern
@ 2012-08-02 21:26                                                   ` Rafael J. Wysocki
  2012-08-03  2:20                                                     ` Alan Stern
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-02 21:26 UTC (permalink / raw)
  To: Alan Stern; +Cc: linux-pci, USB list, Linux PM list

On Thursday, August 02, 2012, Alan Stern wrote:
> On Wed, 1 Aug 2012, Rafael J. Wysocki wrote:
> 
> > What I really thought about was to do _get_noresume(), then check if the
> > device is active and if not, queue up a work item (or another delayed
> > execution in process context) that will do pm_runtime_resume() and
> > then access the hardware.
> > 
> > Why do I think it's better than plain pm_runtime_get()?  Because the code
> > path calling pm_runtime_resume() will know that it is safe to access the
> > hardware after it has returned (unless it returns an error code, but
> > that's exceptional).
> > 
> > In contrast, with pm_runtime_get() there is no way to predict when the
> > device is going to be resumed, so if the device happens to be suspended
> > when pm_runtime_get() returns, the driver kind of has to poll it until
> > it becomes active, or use a wait queue woken up from the resume callback,
> > or do all of the processing in the resume callback itself (like in the
> > example you mentioned).  I'm not sure if the expectation that all driver
> > writers will be able to implement any of these options correctly is a realistic
> > one.
> 
> I don't know about that -- the logic involved in doing the processing 
> within the resume callback isn't terribly complicated.  At least, not 
> much more complicated than the logic involved in setting up a custom 
> work routine as you suggest.

That's why I had the idea of pm_request_resume_and_call(dev, func)
described in this message:

http://marc.info/?l=linux-usb&m=134377126804170&w=4

although perheps it would be better to call it something like
pm_runtime_async_resume_and_call(dev, func).

> Anyway with the existing code, driver writers are free to choose 
> whichever approach they prefer.

I wonder how many instances of pm_runtime_get() used in a wrong way there
are in the kernel right now.  I guess I'll sacrifice some time to check
that.

> > > Check out the sample driver code in section 9 of 
> > > Documentation/power/runtime_pm.txt, especially the foo_read_or_write() 
> > > routine.
> > 
> > Well, that shouldn't need the is_suspended flag at all, methinks, and the
> > reason it does need it is because it uses pm_runtime_get().
> 
> Not so.  Consider your scheme.  When starting an I/O request, you call
> pm_runtime_get_noresume() and then check to see if the device is
> active, say by calling pm_runtime_suspended().  Suppose at that moment
> the suspend callback has just finished and has released the private
> spinlock.  The device's status is still RPM_SUSPENDING, so
> pm_runtime_suspended() returns 0 and you try to carry out the I/O.
> 
> To fix this problem you have to synchronize the status checking with
> the suspend/resume operations.  This means the status changes have to
> occur under the protection of the private lock, which means a private
> flag is needed.

What about checking if the status is RPM_ACTIVE under dev->power.lock?
That's what rpm_resume() does, more or less.

> >  Moreover,
> > processing requests in the resume callback is not something I'd recommend
> > to anyone, because that's going to happen when the status of the device
> > is RPM_RESUMING, we're holding a reference on the parent (if there's one) etc.
> 
> I don't see any problem with that.  The parent's child_count will be 
> incremented while the requests are being processed regardless.  And if 
> you've been waiting for the device to resume in order to carry out some 
> processing, within the resume callback is the logical place to do the 
> work.

Unless your _driver_ callback is actually executed from within a PM domain
callback, for example, and something else may be waiting for it to complete,
so your data processing is adding latencies to some other threads.  I'm not
making that up, by the way, that really can happen.

And what if you are a parent waited for by a child to resume so that _it_
can process its data?  Would you still want to process your data in the
resume callback in that case?

> It avoids the overhead of a second context switch.

That may be avoided without processing data from within the resume callback,
although not with the current code.

> > So, it looks like I don't really agree with the example. :-)
> 
> Feel free to add your scheme as a second example in the document.  :-)
> But please don't remove the first example, unless you can find 
> something actually wrong with it.

Well, it should work in general, so it is not functionally wrong.  However,
I wouldn't like to regard it as the best thing we can offer. :-)

> > > While you're changing names around, consider also adding a "_runtime"  
> > > somewhere to:
> > > 
> > > 	pm_children_suspended,
> > > 	pm_schedule_suspend,
> > > 	pm_request_idle,
> > > 	pm_request_resume,
> > > 	pm_request_autosuspend.
> > > 
> > > For example, we could have pm_runtime_idle_async instead of 
> > > pm_request_idle.
> > 
> > Well, these are not as misleading as pm_runtime_get(), at least in principle.
> 
> No, but it would be good to be more consistent about our naming.  
> Making sure all the function names contain "_runtime" would help.
> 
> I suppose we could keep pm_runtime_get_sync as is, and just change
> pm_runtime_get to pm_runtime_get_async (and likewise for _put).  That
> could reduce the confusion during the changeover.

Changing pm_runtime_get() to pm_runtime_get_async() would be an improvement,
although pm_runtime_get_but_do_not_resume_immediately() might even be better.
Or even pm_runtime_get_but_do_not_access_hardware_when_it_returns(). ;-)

I see no reason to have any of those things, though.  Yes, they _may_ be
useful to someone knowing the runtime PM core internals to save a few lines
of code in some places, but generally they lead people to make serious mistakes
that tend to be difficult to debug.  For this very reason pm_runtime_get() is a
bad interface and I wish we hadn't introduced it at all.  Even if we give it
a more descriptive name, it won't be much better.

And note how that doesn't apply to the pm_runtime_put*() family.  After all,
doing pm_runtime_put() instead of pm_runtime_put_sync() will never lead to
accessing registers of a suspended device.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-02 21:26                                                   ` Rafael J. Wysocki
@ 2012-08-03  2:20                                                     ` Alan Stern
  2012-08-03  3:37                                                       ` Ming Lei
                                                                         ` (2 more replies)
  0 siblings, 3 replies; 89+ messages in thread
From: Alan Stern @ 2012-08-03  2:20 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: linux-pci, USB list, Linux PM list

On Thu, 2 Aug 2012, Rafael J. Wysocki wrote:

> > I don't know about that -- the logic involved in doing the processing 
> > within the resume callback isn't terribly complicated.  At least, not 
> > much more complicated than the logic involved in setting up a custom 
> > work routine as you suggest.
> 
> That's why I had the idea of pm_request_resume_and_call(dev, func)
> described in this message:
> 
> http://marc.info/?l=linux-usb&m=134377126804170&w=4
> 
> although perheps it would be better to call it something like
> pm_runtime_async_resume_and_call(dev, func).

Hmmm.  You'd probably want a version that does a "get" at the same 
time.  I suppose you would call func directly if the device was already 
resumed, without going through the workqueue?

Yes, I agree this would be a better interface then pm_runtime_get.

> > > Well, that shouldn't need the is_suspended flag at all, methinks, and the
> > > reason it does need it is because it uses pm_runtime_get().
> > 
> > Not so.  Consider your scheme.  When starting an I/O request, you call
> > pm_runtime_get_noresume() and then check to see if the device is
> > active, say by calling pm_runtime_suspended().  Suppose at that moment
> > the suspend callback has just finished and has released the private
> > spinlock.  The device's status is still RPM_SUSPENDING, so
> > pm_runtime_suspended() returns 0 and you try to carry out the I/O.
> > 
> > To fix this problem you have to synchronize the status checking with
> > the suspend/resume operations.  This means the status changes have to
> > occur under the protection of the private lock, which means a private
> > flag is needed.
> 
> What about checking if the status is RPM_ACTIVE under dev->power.lock?
> That's what rpm_resume() does, more or less.

That wouldn't solve the race described above.

> > >  Moreover,
> > > processing requests in the resume callback is not something I'd recommend
> > > to anyone, because that's going to happen when the status of the device
> > > is RPM_RESUMING, we're holding a reference on the parent (if there's one) etc.
> > 
> > I don't see any problem with that.  The parent's child_count will be 
> > incremented while the requests are being processed regardless.  And if 
> > you've been waiting for the device to resume in order to carry out some 
> > processing, within the resume callback is the logical place to do the 
> > work.
> 
> Unless your _driver_ callback is actually executed from within a PM domain
> callback, for example, and something else may be waiting for it to complete,
> so your data processing is adding latencies to some other threads.  I'm not
> making that up, by the way, that really can happen.
> 
> And what if you are a parent waited for by a child to resume so that _it_
> can process its data?  Would you still want to process your data in the
> resume callback in that case?

Okay, those are valid reasons.  (Although a device handling I/O 
requests isn't likely to have a child with its own independent I/O 
handling.)

> > I suppose we could keep pm_runtime_get_sync as is, and just change
> > pm_runtime_get to pm_runtime_get_async (and likewise for _put).  That
> > could reduce the confusion during the changeover.
> 
> Changing pm_runtime_get() to pm_runtime_get_async() would be an improvement,
> although pm_runtime_get_but_do_not_resume_immediately() might even be better.
> Or even pm_runtime_get_but_do_not_access_hardware_when_it_returns(). ;-)
> 
> I see no reason to have any of those things, though.  Yes, they _may_ be
> useful to someone knowing the runtime PM core internals to save a few lines
> of code in some places, but generally they lead people to make serious mistakes
> that tend to be difficult to debug.  For this very reason pm_runtime_get() is a
> bad interface and I wish we hadn't introduced it at all.  Even if we give it
> a more descriptive name, it won't be much better.
> 
> And note how that doesn't apply to the pm_runtime_put*() family.  After all,
> doing pm_runtime_put() instead of pm_runtime_put_sync() will never lead to
> accessing registers of a suspended device.

All right.  But I still think "pm_runtime_put_async" is better than
"pm_runtime_put".  At least it forces people to think about what
they're doing.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-03  2:20                                                     ` Alan Stern
@ 2012-08-03  3:37                                                       ` Ming Lei
  2012-08-03 14:28                                                         ` Alan Stern
  2012-08-04 19:47                                                         ` Rafael J. Wysocki
  2012-08-03 14:05                                                       ` Alan Stern
  2012-08-04 19:35                                                       ` Rafael J. Wysocki
  2 siblings, 2 replies; 89+ messages in thread
From: Ming Lei @ 2012-08-03  3:37 UTC (permalink / raw)
  To: Alan Stern; +Cc: Rafael J. Wysocki, linux-pci, USB list, Linux PM list

On Fri, Aug 3, 2012 at 10:20 AM, Alan Stern <stern@rowland.harvard.edu> wrote:
> On Thu, 2 Aug 2012, Rafael J. Wysocki wrote:

> Hmmm.  You'd probably want a version that does a "get" at the same
> time.  I suppose you would call func directly if the device was already
> resumed, without going through the workqueue?

Maybe it isn't good, the 'func' might not be run in the current context
(irq context or some spinlock is held).

>> And what if you are a parent waited for by a child to resume so that _it_
>> can process its data?  Would you still want to process your data in the
>> resume callback in that case?

Looks the child is always resumed after its parent completes the resuming,
and current pm_runtime_get doesn't delay the resume of the parent, and
just make the .runtime_resume run in another context.

Also are there actual examples about the above situation?

>
> Okay, those are valid reasons.  (Although a device handling I/O
> requests isn't likely to have a child with its own independent I/O
> handling.)

IMO, the .runtime_resume callback can handle the IO things easily
via scheduling worker function if the driver don't want to delay
its children's resume.

Thanks,
-- 
Ming Lei

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-03  2:20                                                     ` Alan Stern
  2012-08-03  3:37                                                       ` Ming Lei
@ 2012-08-03 14:05                                                       ` Alan Stern
  2012-08-04 20:08                                                         ` Rafael J. Wysocki
  2012-08-04 19:35                                                       ` Rafael J. Wysocki
  2 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-08-03 14:05 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: linux-pci, USB list, Linux PM list

On Thu, 2 Aug 2012, Alan Stern wrote:

> On Thu, 2 Aug 2012, Rafael J. Wysocki wrote:
> 
> > > I don't know about that -- the logic involved in doing the processing 
> > > within the resume callback isn't terribly complicated.  At least, not 
> > > much more complicated than the logic involved in setting up a custom 
> > > work routine as you suggest.
> > 
> > That's why I had the idea of pm_request_resume_and_call(dev, func)
> > described in this message:
> > 
> > http://marc.info/?l=linux-usb&m=134377126804170&w=4
> > 
> > although perheps it would be better to call it something like
> > pm_runtime_async_resume_and_call(dev, func).
> 
> Hmmm.  You'd probably want a version that does a "get" at the same 
> time.  I suppose you would call func directly if the device was already 
> resumed, without going through the workqueue?
> 
> Yes, I agree this would be a better interface then pm_runtime_get.

A few problems:

What happens if a system sleep begins after 
pm_runtime_async_resume_and_call() but before the runtime resume?  When 
the system wakes up, the device will be back at full power without ever 
executing a runtime PM call.  Then how will func get invoked?

What happens if the driver is unbound after
pm_runtime_async_resume_and_call() but before the runtime resume?  The
When the runtime resume occurs, func will be invoked -- and the driver
might not even be in memory any more.

Alan Stern



^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-03  3:37                                                       ` Ming Lei
@ 2012-08-03 14:28                                                         ` Alan Stern
  2012-08-04 19:47                                                         ` Rafael J. Wysocki
  1 sibling, 0 replies; 89+ messages in thread
From: Alan Stern @ 2012-08-03 14:28 UTC (permalink / raw)
  To: Ming Lei; +Cc: Rafael J. Wysocki, linux-pci, USB list, Linux PM list

On Fri, 3 Aug 2012, Ming Lei wrote:

> On Fri, Aug 3, 2012 at 10:20 AM, Alan Stern <stern@rowland.harvard.edu> wrote:
> > On Thu, 2 Aug 2012, Rafael J. Wysocki wrote:
> 
> > Hmmm.  You'd probably want a version that does a "get" at the same
> > time.  I suppose you would call func directly if the device was already
> > resumed, without going through the workqueue?
> 
> Maybe it isn't good, the 'func' might not be run in the current context
> (irq context or some spinlock is held).

True.  But we don't want to wait for the workqueue if the device is 
already at full power.  Suggestions?

> >> And what if you are a parent waited for by a child to resume so that _it_
> >> can process its data?  Would you still want to process your data in the
> >> resume callback in that case?
> 
> Looks the child is always resumed after its parent completes the resuming,
> and current pm_runtime_get doesn't delay the resume of the parent, and
> just make the .runtime_resume run in another context.

I don't understand.  Rafael's point was that if the parent's resume
callback did a bunch of real work, it would delay resuming the child 
because the child can't be resumed until the parent's resume callback 
returns.

> Also are there actual examples about the above situation?

I don't know of any.  I suspect there aren't many examples.  It 
wouldn't make much sense.

> IMO, the .runtime_resume callback can handle the IO things easily
> via scheduling worker function if the driver don't want to delay
> its children's resume.

That was one of Rafael's other suggestions.  But calling 
pm_request_resume_and_call() is easier than scheduling a worker 
function.

Alan Stern



^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-03  2:20                                                     ` Alan Stern
  2012-08-03  3:37                                                       ` Ming Lei
  2012-08-03 14:05                                                       ` Alan Stern
@ 2012-08-04 19:35                                                       ` Rafael J. Wysocki
  2 siblings, 0 replies; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-04 19:35 UTC (permalink / raw)
  To: Alan Stern; +Cc: linux-pci, USB list, Linux PM list

On Friday, August 03, 2012, Alan Stern wrote:
> On Thu, 2 Aug 2012, Rafael J. Wysocki wrote:
> 
> > > I don't know about that -- the logic involved in doing the processing 
> > > within the resume callback isn't terribly complicated.  At least, not 
> > > much more complicated than the logic involved in setting up a custom 
> > > work routine as you suggest.
> > 
> > That's why I had the idea of pm_request_resume_and_call(dev, func)
> > described in this message:
> > 
> > http://marc.info/?l=linux-usb&m=134377126804170&w=4
> > 
> > although perheps it would be better to call it something like
> > pm_runtime_async_resume_and_call(dev, func).
> 
> Hmmm.  You'd probably want a version that does a "get" at the same 
> time.  I suppose you would call func directly if the device was already 
> resumed, without going through the workqueue?

Yes.

> Yes, I agree this would be a better interface then pm_runtime_get.

OK

> > > > Well, that shouldn't need the is_suspended flag at all, methinks, and the
> > > > reason it does need it is because it uses pm_runtime_get().
> > > 
> > > Not so.  Consider your scheme.  When starting an I/O request, you call
> > > pm_runtime_get_noresume() and then check to see if the device is
> > > active, say by calling pm_runtime_suspended().  Suppose at that moment
> > > the suspend callback has just finished and has released the private
> > > spinlock.  The device's status is still RPM_SUSPENDING, so
> > > pm_runtime_suspended() returns 0 and you try to carry out the I/O.
> > > 
> > > To fix this problem you have to synchronize the status checking with
> > > the suspend/resume operations.  This means the status changes have to
> > > occur under the protection of the private lock, which means a private
> > > flag is needed.
> > 
> > What about checking if the status is RPM_ACTIVE under dev->power.lock?
> > That's what rpm_resume() does, more or less.
> 
> That wouldn't solve the race described above.
> 
> > > >  Moreover,
> > > > processing requests in the resume callback is not something I'd recommend
> > > > to anyone, because that's going to happen when the status of the device
> > > > is RPM_RESUMING, we're holding a reference on the parent (if there's one) etc.
> > > 
> > > I don't see any problem with that.  The parent's child_count will be 
> > > incremented while the requests are being processed regardless.  And if 
> > > you've been waiting for the device to resume in order to carry out some 
> > > processing, within the resume callback is the logical place to do the 
> > > work.
> > 
> > Unless your _driver_ callback is actually executed from within a PM domain
> > callback, for example, and something else may be waiting for it to complete,
> > so your data processing is adding latencies to some other threads.  I'm not
> > making that up, by the way, that really can happen.
> > 
> > And what if you are a parent waited for by a child to resume so that _it_
> > can process its data?  Would you still want to process your data in the
> > resume callback in that case?
> 
> Okay, those are valid reasons.  (Although a device handling I/O 
> requests isn't likely to have a child with its own independent I/O 
> handling.)

I agree that this isn't very likely in practice.

> > > I suppose we could keep pm_runtime_get_sync as is, and just change
> > > pm_runtime_get to pm_runtime_get_async (and likewise for _put).  That
> > > could reduce the confusion during the changeover.
> > 
> > Changing pm_runtime_get() to pm_runtime_get_async() would be an improvement,
> > although pm_runtime_get_but_do_not_resume_immediately() might even be better.
> > Or even pm_runtime_get_but_do_not_access_hardware_when_it_returns(). ;-)
> > 
> > I see no reason to have any of those things, though.  Yes, they _may_ be
> > useful to someone knowing the runtime PM core internals to save a few lines
> > of code in some places, but generally they lead people to make serious mistakes
> > that tend to be difficult to debug.  For this very reason pm_runtime_get() is a
> > bad interface and I wish we hadn't introduced it at all.  Even if we give it
> > a more descriptive name, it won't be much better.
> > 
> > And note how that doesn't apply to the pm_runtime_put*() family.  After all,
> > doing pm_runtime_put() instead of pm_runtime_put_sync() will never lead to
> > accessing registers of a suspended device.
> 
> All right.  But I still think "pm_runtime_put_async" is better than
> "pm_runtime_put".  At least it forces people to think about what
> they're doing.

I agree.

My current plan is to provide a better alternative interface, then to change
the name of "pm_runtime_put" to "pm_runtime_put_async" and to document that
it's going to be deprecated in future.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-03  3:37                                                       ` Ming Lei
  2012-08-03 14:28                                                         ` Alan Stern
@ 2012-08-04 19:47                                                         ` Rafael J. Wysocki
  2012-08-04 20:25                                                           ` Alan Stern
  1 sibling, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-04 19:47 UTC (permalink / raw)
  To: Ming Lei; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Friday, August 03, 2012, Ming Lei wrote:
> On Fri, Aug 3, 2012 at 10:20 AM, Alan Stern <stern@rowland.harvard.edu> wrote:
> > On Thu, 2 Aug 2012, Rafael J. Wysocki wrote:
> 
> > Hmmm.  You'd probably want a version that does a "get" at the same
> > time.  I suppose you would call func directly if the device was already
> > resumed, without going through the workqueue?
> 
> Maybe it isn't good, the 'func' might not be run in the current context
> (irq context or some spinlock is held).

Then I'd say don't use this interface.  If you have code that needs to
be run in a different context, then you have to use a work item (or
equivalent) anyway and you can do a synchronous runtime resume from there.

The problem we want to address here is when there's code that should be
run as soon as the device is active, preferably _without_ a context switch.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-03 14:05                                                       ` Alan Stern
@ 2012-08-04 20:08                                                         ` Rafael J. Wysocki
  2012-08-04 20:42                                                           ` Alan Stern
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-04 20:08 UTC (permalink / raw)
  To: Alan Stern; +Cc: linux-pci, USB list, Linux PM list

On Friday, August 03, 2012, Alan Stern wrote:
> On Thu, 2 Aug 2012, Alan Stern wrote:
> 
> > On Thu, 2 Aug 2012, Rafael J. Wysocki wrote:
> > 
> > > > I don't know about that -- the logic involved in doing the processing 
> > > > within the resume callback isn't terribly complicated.  At least, not 
> > > > much more complicated than the logic involved in setting up a custom 
> > > > work routine as you suggest.
> > > 
> > > That's why I had the idea of pm_request_resume_and_call(dev, func)
> > > described in this message:
> > > 
> > > http://marc.info/?l=linux-usb&m=134377126804170&w=4
> > > 
> > > although perheps it would be better to call it something like
> > > pm_runtime_async_resume_and_call(dev, func).
> > 
> > Hmmm.  You'd probably want a version that does a "get" at the same 
> > time.  I suppose you would call func directly if the device was already 
> > resumed, without going through the workqueue?
> > 
> > Yes, I agree this would be a better interface then pm_runtime_get.
> 
> A few problems:

Which are good points. :-)

> What happens if a system sleep begins after 
> pm_runtime_async_resume_and_call() but before the runtime resume?  When 
> the system wakes up, the device will be back at full power without ever 
> executing a runtime PM call.  Then how will func get invoked?

The pm_runtime_barrier() at the beginning of __device_suspend() guarantees
that if there's a resume request pending at this point, the resume will be
carried out.  I think we can modify rpm_resume() to return only after func()
is executed, so there shouldn't be a problem with that particular case.

> What happens if the driver is unbound after
> pm_runtime_async_resume_and_call() but before the runtime resume?  The
> When the runtime resume occurs, func will be invoked -- and the driver
> might not even be in memory any more.

We have no protection against that at the moment, but then there is no
guarantee that the driver's runtime PM callbacks won't be run after
it's been unbound.

Perhaps we should run pm_runtime_barrier() before executing .remove()
in __device_release_driver() (and perhaps run pm_runtime_idle() afterwards
in case the subsystem or PM domain knows how to power down the device even if
there's no driver) and expect .remove() to clean up?

Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-04 19:47                                                         ` Rafael J. Wysocki
@ 2012-08-04 20:25                                                           ` Alan Stern
  2012-08-04 20:48                                                             ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-08-04 20:25 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: Ming Lei, linux-pci, USB list, Linux PM list

On Sat, 4 Aug 2012, Rafael J. Wysocki wrote:

> On Friday, August 03, 2012, Ming Lei wrote:
> > On Fri, Aug 3, 2012 at 10:20 AM, Alan Stern <stern@rowland.harvard.edu> wrote:
> > > On Thu, 2 Aug 2012, Rafael J. Wysocki wrote:
> > 
> > > Hmmm.  You'd probably want a version that does a "get" at the same
> > > time.  I suppose you would call func directly if the device was already
> > > resumed, without going through the workqueue?
> > 
> > Maybe it isn't good, the 'func' might not be run in the current context
> > (irq context or some spinlock is held).
> 
> Then I'd say don't use this interface.  If you have code that needs to
> be run in a different context, then you have to use a work item (or
> equivalent) anyway and you can do a synchronous runtime resume from there.

That wasn't what he meant.  What if the code needs to run in the _same_
context as the caller?  For example, with a spinlock held.  func would
need to know whether it should acquire the lock, which means it needs
to know whether it was called directly or through the workqueue.

I suppose we could pass it an extra argument with this information...  
but that's a little ugly.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-04 20:08                                                         ` Rafael J. Wysocki
@ 2012-08-04 20:42                                                           ` Alan Stern
  2012-08-04 20:59                                                             ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-08-04 20:42 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: linux-pci, USB list, Linux PM list

On Sat, 4 Aug 2012, Rafael J. Wysocki wrote:

> > What happens if a system sleep begins after 
> > pm_runtime_async_resume_and_call() but before the runtime resume?  When 
> > the system wakes up, the device will be back at full power without ever 
> > executing a runtime PM call.  Then how will func get invoked?
> 
> The pm_runtime_barrier() at the beginning of __device_suspend() guarantees
> that if there's a resume request pending at this point, the resume will be
> carried out.  I think we can modify rpm_resume() to return only after func()
> is executed, so there shouldn't be a problem with that particular case.

A resume request can get added to the workqueue after
__device_suspend() runs, in which case the problem will still exist.

Should pm_runtime_set_active() call func (if such a call is pending)?

> > What happens if the driver is unbound after
> > pm_runtime_async_resume_and_call() but before the runtime resume?  The
> > When the runtime resume occurs, func will be invoked -- and the driver
> > might not even be in memory any more.
> 
> We have no protection against that at the moment, but then there is no
> guarantee that the driver's runtime PM callbacks won't be run after
> it's been unbound.

True.  We have been relying on the subsystem to handle this race, but 
obviously that's no good if the callback is directly to the driver.  I 
don't see any way to fix this in the PM core.

> Perhaps we should run pm_runtime_barrier() before executing .remove()
> in __device_release_driver()

We already do pm_runtime_get_sync() and pm_runtime_put_sync().  I'm not 
sure how necessary those calls really are ... but pm_runtime_barrier() 
would be redundant.

>  (and perhaps run pm_runtime_idle() afterwards
> in case the subsystem or PM domain knows how to power down the device even if
> there's no driver) and expect .remove() to clean up?

I can't think of anything wrong with calling pm_runtime_idle() at the
end of __device_release_driver().

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-04 20:25                                                           ` Alan Stern
@ 2012-08-04 20:48                                                             ` Rafael J. Wysocki
  2012-08-04 20:48                                                               ` Alan Stern
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-04 20:48 UTC (permalink / raw)
  To: Alan Stern; +Cc: Ming Lei, linux-pci, USB list, Linux PM list

On Saturday, August 04, 2012, Alan Stern wrote:
> On Sat, 4 Aug 2012, Rafael J. Wysocki wrote:
> 
> > On Friday, August 03, 2012, Ming Lei wrote:
> > > On Fri, Aug 3, 2012 at 10:20 AM, Alan Stern <stern@rowland.harvard.edu> wrote:
> > > > On Thu, 2 Aug 2012, Rafael J. Wysocki wrote:
> > > 
> > > > Hmmm.  You'd probably want a version that does a "get" at the same
> > > > time.  I suppose you would call func directly if the device was already
> > > > resumed, without going through the workqueue?
> > > 
> > > Maybe it isn't good, the 'func' might not be run in the current context
> > > (irq context or some spinlock is held).
> > 
> > Then I'd say don't use this interface.  If you have code that needs to
> > be run in a different context, then you have to use a work item (or
> > equivalent) anyway and you can do a synchronous runtime resume from there.
> 
> That wasn't what he meant.  What if the code needs to run in the _same_
> context as the caller?  For example, with a spinlock held.

I see.  I think it wouldn't be unreasonable to require that func should take
all of the necessary locks by itself.

However, I believe that if func is going to be called through the workqueue,
the usage counter should be incremented before the preceding resume and
decremented after func returns by the workqueue thread.

> func would need to know whether it should acquire the lock, which means it
> needs to know whether it was called directly or through the workqueue.
> 
> I suppose we could pass it an extra argument with this information...  
> but that's a little ugly.

I agree.  I'd prefer to avoid that, if possible.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-04 20:48                                                             ` Rafael J. Wysocki
@ 2012-08-04 20:48                                                               ` Alan Stern
  2012-08-04 21:15                                                                 ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-08-04 20:48 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: Ming Lei, linux-pci, USB list, Linux PM list

On Sat, 4 Aug 2012, Rafael J. Wysocki wrote:

> > That wasn't what he meant.  What if the code needs to run in the _same_
> > context as the caller?  For example, with a spinlock held.
> 
> I see.  I think it wouldn't be unreasonable to require that func should take
> all of the necessary locks by itself.

But then if func was called directly, because the device was at full
power, it would deadlock trying to acquire a lock the caller already
holds.

> However, I believe that if func is going to be called through the workqueue,
> the usage counter should be incremented before the preceding resume and
> decremented after func returns by the workqueue thread.

Okay.

> > func would need to know whether it should acquire the lock, which means it
> > needs to know whether it was called directly or through the workqueue.
> > 
> > I suppose we could pass it an extra argument with this information...  
> > but that's a little ugly.
> 
> I agree.  I'd prefer to avoid that, if possible.

Do you have any better suggestions?

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-04 20:42                                                           ` Alan Stern
@ 2012-08-04 20:59                                                             ` Rafael J. Wysocki
  0 siblings, 0 replies; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-04 20:59 UTC (permalink / raw)
  To: Alan Stern; +Cc: linux-pci, USB list, Linux PM list

On Saturday, August 04, 2012, Alan Stern wrote:
> On Sat, 4 Aug 2012, Rafael J. Wysocki wrote:
> 
> > > What happens if a system sleep begins after 
> > > pm_runtime_async_resume_and_call() but before the runtime resume?  When 
> > > the system wakes up, the device will be back at full power without ever 
> > > executing a runtime PM call.  Then how will func get invoked?
> > 
> > The pm_runtime_barrier() at the beginning of __device_suspend() guarantees
> > that if there's a resume request pending at this point, the resume will be
> > carried out.  I think we can modify rpm_resume() to return only after func()
> > is executed, so there shouldn't be a problem with that particular case.
> 
> A resume request can get added to the workqueue after
> __device_suspend() runs, in which case the problem will still exist.

In theory.  But the question is who's going to add that request, since
the driver's (and possibly subsystem's/PM domain's) .suspend() callbacks
will be run by __device_suspend() and we can expect those to ensure
certain level of consistency.

Anyway, today we'll have the same problem with a runtime resume request
queued up during system suspend (after __device_suspend() has run).

> Should pm_runtime_set_active() call func (if such a call is pending)?
> 
> > > What happens if the driver is unbound after
> > > pm_runtime_async_resume_and_call() but before the runtime resume?  The
> > > When the runtime resume occurs, func will be invoked -- and the driver
> > > might not even be in memory any more.
> > 
> > We have no protection against that at the moment, but then there is no
> > guarantee that the driver's runtime PM callbacks won't be run after
> > it's been unbound.
> 
> True.  We have been relying on the subsystem to handle this race, but 
> obviously that's no good if the callback is directly to the driver.  I 
> don't see any way to fix this in the PM core.

Me neither.

> > Perhaps we should run pm_runtime_barrier() before executing .remove()
> > in __device_release_driver()
> 
> We already do pm_runtime_get_sync() and pm_runtime_put_sync().  I'm not 
> sure how necessary those calls really are ...

They are, because some platforms do some unusual things in their platform
bus type notifiers.

> but pm_runtime_barrier() would be redundant.
> 
> >  (and perhaps run pm_runtime_idle() afterwards
> > in case the subsystem or PM domain knows how to power down the device even if
> > there's no driver) and expect .remove() to clean up?
> 
> I can't think of anything wrong with calling pm_runtime_idle() at the
> end of __device_release_driver().

Yeah.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-04 20:48                                                               ` Alan Stern
@ 2012-08-04 21:15                                                                 ` Rafael J. Wysocki
  2012-08-04 22:13                                                                   ` Alan Stern
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-04 21:15 UTC (permalink / raw)
  To: Alan Stern; +Cc: Ming Lei, linux-pci, USB list, Linux PM list

On Saturday, August 04, 2012, Alan Stern wrote:
> On Sat, 4 Aug 2012, Rafael J. Wysocki wrote:
> 
> > > That wasn't what he meant.  What if the code needs to run in the _same_
> > > context as the caller?  For example, with a spinlock held.
> > 
> > I see.  I think it wouldn't be unreasonable to require that func should take
> > all of the necessary locks by itself.
> 
> But then if func was called directly, because the device was at full
> power, it would deadlock trying to acquire a lock the caller already
> holds.

I wonder why the caller may want to take any locks beforehand?

> > However, I believe that if func is going to be called through the workqueue,
> > the usage counter should be incremented before the preceding resume and
> > decremented after func returns by the workqueue thread.
> 
> Okay.
> 
> > > func would need to know whether it should acquire the lock, which means it
> > > needs to know whether it was called directly or through the workqueue.
> > > 
> > > I suppose we could pass it an extra argument with this information...  
> > > but that's a little ugly.
> > 
> > I agree.  I'd prefer to avoid that, if possible.
> 
> Do you have any better suggestions?

Hmm.  What about pm_runtime_get_and_call(dev, func_sync, func_async), where
func_sync() is to be called if the device is already active and func_async()
is to be called if it has to be resumed from the workqueue?

Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-04 21:15                                                                 ` Rafael J. Wysocki
@ 2012-08-04 22:13                                                                   ` Alan Stern
  2012-08-05 15:26                                                                     ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-08-04 22:13 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: Ming Lei, linux-pci, USB list, Linux PM list

On Sat, 4 Aug 2012, Rafael J. Wysocki wrote:

> On Saturday, August 04, 2012, Alan Stern wrote:
> > On Sat, 4 Aug 2012, Rafael J. Wysocki wrote:
> > 
> > > > That wasn't what he meant.  What if the code needs to run in the _same_
> > > > context as the caller?  For example, with a spinlock held.
> > > 
> > > I see.  I think it wouldn't be unreasonable to require that func should take
> > > all of the necessary locks by itself.
> > 
> > But then if func was called directly, because the device was at full
> > power, it would deadlock trying to acquire a lock the caller already
> > holds.
> 
> I wonder why the caller may want to take any locks beforehand?

Who knows?  :-)

The best answer may be for the caller not to hold any locks.  In the
runtime-PM document's example driver, the lock would be dropped before
the resume-and-call.

> > Do you have any better suggestions?
> 
> Hmm.  What about pm_runtime_get_and_call(dev, func_sync, func_async), where
> func_sync() is to be called if the device is already active and func_async()
> is to be called if it has to be resumed from the workqueue?

That's a little better but not much.  It would require storing two 
function pointers in the dev->power structure.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-04 22:13                                                                   ` Alan Stern
@ 2012-08-05 15:26                                                                     ` Rafael J. Wysocki
  2012-08-06 13:30                                                                       ` Ming Lei
  2012-08-06 15:48                                                                       ` Alan Stern
  0 siblings, 2 replies; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-05 15:26 UTC (permalink / raw)
  To: Alan Stern; +Cc: Ming Lei, linux-pci, USB list, Linux PM list

On Sunday, August 05, 2012, Alan Stern wrote:
> On Sat, 4 Aug 2012, Rafael J. Wysocki wrote:
> 
> > On Saturday, August 04, 2012, Alan Stern wrote:
> > > On Sat, 4 Aug 2012, Rafael J. Wysocki wrote:
> > > 
> > > > > That wasn't what he meant.  What if the code needs to run in the _same_
> > > > > context as the caller?  For example, with a spinlock held.
> > > > 
> > > > I see.  I think it wouldn't be unreasonable to require that func should take
> > > > all of the necessary locks by itself.
> > > 
> > > But then if func was called directly, because the device was at full
> > > power, it would deadlock trying to acquire a lock the caller already
> > > holds.
> > 
> > I wonder why the caller may want to take any locks beforehand?
> 
> Who knows?  :-)
> 
> The best answer may be for the caller not to hold any locks.  In the
> runtime-PM document's example driver, the lock would be dropped before
> the resume-and-call.
> 
> > > Do you have any better suggestions?
> > 
> > Hmm.  What about pm_runtime_get_and_call(dev, func_sync, func_async), where
> > func_sync() is to be called if the device is already active and func_async()
> > is to be called if it has to be resumed from the workqueue?
> 
> That's a little better but not much.  It would require storing two 
> function pointers in the dev->power structure.

I don't really think we'll need to store func_sync in dev->power.  At least
I don't see a reason to do that at the moment.

I also think that func_sync() would have to be called if runtime PM is
disabled for the given device, so that callers don't have to check that
condition themselves.

What about the appended experimental patch?

Rafael


---
 drivers/base/power/runtime.c |   82 +++++++++++++++++++++++++++++++++++++++----
 include/linux/pm.h           |    1 
 include/linux/pm_runtime.h   |   14 +++++++
 3 files changed, 90 insertions(+), 7 deletions(-)

Index: linux/drivers/base/power/runtime.c
===================================================================
--- linux.orig/drivers/base/power/runtime.c
+++ linux/drivers/base/power/runtime.c
@@ -484,6 +484,15 @@ static int rpm_suspend(struct device *de
 	goto out;
 }
 
+void rpm_queue_up_resume(struct device *dev)
+{
+	dev->power.request = RPM_REQ_RESUME;
+	if (!dev->power.request_pending) {
+		dev->power.request_pending = true;
+		queue_work(pm_wq, &dev->power.work);
+	}
+}
+
 /**
  * rpm_resume - Carry out runtime resume of given device.
  * @dev: Device to resume.
@@ -524,7 +533,9 @@ static int rpm_resume(struct device *dev
 	 * rather than cancelling it now only to restart it again in the near
 	 * future.
 	 */
-	dev->power.request = RPM_REQ_NONE;
+	if (dev->power.request != RPM_REQ_RESUME || !dev->power.func)
+		dev->power.request = RPM_REQ_NONE;
+
 	if (!dev->power.timer_autosuspends)
 		pm_runtime_deactivate_timer(dev);
 
@@ -533,6 +544,12 @@ static int rpm_resume(struct device *dev
 		goto out;
 	}
 
+	if (dev->power.func && (rpmflags & RPM_ASYNC)) {
+		rpm_queue_up_resume(dev);
+		retval = 0;
+		goto out;
+	}
+
 	if (dev->power.runtime_status == RPM_RESUMING
 	    || dev->power.runtime_status == RPM_SUSPENDING) {
 		DEFINE_WAIT(wait);
@@ -591,11 +608,7 @@ static int rpm_resume(struct device *dev
 
 	/* Carry out an asynchronous or a synchronous resume. */
 	if (rpmflags & RPM_ASYNC) {
-		dev->power.request = RPM_REQ_RESUME;
-		if (!dev->power.request_pending) {
-			dev->power.request_pending = true;
-			queue_work(pm_wq, &dev->power.work);
-		}
+		rpm_queue_up_resume(dev);
 		retval = 0;
 		goto out;
 	}
@@ -691,6 +704,7 @@ static int rpm_resume(struct device *dev
 static void pm_runtime_work(struct work_struct *work)
 {
 	struct device *dev = container_of(work, struct device, power.work);
+	void (*func)(struct device *) = NULL;
 	enum rpm_request req;
 
 	spin_lock_irq(&dev->power.lock);
@@ -715,12 +729,24 @@ static void pm_runtime_work(struct work_
 		rpm_suspend(dev, RPM_NOWAIT | RPM_AUTO);
 		break;
 	case RPM_REQ_RESUME:
-		rpm_resume(dev, RPM_NOWAIT);
+		func = dev->power.func;
+		if (func) {
+			dev->power.func = NULL;
+			pm_runtime_get_noresume(dev);
+			rpm_resume(dev, 0);
+		} else {
+			rpm_resume(dev, RPM_NOWAIT);
+		}
 		break;
 	}
 
  out:
 	spin_unlock_irq(&dev->power.lock);
+
+	if (func) {
+		func(dev);
+		pm_runtime_put(dev);
+	}
 }
 
 /**
@@ -877,6 +903,48 @@ int __pm_runtime_resume(struct device *d
 }
 EXPORT_SYMBOL_GPL(__pm_runtime_resume);
 
+int pm_runtime_get_and_call(struct device *dev, void (*func)(struct device *),
+			     void (*func_async)(struct device *))
+{
+	unsigned long flags;
+	int ret;
+
+	atomic_inc(&dev->power.usage_count);
+
+	spin_lock_irqsave(&dev->power.lock, flags);
+
+	ret = dev->power.runtime_error;
+	if (ret) {
+		func = NULL;
+		goto out;
+	}
+
+	if (dev->power.runtime_status != RPM_ACTIVE
+	    && dev->power.disable_depth == 0)
+		func = NULL;
+
+	if (!func && func_async) {
+		if (dev->power.func) {
+			ret = -EAGAIN;
+			goto out;
+		} else {
+			dev->power.func = func_async;
+		}
+	}
+
+	rpm_resume(dev, RPM_ASYNC);
+
+ out:
+	spin_unlock_irqrestore(&dev->power.lock, flags);
+
+	if (func) {
+		func(dev);
+		return 1;
+	}
+
+	return ret;
+}
+
 /**
  * __pm_runtime_set_status - Set runtime PM status of a device.
  * @dev: Device to handle.
Index: linux/include/linux/pm.h
===================================================================
--- linux.orig/include/linux/pm.h
+++ linux/include/linux/pm.h
@@ -547,6 +547,7 @@ struct dev_pm_info {
 	unsigned long		suspended_jiffies;
 	unsigned long		accounting_timestamp;
 	struct dev_pm_qos_request *pq_req;
+	void			(*func)(struct device *);
 #endif
 	struct pm_subsys_data	*subsys_data;  /* Owned by the subsystem. */
 	struct pm_qos_constraints *constraints;
Index: linux/include/linux/pm_runtime.h
===================================================================
--- linux.orig/include/linux/pm_runtime.h
+++ linux/include/linux/pm_runtime.h
@@ -47,6 +47,9 @@ extern void pm_runtime_set_autosuspend_d
 extern unsigned long pm_runtime_autosuspend_expiration(struct device *dev);
 extern void pm_runtime_update_max_time_suspended(struct device *dev,
 						 s64 delta_ns);
+extern int pm_runtime_get_and_call(struct device *dev,
+				   void (*func)(struct device *),
+				   void (*func_async)(struct device *));
 
 static inline bool pm_children_suspended(struct device *dev)
 {
@@ -150,6 +153,17 @@ static inline void pm_runtime_set_autosu
 static inline unsigned long pm_runtime_autosuspend_expiration(
 				struct device *dev) { return 0; }
 
+static inline int pm_runtime_get_and_call(struct device *dev,
+					  void (*func)(struct device *),
+					  void (*func_async)(struct device *))
+{
+	if (func) {
+		func(dev);
+		return 1;
+	}
+	return 0;
+}
+
 #endif /* !CONFIG_PM_RUNTIME */
 
 static inline int pm_runtime_idle(struct device *dev)

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-05 15:26                                                                     ` Rafael J. Wysocki
@ 2012-08-06 13:30                                                                       ` Ming Lei
  2012-08-06 14:47                                                                         ` Alan Stern
  2012-08-06 15:48                                                                       ` Alan Stern
  1 sibling, 1 reply; 89+ messages in thread
From: Ming Lei @ 2012-08-06 13:30 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Sun, Aug 5, 2012 at 11:26 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:

> ---
>  drivers/base/power/runtime.c |   82 +++++++++++++++++++++++++++++++++++++++----
>  include/linux/pm.h           |    1
>  include/linux/pm_runtime.h   |   14 +++++++
>  3 files changed, 90 insertions(+), 7 deletions(-)
>
> Index: linux/drivers/base/power/runtime.c
> ===================================================================
> --- linux.orig/drivers/base/power/runtime.c
> +++ linux/drivers/base/power/runtime.c
> @@ -484,6 +484,15 @@ static int rpm_suspend(struct device *de
>         goto out;
>  }
>
> +void rpm_queue_up_resume(struct device *dev)
> +{
> +       dev->power.request = RPM_REQ_RESUME;
> +       if (!dev->power.request_pending) {
> +               dev->power.request_pending = true;
> +               queue_work(pm_wq, &dev->power.work);
> +       }
> +}
> +
>  /**
>   * rpm_resume - Carry out runtime resume of given device.
>   * @dev: Device to resume.
> @@ -524,7 +533,9 @@ static int rpm_resume(struct device *dev
>          * rather than cancelling it now only to restart it again in the near
>          * future.
>          */
> -       dev->power.request = RPM_REQ_NONE;
> +       if (dev->power.request != RPM_REQ_RESUME || !dev->power.func)
> +               dev->power.request = RPM_REQ_NONE;
> +
>         if (!dev->power.timer_autosuspends)
>                 pm_runtime_deactivate_timer(dev);
>
> @@ -533,6 +544,12 @@ static int rpm_resume(struct device *dev
>                 goto out;
>         }
>
> +       if (dev->power.func && (rpmflags & RPM_ASYNC)) {
> +               rpm_queue_up_resume(dev);
> +               retval = 0;
> +               goto out;
> +       }
> +
>         if (dev->power.runtime_status == RPM_RESUMING
>             || dev->power.runtime_status == RPM_SUSPENDING) {
>                 DEFINE_WAIT(wait);
> @@ -591,11 +608,7 @@ static int rpm_resume(struct device *dev
>
>         /* Carry out an asynchronous or a synchronous resume. */
>         if (rpmflags & RPM_ASYNC) {
> -               dev->power.request = RPM_REQ_RESUME;
> -               if (!dev->power.request_pending) {
> -                       dev->power.request_pending = true;
> -                       queue_work(pm_wq, &dev->power.work);
> -               }
> +               rpm_queue_up_resume(dev);
>                 retval = 0;
>                 goto out;
>         }
> @@ -691,6 +704,7 @@ static int rpm_resume(struct device *dev
>  static void pm_runtime_work(struct work_struct *work)
>  {
>         struct device *dev = container_of(work, struct device, power.work);
> +       void (*func)(struct device *) = NULL;
>         enum rpm_request req;
>
>         spin_lock_irq(&dev->power.lock);
> @@ -715,12 +729,24 @@ static void pm_runtime_work(struct work_
>                 rpm_suspend(dev, RPM_NOWAIT | RPM_AUTO);
>                 break;
>         case RPM_REQ_RESUME:
> -               rpm_resume(dev, RPM_NOWAIT);
> +               func = dev->power.func;
> +               if (func) {
> +                       dev->power.func = NULL;
> +                       pm_runtime_get_noresume(dev);
> +                       rpm_resume(dev, 0);
> +               } else {
> +                       rpm_resume(dev, RPM_NOWAIT);
> +               }
>                 break;
>         }
>
>   out:
>         spin_unlock_irq(&dev->power.lock);
> +
> +       if (func) {
> +               func(dev);
> +               pm_runtime_put(dev);
> +       }
>  }
>
>  /**
> @@ -877,6 +903,48 @@ int __pm_runtime_resume(struct device *d
>  }
>  EXPORT_SYMBOL_GPL(__pm_runtime_resume);
>
> +int pm_runtime_get_and_call(struct device *dev, void (*func)(struct device *),
> +                            void (*func_async)(struct device *))
> +{
> +       unsigned long flags;
> +       int ret;
> +
> +       atomic_inc(&dev->power.usage_count);
> +
> +       spin_lock_irqsave(&dev->power.lock, flags);
> +
> +       ret = dev->power.runtime_error;
> +       if (ret) {
> +               func = NULL;
> +               goto out;
> +       }
> +
> +       if (dev->power.runtime_status != RPM_ACTIVE
> +           && dev->power.disable_depth == 0)
> +               func = NULL;

Looks the above is a bit odd, and your motivation is to call 'func'
for a suspended and runtime-PM enabled device in irq context, isn't it?

> +
> +       if (!func && func_async) {
> +               if (dev->power.func) {
> +                       ret = -EAGAIN;
> +                       goto out;
> +               } else {
> +                       dev->power.func = func_async;
> +               }
> +       }
> +
> +       rpm_resume(dev, RPM_ASYNC);
> +
> + out:
> +       spin_unlock_irqrestore(&dev->power.lock, flags);
> +
> +       if (func) {
> +               func(dev);

Maybe the return value should be passed to caller. Also the race between
'func' and its .runtime_resume callback should be stated in comment.

In fact, maybe it is better to call 'func' always first, then call
' rpm_resume(dev, RPM_ASYNC);', otherwise the driver may
be confused about the order between 'func' and its .runtime_resume
callback.

> +               return 1;
> +       }
> +
> +       return ret;
> +}
> +
>  /**
>   * __pm_runtime_set_status - Set runtime PM status of a device.
>   * @dev: Device to handle.
> Index: linux/include/linux/pm.h
> ===================================================================
> --- linux.orig/include/linux/pm.h
> +++ linux/include/linux/pm.h
> @@ -547,6 +547,7 @@ struct dev_pm_info {
>         unsigned long           suspended_jiffies;
>         unsigned long           accounting_timestamp;
>         struct dev_pm_qos_request *pq_req;
> +       void                    (*func)(struct device *);

Another way is to define 'func' as 'runtime_pre_resume'
in 'struct dev_pm_ops', and there are some advantages about this way:

        - save one pointer in 'struct devices, since most of devices
don't need the 'func'
        - well documents on 'runtime_pre_resume'
        - caller of pm_runtime_get_and_call may be happier, maybe just
        pm_runtime_get or *_aync is enough.


Thanks,
-- 
Ming Lei

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-06 13:30                                                                       ` Ming Lei
@ 2012-08-06 14:47                                                                         ` Alan Stern
  2012-08-07  1:35                                                                           ` Ming Lei
  0 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-08-06 14:47 UTC (permalink / raw)
  To: Ming Lei; +Cc: Rafael J. Wysocki, linux-pci, USB list, Linux PM list

On Mon, 6 Aug 2012, Ming Lei wrote:

> Maybe the return value should be passed to caller. Also the race between
> 'func' and its .runtime_resume callback should be stated in comment.
> 
> In fact, maybe it is better to call 'func' always first, then call
> ' rpm_resume(dev, RPM_ASYNC);', otherwise the driver may
> be confused about the order between 'func' and its .runtime_resume
> callback.

> Another way is to define 'func' as 'runtime_pre_resume'
> in 'struct dev_pm_ops', and there are some advantages about this way:
> 
>         - save one pointer in 'struct devices, since most of devices
> don't need the 'func'
>         - well documents on 'runtime_pre_resume'
>         - caller of pm_runtime_get_and_call may be happier, maybe just
>         pm_runtime_get or *_aync is enough.

No, no, you have completely misunderstood the whole point of this 
change.

The idea is for "func" to be called at a time when it is known that the 
device is at full power.  That means it _has_ to be called after the 
runtime_resume callback returns.

Also, "func" should not be stored in dev_pm_ops because it won't be a 
read-only value.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-05 15:26                                                                     ` Rafael J. Wysocki
  2012-08-06 13:30                                                                       ` Ming Lei
@ 2012-08-06 15:48                                                                       ` Alan Stern
  2012-08-06 20:30                                                                         ` Rafael J. Wysocki
  1 sibling, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-08-06 15:48 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: Ming Lei, linux-pci, USB list, Linux PM list

On Sun, 5 Aug 2012, Rafael J. Wysocki wrote:

> I don't really think we'll need to store func_sync in dev->power.  At least
> I don't see a reason to do that at the moment.

You're right; I wasn't thinking straight.

> I also think that func_sync() would have to be called if runtime PM is
> disabled for the given device, so that callers don't have to check that
> condition themselves.

Yes.

> What about the appended experimental patch?

For now, I think it would be best to start with a single func argument.  
If it turns out that anyone really needs to have two separate
arguments, the single-argument form can be reimplemented on top of the
two-argument form easily enough.

> @@ -484,6 +484,15 @@ static int rpm_suspend(struct device *de
>  	goto out;
>  }
>  
> +void rpm_queue_up_resume(struct device *dev)
> +{
> +	dev->power.request = RPM_REQ_RESUME;
> +	if (!dev->power.request_pending) {
> +		dev->power.request_pending = true;
> +		queue_work(pm_wq, &dev->power.work);
> +	}
> +}
> +
>  /**
>   * rpm_resume - Carry out runtime resume of given device.
>   * @dev: Device to resume.
> @@ -524,7 +533,9 @@ static int rpm_resume(struct device *dev
>  	 * rather than cancelling it now only to restart it again in the near
>  	 * future.
>  	 */
> -	dev->power.request = RPM_REQ_NONE;
> +	if (dev->power.request != RPM_REQ_RESUME || !dev->power.func)
> +		dev->power.request = RPM_REQ_NONE;
> +
>  	if (!dev->power.timer_autosuspends)
>  		pm_runtime_deactivate_timer(dev);
>  
> @@ -533,6 +544,12 @@ static int rpm_resume(struct device *dev
>  		goto out;
>  	}
>  
> +	if (dev->power.func && (rpmflags & RPM_ASYNC)) {
> +		rpm_queue_up_resume(dev);
> +		retval = 0;
> +		goto out;
> +	}
> +
>  	if (dev->power.runtime_status == RPM_RESUMING
>  	    || dev->power.runtime_status == RPM_SUSPENDING) {
>  		DEFINE_WAIT(wait);

All those changes (and some of the following ones) are symptoms of a
basic mistake in this approach.  The idea of this new feature is to
call "func" as soon as we know the device is at full power, no matter
how it got there.  That means we should call it near the end of
rpm_resume() (just before the rpm_idle() call), not from within
pm_runtime_work().

Doing it this way will be more efficient and (I think) will remove
some races.

> @@ -877,6 +903,48 @@ int __pm_runtime_resume(struct device *d
>  }
>  EXPORT_SYMBOL_GPL(__pm_runtime_resume);
>  
> +int pm_runtime_get_and_call(struct device *dev, void (*func)(struct device *),
> +			     void (*func_async)(struct device *))
> +{
> +	unsigned long flags;
> +	int ret;
> +
> +	atomic_inc(&dev->power.usage_count);
> +
> +	spin_lock_irqsave(&dev->power.lock, flags);
> +
> +	ret = dev->power.runtime_error;
> +	if (ret) {
> +		func = NULL;
> +		goto out;
> +	}
> +
> +	if (dev->power.runtime_status != RPM_ACTIVE
> +	    && dev->power.disable_depth == 0)
> +		func = NULL;
> +
> +	if (!func && func_async) {
> +		if (dev->power.func) {
> +			ret = -EAGAIN;
> +			goto out;
> +		} else {
> +			dev->power.func = func_async;
> +		}
> +	}

The logic here is kind of hard to follow.  It will be simpler when
there's only one "func":

	If the status is RPM_ACTIVE or disable_depth > 0 then
	call "func" directly.

	Otherwise save "func" in dev.power and do an async resume.

Some more things:

In __pm_runtime_set_status(), if power.func is set then I think we
should call it if the new status is ACTIVE.  Likwise at the end of
rpm_suspend(), if the suspend fails.  There should be an invariant:
Whenever the status is RPM_ACTIVE, power.func must be NULL.

pm_runtime_barrier() should always clear power.func, even if the 
rpm_resume() call fails.

The documentation should explain that in some ways, "func" is like a
workqueue callback routine: Subsystems and drivers have to be careful
to make sure that it can't be invoked after the device is unbound.  A
safe way to do this is to call pm_runtime_barrier() from within the
driver's .remove() routine, after making sure that
pm_runtime_get_and_call() won't be invoked any more.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-06 15:48                                                                       ` Alan Stern
@ 2012-08-06 20:30                                                                         ` Rafael J. Wysocki
  2012-08-07 12:28                                                                           ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-06 20:30 UTC (permalink / raw)
  To: Alan Stern; +Cc: Ming Lei, linux-pci, USB list, Linux PM list

On Monday, August 06, 2012, Alan Stern wrote:
> On Sun, 5 Aug 2012, Rafael J. Wysocki wrote:
[...]
> 
> > What about the appended experimental patch?
> 
> For now, I think it would be best to start with a single func argument.  
> If it turns out that anyone really needs to have two separate
> arguments, the single-argument form can be reimplemented on top of the
> two-argument form easily enough.

All right.

> > @@ -484,6 +484,15 @@ static int rpm_suspend(struct device *de
> >  	goto out;
> >  }
> >  
> > +void rpm_queue_up_resume(struct device *dev)
> > +{
> > +	dev->power.request = RPM_REQ_RESUME;
> > +	if (!dev->power.request_pending) {
> > +		dev->power.request_pending = true;
> > +		queue_work(pm_wq, &dev->power.work);
> > +	}
> > +}
> > +
> >  /**
> >   * rpm_resume - Carry out runtime resume of given device.
> >   * @dev: Device to resume.
> > @@ -524,7 +533,9 @@ static int rpm_resume(struct device *dev
> >  	 * rather than cancelling it now only to restart it again in the near
> >  	 * future.
> >  	 */
> > -	dev->power.request = RPM_REQ_NONE;
> > +	if (dev->power.request != RPM_REQ_RESUME || !dev->power.func)
> > +		dev->power.request = RPM_REQ_NONE;
> > +
> >  	if (!dev->power.timer_autosuspends)
> >  		pm_runtime_deactivate_timer(dev);
> >  
> > @@ -533,6 +544,12 @@ static int rpm_resume(struct device *dev
> >  		goto out;
> >  	}
> >  
> > +	if (dev->power.func && (rpmflags & RPM_ASYNC)) {
> > +		rpm_queue_up_resume(dev);
> > +		retval = 0;
> > +		goto out;
> > +	}
> > +
> >  	if (dev->power.runtime_status == RPM_RESUMING
> >  	    || dev->power.runtime_status == RPM_SUSPENDING) {
> >  		DEFINE_WAIT(wait);
> 
> All those changes (and some of the following ones) are symptoms of a
> basic mistake in this approach.

Every time you say something like this (i.e. liks someone who knows better)
I kind of feel like being under attach, which I hope is not your intention.
Never mind. :-)

Those changes are there for different reasons rather unrelated to the way
func() is being called, so let me explain.

First, rpm_queue_up_resume() is introduced, because the code it contains will
have to be called in two different places.  At least I don't see how to avoid
that without increasing the code complexity too much.

Second, the following change in rpm_resume()

-	dev->power.request = RPM_REQ_NONE;
+	if (dev->power.request != RPM_REQ_RESUME || !dev->power.func)
+		dev->power.request = RPM_REQ_NONE;

is made to prevent rpm_resume() from canceling the execution of func() queued
up by an earlier pm_runtime_get_and_call().  I believe it is necessary.

Finally, this change in rpm_resume():
 
+	if (dev->power.func && (rpmflags & RPM_ASYNC)) {
+		rpm_queue_up_resume(dev);
+		retval = 0;
+		goto out;
+	}

is not strictly necessary if pm_runtime_get_and_call() is modified to run
rpm_queue_up_resume() directly, like in the new version of the patch which is
appended.  This reduces the number of checks overall, so perhaps it's better.

> The idea of this new feature is to
> call "func" as soon as we know the device is at full power, no matter
> how it got there.

Yes, it is so.

> That means we should call it near the end of
> rpm_resume() (just before the rpm_idle() call), not from within
> pm_runtime_work().
> 
> Doing it this way will be more efficient and (I think) will remove
> some races.

Except that func() shouldn't be executed under dev->power.lock, which makes it
rather difficult to call it from rpm_resume().  Or at least it seems so.

Moreover, it should be called after we've changed the status to RPM_ACTIVE
_and_ dropped the lock.

Besides, I'd like to know what races you're referring to (perhaps you're seeing
some more of them than I am).

> > @@ -877,6 +903,48 @@ int __pm_runtime_resume(struct device *d
> >  }
> >  EXPORT_SYMBOL_GPL(__pm_runtime_resume);
> >  
> > +int pm_runtime_get_and_call(struct device *dev, void (*func)(struct device *),
> > +			     void (*func_async)(struct device *))
> > +{
> > +	unsigned long flags;
> > +	int ret;
> > +
> > +	atomic_inc(&dev->power.usage_count);
> > +
> > +	spin_lock_irqsave(&dev->power.lock, flags);
> > +
> > +	ret = dev->power.runtime_error;
> > +	if (ret) {
> > +		func = NULL;
> > +		goto out;
> > +	}
> > +
> > +	if (dev->power.runtime_status != RPM_ACTIVE
> > +	    && dev->power.disable_depth == 0)
> > +		func = NULL;
> > +
> > +	if (!func && func_async) {
> > +		if (dev->power.func) {
> > +			ret = -EAGAIN;
> > +			goto out;
> > +		} else {
> > +			dev->power.func = func_async;
> > +		}
> > +	}
> 
> The logic here is kind of hard to follow.  It will be simpler when
> there's only one "func":
> 
> 	If the status is RPM_ACTIVE or disable_depth > 0 then
> 	call "func" directly.
> 
> 	Otherwise save "func" in dev.power and do an async resume.

Yeah.  But do not run func() under power.lock, right?

> Some more things:
> 
> In __pm_runtime_set_status(), if power.func is set then I think we
> should call it if the new status is ACTIVE.

__pm_runtime_set_status() only works if power.disable_depth > 0, in
which case func() will be called synchronously.  Now, if someone does
pm_runtime_get_and_call() immediately followed by pm_runtime_disable()
(perhaps from a different thread), then __pm_runtime_disable() will
resume the device, so it's better to call func() from there if set,
I think.  And clear power.func afterwards.

> Likwise at the end of rpm_suspend(), if the suspend fails.

The only way by which power.func can be different from NULL at that point
is when the work item queued up by pm_runtime_get_and_call() runs after
rpm_suspend() has changed the status to RPM_SUSPENDING.  However in that
case the rpm_resume(dev, 0) in pm_runtime_work() will wait for the
suspend to complete (or fail) and func() will be run when it returns.

> There should be an invariant: Whenever the status is RPM_ACTIVE,
> power.func must be NULL.

Well, maybe so, but I don't see why right now.

> pm_runtime_barrier() should always clear power.func, even if the 
> rpm_resume() call fails.

Wouldn't it be better to wait for func() to be run?

> The documentation should explain that in some ways, "func" is like a
> workqueue callback routine: Subsystems and drivers have to be careful
> to make sure that it can't be invoked after the device is unbound.  A
> safe way to do this is to call pm_runtime_barrier() from within the
> driver's .remove() routine, after making sure that
> pm_runtime_get_and_call() won't be invoked any more.

Sure.

The appended patch doesn't contain any changes in __pm_runtime_disable()
or pm_runtime_barrier(), but you are right that they are necessary.

Thanks,
Rafael


---
 drivers/base/power/runtime.c |   98 ++++++++++++++++++++++++++++++++++++++-----
 include/linux/pm.h           |    1 
 include/linux/pm_runtime.h   |   11 ++++
 3 files changed, 99 insertions(+), 11 deletions(-)

Index: linux/drivers/base/power/runtime.c
===================================================================
--- linux.orig/drivers/base/power/runtime.c
+++ linux/drivers/base/power/runtime.c
@@ -484,6 +484,15 @@ static int rpm_suspend(struct device *de
 	goto out;
 }
 
+void rpm_queue_up_resume(struct device *dev)
+{
+	dev->power.request = RPM_REQ_RESUME;
+	if (!dev->power.request_pending) {
+		dev->power.request_pending = true;
+		queue_work(pm_wq, &dev->power.work);
+	}
+}
+
 /**
  * rpm_resume - Carry out runtime resume of given device.
  * @dev: Device to resume.
@@ -519,12 +528,18 @@ static int rpm_resume(struct device *dev
 		goto out;
 
 	/*
-	 * Other scheduled or pending requests need to be canceled.  Small
-	 * optimization: If an autosuspend timer is running, leave it running
-	 * rather than cancelling it now only to restart it again in the near
-	 * future.
+	 * Other scheduled or pending requests need to be canceled.  If the
+	 * execution of a function is queued up along with a resume request,
+	 * do not cancel it.
+	 */
+	if (dev->power.request != RPM_REQ_RESUME || !dev->power.func)
+		dev->power.request = RPM_REQ_NONE;
+
+	/*
+	 * Small optimization: If an autosuspend timer is running, leave it
+	 * running rather than cancelling it now only to restart it again in the
+	 * near future.
 	 */
-	dev->power.request = RPM_REQ_NONE;
 	if (!dev->power.timer_autosuspends)
 		pm_runtime_deactivate_timer(dev);
 
@@ -591,11 +606,7 @@ static int rpm_resume(struct device *dev
 
 	/* Carry out an asynchronous or a synchronous resume. */
 	if (rpmflags & RPM_ASYNC) {
-		dev->power.request = RPM_REQ_RESUME;
-		if (!dev->power.request_pending) {
-			dev->power.request_pending = true;
-			queue_work(pm_wq, &dev->power.work);
-		}
+		rpm_queue_up_resume(dev);
 		retval = 0;
 		goto out;
 	}
@@ -691,6 +702,7 @@ static int rpm_resume(struct device *dev
 static void pm_runtime_work(struct work_struct *work)
 {
 	struct device *dev = container_of(work, struct device, power.work);
+	void (*func)(struct device *) = NULL;
 	enum rpm_request req;
 
 	spin_lock_irq(&dev->power.lock);
@@ -715,12 +727,24 @@ static void pm_runtime_work(struct work_
 		rpm_suspend(dev, RPM_NOWAIT | RPM_AUTO);
 		break;
 	case RPM_REQ_RESUME:
-		rpm_resume(dev, RPM_NOWAIT);
+		func = dev->power.func;
+		if (func) {
+			dev->power.func = NULL;
+			pm_runtime_get_noresume(dev);
+			rpm_resume(dev, 0);
+		} else {
+			rpm_resume(dev, RPM_NOWAIT);
+		}
 		break;
 	}
 
  out:
 	spin_unlock_irq(&dev->power.lock);
+
+	if (func) {
+		func(dev);
+		pm_runtime_put(dev);
+	}
 }
 
 /**
@@ -877,6 +901,58 @@ int __pm_runtime_resume(struct device *d
 }
 EXPORT_SYMBOL_GPL(__pm_runtime_resume);
 
+int pm_runtime_get_and_call(struct device *dev, void (*func)(struct device *))
+{
+	unsigned long flags;
+	bool sync = false;
+	int ret;
+
+	atomic_inc(&dev->power.usage_count);
+
+	spin_lock_irqsave(&dev->power.lock, flags);
+
+	ret = dev->power.runtime_error;
+	if (ret)
+		goto out;
+
+	sync = dev->power.runtime_status == RPM_ACTIVE
+		|| dev->power.disable_depth > 0;
+
+	if (!sync) {
+		if (dev->power.func) {
+			ret = -EAGAIN;
+			goto out;
+		} else {
+			dev->power.func = func;
+		}
+	}
+
+	/*
+	 * The approach here is the same as in rpm_suspend(): autosuspend timers
+	 * will be activated shortly anyway, so it is pointless to cancel them
+	 * now.
+	 */
+	if (!dev->power.timer_autosuspends)
+		pm_runtime_deactivate_timer(dev);
+
+	if (sync)
+		dev->power.request = RPM_REQ_NONE;
+	else
+		rpm_queue_up_resume(dev);
+
+ out:
+	spin_unlock_irqrestore(&dev->power.lock, flags);
+
+	if (sync) {
+		if (func)
+			func(dev);
+
+		return 1;
+	}
+
+	return ret;
+}
+
 /**
  * __pm_runtime_set_status - Set runtime PM status of a device.
  * @dev: Device to handle.
Index: linux/include/linux/pm.h
===================================================================
--- linux.orig/include/linux/pm.h
+++ linux/include/linux/pm.h
@@ -547,6 +547,7 @@ struct dev_pm_info {
 	unsigned long		suspended_jiffies;
 	unsigned long		accounting_timestamp;
 	struct dev_pm_qos_request *pq_req;
+	void			(*func)(struct device *);
 #endif
 	struct pm_subsys_data	*subsys_data;  /* Owned by the subsystem. */
 	struct pm_qos_constraints *constraints;
Index: linux/include/linux/pm_runtime.h
===================================================================
--- linux.orig/include/linux/pm_runtime.h
+++ linux/include/linux/pm_runtime.h
@@ -47,6 +47,8 @@ extern void pm_runtime_set_autosuspend_d
 extern unsigned long pm_runtime_autosuspend_expiration(struct device *dev);
 extern void pm_runtime_update_max_time_suspended(struct device *dev,
 						 s64 delta_ns);
+extern int pm_runtime_get_and_call(struct device *dev,
+				   void (*func)(struct device *));
 
 static inline bool pm_children_suspended(struct device *dev)
 {
@@ -150,6 +152,15 @@ static inline void pm_runtime_set_autosu
 static inline unsigned long pm_runtime_autosuspend_expiration(
 				struct device *dev) { return 0; }
 
+static inline int pm_runtime_get_and_call(struct device *dev,
+					  void (*func)(struct device *))
+{
+	if (func)
+		func(dev);
+
+	return 1;
+}
+
 #endif /* !CONFIG_PM_RUNTIME */
 
 static inline int pm_runtime_idle(struct device *dev)


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-06 14:47                                                                         ` Alan Stern
@ 2012-08-07  1:35                                                                           ` Ming Lei
  2012-08-07 11:23                                                                             ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Ming Lei @ 2012-08-07  1:35 UTC (permalink / raw)
  To: Alan Stern; +Cc: Rafael J. Wysocki, linux-pci, USB list, Linux PM list

On Mon, Aug 6, 2012 at 10:47 PM, Alan Stern <stern@rowland.harvard.edu> wrote:
> No, no, you have completely misunderstood the whole point of this
> change.

Sorry, you are right. And the callback should be renamed as
'.runtime_post_resume', which should do something I/O related in
theory just after device becomes active.

>
> The idea is for "func" to be called at a time when it is known that the
> device is at full power.  That means it _has_ to be called after the
> runtime_resume callback returns.

Yes, I agree, but I don't think it may make .runtime_post_resume
not doable, do I?

>
> Also, "func" should not be stored in dev_pm_ops because it won't be a
> read-only value.

Sorry, could you explain it in detail that why the 'func'
has to switch to multiple functions dynamically? I understand
one function is enough and sometimes it can be bypassed with
one flag(such as, ignore_post_resume is introduced in dev_pm_info)
set.  Also, the driver can store the device specific states
in its own device instance to deal with different situations in the callback.

If the idea is doable, we can save one pointer in 'struct device',
since the 'func' may not be used by more than 90% devices, also
have document benefit, even may simplify implementation of the
mechanism.


Thanks,
-- 
Ming Lei

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-07  1:35                                                                           ` Ming Lei
@ 2012-08-07 11:23                                                                             ` Rafael J. Wysocki
  2012-08-07 15:14                                                                               ` Ming Lei
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-07 11:23 UTC (permalink / raw)
  To: Ming Lei; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Tuesday, August 07, 2012, Ming Lei wrote:
> On Mon, Aug 6, 2012 at 10:47 PM, Alan Stern <stern@rowland.harvard.edu> wrote:
> > No, no, you have completely misunderstood the whole point of this
> > change.
> 
> Sorry, you are right. And the callback should be renamed as
> '.runtime_post_resume', which should do something I/O related in
> theory just after device becomes active.
> 
> >
> > The idea is for "func" to be called at a time when it is known that the
> > device is at full power.  That means it _has_ to be called after the
> > runtime_resume callback returns.
> 
> Yes, I agree, but I don't think it may make .runtime_post_resume
> not doable, do I?

No more device PM callbacks, please.

Besides, callbacks in struct dev_pm_ops are not only for drivers.

> > Also, "func" should not be stored in dev_pm_ops because it won't be a
> > read-only value.
> 
> Sorry, could you explain it in detail that why the 'func'
> has to switch to multiple functions dynamically? I understand
> one function is enough and sometimes it can be bypassed with
> one flag(such as, ignore_post_resume is introduced in dev_pm_info)
> set.  Also, the driver can store the device specific states
> in its own device instance to deal with different situations in the callback.
> 
> If the idea is doable, we can save one pointer in 'struct device',
> since the 'func' may not be used by more than 90% devices, also
> have document benefit, even may simplify implementation of the
> mechanism.

And how many struct device objects there are for the one extra pointer to
matter, let alone the fact that you want to replace it by one extra pointer
somewhere else?

Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-06 20:30                                                                         ` Rafael J. Wysocki
@ 2012-08-07 12:28                                                                           ` Rafael J. Wysocki
  2012-08-07 17:15                                                                             ` Alan Stern
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-07 12:28 UTC (permalink / raw)
  To: Alan Stern; +Cc: Ming Lei, linux-pci, USB list, Linux PM list

On Monday, August 06, 2012, Rafael J. Wysocki wrote:
> On Monday, August 06, 2012, Alan Stern wrote:
> > On Sun, 5 Aug 2012, Rafael J. Wysocki wrote:
> [...]
> > 
> > > What about the appended experimental patch?
> > 
> > For now, I think it would be best to start with a single func argument.  
> > If it turns out that anyone really needs to have two separate
> > arguments, the single-argument form can be reimplemented on top of the
> > two-argument form easily enough.
> 
> All right.
> 
> > > @@ -484,6 +484,15 @@ static int rpm_suspend(struct device *de
> > >  	goto out;
> > >  }
> > >  
> > > +void rpm_queue_up_resume(struct device *dev)
> > > +{
> > > +	dev->power.request = RPM_REQ_RESUME;
> > > +	if (!dev->power.request_pending) {
> > > +		dev->power.request_pending = true;
> > > +		queue_work(pm_wq, &dev->power.work);
> > > +	}
> > > +}
> > > +
> > >  /**
> > >   * rpm_resume - Carry out runtime resume of given device.
> > >   * @dev: Device to resume.
> > > @@ -524,7 +533,9 @@ static int rpm_resume(struct device *dev
> > >  	 * rather than cancelling it now only to restart it again in the near
> > >  	 * future.
> > >  	 */
> > > -	dev->power.request = RPM_REQ_NONE;
> > > +	if (dev->power.request != RPM_REQ_RESUME || !dev->power.func)
> > > +		dev->power.request = RPM_REQ_NONE;
> > > +
> > >  	if (!dev->power.timer_autosuspends)
> > >  		pm_runtime_deactivate_timer(dev);
> > >  
> > > @@ -533,6 +544,12 @@ static int rpm_resume(struct device *dev
> > >  		goto out;
> > >  	}
> > >  
> > > +	if (dev->power.func && (rpmflags & RPM_ASYNC)) {
> > > +		rpm_queue_up_resume(dev);
> > > +		retval = 0;
> > > +		goto out;
> > > +	}
> > > +
> > >  	if (dev->power.runtime_status == RPM_RESUMING
> > >  	    || dev->power.runtime_status == RPM_SUSPENDING) {
> > >  		DEFINE_WAIT(wait);
> > 
> > All those changes (and some of the following ones) are symptoms of a
> > basic mistake in this approach.
> 
> Every time you say something like this (i.e. liks someone who knows better)

s/liks/like/

> I kind of feel like being under attach, which I hope is not your intention.

s/attach/attack/

Two typos in one sentence, I guess it could have been worse ...

> Never mind. :-)
> 
> Those changes are there for different reasons rather unrelated to the way
> func() is being called, so let me explain.
> 
> First, rpm_queue_up_resume() is introduced, because the code it contains will
> have to be called in two different places.  At least I don't see how to avoid
> that without increasing the code complexity too much.
> 
> Second, the following change in rpm_resume()
> 
> -	dev->power.request = RPM_REQ_NONE;
> +	if (dev->power.request != RPM_REQ_RESUME || !dev->power.func)
> +		dev->power.request = RPM_REQ_NONE;
> 
> is made to prevent rpm_resume() from canceling the execution of func() queued
> up by an earlier pm_runtime_get_and_call().  I believe it is necessary.
> 
> Finally, this change in rpm_resume():
>  
> +	if (dev->power.func && (rpmflags & RPM_ASYNC)) {
> +		rpm_queue_up_resume(dev);
> +		retval = 0;
> +		goto out;
> +	}
> 
> is not strictly necessary if pm_runtime_get_and_call() is modified to run
> rpm_queue_up_resume() directly, like in the new version of the patch which is
> appended.  This reduces the number of checks overall, so perhaps it's better.
> 
> > The idea of this new feature is to
> > call "func" as soon as we know the device is at full power, no matter
> > how it got there.
> 
> Yes, it is so.
> 
> > That means we should call it near the end of
> > rpm_resume() (just before the rpm_idle() call), not from within
> > pm_runtime_work().
> > 
> > Doing it this way will be more efficient and (I think) will remove
> > some races.
> 
> Except that func() shouldn't be executed under dev->power.lock, which makes it
> rather difficult to call it from rpm_resume().  Or at least it seems so.
> 
> Moreover, it should be called after we've changed the status to RPM_ACTIVE
> _and_ dropped the lock.

So we could drop the lock right before returning, execute func() and acquire
the lock once again, but then func() might be executed by any thread that
happened to resume the device.  In that case the caller of
pm_runtime_get_and_call() would have to worry about locks that such threads
might acquire and it would have to make sure that func() didn't try to acquire
them too.  That may not be a big deal, but if func() is executed by
pm_runtime_work(), that issue simply goes away.

Then, however, there's another issue: what should happen if
pm_runtime_get_and_call() finds that it cannot execute func() right away,
so it queues up resume and the execution of it, in the meantime some other
thread resumes the device synchronously and pm_runtime_get_and_call() is
run again.  I think in that case func() should be executed synchronously
and the one waiting for execution should be canceled.  The alternative
would be to return -EAGAIN from pm_runtime_get_and_call() and expect the
caller to cope with that, which isn't too attractive.

This actually is analogous to the case when pm_runtime_get_and_call()
sees that power.func is not NULL.  In my experimental patches it returned
-EAGAIN in that case, but perhaps it's better to replace the existing
power.func with the new one.  Then, by doing pm_runtime_get_and_call(dev, NULL)
we can ensure that either the previous func() has run already or it will never
run, which may be useful.

Well, I guess I need to prepare a new experimantal patch. :-)

I'll send it in a new thread I think.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-07 11:23                                                                             ` Rafael J. Wysocki
@ 2012-08-07 15:14                                                                               ` Ming Lei
  2012-08-07 15:42                                                                                 ` Alan Stern
  2012-08-07 20:45                                                                                 ` Rafael J. Wysocki
  0 siblings, 2 replies; 89+ messages in thread
From: Ming Lei @ 2012-08-07 15:14 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Tue, Aug 7, 2012 at 7:23 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
>> Yes, I agree, but I don't think it may make .runtime_post_resume
>> not doable, do I?
>
> No more device PM callbacks, please.

IMO, what the patch is doing is to introduce one callback which
is just called after .runtime_resume is completed, and you want
to move something out of previous .runtime_resume and do it
in another new callback to speedup resume, so it should be
reasonable to introduce the .runtime_post_resume callback in logic.

Also, the 'func' should be per driver, not per device since only one
'func' is enough for all same kind of devices driven by one same
driver.

> Besides, callbacks in struct dev_pm_ops are not only for drivers.

All the current 3 runtime callbacks are for devices. If you mean
they can be defined in bus/power_domain/device_type, .runtime_post_resume
still can be defined there too.

>
>> > Also, "func" should not be stored in dev_pm_ops because it won't be a
>> > read-only value.
>>
>> Sorry, could you explain it in detail that why the 'func'
>> has to switch to multiple functions dynamically? I understand
>> one function is enough and sometimes it can be bypassed with
>> one flag(such as, ignore_post_resume is introduced in dev_pm_info)
>> set.  Also, the driver can store the device specific states
>> in its own device instance to deal with different situations in the callback.
>>
>> If the idea is doable, we can save one pointer in 'struct device',
>> since the 'func' may not be used by more than 90% devices, also
>> have document benefit, even may simplify implementation of the
>> mechanism.
>
> And how many struct device objects there are for the one extra pointer to
> matter, let alone the fact that you want to replace it by one extra pointer
> somewhere else?

For example, in the pandaboard(one omap4 based small system), follows
the count of device instances:

         [root@root]#dmesg | grep device_add | wc -l
        471

The above is just a simple configuration(no graphics, no video/video, only
console enabled) on Pandaboard.

If the callback may be defined in dev_pm_info, not only memory can be saved,
also there are other advantages described before.


Thanks,
-- 
Ming Lei

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-07 15:14                                                                               ` Ming Lei
@ 2012-08-07 15:42                                                                                 ` Alan Stern
  2012-08-07 16:30                                                                                   ` Ming Lei
  2012-08-07 20:45                                                                                 ` Rafael J. Wysocki
  1 sibling, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-08-07 15:42 UTC (permalink / raw)
  To: Ming Lei; +Cc: Rafael J. Wysocki, linux-pci, USB list, Linux PM list

On Tue, 7 Aug 2012, Ming Lei wrote:

> On Tue, Aug 7, 2012 at 7:23 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> >> Yes, I agree, but I don't think it may make .runtime_post_resume
> >> not doable, do I?
> >
> > No more device PM callbacks, please.
> 
> IMO, what the patch is doing is to introduce one callback which
> is just called after .runtime_resume is completed, and you want
> to move something out of previous .runtime_resume and do it
> in another new callback to speedup resume, so it should be
> reasonable to introduce the .runtime_post_resume callback in logic.

No, that's really not what the patch is doing.

The idea behind the new API is that "func" will be called as soon as we
know the device is at full power.  That could be after the next runtime
resume or it could be right away.  This is a one-time call; it should
not be made _every_ time the device resumes.

> Also, the 'func' should be per driver, not per device since only one
> 'func' is enough for all same kind of devices driven by one same
> driver.

But what if the subsystem defines its own dev_pm_info structure?  Then
the driver's dev_pm_info will be ignored by the runtime PM core.  All
the subsystems would have to be changed.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-07 15:42                                                                                 ` Alan Stern
@ 2012-08-07 16:30                                                                                   ` Ming Lei
  2012-08-07 20:57                                                                                     ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Ming Lei @ 2012-08-07 16:30 UTC (permalink / raw)
  To: Alan Stern; +Cc: Rafael J. Wysocki, linux-pci, USB list, Linux PM list

On Tue, Aug 7, 2012 at 11:42 PM, Alan Stern <stern@rowland.harvard.edu> wrote:
>
> No, that's really not what the patch is doing.
>
> The idea behind the new API is that "func" will be called as soon as we
> know the device is at full power.  That could be after the next runtime
> resume or it could be right away.  This is a one-time call; it should

IMO, in the both two cases, the 'func' should be very similar with
.runtime_post_resume from view of the caller because the caller
don't know what the power state of the device is now, so they may
always think the 'func' should do something similar done in
.runtime_post_resume.

Also .runtime_post_resume always knows the device's power state
is active, which is same with 'func'. In fact, it doesn't matter if the active
state is the 1st time or other times, does it?

> not be made _every_ time the device resumes.

Suppose the device is always resumed in the path(such as irq context),
the callback is still called every time.

If the .runtime_post_resume is to be a one-time call, that is easy to do it.
Also I am wondering why the callback shouldn't be called after resume
in sync context, and it may simplify implementation if the two contexts
(sync vs. async) are kept consistent.

>
>> Also, the 'func' should be per driver, not per device since only one
>> 'func' is enough for all same kind of devices driven by one same
>> driver.
>
> But what if the subsystem defines its own dev_pm_info structure?  Then
> the driver's dev_pm_info will be ignored by the runtime PM core.  All
> the subsystems would have to be changed.

Suppose .runtime_post_resume is introduced, the priority of
dev_pm_info for .runtime_post_resume callback can be changed to
adapt to the situation.


Thanks,
-- 
Ming Lei

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-07 12:28                                                                           ` Rafael J. Wysocki
@ 2012-08-07 17:15                                                                             ` Alan Stern
  2012-08-07 21:31                                                                               ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-08-07 17:15 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: Ming Lei, linux-pci, USB list, Linux PM list

On Tue, 7 Aug 2012, Rafael J. Wysocki wrote:

> > > All those changes (and some of the following ones) are symptoms of a
> > > basic mistake in this approach.
> > 
> > Every time you say something like this (i.e. liks someone who knows better)
> 
> s/liks/like/
> 
> > I kind of feel like being under attach, which I hope is not your intention.
> 
> s/attach/attack/

Sorry; you're right.  It's all too easy to get very arrogant in email 
messages.  I'll try not to attack so strongly in the future.

> > > The idea of this new feature is to
> > > call "func" as soon as we know the device is at full power, no matter
> > > how it got there.
> > 
> > Yes, it is so.

Incidentally, that sentence is the justification for the invariance
condition mentioned later.  power.func should be called as soon as we
know the device is at full power; therefore when the status changes to
RPM_ACTIVE it should be called and then cleared (if it was set), and it
should never get set while the status is RPM_ACTIVE.  Therefore it
should never be true that power.func is set _and_ the status is
RPM_ACTIVE.

> > > That means we should call it near the end of
> > > rpm_resume() (just before the rpm_idle() call), not from within
> > > pm_runtime_work().
> > > 
> > > Doing it this way will be more efficient and (I think) will remove
> > > some races.
> > 
> > Except that func() shouldn't be executed under dev->power.lock, which makes it
> > rather difficult to call it from rpm_resume().  Or at least it seems so.
> > 
> > Moreover, it should be called after we've changed the status to RPM_ACTIVE
> > _and_ dropped the lock.
> 
> So we could drop the lock right before returning, execute func() and acquire
> the lock once again,

Yes; that's what I had in mind.  We already do something similar when 
calling pm_runtime_put(parent).

>  but then func() might be executed by any thread that
> happened to resume the device.  In that case the caller of
> pm_runtime_get_and_call() would have to worry about locks that such threads
> might acquire and it would have to make sure that func() didn't try to acquire
> them too.  That may not be a big deal, but if func() is executed by
> pm_runtime_work(), that issue simply goes away.

But then you have to worry about races between pm_runtime_resume() and
the workqueue.  If the device is resumed by some other thread, it
could be suspended again before "func" is called.

> Then, however, there's another issue: what should happen if
> pm_runtime_get_and_call() finds that it cannot execute func() right away,
> so it queues up resume and the execution of it, in the meantime some other
> thread resumes the device synchronously and pm_runtime_get_and_call() is
> run again.  I think in that case func() should be executed synchronously
> and the one waiting for execution should be canceled.  The alternative
> would be to return -EAGAIN from pm_runtime_get_and_call() and expect the
> caller to cope with that, which isn't too attractive.
> 
> This actually is analogous to the case when pm_runtime_get_and_call()
> sees that power.func is not NULL.  In my experimental patches it returned
> -EAGAIN in that case, but perhaps it's better to replace the existing
> power.func with the new one.  Then, by doing pm_runtime_get_and_call(dev, NULL)
> we can ensure that either the previous func() has run already or it will never
> run, which may be useful.

A good point.  I agree that pm_runtime_get_and_call() should always 
overwrite the existing power.func value.

There are a couple of other issues remaining.

What's the best approach when disable_count > 0?  My feeling is that we
should still rely on power.runtime_status as the best approximation to
the device's state, so we shouldn't call "func" directly unless the
status is already RPM_ACTIVE.  If the status is something else, we
can't queue an async resume request.  So we just set power.func and
return.  Eventually the driver will either call pm_runtime_set_active()
or pm_runtime_enable() followed by pm_runtime_resume(), at which time
we would call power.func.

Also, what should happen when power.runtime_error is set?  The same as
when disable_depth > 0?

You mentioned that pm_runtime_disable() does a resume if there's a 
pending resume request.  I had forgotten about this.  It worries me, 
because subsystems use code sequences like this:

	pm_runtime_disable(dev);
	pm_runtime_set_active(dev);
	pm_runtime_enable(dev);

in their system resume routines (in fact, we advise them to do so in
the Documentation file).  Now, it is unlikely for a resume request to
be pending during system sleep, but it doesn't seem to be impossible.  
When there is such a pending request, the pm_runtime_disable() call
will try to do a runtime resume at a time when the device has just been
restored to full power.  That's not good.

Probably this pattern occurs in few enough places that we could go
through and fix them all.  But how?  Should there be a new function:
pm_adjust_runtime_status_after_system_resume()?

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-07 15:14                                                                               ` Ming Lei
  2012-08-07 15:42                                                                                 ` Alan Stern
@ 2012-08-07 20:45                                                                                 ` Rafael J. Wysocki
  2012-08-08  2:02                                                                                   ` Ming Lei
  1 sibling, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-07 20:45 UTC (permalink / raw)
  To: Ming Lei; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Tuesday, August 07, 2012, Ming Lei wrote:
> On Tue, Aug 7, 2012 at 7:23 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> >> Yes, I agree, but I don't think it may make .runtime_post_resume
> >> not doable, do I?
> >
> > No more device PM callbacks, please.
> 
> IMO, what the patch is doing is to introduce one callback which
> is just called after .runtime_resume is completed,

No, it is not a callback.  It is a function to be run _once_ when the device is
known to be active.  It is not a member of a data type or anything like this.

It's kind of disappointing that you don't see a difference between that and a
callback.

> and you want to move something out of previous .runtime_resume

No, I don't.  Where did you get that idea from?

> and do it in another new callback to speedup resume,

No, not to speed up resume.  The idea is to allow drivers to run something
when the resume is complete, so that they don't have to implement a "resume
detection" logic or use .runtime_resume() to run things that don't belong
there.

> so it should be reasonable to introduce the .runtime_post_resume callback in
> logic.

No.  This doesn't have anything to do with callbacks!

If you want a new callback, you should specify what the role of this callback
is, otherwise it is not well defined.  I this case, though, what the role of
func() is depends on the caller and most likely every driver would use it
for something different.  So no, I don't see how it can be a callback.

> Also, the 'func' should be per driver, not per device since only one
> 'func' is enough for all same kind of devices driven by one same
> driver.

It isn't per device!  It is per _caller_.  The fact that the pointer is
stored _temporarily_ in struct device doesn't mean that it is per device
and that it is a callback.  From the struct device point of view it is _data_,
not a member function.

> > Besides, callbacks in struct dev_pm_ops are not only for drivers.
> 
> All the current 3 runtime callbacks are for devices. If you mean
> they can be defined in bus/power_domain/device_type, .runtime_post_resume
> still can be defined there too.

No, it wouldn't make _any_ _sense_ there, because its role there cannot be
defined in any sane way.

> >> > Also, "func" should not be stored in dev_pm_ops because it won't be a
> >> > read-only value.
> >>
> >> Sorry, could you explain it in detail that why the 'func'
> >> has to switch to multiple functions dynamically? I understand
> >> one function is enough and sometimes it can be bypassed with
> >> one flag(such as, ignore_post_resume is introduced in dev_pm_info)
> >> set.  Also, the driver can store the device specific states
> >> in its own device instance to deal with different situations in the callback.
> >>
> >> If the idea is doable, we can save one pointer in 'struct device',
> >> since the 'func' may not be used by more than 90% devices, also
> >> have document benefit, even may simplify implementation of the
> >> mechanism.
> >
> > And how many struct device objects there are for the one extra pointer to
> > matter, let alone the fact that you want to replace it by one extra pointer
> > somewhere else?
> 
> For example, in the pandaboard(one omap4 based small system), follows
> the count of device instances:
> 
>          [root@root]#dmesg | grep device_add | wc -l
>         471
> 
> The above is just a simple configuration(no graphics, no video/video, only
> console enabled) on Pandaboard.
> 
> If the callback may be defined in dev_pm_info,

I guess you mean struct dev_pm_ops, right?

> not only memory can be saved, also there are other advantages described
> before.

So now please count how many struct dev_pm_ops objects there are on that system
and compute the differece.  And please note that drivers that don't use
struct dev_pm_ops for power management will do that in the future.

Also please note that the caller of pm_runtime_get_and_call() need not be
a driver, at least in theory.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-07 16:30                                                                                   ` Ming Lei
@ 2012-08-07 20:57                                                                                     ` Rafael J. Wysocki
  0 siblings, 0 replies; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-07 20:57 UTC (permalink / raw)
  To: Ming Lei; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Tuesday, August 07, 2012, Ming Lei wrote:
> On Tue, Aug 7, 2012 at 11:42 PM, Alan Stern <stern@rowland.harvard.edu> wrote:
> >
> > No, that's really not what the patch is doing.
> >
> > The idea behind the new API is that "func" will be called as soon as we
> > know the device is at full power.  That could be after the next runtime
> > resume or it could be right away.  This is a one-time call; it should
> 
> IMO, in the both two cases, the 'func' should be very similar with
> .runtime_post_resume from view of the caller because the caller
> don't know what the power state of the device is now, so they may
> always think the 'func' should do something similar done in
> .runtime_post_resume.
> 
> Also .runtime_post_resume always knows the device's power state
> is active, which is same with 'func'. In fact, it doesn't matter if the active
> state is the 1st time or other times, does it?

What Alan wanted to say, I think, was that .runtime_post_resume() would have
to be always identical, where func() need not be always the same function.
Moreover, .runtime_post_resume() would _always_ be run after a device resume,
while func() is run only _once_.

> > not be made _every_ time the device resumes.
> 
> Suppose the device is always resumed in the path(such as irq context),
> the callback is still called every time.

Yes, but what if you have _two_ code paths and you want to call different
code as func() in each of them?

> If the .runtime_post_resume is to be a one-time call, that is easy to do it.

No, it isn't.

> Also I am wondering why the callback shouldn't be called after resume
> in sync context, and it may simplify implementation if the two contexts
> (sync vs. async) are kept consistent.

I have no idea what you're talking about.

We actually have a callback that is run every time a device is resumed.
It is called .runtime_idle().  Does it help, though?  No, it doesn't.

> >> Also, the 'func' should be per driver, not per device since only one
> >> 'func' is enough for all same kind of devices driven by one same
> >> driver.
> >
> > But what if the subsystem defines its own dev_pm_info structure?  Then
> > the driver's dev_pm_info will be ignored by the runtime PM core.  All
> > the subsystems would have to be changed.
> 
> Suppose .runtime_post_resume is introduced,

It is not going to be introduced, period.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-07 17:15                                                                             ` Alan Stern
@ 2012-08-07 21:31                                                                               ` Rafael J. Wysocki
  0 siblings, 0 replies; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-07 21:31 UTC (permalink / raw)
  To: Alan Stern; +Cc: Ming Lei, linux-pci, USB list, Linux PM list

On Tuesday, August 07, 2012, Alan Stern wrote:
> On Tue, 7 Aug 2012, Rafael J. Wysocki wrote:
> 
> > > > All those changes (and some of the following ones) are symptoms of a
> > > > basic mistake in this approach.
> > > 
> > > Every time you say something like this (i.e. liks someone who knows better)
> > 
> > s/liks/like/
> > 
> > > I kind of feel like being under attach, which I hope is not your intention.
> > 
> > s/attach/attack/
> 
> Sorry; you're right.  It's all too easy to get very arrogant in email 
> messages.  I'll try not to attack so strongly in the future.

Thanks!

> > > > The idea of this new feature is to
> > > > call "func" as soon as we know the device is at full power, no matter
> > > > how it got there.
> > > 
> > > Yes, it is so.
> 
> Incidentally, that sentence is the justification for the invariance
> condition mentioned later.

:-)

> power.func should be called as soon as we
> know the device is at full power; therefore when the status changes to
> RPM_ACTIVE it should be called and then cleared (if it was set), and it
> should never get set while the status is RPM_ACTIVE.  Therefore it
> should never be true that power.func is set _and_ the status is
> RPM_ACTIVE.

I guess with the patch I've just sent:

http://marc.info/?l=linux-pm&m=134437366811066&w=4

it's almost the case, except when a synchronous resume happens before the work
item scheduled by __pm_runtime_get_and_call() is run.  However, I don't think
it is a problem in that case, because the device won't be suspended before the
execution of that work item starts (rpm_check_suspend_allowed() will see that
power.request_pending is set and that power.request is RPM_REQ_RESUME, so it
will return -EAGAIN).

> > > > That means we should call it near the end of
> > > > rpm_resume() (just before the rpm_idle() call), not from within
> > > > pm_runtime_work().
> > > > 
> > > > Doing it this way will be more efficient and (I think) will remove
> > > > some races.
> > > 
> > > Except that func() shouldn't be executed under dev->power.lock, which makes it
> > > rather difficult to call it from rpm_resume().  Or at least it seems so.
> > > 
> > > Moreover, it should be called after we've changed the status to RPM_ACTIVE
> > > _and_ dropped the lock.
> > 
> > So we could drop the lock right before returning, execute func() and acquire
> > the lock once again,
> 
> Yes; that's what I had in mind.  We already do something similar when 
> calling pm_runtime_put(parent).

Yes, we do.  However, I still don't think it's really safe to call func()
from rpm_resume(), because it may be run synchronously from a context
quite unrelated to the caller of __pm_runtime_get_and_call() (for example,
from the pm_runtime_barrier() in __device_suspend()).

> >  but then func() might be executed by any thread that
> > happened to resume the device.  In that case the caller of
> > pm_runtime_get_and_call() would have to worry about locks that such threads
> > might acquire and it would have to make sure that func() didn't try to acquire
> > them too.  That may not be a big deal, but if func() is executed by
> > pm_runtime_work(), that issue simply goes away.
> 
> But then you have to worry about races between pm_runtime_resume() and
> the workqueue.  If the device is resumed by some other thread, it
> could be suspended again before "func" is called.

No, it can't, if the device's usage count is incremented before dropping
power.lock after rpm_resume(dev, 0) has returned.

> > Then, however, there's another issue: what should happen if
> > pm_runtime_get_and_call() finds that it cannot execute func() right away,
> > so it queues up resume and the execution of it, in the meantime some other
> > thread resumes the device synchronously and pm_runtime_get_and_call() is
> > run again.  I think in that case func() should be executed synchronously
> > and the one waiting for execution should be canceled.  The alternative
> > would be to return -EAGAIN from pm_runtime_get_and_call() and expect the
> > caller to cope with that, which isn't too attractive.
> > 
> > This actually is analogous to the case when pm_runtime_get_and_call()
> > sees that power.func is not NULL.  In my experimental patches it returned
> > -EAGAIN in that case, but perhaps it's better to replace the existing
> > power.func with the new one.  Then, by doing pm_runtime_get_and_call(dev, NULL)
> > we can ensure that either the previous func() has run already or it will never
> > run, which may be useful.
> 
> A good point.  I agree that pm_runtime_get_and_call() should always 
> overwrite the existing power.func value.
> 
> There are a couple of other issues remaining.
> 
> What's the best approach when disable_count > 0?  My feeling is that we
> should still rely on power.runtime_status as the best approximation to
> the device's state, so we shouldn't call "func" directly unless the
> status is already RPM_ACTIVE.

Well, that's one possibility.  In that case, though, the caller may want
to run func() regardless of whether or not runtime PM is enabled for the given
device and that would require some serious trickery.  For this reason, in
the newest patch (http://marc.info/?l=linux-pm&m=134437366811066&w=4) the
caller can choose what to do. 

> If the status is something else, we
> can't queue an async resume request.  So we just set power.func and
> return.  Eventually the driver will either call pm_runtime_set_active()
> or pm_runtime_enable() followed by pm_runtime_resume(), at which time
> we would call power.func.
> 
> Also, what should happen when power.runtime_error is set?  The same as
> when disable_depth > 0?

I think so.

> You mentioned that pm_runtime_disable() does a resume if there's a 
> pending resume request.  I had forgotten about this.  It worries me, 
> because subsystems use code sequences like this:
> 
> 	pm_runtime_disable(dev);
> 	pm_runtime_set_active(dev);
> 	pm_runtime_enable(dev);
> 
> in their system resume routines (in fact, we advise them to do so in
> the Documentation file).  Now, it is unlikely for a resume request to
> be pending during system sleep, but it doesn't seem to be impossible.  
> When there is such a pending request, the pm_runtime_disable() call
> will try to do a runtime resume at a time when the device has just been
> restored to full power.  That's not good.

Well, they should do __pm_runtime_disable(dev, false), then.

> Probably this pattern occurs in few enough places that we could go
> through and fix them all.  But how?  Should there be a new function:
> pm_adjust_runtime_status_after_system_resume()?

I think the above would suffice.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-07 20:45                                                                                 ` Rafael J. Wysocki
@ 2012-08-08  2:02                                                                                   ` Ming Lei
  2012-08-08 18:42                                                                                     ` Alan Stern
  2012-08-08 22:27                                                                                     ` Rafael J. Wysocki
  0 siblings, 2 replies; 89+ messages in thread
From: Ming Lei @ 2012-08-08  2:02 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Wed, Aug 8, 2012 at 4:45 AM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> On Tuesday, August 07, 2012, Ming Lei wrote:
>> On Tue, Aug 7, 2012 at 7:23 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
>> >> Yes, I agree, but I don't think it may make .runtime_post_resume
>> >> not doable, do I?
>> >
>> > No more device PM callbacks, please.
>>
>> IMO, what the patch is doing is to introduce one callback which
>> is just called after .runtime_resume is completed,
>
> No, it is not a callback.  It is a function to be run _once_ when the device is
> known to be active.  It is not a member of a data type or anything like this.

Looks it was said by Alan, not me, :-)

"The documentation should explain that in some ways, "func" is like a
workqueue callback routine:".

See http://marc.info/?l=linux-usb&m=134426838507799&w=2

>
> It's kind of disappointing that you don't see a difference between that and a
> callback.
>
>> and you want to move something out of previous .runtime_resume
>
> No, I don't.  Where did you get that idea from?

If so, I am wondering why the 'func' can't be called in .runtime_resume
directly, and follow the below inside caller at the same time?

             if (device is active or disabled)
                      call func(device).

>
>> and do it in another new callback to speedup resume,
>
> No, not to speed up resume.  The idea is to allow drivers to run something
> when the resume is complete, so that they don't have to implement a "resume
> detection" logic or use .runtime_resume() to run things that don't belong
> there.

Looks it was said by you, :-)

"Unless your _driver_ callback is actually executed from within a PM domain
callback, for example, and something else may be waiting for it to complete,
so your data processing is adding latencies to some other threads.  I'm not
making that up, by the way, that really can happen."

See http://marc.info/?l=linux-pm&m=134394271517527&w=2

Alan also said "Okay, those are valid reasons" for the idea. Except for
this one, I don't see other obvious advantages about the patch.

>
>> so it should be reasonable to introduce the .runtime_post_resume callback in
>> logic.
>
> No.  This doesn't have anything to do with callbacks!
>
> If you want a new callback, you should specify what the role of this callback
> is, otherwise it is not well defined.  I this case, though, what the role of
> func() is depends on the caller and most likely every driver would use it
> for something different.  So no, I don't see how it can be a callback.
>
>> Also, the 'func' should be per driver, not per device since only one
>> 'func' is enough for all same kind of devices driven by one same
>> driver.
>
> It isn't per device!  It is per _caller_.  The fact that the pointer is
> stored _temporarily_ in struct device doesn't mean that it is per device
> and that it is a callback.  From the struct device point of view it is _data_,
> not a member function.

The fact is that it will become per-device one you store it in 'struct device'.

Suppose one driver may drive 10000 same devices, the same data will be
stored inside all the 10000 device instances, it is a good way to do it?

Not mention 90% devices mayn't use the _temporarily_ data at all.

>
>> > Besides, callbacks in struct dev_pm_ops are not only for drivers.
>>
>> All the current 3 runtime callbacks are for devices. If you mean
>> they can be defined in bus/power_domain/device_type, .runtime_post_resume
>> still can be defined there too.
>
> No, it wouldn't make _any_ _sense_ there, because its role there cannot be
> defined in any sane way.
>
>> >> > Also, "func" should not be stored in dev_pm_ops because it won't be a
>> >> > read-only value.
>> >>
>> >> Sorry, could you explain it in detail that why the 'func'
>> >> has to switch to multiple functions dynamically? I understand
>> >> one function is enough and sometimes it can be bypassed with
>> >> one flag(such as, ignore_post_resume is introduced in dev_pm_info)
>> >> set.  Also, the driver can store the device specific states
>> >> in its own device instance to deal with different situations in the callback.
>> >>
>> >> If the idea is doable, we can save one pointer in 'struct device',
>> >> since the 'func' may not be used by more than 90% devices, also
>> >> have document benefit, even may simplify implementation of the
>> >> mechanism.
>> >
>> > And how many struct device objects there are for the one extra pointer to
>> > matter, let alone the fact that you want to replace it by one extra pointer
>> > somewhere else?
>>
>> For example, in the pandaboard(one omap4 based small system), follows
>> the count of device instances:
>>
>>          [root@root]#dmesg | grep device_add | wc -l
>>         471
>>
>> The above is just a simple configuration(no graphics, no video/video, only
>> console enabled) on Pandaboard.
>>
>> If the callback may be defined in dev_pm_info,
>
> I guess you mean struct dev_pm_ops, right?

Sorry, it is a typo.

>
>> not only memory can be saved, also there are other advantages described
>> before.
>
> So now please count how many struct dev_pm_ops objects there are on that system
> and compute the differece.  And please note that drivers that don't use
> struct dev_pm_ops for power management will do that in the future.

Most of dev_pm_ops stays inside module image, and not in ram.
It is a bit difficult to get the count of all dev_pm_ops objects in ram
since it is defined statically.

For example, in USB subsystem, there are only 2 dev_pm_ops
objects in RAM for a normal system, but there may have hundreds of
usb devices in the system(usb_device, usb_interface, ep_device, ...).

>
> Also please note that the caller of pm_runtime_get_and_call() need not be
> a driver, at least in theory.

I admit it in theory,  :-)


Thanks,
-- 
Ming Lei

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-08  2:02                                                                                   ` Ming Lei
@ 2012-08-08 18:42                                                                                     ` Alan Stern
  2012-08-08 20:16                                                                                       ` Rafael J. Wysocki
  2012-08-08 22:27                                                                                     ` Rafael J. Wysocki
  1 sibling, 1 reply; 89+ messages in thread
From: Alan Stern @ 2012-08-08 18:42 UTC (permalink / raw)
  To: Ming Lei; +Cc: Rafael J. Wysocki, linux-pci, USB list, Linux PM list

On Wed, 8 Aug 2012, Ming Lei wrote:

> On Wed, Aug 8, 2012 at 4:45 AM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> > On Tuesday, August 07, 2012, Ming Lei wrote:
> >> On Tue, Aug 7, 2012 at 7:23 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> >> >> Yes, I agree, but I don't think it may make .runtime_post_resume
> >> >> not doable, do I?
> >> >
> >> > No more device PM callbacks, please.
> >>
> >> IMO, what the patch is doing is to introduce one callback which
> >> is just called after .runtime_resume is completed,
> >
> > No, it is not a callback.  It is a function to be run _once_ when the device is
> > known to be active.  It is not a member of a data type or anything like this.
> 
> Looks it was said by Alan, not me, :-)
> 
> "The documentation should explain that in some ways, "func" is like a
> workqueue callback routine:".
> 
> See http://marc.info/?l=linux-usb&m=134426838507799&w=2

I didn't say it _was_ a callback; I said it was _like_ a callback in 
some ways.

> If so, I am wondering why the 'func' can't be called in .runtime_resume
> directly, and follow the below inside caller at the same time?
> 
>              if (device is active or disabled)
>                       call func(device).

That was my original suggestion.  Rafael pointed out that having a 
single function call to do this would make it easier for driver writers 
to get it right.

> > No, not to speed up resume.  The idea is to allow drivers to run something
> > when the resume is complete, so that they don't have to implement a "resume
> > detection" logic or use .runtime_resume() to run things that don't belong
> > there.
> 
> Looks it was said by you, :-)
> 
> "Unless your _driver_ callback is actually executed from within a PM domain
> callback, for example, and something else may be waiting for it to complete,
> so your data processing is adding latencies to some other threads.  I'm not
> making that up, by the way, that really can happen."
> 
> See http://marc.info/?l=linux-pm&m=134394271517527&w=2
> 
> Alan also said "Okay, those are valid reasons" for the idea. Except for
> this one, I don't see other obvious advantages about the patch.

Those _are_ the two advantages:

	The runtime-resume method does nothing but bring the device
	back to full power; it doesn't do any other processing;

	It's easier than calling pm_runtime_get() followed by a test
	to see whether the device is active.

> Suppose one driver may drive 10000 same devices, the same data will be
> stored inside all the 10000 device instances, it is a good way to do it?

If you've got a system with 10000 device instances, you can probably 
spare the memory needed to store these function pointers.  But you're 
right that this is a disadvantage.

Alan Stern


^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-08 18:42                                                                                     ` Alan Stern
@ 2012-08-08 20:16                                                                                       ` Rafael J. Wysocki
  2012-08-09  5:55                                                                                         ` Ming Lei
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-08 20:16 UTC (permalink / raw)
  To: Alan Stern; +Cc: Ming Lei, linux-pci, USB list, Linux PM list

On Wednesday, August 08, 2012, Alan Stern wrote:
> On Wed, 8 Aug 2012, Ming Lei wrote:
[...]
> > If so, I am wondering why the 'func' can't be called in .runtime_resume
> > directly, and follow the below inside caller at the same time?
> > 
> >              if (device is active or disabled)
> >                       call func(device).
> 
> That was my original suggestion.  Rafael pointed out that having a 
> single function call to do this would make it easier for driver writers 
> to get it right.

Not only would it be easier to get it right, in my opinion, but also in the
example above func() may be called in some places where the driver may not
want it to be called and which are very difficult to detect (like a resume
from __device_suspend() during system suspend).  Moreover, if the driver
callback is not executed directly by the PM core, but instead it is executed by
a subsystem or PM domain callback, there's no guarantee that the device _can_
be used for processing regular I/O before the driver callback returns (the
subsystem callback may still need to do something _after_ that happens).

So, this appears to be a matter of correctness too.

> > > No, not to speed up resume.  The idea is to allow drivers to run something
> > > when the resume is complete, so that they don't have to implement a "resume
> > > detection" logic or use .runtime_resume() to run things that don't belong
> > > there.
> > 
> > Looks it was said by you, :-)
> > 
> > "Unless your _driver_ callback is actually executed from within a PM domain
> > callback, for example, and something else may be waiting for it to complete,
> > so your data processing is adding latencies to some other threads.  I'm not
> > making that up, by the way, that really can happen."
> > 
> > See http://marc.info/?l=linux-pm&m=134394271517527&w=2
> > 
> > Alan also said "Okay, those are valid reasons" for the idea. Except for
> > this one, I don't see other obvious advantages about the patch.
> 
> Those _are_ the two advantages:
> 
> 	The runtime-resume method does nothing but bring the device
> 	back to full power; it doesn't do any other processing;
> 
> 	It's easier than calling pm_runtime_get() followed by a test
> 	to see whether the device is active.
> 
> > Suppose one driver may drive 10000 same devices, the same data will be
> > stored inside all the 10000 device instances, it is a good way to do it?
> 
> If you've got a system with 10000 device instances, you can probably 
> spare the memory needed to store these function pointers.  But you're 
> right that this is a disadvantage.

Yes, it is in grand general, but that also is a matter of alignment and of
the way the slab allocator works.  For example, if every struct device
object were stored at the beginning of a new memory page, then its size
wouldn't matter a lot as long as it were smaller than PAGE_SIZE.

I haven't checked the details, but I'm pretty sure that focusing on the size
alone doesn't give us the whole picture here.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-08  2:02                                                                                   ` Ming Lei
  2012-08-08 18:42                                                                                     ` Alan Stern
@ 2012-08-08 22:27                                                                                     ` Rafael J. Wysocki
  1 sibling, 0 replies; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-08 22:27 UTC (permalink / raw)
  To: Ming Lei; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Wednesday, August 08, 2012, Ming Lei wrote:
> On Wed, Aug 8, 2012 at 4:45 AM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> > On Tuesday, August 07, 2012, Ming Lei wrote:
> >> On Tue, Aug 7, 2012 at 7:23 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
[...]
> >
> >> and you want to move something out of previous .runtime_resume
> >
> > No, I don't.  Where did you get that idea from?
> 
> If so, I am wondering why the 'func' can't be called in .runtime_resume
> directly, and follow the below inside caller at the same time?
> 
>              if (device is active or disabled)
>                       call func(device).

This was covered in my last reply to Alan.

> >> and do it in another new callback to speedup resume,
> >
> > No, not to speed up resume.  The idea is to allow drivers to run something
> > when the resume is complete, so that they don't have to implement a "resume
> > detection" logic or use .runtime_resume() to run things that don't belong
> > there.
> 
> Looks it was said by you, :-)
> 
> "Unless your _driver_ callback is actually executed from within a PM domain
> callback, for example, and something else may be waiting for it to complete,
> so your data processing is adding latencies to some other threads.  I'm not
> making that up, by the way, that really can happen."
> 
> See http://marc.info/?l=linux-pm&m=134394271517527&w=2

We were discussing specific pseudo-code in the documentation and you
conveniently took the above out of context.  Never mind. :-)

I was trying to illustrate my point with a convincing example and I admit I
could do better.

Anyway the point was that the purpose of .runtime_resume() was not to
process random I/O.  Its purpose is to _resume_ a suspended device,
no less, no more.  Which the "so that they don't have to [...] use
.runtime_resume() to run things that don't belong there." sentence above is
about.  So I've been saying the same thing all the time and it's never been
specifically about speedup (or rather about latencies added by random I/O
processing in drivers' runtime resume callbacks).

> Alan also said "Okay, those are valid reasons" for the idea. Except for
> this one, I don't see other obvious advantages about the patch.
> 
> >
> >> so it should be reasonable to introduce the .runtime_post_resume callback in
> >> logic.
> >
> > No.  This doesn't have anything to do with callbacks!
> >
> > If you want a new callback, you should specify what the role of this callback
> > is, otherwise it is not well defined.  I this case, though, what the role of
> > func() is depends on the caller and most likely every driver would use it
> > for something different.  So no, I don't see how it can be a callback.
> >
> >> Also, the 'func' should be per driver, not per device since only one
> >> 'func' is enough for all same kind of devices driven by one same
> >> driver.
> >
> > It isn't per device!  It is per _caller_.  The fact that the pointer is
> > stored _temporarily_ in struct device doesn't mean that it is per device
> > and that it is a callback.  From the struct device point of view it is _data_,
> > not a member function.
> 
> The fact is that it will become per-device one you store it in 'struct device'.
> 
> Suppose one driver may drive 10000 same devices,

Do you have any specific example of that?  If not, then please don't make up
arguments.

> the same data will be stored inside all the 10000 device instances, it is a
> good way to do it?
> 
> Not mention 90% devices mayn't use the _temporarily_ data at all.

It may be unused just as well as an additional callback pointer in a driver
object.

[...]
> >
> > So now please count how many struct dev_pm_ops objects there are on that system
> > and compute the differece.  And please note that drivers that don't use
> > struct dev_pm_ops for power management will do that in the future.
> 
> Most of dev_pm_ops stays inside module image, and not in ram.

Care to explain?  I'm not sure I understand the above correctly.

> It is a bit difficult to get the count of all dev_pm_ops objects in ram
> since it is defined statically.

Still, they are occupying memory, aren't they?  So you really can't tell
the difference between storing pointers in device driver objects and
struct device objects.

> For example, in USB subsystem, there are only 2 dev_pm_ops
> objects in RAM for a normal system, but there may have hundreds of
> usb devices in the system(usb_device, usb_interface, ep_device, ...).

Yes, USB is kind of exceptional, but also this means that your "let's
put that pointer into struct dev_pm_ops" idea won't work for USB drivers,
precisely because they don't use struct dev_pm_ops objects.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-08 20:16                                                                                       ` Rafael J. Wysocki
@ 2012-08-09  5:55                                                                                         ` Ming Lei
  2012-08-09 10:46                                                                                           ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Ming Lei @ 2012-08-09  5:55 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Thu, Aug 9, 2012 at 4:16 AM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> On Wednesday, August 08, 2012, Alan Stern wrote:

To be honest, I agree on the idea:

       The runtime-resume method does nothing but bring the device
        back to full power; it doesn't do any other processing, which
        should be done in 'func' or some kind of callback.

I just think it is not good to introduce one extra field of 'func' in
dev_pm_info which is embedded into struct device in the patch,
see the reasons in the last part of the reply.

>> That was my original suggestion.  Rafael pointed out that having a
>> single function call to do this would make it easier for driver writers
>> to get it right.
>
> Not only would it be easier to get it right, in my opinion, but also in the
> example above func() may be called in some places where the driver may not
> want it to be called and which are very difficult to detect (like a resume
> from __device_suspend() during system suspend).  Moreover, if the driver

IMO, func() does some driver specific things, which PM core shouldn't pay
special attention to in theory.

> callback is not executed directly by the PM core, but instead it is executed by
> a subsystem or PM domain callback, there's no guarantee that the device _can_
> be used for processing regular I/O before the driver callback returns (the
> subsystem callback may still need to do something _after_ that happens).

driver->runtime_resume should be allowed to do I/O things after
the device has been woken up inside driver callback, shouldn't it? If it
isn't allowed, something wrong should have be reported before.

> So, this appears to be a matter of correctness too.


>> If you've got a system with 10000 device instances, you can probably
>> spare the memory needed to store these function pointers.  But you're
>> right that this is a disadvantage.
>
> Yes, it is in grand general, but that also is a matter of alignment and of
> the way the slab allocator works.  For example, if every struct device
> object were stored at the beginning of a new memory page, then its size
> wouldn't matter a lot as long as it were smaller than PAGE_SIZE.
>
> I haven't checked the details, but I'm pretty sure that focusing on the size
> alone doesn't give us the whole picture here.

The allocation by kmalloc is not page aligned, and it is 2-power
aligned, for example 16, 32, 64,..., also it is at least hardware
L1 cache size aligned.

Firstly, introduce one extra pointer in device may increase memory
consume for device allocation, also it reflects one design drawback
in the patch, see below.

More importantly, the implementation violates some software design
principle in object oriented design. The driver core takes much
object oriented idea in its design and implementation, and introduces
device / driver / bus class. One class is an abstraction of one kind of
objects or instances with same attributes, so one class may include
many objects, for example, usb_device(class) is abstraction for all usb
devices, and there may have many many usb devices in a system, but only
one usb_device structure defined.

One specific driver class is a special class since it may only have one
driver object , which is basically read only. In OO world, it might be called
static class.

Generally, one driver object serves one specific device class, instead
of one device object. For example, usb_storage_driver is a driver object,
which serves all usb mass storage devices which all belongs to usb mass
storage class).

The 'func' to be introduced is a function pointer, which should be
driver related thing and should serve one specific device class in theory,
and it shouldn't serve only one concrete device object, so it is not good
to include it into 'struct device'.

The only function pointer in struct device:

              void    (*release)(struct device *dev)

should be removed.  All its users should convert to use device_type to
define release handler for its 'device class', instead of device object.

So suggest to improve it.


Thanks,
-- 
Ming Lei

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-09  5:55                                                                                         ` Ming Lei
@ 2012-08-09 10:46                                                                                           ` Rafael J. Wysocki
  2012-08-09 10:55                                                                                             ` Ming Lei
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-09 10:46 UTC (permalink / raw)
  To: Ming Lei; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Thursday, August 09, 2012, Ming Lei wrote:
> On Thu, Aug 9, 2012 at 4:16 AM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> > On Wednesday, August 08, 2012, Alan Stern wrote:
> 
> To be honest, I agree on the idea:
> 
>        The runtime-resume method does nothing but bring the device
>         back to full power; it doesn't do any other processing, which
>         should be done in 'func' or some kind of callback.

Good. :-)

> I just think it is not good to introduce one extra field of 'func' in
> dev_pm_info which is embedded into struct device in the patch,
> see the reasons in the last part of the reply.
> 
> >> That was my original suggestion.  Rafael pointed out that having a
> >> single function call to do this would make it easier for driver writers
> >> to get it right.
> >
> > Not only would it be easier to get it right, in my opinion, but also in the
> > example above func() may be called in some places where the driver may not
> > want it to be called and which are very difficult to detect (like a resume
> > from __device_suspend() during system suspend).  Moreover, if the driver
> 
> IMO, func() does some driver specific things, which PM core shouldn't pay
> special attention to in theory.

But also it shouldn't execute that code, right?

> > callback is not executed directly by the PM core, but instead it is executed by
> > a subsystem or PM domain callback, there's no guarantee that the device _can_
> > be used for processing regular I/O before the driver callback returns (the
> > subsystem callback may still need to do something _after_ that happens).
> 
> driver->runtime_resume should be allowed to do I/O things after
> the device has been woken up inside driver callback, shouldn't it? If it
> isn't allowed, something wrong should have be reported before.

Well, the lack of reports doesn't mean there are no bugs. :-)

People may actually see those bugs, but they don't report them or they
report them as system suspend/resume bugs, for example.

> > So, this appears to be a matter of correctness too.
> 
> 
> >> If you've got a system with 10000 device instances, you can probably
> >> spare the memory needed to store these function pointers.  But you're
> >> right that this is a disadvantage.
> >
> > Yes, it is in grand general, but that also is a matter of alignment and of
> > the way the slab allocator works.  For example, if every struct device
> > object were stored at the beginning of a new memory page, then its size
> > wouldn't matter a lot as long as it were smaller than PAGE_SIZE.
> >
> > I haven't checked the details, but I'm pretty sure that focusing on the size
> > alone doesn't give us the whole picture here.
> 
> The allocation by kmalloc is not page aligned, and it is 2-power
> aligned, for example 16, 32, 64,..., also it is at least hardware
> L1 cache size aligned.

Sure, that's why I used the conditional above.  And it doesn't mean I didn't
have the point.

> Firstly, introduce one extra pointer in device may increase memory
> consume for device allocation,

Yes, it does, which may or may not matter depending on the actual size of
struct device and the CPU cache line size on the given machine, right?

> also it reflects one design drawback in the patch, see below.
> 
> More importantly, the implementation violates some software design
> principle in object oriented design.

It doesn't violate anything and you're just ignoring what you've been told.
That makes discussing with you rather difficult, but I'll try again
nevertheless.

If you look at the actual patch I've just posted:

https://patchwork.kernel.org/patch/1299781/

you can see that power.func is never run directly.  Moreover, the pointer it
contains is only used to run a function in pm_runtime_work() and note that
pm_runtime_work() reads that pointer _twice_, because it may be changed in the
meantime by a concurrent thread.

All of this means what I've told you already at least once: power.func is
_not_ _a_ _member_ _function_.

IOW, if it were C++, power.func would still be a function pointer, not a method,
because it is _data_, although its data type happens to be "void function
taking a struct device pointer argument".  It is a data field used to pass
information to a work function, pm_runtime_work(), from a piece of code that
schedules its execution, __pm_runtime_get_and_call().  See now?

> The driver core takes much
> object oriented idea in its design and implementation, and introduces
> device / driver / bus class. One class is an abstraction of one kind of
> objects or instances with same attributes, so one class may include
> many objects, for example, usb_device(class) is abstraction for all usb
> devices, and there may have many many usb devices in a system, but only
> one usb_device structure defined.
> 
> One specific driver class is a special class since it may only have one
> driver object , which is basically read only. In OO world, it might be called
> static class.
> 
> Generally, one driver object serves one specific device class, instead
> of one device object. For example, usb_storage_driver is a driver object,
> which serves all usb mass storage devices which all belongs to usb mass
> storage class).
> 
> The 'func' to be introduced is a function pointer, which should be
> driver related thing and should serve one specific device class in theory,
> and it shouldn't serve only one concrete device object, so it is not good
> to include it into 'struct device'.

No.  You're confusing function pointers with member functions.

Yes, we use function pointers to store the addresses of member functions,
because that's what we can do in C.  Again, in C++ member functions would be
methods, but we still might use function pointers for other purposes.

Whether or not it is "elegant" or "clean" (or whatever you call it) function
pointers for other purposes is a different matter and I think it is highly
subjective.

> The only function pointer in struct device:
> 
>               void    (*release)(struct device *dev)
> 
> should be removed.  All its users should convert to use device_type to
> define release handler for its 'device class', instead of device object.
> 
> So suggest to improve it.

What I want the callers to be able to do is "run this code right now if the
device is operational or schedule the resume of it and run this code when
it's been resumed".  So, the pointer is used to tell the workqueue code
which code to run (if any) when the device has been resumed.

I don't want a "run this code every time the device is resumed" kind of thing,
which I agree that a callback would be more suitable for.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-09 10:46                                                                                           ` Rafael J. Wysocki
@ 2012-08-09 10:55                                                                                             ` Ming Lei
  2012-08-09 19:41                                                                                               ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Ming Lei @ 2012-08-09 10:55 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Thu, Aug 9, 2012 at 6:46 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> On Thursday, August 09, 2012, Ming Lei wrote:

>> driver->runtime_resume should be allowed to do I/O things after
>> the device has been woken up inside driver callback, shouldn't it? If it
>> isn't allowed, something wrong should have be reported before.
>
> Well, the lack of reports doesn't mean there are no bugs. :-)
>
> People may actually see those bugs, but they don't report them or they
> report them as system suspend/resume bugs, for example.

Also, I am still wondering why subsystem PM callback need to do
something after driver->runtime_resume completes, could you explain
it?

>> Firstly, introduce one extra pointer in device may increase memory
>> consume for device allocation,
>
> Yes, it does, which may or may not matter depending on the actual size of
> struct device and the CPU cache line size on the given machine, right?

It may double memory allocation size in some cases. And it is very possible
since there are so many device objects in system.

>
>> also it reflects one design drawback in the patch, see below.
>>
>> More importantly, the implementation violates some software design
>> principle in object oriented design.
>
> It doesn't violate anything and you're just ignoring what you've been told.
> That makes discussing with you rather difficult, but I'll try again
> nevertheless.
>
> If you look at the actual patch I've just posted:
>
> https://patchwork.kernel.org/patch/1299781/
>
> you can see that power.func is never run directly.  Moreover, the pointer it
> contains is only used to run a function in pm_runtime_work() and note that
> pm_runtime_work() reads that pointer _twice_, because it may be changed in the
> meantime by a concurrent thread.

I have explained it before, it is enough to keep the pointer read only
since driver can maintain its internal state in its specific device instance
(for example, usb_interface objects) and decide what to do in 'func'
for situations, right?



Thanks,
-- 
Ming Lei

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-09 10:55                                                                                             ` Ming Lei
@ 2012-08-09 19:41                                                                                               ` Rafael J. Wysocki
  2012-08-10  3:19                                                                                                 ` Ming Lei
  0 siblings, 1 reply; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-09 19:41 UTC (permalink / raw)
  To: Ming Lei; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Thursday, August 09, 2012, Ming Lei wrote:
> On Thu, Aug 9, 2012 at 6:46 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> > On Thursday, August 09, 2012, Ming Lei wrote:
> 
> >> driver->runtime_resume should be allowed to do I/O things after
> >> the device has been woken up inside driver callback, shouldn't it? If it
> >> isn't allowed, something wrong should have be reported before.
> >
> > Well, the lack of reports doesn't mean there are no bugs. :-)
> >
> > People may actually see those bugs, but they don't report them or they
> > report them as system suspend/resume bugs, for example.
> 
> Also, I am still wondering why subsystem PM callback need to do
> something after driver->runtime_resume completes, could you explain
> it?

It just isn't guaranteed that the subsystem callback won't do anything
after driver->runtime_resume completion.  I agree that it isn't likely
to happen.

> >> Firstly, introduce one extra pointer in device may increase memory
> >> consume for device allocation,
> >
> > Yes, it does, which may or may not matter depending on the actual size of
> > struct device and the CPU cache line size on the given machine, right?
> 
> It may double memory allocation size in some cases. And it is very possible
> since there are so many device objects in system.

Numbers, please?  If you don't have them, it's just waving your hands.

> >> also it reflects one design drawback in the patch, see below.
> >>
> >> More importantly, the implementation violates some software design
> >> principle in object oriented design.
> >
> > It doesn't violate anything and you're just ignoring what you've been told.
> > That makes discussing with you rather difficult, but I'll try again
> > nevertheless.
> >
> > If you look at the actual patch I've just posted:
> >
> > https://patchwork.kernel.org/patch/1299781/
> >
> > you can see that power.func is never run directly.  Moreover, the pointer it
> > contains is only used to run a function in pm_runtime_work() and note that
> > pm_runtime_work() reads that pointer _twice_, because it may be changed in the
> > meantime by a concurrent thread.
> 
> I have explained it before, it is enough to keep the pointer read only
> since driver can maintain its internal state in its specific device instance
> (for example, usb_interface objects) and decide what to do in 'func'
> for situations, right?

Yes, it is.  I actually have a patch that does something similar (I'll post it
shortly).

Of course, it is based on the assumption that func() will always be the same
pointer for the given driver, which doesn't seem to be proven, but perhaps
it is sufficient.  At least I'm not aware of use cases where it wouldn't be.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-09 19:41                                                                                               ` Rafael J. Wysocki
@ 2012-08-10  3:19                                                                                                 ` Ming Lei
  2012-08-10 20:29                                                                                                   ` Rafael J. Wysocki
  0 siblings, 1 reply; 89+ messages in thread
From: Ming Lei @ 2012-08-10  3:19 UTC (permalink / raw)
  To: Rafael J. Wysocki; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Fri, Aug 10, 2012 at 3:41 AM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
>
> It just isn't guaranteed that the subsystem callback won't do anything
> after driver->runtime_resume completion.  I agree that it isn't likely
> to happen.

In fact, the subsystem callback should make sure that don't happen, see
below comments on .runtime_resume:

 * @runtime_resume: Put the device into the fully active state in response to a
 *      wakeup event generated by hardware or at the request of software.  If
 *      necessary, put the device into the full-power state and restore its
 *      registers, so that it is fully operational.

So once driver->runtime_resume completes, the device should be fully operational
from the view of driver.

>
>> >> Firstly, introduce one extra pointer in device may increase memory
>> >> consume for device allocation,
>> >
>> > Yes, it does, which may or may not matter depending on the actual size of
>> > struct device and the CPU cache line size on the given machine, right?
>>
>> It may double memory allocation size in some cases. And it is very possible
>> since there are so many device objects in system.
>
> Numbers, please?  If you don't have them, it's just waving your hands.

It is easily observed and proved. Suppose sizeof(struct foo_dev) is 508bytes,
it will become 516bytes after your patch applies on 64bit arch, so
ksize(foo_dev_ptr)
will become 1024 and the memory consumption of the object is doubled.

>> I have explained it before, it is enough to keep the pointer read only
>> since driver can maintain its internal state in its specific device instance
>> (for example, usb_interface objects) and decide what to do in 'func'
>> for situations, right?
>
> Yes, it is.  I actually have a patch that does something similar (I'll post it
> shortly).

I have seen your patch which moves the 'func' from device into device_driver.
It is much better than before.

> Of course, it is based on the assumption that func() will always be the same
> pointer for the given driver, which doesn't seem to be proven, but perhaps
> it is sufficient.  At least I'm not aware of use cases where it wouldn't be.

Since you have moved 'func' into device_driver, and you still thought the
pointer can't be changed after it is set, so why not implement it as callback?

IMO, it is a bit weird to just store a function pointer data(not for
callback) in
driver object, but anyway, it is better than before, :-)

Thanks,
-- 
Ming Lei

^ permalink raw reply	[flat|nested] 89+ messages in thread

* Re: Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...)
  2012-08-10  3:19                                                                                                 ` Ming Lei
@ 2012-08-10 20:29                                                                                                   ` Rafael J. Wysocki
  0 siblings, 0 replies; 89+ messages in thread
From: Rafael J. Wysocki @ 2012-08-10 20:29 UTC (permalink / raw)
  To: Ming Lei; +Cc: Alan Stern, linux-pci, USB list, Linux PM list

On Friday, August 10, 2012, Ming Lei wrote:
> On Fri, Aug 10, 2012 at 3:41 AM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> >
> > It just isn't guaranteed that the subsystem callback won't do anything
> > after driver->runtime_resume completion.  I agree that it isn't likely
> > to happen.
> 
> In fact, the subsystem callback should make sure that don't happen, see
> below comments on .runtime_resume:
> 
>  * @runtime_resume: Put the device into the fully active state in response to a
>  *      wakeup event generated by hardware or at the request of software.  If
>  *      necessary, put the device into the full-power state and restore its
>  *      registers, so that it is fully operational.
> 
> So once driver->runtime_resume completes, the device should be fully operational
> from the view of driver.

This comment only applies literally to drivers whose callbacks are run directly
by the PM core.  If subsystems and/or PM domains are involved, the interactions
between different layers of callbacks obviously have to be taken into account.

Please note, however, that this comment doesn't say anything about processing
I/O by the callback (hint: the callback is _not_ _supposed_ to do that).

> >> >> Firstly, introduce one extra pointer in device may increase memory
> >> >> consume for device allocation,
> >> >
> >> > Yes, it does, which may or may not matter depending on the actual size of
> >> > struct device and the CPU cache line size on the given machine, right?
> >>
> >> It may double memory allocation size in some cases. And it is very possible
> >> since there are so many device objects in system.
> >
> > Numbers, please?  If you don't have them, it's just waving your hands.
> 
> It is easily observed and proved. Suppose sizeof(struct foo_dev) is 508bytes,
> it will become 516bytes after your patch applies on 64bit arch, so
> ksize(foo_dev_ptr)
> will become 1024 and the memory consumption of the object is doubled.

I meant real numbers, not made-up ones.

> >> I have explained it before, it is enough to keep the pointer read only
> >> since driver can maintain its internal state in its specific device instance
> >> (for example, usb_interface objects) and decide what to do in 'func'
> >> for situations, right?
> >
> > Yes, it is.  I actually have a patch that does something similar (I'll post it
> > shortly).
> 
> I have seen your patch which moves the 'func' from device into device_driver.
> It is much better than before.

Oh, thanks for letting me know.

> > Of course, it is based on the assumption that func() will always be the same
> > pointer for the given driver, which doesn't seem to be proven, but perhaps
> > it is sufficient.  At least I'm not aware of use cases where it wouldn't be.
> 
> Since you have moved 'func' into device_driver, and you still thought the
> pointer can't be changed after it is set, so why not implement it as callback?

I don't understand what you mean.

It _is_ a callback now in fact.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 89+ messages in thread

end of thread, other threads:[~2012-08-10 20:23 UTC | newest]

Thread overview: 89+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <Pine.LNX.4.44L0.1207241312050.1164-100000@iolanthe.rowland.org>
     [not found] ` <87r4s0opck.fsf@nemi.mork.no>
2012-07-25  4:08   ` bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure Bjørn Mork
2012-07-25  4:34     ` Huang Ying
2012-07-25  9:58       ` Bjørn Mork
2012-07-25 13:30         ` huang ying
2012-07-25 13:58           ` Bjørn Mork
2012-07-25 18:56             ` Rafael J. Wysocki
2012-07-25 20:02             ` Rafael J. Wysocki
2012-07-25 22:36               ` Bjørn Mork
2012-07-26  2:38                 ` Huang Ying
2012-07-26  2:38               ` Huang Ying
2012-07-26  8:54             ` Huang Ying
2012-07-26 10:35               ` Bjørn Mork
2012-07-26 11:02                 ` Bjørn Mork
2012-07-26 12:04                   ` Bjørn Mork
2012-07-26 15:03                 ` Alan Stern
2012-07-26 16:24                   ` Bjørn Mork
2012-07-27  5:35                 ` Huang Ying
2012-07-27  9:11                   ` Bjørn Mork
2012-07-30  3:15                     ` Huang Ying
2012-07-30  8:08                       ` Bjørn Mork
2012-07-30 13:31                         ` huang ying
2012-07-30 16:57                           ` Bjørn Mork
2012-07-31  0:22                             ` Huang Ying
2012-07-30 14:19                       ` Alan Stern
2012-07-31  0:24                         ` Huang Ying
2012-07-31  3:18                         ` Huang Ying
2012-07-31 17:07                           ` Alan Stern
2012-07-27 15:03                   ` Alan Stern
2012-07-27 19:11                     ` Rafael J. Wysocki
2012-07-27 19:39                       ` Alan Stern
2012-07-27 19:54                         ` Rafael J. Wysocki
2012-07-28 16:12                           ` Alan Stern
2012-07-28 20:26                             ` Rafael J. Wysocki
2012-07-28 21:12                               ` Alan Stern
2012-07-29 13:55                                 ` Rafael J. Wysocki
2012-07-29 14:55                                   ` Alan Stern
2012-07-29 19:18                                     ` Rafael J. Wysocki
2012-07-31 20:31                                       ` Do we need asynchronous pm_runtime_get()? (was: Re: bisected regression ...) Rafael J. Wysocki
2012-07-31 21:05                                         ` Alan Stern
2012-07-31 21:34                                           ` Rafael J. Wysocki
2012-07-31 21:49                                             ` Rafael J. Wysocki
2012-08-01 14:36                                             ` Alan Stern
2012-08-01 21:24                                               ` Rafael J. Wysocki
2012-08-02 20:16                                                 ` Alan Stern
2012-08-02 21:26                                                   ` Rafael J. Wysocki
2012-08-03  2:20                                                     ` Alan Stern
2012-08-03  3:37                                                       ` Ming Lei
2012-08-03 14:28                                                         ` Alan Stern
2012-08-04 19:47                                                         ` Rafael J. Wysocki
2012-08-04 20:25                                                           ` Alan Stern
2012-08-04 20:48                                                             ` Rafael J. Wysocki
2012-08-04 20:48                                                               ` Alan Stern
2012-08-04 21:15                                                                 ` Rafael J. Wysocki
2012-08-04 22:13                                                                   ` Alan Stern
2012-08-05 15:26                                                                     ` Rafael J. Wysocki
2012-08-06 13:30                                                                       ` Ming Lei
2012-08-06 14:47                                                                         ` Alan Stern
2012-08-07  1:35                                                                           ` Ming Lei
2012-08-07 11:23                                                                             ` Rafael J. Wysocki
2012-08-07 15:14                                                                               ` Ming Lei
2012-08-07 15:42                                                                                 ` Alan Stern
2012-08-07 16:30                                                                                   ` Ming Lei
2012-08-07 20:57                                                                                     ` Rafael J. Wysocki
2012-08-07 20:45                                                                                 ` Rafael J. Wysocki
2012-08-08  2:02                                                                                   ` Ming Lei
2012-08-08 18:42                                                                                     ` Alan Stern
2012-08-08 20:16                                                                                       ` Rafael J. Wysocki
2012-08-09  5:55                                                                                         ` Ming Lei
2012-08-09 10:46                                                                                           ` Rafael J. Wysocki
2012-08-09 10:55                                                                                             ` Ming Lei
2012-08-09 19:41                                                                                               ` Rafael J. Wysocki
2012-08-10  3:19                                                                                                 ` Ming Lei
2012-08-10 20:29                                                                                                   ` Rafael J. Wysocki
2012-08-08 22:27                                                                                     ` Rafael J. Wysocki
2012-08-06 15:48                                                                       ` Alan Stern
2012-08-06 20:30                                                                         ` Rafael J. Wysocki
2012-08-07 12:28                                                                           ` Rafael J. Wysocki
2012-08-07 17:15                                                                             ` Alan Stern
2012-08-07 21:31                                                                               ` Rafael J. Wysocki
2012-08-03 14:05                                                       ` Alan Stern
2012-08-04 20:08                                                         ` Rafael J. Wysocki
2012-08-04 20:42                                                           ` Alan Stern
2012-08-04 20:59                                                             ` Rafael J. Wysocki
2012-08-04 19:35                                                       ` Rafael J. Wysocki
2012-07-29 20:12                                     ` bisected regression, v3.5 -> next-20120724: PCI PM causes USB hotplug failure Jassi Brar
2012-07-29 21:44                                       ` Alan Stern
2012-07-25 19:51           ` [PATCH] PCI / PM: Fix messages printed by acpi_pci_set_power_state() Rafael J. Wysocki
2012-07-25 20:02             ` Alan Stern
2012-07-25 20:48               ` [PATCH][update] " Rafael J. Wysocki

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.