Re: [Bug 43247] O2 micro SD/MMC+1394 controller: 1394 device can't work (Register access failure)

Re: [Bug 43247] O2 micro SD/MMC+1394 controller: 1394 device can't work (Register access failure)

[Stefan Richter]

On Jul 10 bugzilla-daemon@bugzilla.kernel.org wrote:
> https://bugzilla.kernel.org/show_bug.cgi?id=43247
> 
> --- Comment #19 from jennifer <jennifer.li@o2micro.com>  2012-07-10 11:02:16 ---
[...]
> > Do you mean by that that
> > the original Linux driver works for you if you boot Linux but do not let
> > the driver be automatically loaded during boot, but instead load the driver
> > later by 'modprobe firewire-ohci' after the rest of the system has finished
> > booting up?
> 
> Yes.
> 
> > 
> > And if yes, did you also need to load the sdhci-pci driver manually this
> > way?  
> 
> There are 4 ways which can pass the issue.
> 1. Load 1394 by OS + Load sdhci-pci by manually.
> 2. Load sdhci-pci by OS + Load 1394 by manually.
> 3. Load 1394 by manually + load sdhci-pci by manually.
> 4. Load sdhci-pci by manually + load 1394 by manually.
> 
> > And further, does it matter whether sdhci-pci is loaded before
> > firewire-ohci or the other way around?
> 
> According to our test result, if we load the driver by manually and the issue
> will disappear. It didn't has the relationship about the loaded priority.
> But, if we load those drivers by OS and the issue will happen.


Could somebody at linux-pci@vger.kernel.org please advise?

1.)  Is there a kernel parameter which Jennifer could try in order to
force serialized PCI driver probing?

2.)  If there is one and if this turns out to cure the issue in testing:
How can I implement serialization between the O2Micro FireWire .probe()
and .resume() on one hand and the O2Micro SDHCI .probe() and .resume() on
the other hand?


[If you reply to this via bugzilla mail, please add
    Cc: linux-pci@vger.kernel.org
in your reply.  I am not aware of a way to add it to bugzilla.kernel.org's
Cc list of bug 43247.]
-- 
Stefan Richter
-=====-===-- -=== -===-
http://arcgraph.de/sr/

Re: [Bug 43247] O2 micro SD/MMC+1394 controller: 1394 device can't work (Register access failure)

[Bjorn Helgaas]

> Jennifer and the other reporters of the issue most certainly all used
> modular kernels, i.e. got the drivers loaded by udev.
>
> module_init() doesn't matter, but
>   - .probe(),
>   - .resume()
> do.  So as you say, if the drivers were statically linked, then at least
> .probe() would be performed serially, one PCI device after another,
> right?  But AFAIK .resume() would still be performed in parallel by a
> pool of kernel threads in current kernels.

OK, I see.  When both drivers are loaded near the same time via udev,
the firewire driver doesn't work correctly.  I don't know of anything
that serializes driver registration from dynamically loaded modules.

I wouldn't bother with the resume issue until after the load-time
issue is resolved because suspend/resume makes things harder to debug.
 You might be able to try loading/unloading the modules in a loop to
reproduce the issue in a single boot.

> From what I can tell, there is no driver bug but an unfortunate interaction
> between parts of a combo controller which in theory should functionally be
> totally unrelated.

That's plausible.  But I can't think of anything useful the PCI core
can do here.  If you can identify a hardware issue in the device, it's
possible you could write a quirk to work around it.  But after the
driver has called pci_enable_device() (the last point at which quirks
are applied), the PCI core is pretty much out of the picture.

Maybe something could be added to one or both drivers to look for
these devices and try deal with them?  I'm afraid I don't have any
good ideas here.

Re: [Bug 43247] O2 micro SD/MMC+1394 controller: 1394 device can't work (Register access failure)

[Stefan Richter]

On Jul 17 Bjorn Helgaas wrote:
> > 1.)  Is there a kernel parameter which Jennifer could try in order
> >      to force serialized PCI driver probing?
> 
> I don't understand the question here.  As far as I know, drivers
> compiled statically into the kernel are already initialized serially,
> via the do_initcalls() -> do_initcall_level(6) path.  The PCI core
> enumerates all the devices, then when we call each driver's
> module_init() function (serially), the module_init() function will
> register the driver, and the driver core will call the driver's
> .probe() function for every matching PCI device.

Jennifer and the other reporters of the issue most certainly all used
modular kernels, i.e. got the drivers loaded by udev.

module_init() doesn't matter, but
  - .probe(),
  - .resume()
do.  So as you say, if the drivers were statically linked, then at least
.probe() would be performed serially, one PCI device after another,
right?  But AFAIK .resume() would still be performed in parallel by a
pool of kernel threads in current kernels.

> > 2.)  If there is one and if this turns out to cure the issue in
> >      testing:
> >      How can I implement serialization between the O2Micro FireWire
> >      .probe() and .resume() on one hand and the O2Micro SDHCI
> >      .probe() and .resume() on the other hand?
> 
> If serialization is required between two drivers, that sounds like a
> driver bug.  What are the two drivers involved?

>From what I can tell, there is no driver bug but an unfortunate interaction
between parts of a combo controller which in theory should functionally be
totally unrelated.

First we had lots and lots of slightly different and inconclusive reports
from various people about the O2Micro FireWire part (or/and the SDHCI
part) failing to work after boot or after PM resume.  Jennifer's recent
reports show that this happens if the two devices are being .probed() at
the same time (or perhaps not exactly at the same time but close together;
at least the firewire-ohci .probe() takes long enough to make some overlap
probable of the .probe()s were called asynchronously baut close together).

The involved drivers are:

    drivers/firewire/firewire-ohci.ko (drivers/firewire/ohci.c)
    drivers/mmc/host/sdhci-pci.ko     (drivers/mmc/host/sdhci-pci.c)


The involved device is either this triple combo device:

0b:00.0 FireWire (IEEE 1394) [0c00]: O2 Micro, Inc. Device [1217:11f7] (rev 05) (prog-if 10 [OHCI])
	Subsystem: Dell Device [1028:04a4]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx+
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 42
	Region 0: Memory at e0130000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: <access denied>
	Kernel driver in use: firewire_ohci
	Kernel modules: firewire-ohci

0b:00.1 SD Host controller [0805]: O2 Micro, Inc. Device [1217:8320] (rev 05) (prog-if 01)
	Subsystem: Dell Device [1028:04a4]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 16
	Region 0: Memory at e0120000 (32-bit, non-prefetchable) [size=512]
	Capabilities: <access denied>
	Kernel driver in use: sdhci-pci
	Kernel modules: sdhci-pci

0b:00.2 Mass storage controller [0180]: O2 Micro, Inc. Device [1217:8330] (rev 05)
	Subsystem: Dell Device [1028:04a4]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin C routed to IRQ 10
	Region 0: Memory at e0110000 (32-bit, non-prefetchable) [size=1K]
	Region 2: Memory at e0100000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: <access denied>


Or this dual combo device:

09:00.0 FireWire (IEEE 1394) [0c00]: O2 Micro, Inc. Device [1217:13f7] (rev 05) (prog-if 10 [OHCI])
	Subsystem: Dell Device [1028:04b4]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 17
	Region 0: Memory at e1a30000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: <access denied>
	Kernel driver in use: firewire_ohci
	Kernel modules: firewire-ohci

09:00.1 SD Host controller [0805]: O2 Micro, Inc. Device [1217:8321] (rev 05) (prog-if 01)
	Subsystem: Dell Device [1028:04b4]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 18
	Region 0: Memory at e1a20000 (32-bit, non-prefetchable) [size=512]
	Capabilities: <access denied>
	Kernel driver in use: sdhci-pci
	Kernel modules: sdhci-pci

-- 
Stefan Richter
-=====-===-- -=== =---=
http://arcgraph.de/sr/