Preventing PCI from assigning mem (for MMIO) to bridge device

Preventing PCI from assigning mem (for MMIO) to bridge device

[Rafał Miłecki]

Hello,

I own a home router Linksys WRT300N v1.0 with CardBus slot on PCB and
wireless card attached.

This is MIPS SoC BCM4704 using following code for PCI controller:
1) register_pci_controller
http://lxr.free-electrons.com/source/arch/mips/pci/pci.c?v=3.18#L167
2) ssb_pcicore_init_hostmode
http://lxr.free-electrons.com/source/drivers/ssb/driver_pcicore.c?v=3.18#L317

There are two PCI devices discoverable:
1) 0000:00:00.0 (bridge)
14e4:472d / class 0x068000 / hdr_type 0
2) 0000:00:01.0 (wireless device)
14e4:4329 / class 0x028000 / hdr_type 0

My problem is that PCI subsystem assigns memory to the bridge device:
pci 0000:00:00.0: BAR 1: assigned [mem 0x40000000-0x47ffffff pref]
pci 0000:00:01.0: BAR 0: assigned [mem 0x48000000-0x48003fff]
pci 0000:00:00.0: BAR 0: assigned [mem 0x48004000-0x48005fff]

This ancient & simple PCI controller allows assigning resources to the
wireless device only (slot 1). Trying to assign resources to the
bridge device (writing to slot 0 to registers PCI_BASE_ADDRESS_[0-5])
results in overwriting wireless device (slot 0) configuration!
As you can guess from the above log, the last assignment (targeting
slot 0) will break (overwrite) wireless device (slot 1) configuration.
Trying to access any MMIO register of wireless device will cause "Data
bus error".
For more details please see my comment in OpenWrt Trac:
https://dev.openwrt.org/ticket/12682#comment:11 (there is a nice
configuration dump after every assignment).

So I'm looking for a way to stop PCI subsystem from assigning any
memory to the bridge device (slot 0). I still need it to assign memory
for wireless device (slot 1) as its driver requires MMIO access.

I'm wondering what is the best way to achieve that?

MIPS arch code seems to respect PCI_PROBE_ONLY but this will stop
assigning memory to wireless device (slot 1) too.

I was considering modifying my "struct pci_ops" write callback to
include something like this:
if (extpci_core->cardbusmode && dev == 0 &&
    off >= PCI_BASE_ADDRESS_0 &&
    off <= PCI_BASE_ADDRESS_5)
        return -ENOTSUPP;
It seems to be working fine, all I get with above change is:
pci 0000:00:01.0: BAR 0: assigned [mem 0x40000000-0x40003fff]
, but I need an opinion if this is a correct solution.

-- 
Rafał

Re: Preventing PCI from assigning mem (for MMIO) to bridge device

[Bjorn Helgaas]

Hi Rafał,

On Tue, Apr 7, 2015 at 4:51 AM, Rafał Miłecki <zajec5@gmail.com> wrote:
> Hello,
>
> I own a home router Linksys WRT300N v1.0 with CardBus slot on PCB and
> wireless card attached.
>
> This is MIPS SoC BCM4704 using following code for PCI controller:
> 1) register_pci_controller
> http://lxr.free-electrons.com/source/arch/mips/pci/pci.c?v=3.18#L167
> 2) ssb_pcicore_init_hostmode
> http://lxr.free-electrons.com/source/drivers/ssb/driver_pcicore.c?v=3.18#L317
>
> There are two PCI devices discoverable:
> 1) 0000:00:00.0 (bridge)
> 14e4:472d / class 0x068000 / hdr_type 0
> 2) 0000:00:01.0 (wireless device)
> 14e4:4329 / class 0x028000 / hdr_type 0
>
> My problem is that PCI subsystem assigns memory to the bridge device:
> pci 0000:00:00.0: BAR 1: assigned [mem 0x40000000-0x47ffffff pref]
> pci 0000:00:01.0: BAR 0: assigned [mem 0x48000000-0x48003fff]
> pci 0000:00:00.0: BAR 0: assigned [mem 0x48004000-0x48005fff]
>
> This ancient & simple PCI controller allows assigning resources to the
> wireless device only (slot 1). Trying to assign resources to the
> bridge device (writing to slot 0 to registers PCI_BASE_ADDRESS_[0-5])
> results in overwriting wireless device (slot 0) configuration!
> As you can guess from the above log, the last assignment (targeting
> slot 0) will break (overwrite) wireless device (slot 1) configuration.
> Trying to access any MMIO register of wireless device will cause "Data
> bus error".
> For more details please see my comment in OpenWrt Trac:
> https://dev.openwrt.org/ticket/12682#comment:11 (there is a nice
> configuration dump after every assignment).
>
> So I'm looking for a way to stop PCI subsystem from assigning any
> memory to the bridge device (slot 0). I still need it to assign memory
> for wireless device (slot 1) as its driver requires MMIO access.
>
> I'm wondering what is the best way to achieve that?
>
> MIPS arch code seems to respect PCI_PROBE_ONLY but this will stop
> assigning memory to wireless device (slot 1) too.
>
> I was considering modifying my "struct pci_ops" write callback to
> include something like this:
> if (extpci_core->cardbusmode && dev == 0 &&
>     off >= PCI_BASE_ADDRESS_0 &&
>     off <= PCI_BASE_ADDRESS_5)
>         return -ENOTSUPP;
> It seems to be working fine, all I get with above change is:
> pci 0000:00:01.0: BAR 0: assigned [mem 0x40000000-0x40003fff]
> , but I need an opinion if this is a correct solution.

I'd have to understand the problem better to give you a good opinion.
It sounds like you think there's a hardware defect in the bridge?  If
that's the case, we can often write a quirk to work around it.

Could you open a report at bugzilla.kernel.org against drivers/PCI,
and attach a complete dmesg log and output of "lspci -vv"?  The newer
the kernel you can try, the better.

Bjorn