PCI enumeration without a BIOS

PCI enumeration without a BIOS

[Wesley Terpstra]

[-- Attachment #1: Type: text/plain, Size: 1302 bytes --]

Good morning,

I am trying to clean up PCIe support for the RISC-V port. On this
architecture, PCIe enumeration may not have been performed by the
"BIOS", of which there is none. However, it looks like linux is
expecting a valid PCIe configuration during boot. I've attached the
output of dmesg.

In particular, these are the lines that seem wrong to me:
[    5.920000] pci 0000:00:00.0: bridge configuration invalid ([bus
00-00]), reconfiguring
[    5.930000] pci 0000:01:00.0: bridge configuration invalid ([bus
00-00]), reconfiguring
...
[    6.000000] pci 0000:04:00.0: [Firmware Bug]: reg 0x10: invalid BAR
(can't size)
[    6.010000] pci 0000:06:00.0: [Firmware Bug]: reg 0x10: invalid BAR
(can't size)
...

To my eyes the above looks like linux was expecting a pre-enumerated
bus. How can I tell linux to just enumerate everything and not be
surprised? Ideally, I'd like this to be optional, in case a future
boot loader might enumerate PCIe in order to netboot or whatever.

These warnings are also troubling, but seem to be a general weakness
of the xilinx PCIe bridge:
[    6.210000] pci 0000:02:01.0: BAR 7: no space for [io  size 0x1000]
[    6.210000] pci 0000:02:01.0: BAR 7: failed to assign [io  size 0x1000]

Everything actually works in the end, so this is mostly a cosmetic concern.

[-- Attachment #2: dmesg --]
[-- Type: application/octet-stream, Size: 24376 bytes --]

[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
[    0.000000] Linux version 4.11.0-rc1-661305-g4f97179-dirty (terpstra@gamma03.internal.sifive.com) (gcc version 7.0.1 20170208 (experimental) (GCC) ) #7 SMP Tue Apr 4 11:26:16 PDT 2017
[    0.000000] bootconsole [early0] enabled
[    0.000000] Initial ramdisk at: 0xffffffff80021fa8 (2515363 bytes)
[    0.000000] On node 0 totalpages: 786432
[    0.000000] free_area_init_node: node 0, pgdat ffffffff806e2a80, node_mem_map ffffffffbd3f9000
[    0.000000]   Normal zone: 10752 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 786432 pages, LIFO batch:31
[    0.000000] CPU 0: rv64imacs ... enabled
[    0.000000] CPU 1: rv64imacs ... enabled
[    0.000000] CPU 2: rv64imacs ... enabled
[    0.000000] CPU 3: rv64imacs ... enabled
[    0.000000] percpu: Embedded 14 pages/cpu @ffffffffbd3b1000 s27672 r0 d29672 u57344
[    0.000000] pcpu-alloc: s27672 r0 d29672 u57344 alloc=14*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 775680
[    0.000000] Kernel command line: earlyprintk 
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.000000] Sorting __ex_table...
[    0.000000] Memory: 989688K/3145728K available (3526K kernel code, 183K rwdata, 722K rodata, 2623K init, 265K bss, 2156040K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] 	RCU debugfs-based tracing is enabled.
[    0.000000] 	Build-time adjustment of leaf fanout to 64.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=4
[    0.000000] NR_IRQS:0 nr_irqs:0 0
[    0.000000] riscv,cpu_intc,0: 64 local interrupts mapped
[    0.000000] riscv,cpu_intc,1: 64 local interrupts mapped
[    0.000000] riscv,cpu_intc,2: 64 local interrupts mapped
[    0.000000] riscv,cpu_intc,3: 64 local interrupts mapped
[    0.000000] riscv,plic0,c000000: mapped 10 interrupts to 8/8 handlers
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275 ns
[    0.000000] Calibrating delay loop (skipped), value calculated using timer frequency.. 2.00 BogoMIPS (lpj=10000)
[    0.010000] pid_max: default: 32768 minimum: 301
[    0.010000] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.020000] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.050000] smp: Bringing up secondary CPUs ...
[    0.070000] smp: Brought up 1 node, 4 CPUs
[    0.080000] devtmpfs: initialized
[    0.080000] cpu cpu0: Error -2 creating of_node link
[    0.090000] cpu cpu1: Error -2 creating of_node link
[    0.090000] cpu cpu2: Error -2 creating of_node link
[    0.100000] cpu cpu3: Error -2 creating of_node link
[    0.120000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.130000] futex hash table entries: 1024 (order: 4, 65536 bytes)
[    0.140000] NET: Registered protocol family 16
[    0.220000] vgaarb: loaded
[    0.220000] SCSI subsystem initialized
[    0.230000] libata version 3.00 loaded.
[    0.230000] usbcore: registered new interface driver usbfs
[    0.230000] usbcore: registered new interface driver hub
[    0.240000] usbcore: registered new device driver usb
[    0.240000] pps_core: LinuxPPS API ver. 1 registered
[    0.250000] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.250000] PTP clock support registered
[    0.260000] clocksource: Switched to clocksource riscv_clocksource
[    0.270000] NET: Registered protocol family 2
[    0.290000] TCP established hash table entries: 32768 (order: 6, 262144 bytes)
[    0.300000] TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
[    0.320000] TCP: Hash tables configured (established 32768 bind 32768)
[    0.330000] UDP hash table entries: 2048 (order: 4, 65536 bytes)
[    0.340000] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
[    0.350000] NET: Registered protocol family 1
[    0.360000] PCI: CLS 0 bytes, default 64
[    2.990000] Unpacking initramfs...
[    5.680000] console [sbi_console0] enabled
[    5.690000] bootconsole [early0] disabled
[    5.710000] workingset: timestamp_bits=62 max_order=18 bucket_order=0
[    5.840000] io scheduler noop registered
[    5.850000] io scheduler cfq registered (default)
[    5.850000] io scheduler mq-deadline registered
[    5.870000] xilinx-pcie 50000000.pci: PCIe Link is UP
[    5.880000] OF: PCI: host bridge /soc/pci@50000000 ranges:
[    5.880000] OF: PCI:   No bus range found for /soc/pci@50000000, using [bus 00-ff]
[    5.890000] OF: PCI:   MEM 0x60000000..0x7fffffff -> 0x60000000
[    5.900000] xilinx-pcie 50000000.pci: PCI host bridge to bus 0000:00
[    5.900000] pci_bus 0000:00: root bus resource [bus 00-ff]
[    5.910000] pci_bus 0000:00: root bus resource [mem 0x60000000-0x7fffffff]
[    5.920000] pci_bus 0000:00: scanning bus
[    5.920000] pci 0000:00:00.0: [10ee:7111] type 01 class 0x060400
[    5.920000] pci_bus 0000:00: fixups for bus
[    5.920000] pci 0000:00:00.0: scanning [bus 00-00] behind bridge, pass 0
[    5.920000] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    5.930000] pci 0000:00:00.0: scanning [bus 00-00] behind bridge, pass 1
[    5.930000] pci_bus 0000:01: scanning bus
[    5.930000] pci 0000:01:00.0: [10b5:8632] type 01 class 0x060400
[    5.930000] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0001ffff]
[    5.930000] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    5.930000] pci 0000:01:00.0: PME# disabled
[    5.930000] pci_bus 0000:01: fixups for bus
[    5.930000] pci 0000:01:00.0: scanning [bus 00-00] behind bridge, pass 0
[    5.930000] pci 0000:01:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    5.940000] pci 0000:01:00.0: scanning [bus 00-00] behind bridge, pass 1
[    5.940000] pci_bus 0000:02: scanning bus
[    5.940000] pci 0000:02:00.0: [10b5:8632] type 01 class 0x060400
[    5.940000] pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
[    5.940000] pci 0000:02:00.0: PME# disabled
[    5.940000] pci 0000:02:01.0: [10b5:8632] type 01 class 0x060400
[    5.940000] pci 0000:02:01.0: PME# supported from D0 D3hot D3cold
[    5.940000] pci 0000:02:01.0: PME# disabled
[    5.950000] pci 0000:02:02.0: [10b5:8632] type 01 class 0x060400
[    5.950000] pci 0000:02:02.0: PME# supported from D0 D3hot D3cold
[    5.950000] pci 0000:02:02.0: PME# disabled
[    5.950000] pci 0000:02:03.0: [10b5:8632] type 01 class 0x060400
[    5.950000] pci 0000:02:03.0: PME# supported from D0 D3hot D3cold
[    5.950000] pci 0000:02:03.0: PME# disabled
[    5.950000] pci 0000:02:04.0: [10b5:8632] type 01 class 0x060400
[    5.950000] pci 0000:02:04.0: PME# supported from D0 D3hot D3cold
[    5.950000] pci 0000:02:04.0: PME# disabled
[    5.950000] pci_bus 0000:02: fixups for bus
[    5.960000] pci 0000:02:00.0: scanning [bus 00-00] behind bridge, pass 0
[    5.960000] pci 0000:02:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    5.960000] pci 0000:02:01.0: scanning [bus 00-00] behind bridge, pass 0
[    5.960000] pci 0000:02:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    5.970000] pci 0000:02:02.0: scanning [bus 00-00] behind bridge, pass 0
[    5.970000] pci 0000:02:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    5.980000] pci 0000:02:03.0: scanning [bus 00-00] behind bridge, pass 0
[    5.980000] pci 0000:02:03.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    5.990000] pci 0000:02:04.0: scanning [bus 00-00] behind bridge, pass 0
[    5.990000] pci 0000:02:04.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    6.000000] pci 0000:02:00.0: scanning [bus 00-00] behind bridge, pass 1
[    6.000000] pci_bus 0000:03: scanning bus
[    6.000000] pci 0000:03:00.0: [1912:0015] type 00 class 0x0c0330
[    6.000000] pci 0000:03:00.0: reg 0x10: [mem 0x00000000-0x00001fff 64bit]
[    6.000000] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold
[    6.000000] pci 0000:03:00.0: PME# disabled
[    6.000000] pci_bus 0000:03: fixups for bus
[    6.000000] pci_bus 0000:03: bus scan returning with max=03
[    6.000000] pci_bus 0000:03: busn_res: [bus 03-ff] end is updated to 03
[    6.000000] pci 0000:02:01.0: scanning [bus 00-00] behind bridge, pass 1
[    6.000000] pci_bus 0000:04: scanning bus
[    6.000000] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000
[    6.000000] pci 0000:04:00.0: [Firmware Bug]: reg 0x10: invalid BAR (can't size)
[    6.010000] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref]
[    6.010000] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[    6.010000] pci 0000:04:00.0: supports D1 D2
[    6.010000] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    6.010000] pci 0000:04:00.0: PME# disabled
[    6.010000] pci_bus 0000:04: fixups for bus
[    6.010000] pci_bus 0000:04: bus scan returning with max=04
[    6.010000] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[    6.010000] pci 0000:02:02.0: scanning [bus 00-00] behind bridge, pass 1
[    6.010000] pci_bus 0000:05: scanning bus
[    6.010000] pci_bus 0000:05: fixups for bus
[    6.010000] pci_bus 0000:05: bus scan returning with max=05
[    6.010000] pci_bus 0000:05: busn_res: [bus 05-ff] end is updated to 05
[    6.010000] pci 0000:02:03.0: scanning [bus 00-00] behind bridge, pass 1
[    6.010000] pci_bus 0000:06: scanning bus
[    6.010000] pci 0000:06:00.0: [1b4b:9230] type 00 class 0x010601
[    6.010000] pci 0000:06:00.0: [Firmware Bug]: reg 0x10: invalid BAR (can't size)
[    6.020000] pci 0000:06:00.0: [Firmware Bug]: reg 0x14: invalid BAR (can't size)
[    6.030000] pci 0000:06:00.0: [Firmware Bug]: reg 0x18: invalid BAR (can't size)
[    6.040000] pci 0000:06:00.0: [Firmware Bug]: reg 0x1c: invalid BAR (can't size)
[    6.040000] pci 0000:06:00.0: [Firmware Bug]: reg 0x20: invalid BAR (can't size)
[    6.050000] pci 0000:06:00.0: reg 0x24: [mem 0x00900000-0x009007ff]
[    6.050000] pci 0000:06:00.0: reg 0x30: [mem 0xd0000000-0xd000ffff pref]
[    6.050000] pci 0000:06:00.0: calling quirk_dma_func1_alias+0x0/0x40
[    6.050000] pci 0000:06:00.0: Enabling fixed DMA alias to 00.1
[    6.060000] pci 0000:06:00.0: PME# supported from D3hot
[    6.060000] pci 0000:06:00.0: PME# disabled
[    6.060000] pci_bus 0000:06: fixups for bus
[    6.060000] pci_bus 0000:06: bus scan returning with max=06
[    6.060000] pci_bus 0000:06: busn_res: [bus 06-ff] end is updated to 06
[    6.060000] pci 0000:02:04.0: scanning [bus 00-00] behind bridge, pass 1
[    6.060000] pci_bus 0000:07: scanning bus
[    6.060000] pci 0000:07:00.0: [1002:6779] type 00 class 0x030000
[    6.060000] pci 0000:07:00.0: reg 0x10: [mem 0x00000000-0x0fffffff 64bit pref]
[    6.060000] pci 0000:07:00.0: reg 0x18: [mem 0x00000000-0x0001ffff 64bit]
[    6.060000] pci 0000:07:00.0: [Firmware Bug]: reg 0x20: invalid BAR (can't size)
[    6.070000] pci 0000:07:00.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
[    6.070000] pci 0000:07:00.0: calling quirk_no_pm_reset+0x0/0x30
[    6.070000] pci 0000:07:00.0: supports D1 D2
[    6.070000] pci 0000:07:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[    6.080000] pci 0000:07:00.1: [1002:aa98] type 00 class 0x040300
[    6.080000] pci 0000:07:00.1: reg 0x10: [mem 0x00000000-0x00003fff 64bit]
[    6.080000] pci 0000:07:00.1: supports D1 D2
[    6.080000] pci_bus 0000:07: fixups for bus
[    6.080000] pci_bus 0000:07: bus scan returning with max=07
[    6.080000] pci_bus 0000:07: busn_res: [bus 07-ff] end is updated to 07
[    6.080000] pci_bus 0000:02: bus scan returning with max=07
[    6.080000] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 07
[    6.080000] pci_bus 0000:01: bus scan returning with max=07
[    6.080000] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 07
[    6.080000] pci_bus 0000:00: bus scan returning with max=07
[    6.080000] pci 0000:02:01.0: bridge window [io  0x1000-0x0fff] to [bus 04] add_size 1000
[    6.080000] pci 0000:02:01.0: bridge window [mem 0x00100000-0x001fffff 64bit pref] to [bus 04] add_size 200000 add_align 100000
[    6.080000] pci 0000:02:01.0: bridge window [mem 0x00100000-0x000fffff] to [bus 04] add_size 200000 add_align 100000
[    6.080000] pci 0000:01:00.0: bridge window [io  0x1000-0x0fff] to [bus 02-07] add_size 1000
[    6.080000] pci 0000:01:00.0: bridge window [mem 0x00100000-0x003fffff] to [bus 02-07] add_size 200000 add_align 100000
[    6.090000] pci 0000:00:00.0: bridge window [io  0x1000-0x0fff] to [bus 01-07] add_size 1000
[    6.090000] pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x7bffffff]
[    6.090000] pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
[    6.100000] pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
[    6.100000] pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
[    6.110000] pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
[    6.120000] pci 0000:01:00.0: BAR 9: assigned [mem 0x60000000-0x77ffffff 64bit pref]
[    6.130000] pci 0000:01:00.0: BAR 8: assigned [mem 0x78000000-0x784fffff]
[    6.130000] pci 0000:01:00.0: BAR 0: assigned [mem 0x78500000-0x7851ffff]
[    6.140000] pci 0000:01:00.0: BAR 7: no space for [io  size 0x1000]
[    6.150000] pci 0000:01:00.0: BAR 7: failed to assign [io  size 0x1000]
[    6.150000] pci 0000:01:00.0: BAR 7: no space for [io  size 0x1000]
[    6.160000] pci 0000:01:00.0: BAR 7: failed to assign [io  size 0x1000]
[    6.170000] pci 0000:02:04.0: BAR 9: assigned [mem 0x60000000-0x6fffffff 64bit pref]
[    6.170000] pci 0000:02:00.0: BAR 8: assigned [mem 0x78000000-0x780fffff]
[    6.180000] pci 0000:02:01.0: BAR 8: assigned [mem 0x78100000-0x782fffff]
[    6.190000] pci 0000:02:01.0: BAR 9: assigned [mem 0x70000000-0x702fffff 64bit pref]
[    6.190000] pci 0000:02:03.0: BAR 8: assigned [mem 0x78300000-0x783fffff]
[    6.200000] pci 0000:02:04.0: BAR 8: assigned [mem 0x78400000-0x784fffff]
[    6.210000] pci 0000:02:01.0: BAR 7: no space for [io  size 0x1000]
[    6.210000] pci 0000:02:01.0: BAR 7: failed to assign [io  size 0x1000]
[    6.220000] pci 0000:02:01.0: BAR 7: no space for [io  size 0x1000]
[    6.230000] pci 0000:02:01.0: BAR 7: failed to assign [io  size 0x1000]
[    6.230000] pci 0000:03:00.0: BAR 0: assigned [mem 0x78000000-0x78001fff 64bit]
[    6.240000] pci 0000:02:00.0: PCI bridge to [bus 03]
[    6.250000] pci 0000:02:00.0:   bridge window [mem 0x78000000-0x780fffff]
[    6.250000] pci 0000:04:00.0: BAR 4: assigned [mem 0x70000000-0x70003fff 64bit pref]
[    6.260000] pci 0000:04:00.0: BAR 2: assigned [mem 0x70004000-0x70004fff 64bit pref]
[    6.270000] pci 0000:02:01.0: PCI bridge to [bus 04]
[    6.270000] pci 0000:02:01.0:   bridge window [mem 0x78100000-0x782fffff]
[    6.280000] pci 0000:02:01.0:   bridge window [mem 0x70000000-0x702fffff 64bit pref]
[    6.290000] pci 0000:02:02.0: PCI bridge to [bus 05]
[    6.290000] pci 0000:06:00.0: BAR 6: assigned [mem 0x78300000-0x7830ffff pref]
[    6.300000] pci 0000:06:00.0: BAR 5: assigned [mem 0x78310000-0x783107ff]
[    6.310000] pci 0000:02:03.0: PCI bridge to [bus 06]
[    6.310000] pci 0000:02:03.0:   bridge window [mem 0x78300000-0x783fffff]
[    6.320000] pci 0000:07:00.0: BAR 0: assigned [mem 0x60000000-0x6fffffff 64bit pref]
[    6.330000] pci 0000:07:00.0: BAR 2: assigned [mem 0x78400000-0x7841ffff 64bit]
[    6.330000] pci 0000:07:00.0: BAR 6: assigned [mem 0x78420000-0x7843ffff pref]
[    6.340000] pci 0000:07:00.1: BAR 0: assigned [mem 0x78440000-0x78443fff 64bit]
[    6.350000] pci 0000:02:04.0: PCI bridge to [bus 07]
[    6.350000] pci 0000:02:04.0:   bridge window [mem 0x78400000-0x784fffff]
[    6.360000] pci 0000:02:04.0:   bridge window [mem 0x60000000-0x6fffffff 64bit pref]
[    6.370000] pci 0000:01:00.0: PCI bridge to [bus 02-07]
[    6.370000] pci 0000:01:00.0:   bridge window [mem 0x78000000-0x784fffff]
[    6.380000] pci 0000:01:00.0:   bridge window [mem 0x60000000-0x77ffffff 64bit pref]
[    6.390000] pci 0000:00:00.0: PCI bridge to [bus 01-07]
[    6.390000] pci 0000:00:00.0:   bridge window [mem 0x60000000-0x7bffffff]
[    6.400000] pci 0000:00:00.0: fixup irq: got 0
[    6.400000] pci 0000:00:00.0: assigning IRQ 00
[    6.400000] pci 0000:01:00.0: fixup irq: got 27
[    6.400000] pci 0000:01:00.0: assigning IRQ 27
[    6.400000] pci 0000:02:00.0: fixup irq: got 27
[    6.400000] pci 0000:02:00.0: assigning IRQ 27
[    6.400000] pci 0000:02:01.0: fixup irq: got 28
[    6.400000] pci 0000:02:01.0: assigning IRQ 28
[    6.400000] pci 0000:02:02.0: fixup irq: got 29
[    6.400000] pci 0000:02:02.0: assigning IRQ 29
[    6.400000] pci 0000:02:03.0: fixup irq: got 30
[    6.400000] pci 0000:02:03.0: assigning IRQ 30
[    6.410000] pci 0000:02:04.0: fixup irq: got 27
[    6.410000] pci 0000:02:04.0: assigning IRQ 27
[    6.410000] pci 0000:03:00.0: fixup irq: got 27
[    6.410000] pci 0000:03:00.0: assigning IRQ 27
[    6.410000] pci 0000:04:00.0: fixup irq: got 28
[    6.410000] pci 0000:04:00.0: assigning IRQ 28
[    6.410000] pci 0000:06:00.0: fixup irq: got 30
[    6.410000] pci 0000:06:00.0: assigning IRQ 30
[    6.410000] pci 0000:07:00.0: fixup irq: got 27
[    6.410000] pci 0000:07:00.0: assigning IRQ 27
[    6.410000] pci 0000:07:00.1: fixup irq: got 28
[    6.410000] pci 0000:07:00.1: assigning IRQ 28
[    6.410000] pci 0000:03:00.0: calling quirk_usb_early_handoff+0x0/0x6b0
[    6.410000] pci 0000:00:00.0: enabling device (0000 -> 0002)
[    6.410000] pci 0000:00:00.0: enabling bus mastering
[    6.410000] pci 0000:01:00.0: enabling device (0000 -> 0002)
[    6.420000] pci 0000:01:00.0: enabling bus mastering
[    6.420000] pci 0000:02:00.0: enabling device (0000 -> 0002)
[    6.430000] pci 0000:02:00.0: enabling bus mastering
[    6.430000] pci 0000:03:00.0: enabling device (0000 -> 0002)
[    7.090000] ahci 0000:06:00.0: version 3.0
[    7.090000] pci 0000:02:03.0: enabling device (0000 -> 0002)
[    7.090000] pci 0000:02:03.0: enabling bus mastering
[    7.090000] ahci 0000:06:00.0: controller can do FBS, turning on CAP_FBS
[    7.120000] ahci 0000:06:00.0: AHCI 0001.0200 32 slots 8 ports 6 Gbps 0xff impl SATA mode
[    7.120000] ahci 0000:06:00.0: flags: 64bit ncq fbs pio 
[    7.130000] ahci 0000:06:00.0: enabling bus mastering
[    7.170000] scsi host0: ahci
[    7.180000] scsi host1: ahci
[    7.180000] scsi host2: ahci
[    7.190000] scsi host3: ahci
[    7.200000] scsi host4: ahci
[    7.200000] scsi host5: ahci
[    7.210000] scsi host6: ahci
[    7.220000] scsi host7: ahci
[    7.220000] ata1: SATA max UDMA/133 abar m2048@0x78310000 port 0x78310100 irq 31
[    7.230000] ata2: SATA max UDMA/133 abar m2048@0x78310000 port 0x78310180 irq 31
[    7.240000] ata3: SATA max UDMA/133 abar m2048@0x78310000 port 0x78310200 irq 31
[    7.240000] ata4: SATA max UDMA/133 abar m2048@0x78310000 port 0x78310280 irq 31
[    7.250000] ata5: SATA max UDMA/133 abar m2048@0x78310000 port 0x78310300 irq 31
[    7.260000] ata6: SATA max UDMA/133 abar m2048@0x78310000 port 0x78310380 irq 31
[    7.270000] ata7: SATA max UDMA/133 abar m2048@0x78310000 port 0x78310400 irq 31
[    7.270000] ata8: SATA max UDMA/133 abar m2048@0x78310000 port 0x78310480 irq 31
[    7.280000] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    7.290000] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    7.300000] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[    7.300000] pci 0000:02:01.0: enabling device (0000 -> 0002)
[    7.310000] pci 0000:02:01.0: enabling bus mastering
[    7.310000] r8169 0000:04:00.0: enabling device (0000 -> 0002)
[    7.310000] r8169 0000:04:00.0: enabling Mem-Wr-Inval
[    7.310000] r8169 0000:04:00.0: enabling bus mastering
[    7.320000] r8169 0000:04:00.0 eth0: RTL8168evl/8111evl at 0xffffffff44009000, 00:0a:cd:29:9e:45, XID 0c900800 IRQ 32
[    7.330000] r8169 0000:04:00.0 eth0: jumbo features [frames: 9200 bytes, tx checksumming: ko]
[    7.340000] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    7.350000] ehci-pci: EHCI PCI platform driver
[    7.350000] xhci_hcd 0000:03:00.0: enabling bus mastering
[    7.350000] xhci_hcd 0000:03:00.0: xHCI Host Controller
[    7.360000] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 1
[    7.370000] xhci_hcd 0000:03:00.0: hcc params 0x014050cf hci version 0x100 quirks 0x00000090
[    7.380000] xhci_hcd 0000:03:00.0: enabling Mem-Wr-Inval
[    7.390000] hub 1-0:1.0: USB hub found
[    7.390000] hub 1-0:1.0: 2 ports detected
[    7.400000] xhci_hcd 0000:03:00.0: xHCI Host Controller
[    7.410000] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 2
[    7.420000] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    7.430000] hub 2-0:1.0: USB hub found
[    7.430000] hub 2-0:1.0: 2 ports detected
[    7.440000] usbcore: registered new interface driver usb-storage
[    7.450000] usbcore: registered new interface driver usbhid
[    7.450000] usbhid: USB HID core driver
[    7.460000] NET: Registered protocol family 17
[    7.630000] ata5: SATA link down (SStatus 0 SControl 300)
[    7.630000] ata6: SATA link down (SStatus 0 SControl 300)
[    7.640000] ata3: SATA link down (SStatus 0 SControl 300)
[    7.640000] ata4: SATA link down (SStatus 0 SControl 300)
[    7.650000] ata2: SATA link down (SStatus 0 SControl 300)
[    7.650000] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    7.660000] ata7: SATA link down (SStatus 0 SControl 300)
[    7.660000] ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    7.670000] ata8.00: ATAPI: MARVELL VIRTUALL, 1.09, max UDMA/66
[    7.680000] ata1.00: ATA-10: Patriot Blaze, S9FM02.8, max UDMA/133
[    7.680000] ata1.00: 234441648 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[    7.690000] ata8.00: configured for UDMA/66
[    7.700000] ata1.00: configured for UDMA/133
[    7.710000] scsi 0:0:0:0: Direct-Access     ATA      Patriot Blaze    02.8 PQ: 0 ANSI: 5
[    7.730000] sd 0:0:0:0: [sda] 234441648 512-byte logical blocks: (120 GB/112 GiB)
[    7.730000] scsi 7:0:0:0: Processor         Marvell  Console          1.01 PQ: 0 ANSI: 5
[    7.740000] sd 0:0:0:0: [sda] 4096-byte physical blocks
[    7.750000] sd 0:0:0:0: [sda] Write Protect is off
[    7.750000] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    7.750000] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    7.760000] usb 1-1: new low-speed USB device number 2 using xhci_hcd
[    7.790000]  sda: sda1
[    7.800000] sd 0:0:0:0: [sda] Attached SCSI disk
[    7.860000] Freeing unused kernel memory: 2620K
[    7.860000] This architecture does not have kernel memory protection.
[    8.040000] input: CHICONY USB Keyboard as /devices/platform/soc/50000000.pci/pci0000:00/0000:00:00.0/0000:01:00.0/0000:02:00.0/0000:03:00.0/usb1/1-1/1-1:1.0/0003:04F2:0833.0001/input/input0
[    8.060000] hid-generic 0003:04F2:0833.0001: input: USB HID v1.11 Keyboard [CHICONY USB Keyboard] on usb-0000:03:00.0-1/input0
[    8.140000] input: CHICONY USB Keyboard as /devices/platform/soc/50000000.pci/pci0000:00/0000:00:00.0/0000:01:00.0/0000:02:00.0/0000:03:00.0/usb1/1-1/1-1:1.1/0003:04F2:0833.0002/input/input1
[    8.160000] hid-generic 0003:04F2:0833.0002: input: USB HID v1.11 Device [CHICONY USB Keyboard] on usb-0000:03:00.0-1/input1
[   16.280000] random: dd: uninitialized urandom read (512 bytes read)
[   16.830000] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168e-3.fw failed with error -2
[   16.840000] r8169 0000:04:00.0 eth0: unable to load firmware patch rtl_nic/rtl8168e-3.fw (-2)
[   16.880000] r8169 0000:04:00.0 eth0: link down
[   16.880000] r8169 0000:04:00.0 eth0: link down
[   20.340000] r8169 0000:04:00.0 eth0: link up
[   24.120000] random: dropbear: uninitialized urandom read (32 bytes read)
[  683.910000] random: fast init done

Re: PCI enumeration without a BIOS

[Bjorn Helgaas]

Hi Wesley,

On Tue, Apr 04, 2017 at 12:31:57PM -0700, Wesley Terpstra wrote:
> Good morning,
> 
> I am trying to clean up PCIe support for the RISC-V port. On this
> architecture, PCIe enumeration may not have been performed by the
> "BIOS", of which there is none. However, it looks like linux is
> expecting a valid PCIe configuration during boot. I've attached the
> output of dmesg.

If firmware has already configured PCI, Linux generally will use that
configuration.  Linux *should* be able to enumerate PCI devices and
configure them even without any prior firmware configuration.  But
there may be bugs there, since those paths aren't exercised as much.

> In particular, these are the lines that seem wrong to me:
> [    5.920000] pci 0000:00:00.0: bridge configuration invalid ([bus
> 00-00]), reconfiguring
> [    5.930000] pci 0000:01:00.0: bridge configuration invalid ([bus
> 00-00]), reconfiguring

This is normal but possibly worded more alarmingly than necessary.
Bridges power up with secondary/subordinate bus numbers as zero, so
this just means nothing has changed their configuration from the
power-up default.

> [    6.000000] pci 0000:04:00.0: [Firmware Bug]: reg 0x10: invalid BAR
> (can't size)
> [    6.010000] pci 0000:06:00.0: [Firmware Bug]: reg 0x10: invalid BAR
> (can't size)

These are more worrisome.  We discover the size of a BAR by writing
all ones to it and reading it back, which tells us how many bits of
the BAR are hard-wired to zero.  This behavior is prescribed by the
PCI specs, so it's likely a hardware defect.

It's tagged as a "Firmware Bug" because sometimes this problem is a
consequence of a BIOS forgetting to configure something in the device
or forgetting to hide the device from the OS completely.  Possibly the
tag is a little too aggressive -- it's making assumptions that aren't
always true.

> To my eyes the above looks like linux was expecting a pre-enumerated
> bus. How can I tell linux to just enumerate everything and not be
> surprised? Ideally, I'd like this to be optional, in case a future
> boot loader might enumerate PCIe in order to netboot or whatever.
> 
> These warnings are also troubling, but seem to be a general weakness
> of the xilinx PCIe bridge:
> [    6.210000] pci 0000:02:01.0: BAR 7: no space for [io  size 0x1000]
> [    6.210000] pci 0000:02:01.0: BAR 7: failed to assign [io  size 0x1000]

This is because the xilinx host bridge doesn't support I/O space (it
could be that the hardware itself doesn't support it, or maybe the
Linux driver just doesn't support it).  In any event, you have no PCI
devices that actually require I/O space, so this doesn't hurt
anything.  The messages about "BAR 7" are really referring to the
bridge I/O windows, which Linux tries to configure by default.

I wish we had more informative messages about this situation, but it's
a little subtle to detect this situation, so we don't.

> Everything actually works in the end, so this is mostly a cosmetic concern.

Yeah, everything seems mostly working.  The "invalid BAR" things
*could* be an issue -- those registers are not what the PCI spec says
they should be.  Occasionally people put non-BAR registers where BARs
are supposed to be.  We write 0xffffffff to those registers (as the
spec says we must), and if they are non-BARs, we have no idea what
that might do, so it's possible it could break something.  See the
quirks that set non_compliant_bars for examples.

Bjorn

Re: PCI enumeration without a BIOS

[Wesley Terpstra]

[-- Attachment #1: Type: text/plain, Size: 4365 bytes --]

Thank you very much for your detailed analysis of the log!

On Tue, Apr 4, 2017 at 1:56 PM, Bjorn Helgaas <helgaas@kernel.org> wrote:
>> In particular, these are the lines that seem wrong to me:
>> [    5.920000] pci 0000:00:00.0: bridge configuration invalid ([bus
>> 00-00]), reconfiguring
>> [    5.930000] pci 0000:01:00.0: bridge configuration invalid ([bus
>> 00-00]), reconfiguring
>
> This is normal but possibly worded more alarmingly than necessary.
> Bridges power up with secondary/subordinate bus numbers as zero, so
> this just means nothing has changed their configuration from the
> power-up default.

Right. If the firmware had enumerated the bridge already, though, you
would not see this message because it would have been assigned a bus
number already. Right?

>> [    6.000000] pci 0000:04:00.0: [Firmware Bug]: reg 0x10: invalid BAR
>> (can't size)
>> [    6.010000] pci 0000:06:00.0: [Firmware Bug]: reg 0x10: invalid BAR
>> (can't size)
>
> These are more worrisome.  We discover the size of a BAR by writing
> all ones to it and reading it back, which tells us how many bits of
> the BAR are hard-wired to zero.  This behavior is prescribed by the
> PCI specs, so it's likely a hardware defect.

So, you would say it's a hardware problem of the PCIe cards in
question and I can safely ignore it?

> This is because the xilinx host bridge doesn't support I/O space

Yeah. I knew this, but thanks for confirming.

FYI, there is a bug in the pcie-xilinx bridge wrt legacy interrupts.
I've seen several people discussing the same problem for the altera
bridge and the "NW" xilinx bridge, but somehow this bridge still has
the issue. I've attached a patch that solved the problem for me.
Without it I see:

[    6.230000] ------------[ cut here ]------------
[    6.230000] WARNING: CPU: 0 PID: 1 at
/scratch/terpstra/freedom-u-sdk/linux/kernel/irq/irqdomain.c:365
irq_domain_associate+0x190/0x200
[    6.240000] error: hwirq 0x4 is too large for dummy
[    6.250000] CPU: 0 PID: 1 Comm: swapper Not tainted
4.11.0-rc1-661305-g4f97179 #12
[    6.250000] Call Trace:
[    6.260000] [<ffffffff80288660>] walk_stackframe+0x0/0x104
[    6.260000] [<ffffffff80288800>] show_stack+0x38/0x50
[    6.270000] [<ffffffff803c6e30>] dump_stack+0x2c/0x40
[    6.270000] [<ffffffff8028c600>] __warn+0x118/0x130
[    6.280000] [<ffffffff8028c658>] warn_slowpath_fmt+0x40/0x54
[    6.280000] [<ffffffff802c02a8>] irq_domain_associate+0x18c/0x200
[    6.290000] [<ffffffff802c0a3c>] irq_create_mapping+0x90/0xe4
[    6.300000] [<ffffffff802c0be4>] irq_create_fwspec_mapping+0x154/0x288
[    6.300000] [<ffffffff802c0d7c>] irq_create_of_mapping+0x64/0x84
[    6.310000] [<ffffffff804f9cb8>] of_irq_parse_and_map_pci+0x38/0x50
[    6.310000] [<ffffffff80407e00>] pci_fixup_irqs+0x6c/0x114
[    6.320000] [<ffffffff80408e64>] xilinx_pcie_probe+0x308/0x3f0
[    6.330000] [<ffffffff8042cba4>] platform_drv_probe+0x3c/0x88
[    6.330000] [<ffffffff8042aec0>] really_probe+0xbc/0x260
[    6.340000] [<ffffffff8042b138>] __driver_attach+0xd4/0xdc
[    6.340000] [<ffffffff80429200>] bus_for_each_dev+0x68/0xb8
[    6.350000] [<ffffffff8042b640>] driver_attach+0x24/0x38
[    6.350000] [<ffffffff80429db8>] bus_add_driver+0x1b4/0x22c
[    6.360000] [<ffffffff8042bdc0>] driver_register+0x68/0x12c
[    6.360000] [<ffffffff8042da78>] __platform_driver_register+0x48/0x5c
[    6.370000] [<ffffffff8000db38>] xilinx_pcie_driver_init+0x20/0x34
[    6.380000] [<ffffffff80000d48>] do_one_initcall+0x98/0x140
[    6.380000] [<ffffffff80000f38>] kernel_init_freeable+0x148/0x218
[    6.390000] [<ffffffff805ad19c>] kernel_init+0x18/0x114
[    6.390000] [<ffffffff80286cac>] ret_from_syscall+0xc/0x10
[    6.400000] ---[ end trace 8023adf5befc91e0 ]---

... that said, I am not confident my patch is the right fix. So
consider this a bug report + work-around only. :)

> Yeah, everything seems mostly working.  The "invalid BAR" things
> *could* be an issue -- those registers are not what the PCI spec says
> they should be.

The devices in question are:
04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
06:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230 PCIe
SATA 6Gb/s Controller (rev 11)

I am going to plug them in to an Intel machine with 4.11 and see if I
get the same warnings.

[-- Attachment #2: pcie-xilinx.patch --]
[-- Type: application/octet-stream, Size: 776 bytes --]

diff --git a/drivers/pci/host/pcie-xilinx.c b/drivers/pci/host/pcie-xilinx.c
index 7f030f5..619bc67 100644
--- a/drivers/pci/host/pcie-xilinx.c
+++ b/drivers/pci/host/pcie-xilinx.c
@@ -443,7 +443,7 @@ static irqreturn_t xilinx_pcie_intr_handler(int irq, void *data)
 			val = ((val & XILINX_PCIE_RPIFR1_INTR_MASK) >>
 				XILINX_PCIE_RPIFR1_INTR_SHIFT) + 1;
 			generic_handle_irq(irq_find_mapping(port->leg_domain,
-							    val));
+							    val + 1));
 		}
 	}
 
@@ -524,7 +524,7 @@ static int xilinx_pcie_init_irq_domain(struct xilinx_pcie_port *port)
 		return -ENODEV;
 	}
 
-	port->leg_domain = irq_domain_add_linear(pcie_intc_node, 4,
+	port->leg_domain = irq_domain_add_linear(pcie_intc_node, 5,
 						 &intx_domain_ops,
 						 port);
 	if (!port->leg_domain) {

Re: PCI enumeration without a BIOS

[Wesley Terpstra]

On Tue, Apr 4, 2017 at 4:00 PM, Wesley Terpstra <wesley@sifive.com> wrote:
>> Yeah, everything seems mostly working.  The "invalid BAR" things
>> *could* be an issue -- those registers are not what the PCI spec says
>> they should be.
>
> The devices in question are:
> 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
> RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
> 06:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230 PCIe
> SATA 6Gb/s Controller (rev 11)
>
> I am going to plug them in to an Intel machine with 4.11 and see if I
> get the same warnings.

What the devil! I just booted the riscv machine again and the
"Firmware Bug" warnings were gone. All the other warnings remain the
same.

I've tried to reproduce it, but I can't. It's possible that I got the
cards into a bad state from reprogramming my FPGA repeatedly with
sometimes bad firmware. After the cold start that made the warnings go
away, I can't get them back. I'm going to chalk those warnings up to
"only happens when exposed to a bad processor image".

Thanks again for your help.

Re: PCI enumeration without a BIOS

[Bjorn Helgaas]

[-- Attachment #1: Type: text/plain, Size: 4746 bytes --]

[+cc Michal, Ley Foon]

On Tue, Apr 04, 2017 at 04:00:49PM -0700, Wesley Terpstra wrote:
> Thank you very much for your detailed analysis of the log!
> 
> On Tue, Apr 4, 2017 at 1:56 PM, Bjorn Helgaas <helgaas@kernel.org> wrote:
> >> In particular, these are the lines that seem wrong to me:
> >> [    5.920000] pci 0000:00:00.0: bridge configuration invalid ([bus
> >> 00-00]), reconfiguring
> >> [    5.930000] pci 0000:01:00.0: bridge configuration invalid ([bus
> >> 00-00]), reconfiguring
> >
> > This is normal but possibly worded more alarmingly than necessary.
> > Bridges power up with secondary/subordinate bus numbers as zero, so
> > this just means nothing has changed their configuration from the
> > power-up default.
> 
> Right. If the firmware had enumerated the bridge already, though, you
> would not see this message because it would have been assigned a bus
> number already. Right?

Yes.

> >> [    6.000000] pci 0000:04:00.0: [Firmware Bug]: reg 0x10: invalid BAR
> >> (can't size)
> >> [    6.010000] pci 0000:06:00.0: [Firmware Bug]: reg 0x10: invalid BAR
> >> (can't size)
> >
> > These are more worrisome.  We discover the size of a BAR by writing
> > all ones to it and reading it back, which tells us how many bits of
> > the BAR are hard-wired to zero.  This behavior is prescribed by the
> > PCI specs, so it's likely a hardware defect.
> 
> So, you would say it's a hardware problem of the PCIe cards in
> question and I can safely ignore it?

Yes.

> > This is because the xilinx host bridge doesn't support I/O space
> 
> Yeah. I knew this, but thanks for confirming.
> 
> FYI, there is a bug in the pcie-xilinx bridge wrt legacy interrupts.
> I've seen several people discussing the same problem for the altera
> bridge and the "NW" xilinx bridge, but somehow this bridge still has
> the issue. I've attached a patch that solved the problem for me.

This sounds familiar to me, too.  Copying Michal & Ley Foon in case they
know something about it.

> Without it I see:
> 
> [    6.230000] ------------[ cut here ]------------
> [    6.230000] WARNING: CPU: 0 PID: 1 at
> /scratch/terpstra/freedom-u-sdk/linux/kernel/irq/irqdomain.c:365
> irq_domain_associate+0x190/0x200
> [    6.240000] error: hwirq 0x4 is too large for dummy
> [    6.250000] CPU: 0 PID: 1 Comm: swapper Not tainted
> 4.11.0-rc1-661305-g4f97179 #12
> [    6.250000] Call Trace:
> [    6.260000] [<ffffffff80288660>] walk_stackframe+0x0/0x104
> [    6.260000] [<ffffffff80288800>] show_stack+0x38/0x50
> [    6.270000] [<ffffffff803c6e30>] dump_stack+0x2c/0x40
> [    6.270000] [<ffffffff8028c600>] __warn+0x118/0x130
> [    6.280000] [<ffffffff8028c658>] warn_slowpath_fmt+0x40/0x54
> [    6.280000] [<ffffffff802c02a8>] irq_domain_associate+0x18c/0x200
> [    6.290000] [<ffffffff802c0a3c>] irq_create_mapping+0x90/0xe4
> [    6.300000] [<ffffffff802c0be4>] irq_create_fwspec_mapping+0x154/0x288
> [    6.300000] [<ffffffff802c0d7c>] irq_create_of_mapping+0x64/0x84
> [    6.310000] [<ffffffff804f9cb8>] of_irq_parse_and_map_pci+0x38/0x50
> [    6.310000] [<ffffffff80407e00>] pci_fixup_irqs+0x6c/0x114
> [    6.320000] [<ffffffff80408e64>] xilinx_pcie_probe+0x308/0x3f0
> [    6.330000] [<ffffffff8042cba4>] platform_drv_probe+0x3c/0x88
> [    6.330000] [<ffffffff8042aec0>] really_probe+0xbc/0x260
> [    6.340000] [<ffffffff8042b138>] __driver_attach+0xd4/0xdc
> [    6.340000] [<ffffffff80429200>] bus_for_each_dev+0x68/0xb8
> [    6.350000] [<ffffffff8042b640>] driver_attach+0x24/0x38
> [    6.350000] [<ffffffff80429db8>] bus_add_driver+0x1b4/0x22c
> [    6.360000] [<ffffffff8042bdc0>] driver_register+0x68/0x12c
> [    6.360000] [<ffffffff8042da78>] __platform_driver_register+0x48/0x5c
> [    6.370000] [<ffffffff8000db38>] xilinx_pcie_driver_init+0x20/0x34
> [    6.380000] [<ffffffff80000d48>] do_one_initcall+0x98/0x140
> [    6.380000] [<ffffffff80000f38>] kernel_init_freeable+0x148/0x218
> [    6.390000] [<ffffffff805ad19c>] kernel_init+0x18/0x114
> [    6.390000] [<ffffffff80286cac>] ret_from_syscall+0xc/0x10
> [    6.400000] ---[ end trace 8023adf5befc91e0 ]---
> 
> ... that said, I am not confident my patch is the right fix. So
> consider this a bug report + work-around only. :)
> 
> > Yeah, everything seems mostly working.  The "invalid BAR" things
> > *could* be an issue -- those registers are not what the PCI spec says
> > they should be.
> 
> The devices in question are:
> 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
> RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
> 06:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230 PCIe
> SATA 6Gb/s Controller (rev 11)
> 
> I am going to plug them in to an Intel machine with 4.11 and see if I
> get the same warnings.



[-- Attachment #2: pcie-xilinx.patch --]
[-- Type: text/x-diff, Size: 776 bytes --]

diff --git a/drivers/pci/host/pcie-xilinx.c b/drivers/pci/host/pcie-xilinx.c
index 7f030f5..619bc67 100644
--- a/drivers/pci/host/pcie-xilinx.c
+++ b/drivers/pci/host/pcie-xilinx.c
@@ -443,7 +443,7 @@ static irqreturn_t xilinx_pcie_intr_handler(int irq, void *data)
 			val = ((val & XILINX_PCIE_RPIFR1_INTR_MASK) >>
 				XILINX_PCIE_RPIFR1_INTR_SHIFT) + 1;
 			generic_handle_irq(irq_find_mapping(port->leg_domain,
-							    val));
+							    val + 1));
 		}
 	}
 
@@ -524,7 +524,7 @@ static int xilinx_pcie_init_irq_domain(struct xilinx_pcie_port *port)
 		return -ENODEV;
 	}
 
-	port->leg_domain = irq_domain_add_linear(pcie_intc_node, 4,
+	port->leg_domain = irq_domain_add_linear(pcie_intc_node, 5,
 						 &intx_domain_ops,
 						 port);
 	if (!port->leg_domain) {

Re: PCI enumeration without a BIOS

[Ley Foon Tan]

On Wed, 2017-04-05 at 13:37 -0500, Bjorn Helgaas wrote:
> [+cc Michal, Ley Foon]
>=20
> On Tue, Apr 04, 2017 at 04:00:49PM -0700, Wesley Terpstra wrote:
> >=20
> > Thank you very much for your detailed analysis of the log!
> >=20
> > On Tue, Apr 4, 2017 at 1:56 PM, Bjorn Helgaas <helgaas@kernel.org>
> > wrote:
> > >=20
> > > >=20
> > > > In particular, these are the lines that seem wrong to me:
> > > > [=C2=A0=C2=A0=C2=A0=C2=A05.920000] pci 0000:00:00.0: bridge configu=
ration invalid
> > > > ([bus
> > > > 00-00]), reconfiguring
> > > > [=C2=A0=C2=A0=C2=A0=C2=A05.930000] pci 0000:01:00.0: bridge configu=
ration invalid
> > > > ([bus
> > > > 00-00]), reconfiguring
> > > This is normal but possibly worded more alarmingly than
> > > necessary.
> > > Bridges power up with secondary/subordinate bus numbers as zero,
> > > so
> > > this just means nothing has changed their configuration from the
> > > power-up default.
> > Right. If the firmware had enumerated the bridge already, though,
> > you
> > would not see this message because it would have been assigned a
> > bus
> > number already. Right?
> Yes.
>=20
> >=20
> > >=20
> > > >=20
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.000000] pci 0000:04:00.0: [Firmware Bug]=
: reg 0x10:
> > > > invalid BAR
> > > > (can't size)
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.010000] pci 0000:06:00.0: [Firmware Bug]=
: reg 0x10:
> > > > invalid BAR
> > > > (can't size)
> > > These are more worrisome.=C2=A0=C2=A0We discover the size of a BAR by
> > > writing
> > > all ones to it and reading it back, which tells us how many bits
> > > of
> > > the BAR are hard-wired to zero.=C2=A0=C2=A0This behavior is prescribe=
d by
> > > the
> > > PCI specs, so it's likely a hardware defect.
> > So, you would say it's a hardware problem of the PCIe cards in
> > question and I can safely ignore it?
> Yes.
>=20
> >=20
> > >=20
> > > This is because the xilinx host bridge doesn't support I/O space
> > Yeah. I knew this, but thanks for confirming.
> >=20
> > FYI, there is a bug in the pcie-xilinx bridge wrt legacy
> > interrupts.
> > I've seen several people discussing the same problem for the altera
> > bridge and the "NW" xilinx bridge, but somehow this bridge still
> > has
> > the issue. I've attached a patch that solved the problem for me.
> This sounds familiar to me, too.=C2=A0=C2=A0Copying Michal & Ley Foon in =
case
> they
> know something about it.
We have fixed this in last year.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/comm
it/drivers/pci/host/pcie-
altera.c?id=3D99496bd2971fc378226ad4413e5b72c4545714bd

>=20
> >=20
> > Without it I see:
> >=20
> > [=C2=A0=C2=A0=C2=A0=C2=A06.230000] ------------[ cut here ]------------
> > [=C2=A0=C2=A0=C2=A0=C2=A06.230000] WARNING: CPU: 0 PID: 1 at
> > /scratch/terpstra/freedom-u-sdk/linux/kernel/irq/irqdomain.c:365
> > irq_domain_associate+0x190/0x200
> > [=C2=A0=C2=A0=C2=A0=C2=A06.240000] error: hwirq 0x4 is too large for du=
mmy
> > [=C2=A0=C2=A0=C2=A0=C2=A06.250000] CPU: 0 PID: 1 Comm: swapper Not tain=
ted
> > 4.11.0-rc1-661305-g4f97179 #12
> > [=C2=A0=C2=A0=C2=A0=C2=A06.250000] Call Trace:
> > [=C2=A0=C2=A0=C2=A0=C2=A06.260000] [<ffffffff80288660>] walk_stackframe=
+0x0/0x104
> > [=C2=A0=C2=A0=C2=A0=C2=A06.260000] [<ffffffff80288800>] show_stack+0x38=
/0x50
> > [=C2=A0=C2=A0=C2=A0=C2=A06.270000] [<ffffffff803c6e30>] dump_stack+0x2c=
/0x40
> > [=C2=A0=C2=A0=C2=A0=C2=A06.270000] [<ffffffff8028c600>] __warn+0x118/0x=
130
> > [=C2=A0=C2=A0=C2=A0=C2=A06.280000] [<ffffffff8028c658>] warn_slowpath_f=
mt+0x40/0x54
> > [=C2=A0=C2=A0=C2=A0=C2=A06.280000] [<ffffffff802c02a8>]
> > irq_domain_associate+0x18c/0x200
> > [=C2=A0=C2=A0=C2=A0=C2=A06.290000] [<ffffffff802c0a3c>] irq_create_mapp=
ing+0x90/0xe4
> > [=C2=A0=C2=A0=C2=A0=C2=A06.300000] [<ffffffff802c0be4>]
> > irq_create_fwspec_mapping+0x154/0x288
> > [=C2=A0=C2=A0=C2=A0=C2=A06.300000] [<ffffffff802c0d7c>] irq_create_of_m=
apping+0x64/0x84
> > [=C2=A0=C2=A0=C2=A0=C2=A06.310000] [<ffffffff804f9cb8>]
> > of_irq_parse_and_map_pci+0x38/0x50
> > [=C2=A0=C2=A0=C2=A0=C2=A06.310000] [<ffffffff80407e00>] pci_fixup_irqs+=
0x6c/0x114
> > [=C2=A0=C2=A0=C2=A0=C2=A06.320000] [<ffffffff80408e64>] xilinx_pcie_pro=
be+0x308/0x3f0
> > [=C2=A0=C2=A0=C2=A0=C2=A06.330000] [<ffffffff8042cba4>] platform_drv_pr=
obe+0x3c/0x88
> > [=C2=A0=C2=A0=C2=A0=C2=A06.330000] [<ffffffff8042aec0>] really_probe+0x=
bc/0x260
> > [=C2=A0=C2=A0=C2=A0=C2=A06.340000] [<ffffffff8042b138>] __driver_attach=
+0xd4/0xdc
> > [=C2=A0=C2=A0=C2=A0=C2=A06.340000] [<ffffffff80429200>] bus_for_each_de=
v+0x68/0xb8
> > [=C2=A0=C2=A0=C2=A0=C2=A06.350000] [<ffffffff8042b640>] driver_attach+0=
x24/0x38
> > [=C2=A0=C2=A0=C2=A0=C2=A06.350000] [<ffffffff80429db8>] bus_add_driver+=
0x1b4/0x22c
> > [=C2=A0=C2=A0=C2=A0=C2=A06.360000] [<ffffffff8042bdc0>] driver_register=
+0x68/0x12c
> > [=C2=A0=C2=A0=C2=A0=C2=A06.360000] [<ffffffff8042da78>]
> > __platform_driver_register+0x48/0x5c
> > [=C2=A0=C2=A0=C2=A0=C2=A06.370000] [<ffffffff8000db38>]
> > xilinx_pcie_driver_init+0x20/0x34
> > [=C2=A0=C2=A0=C2=A0=C2=A06.380000] [<ffffffff80000d48>] do_one_initcall=
+0x98/0x140
> > [=C2=A0=C2=A0=C2=A0=C2=A06.380000] [<ffffffff80000f38>]
> > kernel_init_freeable+0x148/0x218
> > [=C2=A0=C2=A0=C2=A0=C2=A06.390000] [<ffffffff805ad19c>] kernel_init+0x1=
8/0x114
> > [=C2=A0=C2=A0=C2=A0=C2=A06.390000] [<ffffffff80286cac>] ret_from_syscal=
l+0xc/0x10
> > [=C2=A0=C2=A0=C2=A0=C2=A06.400000] ---[ end trace 8023adf5befc91e0 ]---
> >=20
> > ... that said, I am not confident my patch is the right fix. So
> > consider this a bug report + work-around only. :)
> >=20
> > >=20
> > > Yeah, everything seems mostly working.=C2=A0=C2=A0The "invalid BAR" t=
hings
> > > *could* be an issue -- those registers are not what the PCI spec
> > > says
> > > they should be.
> > The devices in question are:
> > 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
> > RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
> > 06:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230
> > PCIe
> > SATA 6Gb/s Controller (rev 11)
> >=20
> > I am going to plug them in to an Intel machine with 4.11 and see if
> > I
> > get the same warnings.
>=20

Regards
Ley Foon

Re: PCI enumeration without a BIOS

[Ley Foon Tan]

On Wed, 2017-04-05 at 13:37 -0500, Bjorn Helgaas wrote:
> [+cc Michal, Ley Foon]
> 
> On Tue, Apr 04, 2017 at 04:00:49PM -0700, Wesley Terpstra wrote:
> > 
> > Thank you very much for your detailed analysis of the log!
> > 
> > On Tue, Apr 4, 2017 at 1:56 PM, Bjorn Helgaas <helgaas@kernel.org>
> > wrote:
> > > 
> > > > 
> > > > In particular, these are the lines that seem wrong to me:
> > > > [    5.920000] pci 0000:00:00.0: bridge configuration invalid
> > > > ([bus
> > > > 00-00]), reconfiguring
> > > > [    5.930000] pci 0000:01:00.0: bridge configuration invalid
> > > > ([bus
> > > > 00-00]), reconfiguring
> > > This is normal but possibly worded more alarmingly than
> > > necessary.
> > > Bridges power up with secondary/subordinate bus numbers as zero,
> > > so
> > > this just means nothing has changed their configuration from the
> > > power-up default.
> > Right. If the firmware had enumerated the bridge already, though,
> > you
> > would not see this message because it would have been assigned a
> > bus
> > number already. Right?
> Yes.
> 
> > 
> > > 
> > > > 
> > > > [    6.000000] pci 0000:04:00.0: [Firmware Bug]: reg 0x10:
> > > > invalid BAR
> > > > (can't size)
> > > > [    6.010000] pci 0000:06:00.0: [Firmware Bug]: reg 0x10:
> > > > invalid BAR
> > > > (can't size)
> > > These are more worrisome.  We discover the size of a BAR by
> > > writing
> > > all ones to it and reading it back, which tells us how many bits
> > > of
> > > the BAR are hard-wired to zero.  This behavior is prescribed by
> > > the
> > > PCI specs, so it's likely a hardware defect.
> > So, you would say it's a hardware problem of the PCIe cards in
> > question and I can safely ignore it?
> Yes.
> 
> > 
> > > 
> > > This is because the xilinx host bridge doesn't support I/O space
> > Yeah. I knew this, but thanks for confirming.
> > 
> > FYI, there is a bug in the pcie-xilinx bridge wrt legacy
> > interrupts.
> > I've seen several people discussing the same problem for the altera
> > bridge and the "NW" xilinx bridge, but somehow this bridge still
> > has
> > the issue. I've attached a patch that solved the problem for me.
> This sounds familiar to me, too.  Copying Michal & Ley Foon in case
> they
> know something about it.
We have fixed this in last year.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/comm
it/drivers/pci/host/pcie-
altera.c?id=99496bd2971fc378226ad4413e5b72c4545714bd

> 
> > 
> > Without it I see:
> > 
> > [    6.230000] ------------[ cut here ]------------
> > [    6.230000] WARNING: CPU: 0 PID: 1 at
> > /scratch/terpstra/freedom-u-sdk/linux/kernel/irq/irqdomain.c:365
> > irq_domain_associate+0x190/0x200
> > [    6.240000] error: hwirq 0x4 is too large for dummy
> > [    6.250000] CPU: 0 PID: 1 Comm: swapper Not tainted
> > 4.11.0-rc1-661305-g4f97179 #12
> > [    6.250000] Call Trace:
> > [    6.260000] [<ffffffff80288660>] walk_stackframe+0x0/0x104
> > [    6.260000] [<ffffffff80288800>] show_stack+0x38/0x50
> > [    6.270000] [<ffffffff803c6e30>] dump_stack+0x2c/0x40
> > [    6.270000] [<ffffffff8028c600>] __warn+0x118/0x130
> > [    6.280000] [<ffffffff8028c658>] warn_slowpath_fmt+0x40/0x54
> > [    6.280000] [<ffffffff802c02a8>]
> > irq_domain_associate+0x18c/0x200
> > [    6.290000] [<ffffffff802c0a3c>] irq_create_mapping+0x90/0xe4
> > [    6.300000] [<ffffffff802c0be4>]
> > irq_create_fwspec_mapping+0x154/0x288
> > [    6.300000] [<ffffffff802c0d7c>] irq_create_of_mapping+0x64/0x84
> > [    6.310000] [<ffffffff804f9cb8>]
> > of_irq_parse_and_map_pci+0x38/0x50
> > [    6.310000] [<ffffffff80407e00>] pci_fixup_irqs+0x6c/0x114
> > [    6.320000] [<ffffffff80408e64>] xilinx_pcie_probe+0x308/0x3f0
> > [    6.330000] [<ffffffff8042cba4>] platform_drv_probe+0x3c/0x88
> > [    6.330000] [<ffffffff8042aec0>] really_probe+0xbc/0x260
> > [    6.340000] [<ffffffff8042b138>] __driver_attach+0xd4/0xdc
> > [    6.340000] [<ffffffff80429200>] bus_for_each_dev+0x68/0xb8
> > [    6.350000] [<ffffffff8042b640>] driver_attach+0x24/0x38
> > [    6.350000] [<ffffffff80429db8>] bus_add_driver+0x1b4/0x22c
> > [    6.360000] [<ffffffff8042bdc0>] driver_register+0x68/0x12c
> > [    6.360000] [<ffffffff8042da78>]
> > __platform_driver_register+0x48/0x5c
> > [    6.370000] [<ffffffff8000db38>]
> > xilinx_pcie_driver_init+0x20/0x34
> > [    6.380000] [<ffffffff80000d48>] do_one_initcall+0x98/0x140
> > [    6.380000] [<ffffffff80000f38>]
> > kernel_init_freeable+0x148/0x218
> > [    6.390000] [<ffffffff805ad19c>] kernel_init+0x18/0x114
> > [    6.390000] [<ffffffff80286cac>] ret_from_syscall+0xc/0x10
> > [    6.400000] ---[ end trace 8023adf5befc91e0 ]---
> > 
> > ... that said, I am not confident my patch is the right fix. So
> > consider this a bug report + work-around only. :)
> > 
> > > 
> > > Yeah, everything seems mostly working.  The "invalid BAR" things
> > > *could* be an issue -- those registers are not what the PCI spec
> > > says
> > > they should be.
> > The devices in question are:
> > 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
> > RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
> > 06:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230
> > PCIe
> > SATA 6Gb/s Controller (rev 11)
> > 
> > I am going to plug them in to an Intel machine with 4.11 and see if
> > I
> > get the same warnings.
> 

Regards
Ley Foon

Re: PCI enumeration without a BIOS

[Wesley Terpstra]

On Wed, Apr 5, 2017 at 6:11 PM, Ley Foon Tan <ley.foon.tan@intel.com> wrote:
> We have fixed this in last year.

Right, for host/pcie-altera.c
However, the problem persists in host/pcie-xilinx.c and host/pcie-xilinx-nwl.c

> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/comm
> it/drivers/pci/host/pcie-
> altera.c?id=99496bd2971fc378226ad4413e5b72c4545714bd
>
>>
>> >
>> > Without it I see:
>> >
>> > [    6.230000] ------------[ cut here ]------------
>> > [    6.230000] WARNING: CPU: 0 PID: 1 at
>> > /scratch/terpstra/freedom-u-sdk/linux/kernel/irq/irqdomain.c:365
>> > irq_domain_associate+0x190/0x200
>> > [    6.240000] error: hwirq 0x4 is too large for dummy
>> > [    6.250000] CPU: 0 PID: 1 Comm: swapper Not tainted
>> > 4.11.0-rc1-661305-g4f97179 #12
>> > [    6.250000] Call Trace:
>> > [    6.260000] [<ffffffff80288660>] walk_stackframe+0x0/0x104
>> > [    6.260000] [<ffffffff80288800>] show_stack+0x38/0x50
>> > [    6.270000] [<ffffffff803c6e30>] dump_stack+0x2c/0x40
>> > [    6.270000] [<ffffffff8028c600>] __warn+0x118/0x130
>> > [    6.280000] [<ffffffff8028c658>] warn_slowpath_fmt+0x40/0x54
>> > [    6.280000] [<ffffffff802c02a8>]
>> > irq_domain_associate+0x18c/0x200
>> > [    6.290000] [<ffffffff802c0a3c>] irq_create_mapping+0x90/0xe4
>> > [    6.300000] [<ffffffff802c0be4>]
>> > irq_create_fwspec_mapping+0x154/0x288
>> > [    6.300000] [<ffffffff802c0d7c>] irq_create_of_mapping+0x64/0x84
>> > [    6.310000] [<ffffffff804f9cb8>]
>> > of_irq_parse_and_map_pci+0x38/0x50
>> > [    6.310000] [<ffffffff80407e00>] pci_fixup_irqs+0x6c/0x114
>> > [    6.320000] [<ffffffff80408e64>] xilinx_pcie_probe+0x308/0x3f0
>> > [    6.330000] [<ffffffff8042cba4>] platform_drv_probe+0x3c/0x88
>> > [    6.330000] [<ffffffff8042aec0>] really_probe+0xbc/0x260
>> > [    6.340000] [<ffffffff8042b138>] __driver_attach+0xd4/0xdc
>> > [    6.340000] [<ffffffff80429200>] bus_for_each_dev+0x68/0xb8
>> > [    6.350000] [<ffffffff8042b640>] driver_attach+0x24/0x38
>> > [    6.350000] [<ffffffff80429db8>] bus_add_driver+0x1b4/0x22c
>> > [    6.360000] [<ffffffff8042bdc0>] driver_register+0x68/0x12c
>> > [    6.360000] [<ffffffff8042da78>]
>> > __platform_driver_register+0x48/0x5c
>> > [    6.370000] [<ffffffff8000db38>]
>> > xilinx_pcie_driver_init+0x20/0x34
>> > [    6.380000] [<ffffffff80000d48>] do_one_initcall+0x98/0x140
>> > [    6.380000] [<ffffffff80000f38>]
>> > kernel_init_freeable+0x148/0x218
>> > [    6.390000] [<ffffffff805ad19c>] kernel_init+0x18/0x114
>> > [    6.390000] [<ffffffff80286cac>] ret_from_syscall+0xc/0x10
>> > [    6.400000] ---[ end trace 8023adf5befc91e0 ]---
>> >
>> > ... that said, I am not confident my patch is the right fix. So
>> > consider this a bug report + work-around only. :)
>> >
>> > >
>> > > Yeah, everything seems mostly working.  The "invalid BAR" things
>> > > *could* be an issue -- those registers are not what the PCI spec
>> > > says
>> > > they should be.
>> > The devices in question are:
>> > 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
>> > RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
>> > 06:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230
>> > PCIe
>> > SATA 6Gb/s Controller (rev 11)
>> >
>> > I am going to plug them in to an Intel machine with 4.11 and see if
>> > I
>> > get the same warnings.
>>
>
> Regards
> Ley Foon

Re: PCI enumeration without a BIOS

[Ley Foon Tan]

On Wed, 2017-04-05 at 18:59 -0700, Wesley Terpstra wrote:
> On Wed, Apr 5, 2017 at 6:11 PM, Ley Foon Tan <ley.foon.tan@intel.com>
> wrote:
> >=20
> > We have fixed this in last year.
> Right, for host/pcie-altera.c
> However, the problem persists in host/pcie-xilinx.c and host/pcie-
> xilinx-nwl.c

It hasn't integrated to mainline yet.
https://lkml.org/lkml/2016/8/30/198

>=20
> >=20
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/
> > comm
> > it/drivers/pci/host/pcie-
> > altera.c?id=3D99496bd2971fc378226ad4413e5b72c4545714bd
> >=20
> > >=20
> > >=20
> > > >=20
> > > >=20
> > > > Without it I see:
> > > >=20
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.230000] ------------[ cut here ]--------=
----
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.230000] WARNING: CPU: 0 PID: 1 at
> > > > /scratch/terpstra/freedom-u-
> > > > sdk/linux/kernel/irq/irqdomain.c:365
> > > > irq_domain_associate+0x190/0x200
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.240000] error: hwirq 0x4 is too large fo=
r dummy
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.250000] CPU: 0 PID: 1 Comm: swapper Not =
tainted
> > > > 4.11.0-rc1-661305-g4f97179 #12
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.250000] Call Trace:
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.260000] [<ffffffff80288660>] walk_stackf=
rame+0x0/0x104
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.260000] [<ffffffff80288800>] show_stack+=
0x38/0x50
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.270000] [<ffffffff803c6e30>] dump_stack+=
0x2c/0x40
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.270000] [<ffffffff8028c600>] __warn+0x11=
8/0x130
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.280000] [<ffffffff8028c658>] warn_slowpa=
th_fmt+0x40/0x54
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.280000] [<ffffffff802c02a8>]
> > > > irq_domain_associate+0x18c/0x200
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.290000] [<ffffffff802c0a3c>]
> > > > irq_create_mapping+0x90/0xe4
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.300000] [<ffffffff802c0be4>]
> > > > irq_create_fwspec_mapping+0x154/0x288
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.300000] [<ffffffff802c0d7c>]
> > > > irq_create_of_mapping+0x64/0x84
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.310000] [<ffffffff804f9cb8>]
> > > > of_irq_parse_and_map_pci+0x38/0x50
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.310000] [<ffffffff80407e00>] pci_fixup_i=
rqs+0x6c/0x114
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.320000] [<ffffffff80408e64>]
> > > > xilinx_pcie_probe+0x308/0x3f0
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.330000] [<ffffffff8042cba4>]
> > > > platform_drv_probe+0x3c/0x88
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.330000] [<ffffffff8042aec0>] really_prob=
e+0xbc/0x260
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.340000] [<ffffffff8042b138>] __driver_at=
tach+0xd4/0xdc
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.340000] [<ffffffff80429200>] bus_for_eac=
h_dev+0x68/0xb8
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.350000] [<ffffffff8042b640>] driver_atta=
ch+0x24/0x38
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.350000] [<ffffffff80429db8>] bus_add_dri=
ver+0x1b4/0x22c
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.360000] [<ffffffff8042bdc0>] driver_regi=
ster+0x68/0x12c
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.360000] [<ffffffff8042da78>]
> > > > __platform_driver_register+0x48/0x5c
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.370000] [<ffffffff8000db38>]
> > > > xilinx_pcie_driver_init+0x20/0x34
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.380000] [<ffffffff80000d48>] do_one_init=
call+0x98/0x140
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.380000] [<ffffffff80000f38>]
> > > > kernel_init_freeable+0x148/0x218
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.390000] [<ffffffff805ad19c>] kernel_init=
+0x18/0x114
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.390000] [<ffffffff80286cac>] ret_from_sy=
scall+0xc/0x10
> > > > [=C2=A0=C2=A0=C2=A0=C2=A06.400000] ---[ end trace 8023adf5befc91e0 =
]---
> > > >=20
> > > > ... that said, I am not confident my patch is the right fix. So
> > > > consider this a bug report + work-around only. :)
> > > >=20
> > > > >=20
> > > > >=20
> > > > > Yeah, everything seems mostly working.=C2=A0=C2=A0The "invalid BA=
R"
> > > > > things
> > > > > *could* be an issue -- those registers are not what the PCI
> > > > > spec
> > > > > says
> > > > > they should be.
> > > > The devices in question are:
> > > > 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
> > > > RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev
> > > > 06)
> > > > 06:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230
> > > > PCIe
> > > > SATA 6Gb/s Controller (rev 11)
> > > >=20
> > > > I am going to plug them in to an Intel machine with 4.11 and
> > > > see if
> > > > I
> > > > get the same warnings.
> > Regards
> > Ley Foon

Re: PCI enumeration without a BIOS

[Ley Foon Tan]

On Wed, 2017-04-05 at 18:59 -0700, Wesley Terpstra wrote:
> On Wed, Apr 5, 2017 at 6:11 PM, Ley Foon Tan <ley.foon.tan@intel.com>
> wrote:
> > 
> > We have fixed this in last year.
> Right, for host/pcie-altera.c
> However, the problem persists in host/pcie-xilinx.c and host/pcie-
> xilinx-nwl.c

It hasn't integrated to mainline yet.
https://lkml.org/lkml/2016/8/30/198

> 
> > 
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/
> > comm
> > it/drivers/pci/host/pcie-
> > altera.c?id=99496bd2971fc378226ad4413e5b72c4545714bd
> > 
> > > 
> > > 
> > > > 
> > > > 
> > > > Without it I see:
> > > > 
> > > > [    6.230000] ------------[ cut here ]------------
> > > > [    6.230000] WARNING: CPU: 0 PID: 1 at
> > > > /scratch/terpstra/freedom-u-
> > > > sdk/linux/kernel/irq/irqdomain.c:365
> > > > irq_domain_associate+0x190/0x200
> > > > [    6.240000] error: hwirq 0x4 is too large for dummy
> > > > [    6.250000] CPU: 0 PID: 1 Comm: swapper Not tainted
> > > > 4.11.0-rc1-661305-g4f97179 #12
> > > > [    6.250000] Call Trace:
> > > > [    6.260000] [<ffffffff80288660>] walk_stackframe+0x0/0x104
> > > > [    6.260000] [<ffffffff80288800>] show_stack+0x38/0x50
> > > > [    6.270000] [<ffffffff803c6e30>] dump_stack+0x2c/0x40
> > > > [    6.270000] [<ffffffff8028c600>] __warn+0x118/0x130
> > > > [    6.280000] [<ffffffff8028c658>] warn_slowpath_fmt+0x40/0x54
> > > > [    6.280000] [<ffffffff802c02a8>]
> > > > irq_domain_associate+0x18c/0x200
> > > > [    6.290000] [<ffffffff802c0a3c>]
> > > > irq_create_mapping+0x90/0xe4
> > > > [    6.300000] [<ffffffff802c0be4>]
> > > > irq_create_fwspec_mapping+0x154/0x288
> > > > [    6.300000] [<ffffffff802c0d7c>]
> > > > irq_create_of_mapping+0x64/0x84
> > > > [    6.310000] [<ffffffff804f9cb8>]
> > > > of_irq_parse_and_map_pci+0x38/0x50
> > > > [    6.310000] [<ffffffff80407e00>] pci_fixup_irqs+0x6c/0x114
> > > > [    6.320000] [<ffffffff80408e64>]
> > > > xilinx_pcie_probe+0x308/0x3f0
> > > > [    6.330000] [<ffffffff8042cba4>]
> > > > platform_drv_probe+0x3c/0x88
> > > > [    6.330000] [<ffffffff8042aec0>] really_probe+0xbc/0x260
> > > > [    6.340000] [<ffffffff8042b138>] __driver_attach+0xd4/0xdc
> > > > [    6.340000] [<ffffffff80429200>] bus_for_each_dev+0x68/0xb8
> > > > [    6.350000] [<ffffffff8042b640>] driver_attach+0x24/0x38
> > > > [    6.350000] [<ffffffff80429db8>] bus_add_driver+0x1b4/0x22c
> > > > [    6.360000] [<ffffffff8042bdc0>] driver_register+0x68/0x12c
> > > > [    6.360000] [<ffffffff8042da78>]
> > > > __platform_driver_register+0x48/0x5c
> > > > [    6.370000] [<ffffffff8000db38>]
> > > > xilinx_pcie_driver_init+0x20/0x34
> > > > [    6.380000] [<ffffffff80000d48>] do_one_initcall+0x98/0x140
> > > > [    6.380000] [<ffffffff80000f38>]
> > > > kernel_init_freeable+0x148/0x218
> > > > [    6.390000] [<ffffffff805ad19c>] kernel_init+0x18/0x114
> > > > [    6.390000] [<ffffffff80286cac>] ret_from_syscall+0xc/0x10
> > > > [    6.400000] ---[ end trace 8023adf5befc91e0 ]---
> > > > 
> > > > ... that said, I am not confident my patch is the right fix. So
> > > > consider this a bug report + work-around only. :)
> > > > 
> > > > > 
> > > > > 
> > > > > Yeah, everything seems mostly working.  The "invalid BAR"
> > > > > things
> > > > > *could* be an issue -- those registers are not what the PCI
> > > > > spec
> > > > > says
> > > > > they should be.
> > > > The devices in question are:
> > > > 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
> > > > RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev
> > > > 06)
> > > > 06:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230
> > > > PCIe
> > > > SATA 6Gb/s Controller (rev 11)
> > > > 
> > > > I am going to plug them in to an Intel machine with 4.11 and
> > > > see if
> > > > I
> > > > get the same warnings.
> > Regards
> > Ley Foon

Re: PCI enumeration without a BIOS

[Wesley Terpstra]

Is there an equivalent patch for xilinx-pcie.c? That one is for
xilinx-pcie-nwl.c. Also, forgive my ignorance, but if a patch has been
around since August 2016, shouldn't it be in 4.11 by now? (I'm running
4.11-rc1 which has the altera fix but neither of the xilinx changes).

On Wed, Apr 5, 2017 at 7:05 PM, Ley Foon Tan <ley.foon.tan@intel.com> wrote:
> On Wed, 2017-04-05 at 18:59 -0700, Wesley Terpstra wrote:
>> On Wed, Apr 5, 2017 at 6:11 PM, Ley Foon Tan <ley.foon.tan@intel.com>
>> wrote:
>> >
>> > We have fixed this in last year.
>> Right, for host/pcie-altera.c
>> However, the problem persists in host/pcie-xilinx.c and host/pcie-
>> xilinx-nwl.c
>
> It hasn't integrated to mainline yet.
> https://lkml.org/lkml/2016/8/30/198
>
>>
>> >
>> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/
>> > comm
>> > it/drivers/pci/host/pcie-
>> > altera.c?id=99496bd2971fc378226ad4413e5b72c4545714bd
>> >
>> > >
>> > >
>> > > >
>> > > >
>> > > > Without it I see:
>> > > >
>> > > > [    6.230000] ------------[ cut here ]------------
>> > > > [    6.230000] WARNING: CPU: 0 PID: 1 at
>> > > > /scratch/terpstra/freedom-u-
>> > > > sdk/linux/kernel/irq/irqdomain.c:365
>> > > > irq_domain_associate+0x190/0x200
>> > > > [    6.240000] error: hwirq 0x4 is too large for dummy
>> > > > [    6.250000] CPU: 0 PID: 1 Comm: swapper Not tainted
>> > > > 4.11.0-rc1-661305-g4f97179 #12
>> > > > [    6.250000] Call Trace:
>> > > > [    6.260000] [<ffffffff80288660>] walk_stackframe+0x0/0x104
>> > > > [    6.260000] [<ffffffff80288800>] show_stack+0x38/0x50
>> > > > [    6.270000] [<ffffffff803c6e30>] dump_stack+0x2c/0x40
>> > > > [    6.270000] [<ffffffff8028c600>] __warn+0x118/0x130
>> > > > [    6.280000] [<ffffffff8028c658>] warn_slowpath_fmt+0x40/0x54
>> > > > [    6.280000] [<ffffffff802c02a8>]
>> > > > irq_domain_associate+0x18c/0x200
>> > > > [    6.290000] [<ffffffff802c0a3c>]
>> > > > irq_create_mapping+0x90/0xe4
>> > > > [    6.300000] [<ffffffff802c0be4>]
>> > > > irq_create_fwspec_mapping+0x154/0x288
>> > > > [    6.300000] [<ffffffff802c0d7c>]
>> > > > irq_create_of_mapping+0x64/0x84
>> > > > [    6.310000] [<ffffffff804f9cb8>]
>> > > > of_irq_parse_and_map_pci+0x38/0x50
>> > > > [    6.310000] [<ffffffff80407e00>] pci_fixup_irqs+0x6c/0x114
>> > > > [    6.320000] [<ffffffff80408e64>]
>> > > > xilinx_pcie_probe+0x308/0x3f0
>> > > > [    6.330000] [<ffffffff8042cba4>]
>> > > > platform_drv_probe+0x3c/0x88
>> > > > [    6.330000] [<ffffffff8042aec0>] really_probe+0xbc/0x260
>> > > > [    6.340000] [<ffffffff8042b138>] __driver_attach+0xd4/0xdc
>> > > > [    6.340000] [<ffffffff80429200>] bus_for_each_dev+0x68/0xb8
>> > > > [    6.350000] [<ffffffff8042b640>] driver_attach+0x24/0x38
>> > > > [    6.350000] [<ffffffff80429db8>] bus_add_driver+0x1b4/0x22c
>> > > > [    6.360000] [<ffffffff8042bdc0>] driver_register+0x68/0x12c
>> > > > [    6.360000] [<ffffffff8042da78>]
>> > > > __platform_driver_register+0x48/0x5c
>> > > > [    6.370000] [<ffffffff8000db38>]
>> > > > xilinx_pcie_driver_init+0x20/0x34
>> > > > [    6.380000] [<ffffffff80000d48>] do_one_initcall+0x98/0x140
>> > > > [    6.380000] [<ffffffff80000f38>]
>> > > > kernel_init_freeable+0x148/0x218
>> > > > [    6.390000] [<ffffffff805ad19c>] kernel_init+0x18/0x114
>> > > > [    6.390000] [<ffffffff80286cac>] ret_from_syscall+0xc/0x10
>> > > > [    6.400000] ---[ end trace 8023adf5befc91e0 ]---
>> > > >
>> > > > ... that said, I am not confident my patch is the right fix. So
>> > > > consider this a bug report + work-around only. :)
>> > > >
>> > > > >
>> > > > >
>> > > > > Yeah, everything seems mostly working.  The "invalid BAR"
>> > > > > things
>> > > > > *could* be an issue -- those registers are not what the PCI
>> > > > > spec
>> > > > > says
>> > > > > they should be.
>> > > > The devices in question are:
>> > > > 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
>> > > > RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev
>> > > > 06)
>> > > > 06:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230
>> > > > PCIe
>> > > > SATA 6Gb/s Controller (rev 11)
>> > > >
>> > > > I am going to plug them in to an Intel machine with 4.11 and
>> > > > see if
>> > > > I
>> > > > get the same warnings.
>> > Regards
>> > Ley Foon