linux-pci.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Toke Høiland-Jørgensen" <toke@redhat.com>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: "Pali Rohár" <pali@kernel.org>,
	vtolkm@gmail.com, linux-pci@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	"Rob Herring" <robh@kernel.org>,
	"Ilias Apalodimas" <ilias.apalodimas@linaro.org>,
	"Marek Behún" <marek.behun@nic.cz>,
	"Thomas Petazzoni" <thomas.petazzoni@bootlin.com>,
	"Jason Cooper" <jason@lakedaemon.net>
Subject: Re: PCI trouble on mvebu (Turris Omnia)
Date: Thu, 29 Oct 2020 23:09:48 +0100	[thread overview]
Message-ID: <87imasof9v.fsf@toke.dk> (raw)
In-Reply-To: <871rhgpyzj.fsf@toke.dk>

Toke Høiland-Jørgensen <toke@redhat.com> writes:

> Bjorn Helgaas <helgaas@kernel.org> writes:
>
>> Another experiment: build kernel without CONFIG_PCIEASPM, set $ROOT
>> and $NIC appropriately, and try the following:
>>
>>   # Set $ROOT and $NIC (update to match your system):
>>
>>     # ROOT=00:02.0
>>     # NIC=02:00.0
>
> (these matched the ath10k card, so just went with that)

And since Marek's latest email mentioned that the WLE900 is especially
problematic, I also tried with the other slot that has the mt76 in it:

# ROOT=00:03.0
# NIC=03:00.0
# setpci -s$ROOT CAP_EXP+0xc.l
0003ac12
# setpci -s$ROOT CAP_EXP+0x10.w
0040
# setpci -s$ROOT CAP_EXP+0x12.w
1011
# setpci -s$NIC CAP_EXP+0xc.l
0047dc11
# setpci -s$NIC CAP_EXP+0x10.w
0000
# setpci -s$NIC CAP_EXP+0x12.w
1011

# setpci -s$ROOT CAP_EXP+0x10.w=0x0020
# sleep 1
# setpci -s$ROOT CAP_EXP+0x12.w
1011
# setpci -s$NIC CAP_EXP+0x12.w
1011

# setpci -s$NIC CAP_EXP+0x10.w=0x0040
# setpci -s$ROOT CAP_EXP+0x10.w=0x0040
# setpci -s$ROOT CAP_EXP+0x10.w=0x0060
# sleep 1
# setpci -s$ROOT CAP_EXP+0x12.w
1011
# setpci -s$NIC CAP_EXP+0x12.w
1011

And based on this I went back and rebuilt the kernel with PCIEASPM
enabled, and now both the WLE200 and the MT76 works with this output:

[    1.544429] mvebu-pcie soc:pcie: host bridge /soc/pcie ranges:
[    1.544455] mvebu-pcie soc:pcie:      MEM 0x00f1080000..0x00f1081fff -> 0x0000080000
[    1.544471] mvebu-pcie soc:pcie:      MEM 0x00f1040000..0x00f1041fff -> 0x0000040000
[    1.544485] mvebu-pcie soc:pcie:      MEM 0x00f1044000..0x00f1045fff -> 0x0000044000
[    1.544500] mvebu-pcie soc:pcie:      MEM 0x00f1048000..0x00f1049fff -> 0x0000048000
[    1.544513] mvebu-pcie soc:pcie:      MEM 0xffffffffffffffff..0x00fffffffe -> 0x0100000000
[    1.544527] mvebu-pcie soc:pcie:       IO 0xffffffffffffffff..0x00fffffffe -> 0x0100000000
[    1.544540] mvebu-pcie soc:pcie:      MEM 0xffffffffffffffff..0x00fffffffe -> 0x0200000000
[    1.544552] mvebu-pcie soc:pcie:       IO 0xffffffffffffffff..0x00fffffffe -> 0x0200000000
[    1.544565] mvebu-pcie soc:pcie:      MEM 0xffffffffffffffff..0x00fffffffe -> 0x0300000000
[    1.544577] mvebu-pcie soc:pcie:       IO 0xffffffffffffffff..0x00fffffffe -> 0x0300000000
[    1.544590] mvebu-pcie soc:pcie:      MEM 0xffffffffffffffff..0x00fffffffe -> 0x0400000000
[    1.544599] mvebu-pcie soc:pcie:       IO 0xffffffffffffffff..0x00fffffffe -> 0x0400000000
[    1.544768] mvebu-pcie soc:pcie: PCI host bridge to bus 0000:00
[    1.544776] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.544783] pci_bus 0000:00: root bus resource [mem 0xf1080000-0xf1081fff] (bus address [0x00080000-0x00081fff])
[    1.544789] pci_bus 0000:00: root bus resource [mem 0xf1040000-0xf1041fff] (bus address [0x00040000-0x00041fff])
[    1.544795] pci_bus 0000:00: root bus resource [mem 0xf1044000-0xf1045fff] (bus address [0x00044000-0x00045fff])
[    1.544801] pci_bus 0000:00: root bus resource [mem 0xf1048000-0xf1049fff] (bus address [0x00048000-0x00049fff])
[    1.544806] pci_bus 0000:00: root bus resource [mem 0xe0000000-0xe7ffffff]
[    1.544811] pci_bus 0000:00: root bus resource [io  0x1000-0xeffff]
[    1.544882] pci 0000:00:01.0: [11ab:6820] type 01 class 0x060400
[    1.544896] pci 0000:00:01.0: reg 0x38: [mem 0x00000000-0x000007ff pref]
[    1.545073] pci 0000:00:02.0: [11ab:6820] type 01 class 0x060400
[    1.545085] pci 0000:00:02.0: reg 0x38: [mem 0x00000000-0x000007ff pref]
[    1.545237] pci 0000:00:03.0: [11ab:6820] type 01 class 0x060400
[    1.545250] pci 0000:00:03.0: reg 0x38: [mem 0x00000000-0x000007ff pref]
[    1.546030] PCI: bus0: Fast back to back transfers disabled
[    1.546037] pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    1.546045] pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    1.546052] pci 0000:00:03.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    1.546132] pci 0000:01:00.0: [168c:002e] type 00 class 0x028000
[    1.546154] pci 0000:01:00.0: reg 0x10: [mem 0xe8000000-0xe800ffff 64bit]
[    1.546263] pci 0000:01:00.0: supports D1
[    1.546268] pci 0000:01:00.0: PME# supported from D0 D1 D3hot
[    1.546377] pci 0000:00:01.0: ASPM: current common clock configuration is inconsistent, reconfiguring
[    1.602042] PCI: bus1: Fast back to back transfers enabled
[    1.602052] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    1.602146] pci 0000:02:00.0: [168c:003c] type 00 class 0x028000
[    1.602169] pci 0000:02:00.0: reg 0x10: [mem 0xea000000-0xea1fffff 64bit]
[    1.602201] pci 0000:02:00.0: reg 0x30: [mem 0xea200000-0xea20ffff pref]
[    1.602280] pci 0000:02:00.0: supports D1 D2
[    1.602377] pci 0000:00:02.0: ASPM: current common clock configuration is inconsistent, reconfiguring
[    1.632025] PCI: bus2: Fast back to back transfers enabled
[    1.632033] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 02
[    1.632117] pci 0000:03:00.0: [14c3:7612] type 00 class 0x028000
[    1.632141] pci 0000:03:00.0: reg 0x10: [mem 0xec000000-0xec0fffff 64bit]
[    1.632175] pci 0000:03:00.0: reg 0x30: [mem 0xec100000-0xec10ffff pref]
[    1.632262] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold
[    1.632373] pci 0000:00:03.0: ASPM: current common clock configuration is inconsistent, reconfiguring
[    1.662037] PCI: bus3: Fast back to back transfers disabled
[    1.662045] pci_bus 0000:03: busn_res: [bus 03-ff] end is updated to 03
[    1.662078] pci 0000:00:01.0: BAR 8: assigned [mem 0xe0000000-0xe00fffff]
[    1.662086] pci 0000:00:02.0: BAR 8: assigned [mem 0xe0200000-0xe04fffff]
[    1.662093] pci 0000:00:03.0: BAR 8: assigned [mem 0xe0600000-0xe07fffff]
[    1.662101] pci 0000:00:01.0: BAR 6: assigned [mem 0xe0100000-0xe01007ff pref]
[    1.662109] pci 0000:00:02.0: BAR 6: assigned [mem 0xe0500000-0xe05007ff pref]
[    1.662116] pci 0000:00:03.0: BAR 6: assigned [mem 0xe0800000-0xe08007ff pref]
[    1.662124] pci 0000:01:00.0: BAR 0: assigned [mem 0xe0000000-0xe000ffff 64bit]
[    1.662135] pci 0000:00:01.0: PCI bridge to [bus 01]
[    1.662142] pci 0000:00:01.0:   bridge window [mem 0xe0000000-0xe00fffff]
[    1.662151] pci 0000:02:00.0: BAR 0: assigned [mem 0xe0200000-0xe03fffff 64bit]
[    1.662158] pci 0000:02:00.0: BAR 0: error updating (0xe0200004 != 0xffffffff)
[    1.662164] pci 0000:02:00.0: BAR 0: error updating (high 0x000000 != 0xffffffff)
[    1.662170] pci 0000:02:00.0: BAR 6: assigned [mem 0xe0400000-0xe040ffff pref]
[    1.662176] pci 0000:00:02.0: PCI bridge to [bus 02]
[    1.662182] pci 0000:00:02.0:   bridge window [mem 0xe0200000-0xe04fffff]
[    1.662190] pci 0000:03:00.0: BAR 0: assigned [mem 0xe0600000-0xe06fffff 64bit]
[    1.662202] pci 0000:03:00.0: BAR 6: assigned [mem 0xe0700000-0xe070ffff pref]
[    1.662207] pci 0000:00:03.0: PCI bridge to [bus 03]
[    1.662212] pci 0000:00:03.0:   bridge window [mem 0xe0600000-0xe07fffff]


This has me somewhat puzzled. Investigating further, it turns out that
if I *remove* the MT76 card, the WLE200 starts failing again. So with
just the WLE* cards plugged in, I went back and tried the setpci
sequence again with the WLE200 (with PCIEASPM disabled):

# ROOT=00:01.0
# NIC=01:00.0
# setpci -s$ROOT CAP_EXP+0xc.l
0003ac12
# setpci -s$ROOT CAP_EXP+0x10.w
0040
# setpci -s$ROOT CAP_EXP+0x12.w
1011
# setpci -s$NIC CAP_EXP+0xc.l
00033c11
# setpci -s$NIC CAP_EXP+0x10.w
0000
# setpci -s$NIC CAP_EXP+0x12.w
1011
# setpci -s$ROOT CAP_EXP+0x10.w=0x0020
# sleep 1
# setpci -s$ROOT CAP_EXP+0x12.w
1011
# setpci -s$NIC CAP_EXP+0x12.w
1011
# setpci -s$NIC CAP_EXP+0x10.w=0x0040
# setpci -s$ROOT CAP_EXP+0x10.w=0x0040
# setpci -s$ROOT CAP_EXP+0x10.w=0x0060
# sleep 1
# setpci -s$ROOT CAP_EXP+0x12.w
1011
# setpci -s$NIC CAP_EXP+0x12.w
1011

-Toke


  reply	other threads:[~2020-10-29 22:09 UTC|newest]

Thread overview: 62+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-27 15:43 PCI trouble on mvebu (Turris Omnia) Toke Høiland-Jørgensen
2020-10-27 17:20 ` Bjorn Helgaas
2020-10-27 17:44   ` ™֟☻̭҇ Ѽ ҉ ®
2020-10-27 18:59     ` Toke Høiland-Jørgensen
2020-10-27 20:20       ` Toke Høiland-Jørgensen
2020-10-27 21:22         ` ™֟☻̭҇ Ѽ ҉ ®
2020-10-27 21:31           ` Toke Høiland-Jørgensen
2020-10-27 22:01             ` ™֟☻̭҇ Ѽ ҉ ®
2020-10-27 22:12               ` Toke Høiland-Jørgensen
2020-10-27 18:56   ` Toke Høiland-Jørgensen
2020-10-28 13:36     ` Toke Høiland-Jørgensen
2020-10-28 14:42       ` Bjorn Helgaas
2020-10-28 15:08         ` Toke Høiland-Jørgensen
2020-10-28 16:40           ` ™֟☻̭҇ Ѽ ҉ ®
2020-10-28 23:16             ` Bjorn Helgaas
2020-10-29 10:09               ` Pali Rohár
2020-10-29 10:56                 ` ™֟☻̭҇ Ѽ ҉ ®
2020-10-29 11:12                 ` Toke Høiland-Jørgensen
2020-10-29 19:30                   ` Bjorn Helgaas
2020-10-29 19:56                     ` ™֟☻̭҇ Ѽ ҉ ®
2020-10-29 19:57                     ` Andrew Lunn
2020-10-29 21:55                       ` Thomas Petazzoni
2020-10-29 20:18                     ` Toke Høiland-Jørgensen
2020-10-29 22:09                       ` Toke Høiland-Jørgensen [this message]
2020-10-29 20:58                     ` Marek Behun
2020-10-30 10:08                       ` Pali Rohár
2020-10-30 10:45                         ` Marek Behun
2020-10-29 21:54                     ` Thomas Petazzoni
2020-10-29 23:15                       ` Toke Høiland-Jørgensen
2020-10-30  8:23                         ` Thomas Petazzoni
2020-10-30 10:15                         ` Pali Rohár
2020-10-29 10:41               ` Toke Høiland-Jørgensen
2020-10-29 11:18                 ` ™֟☻̭҇ Ѽ ҉ ®
2020-10-30 11:23               ` Pali Rohár
2020-10-30 13:02                 ` Toke Høiland-Jørgensen
2020-10-30 14:23                   ` Pali Rohár
2020-10-30 14:54                     ` ™֟☻̭҇ Ѽ ҉ ®
2020-10-31 12:49                       ` Toke Høiland-Jørgensen
2020-11-02 15:24                         ` Pali Rohár
2020-11-02 15:54                           ` Toke Høiland-Jørgensen
2020-11-02 16:18                             ` ™֟☻̭҇ Ѽ ҉ ®
2020-11-02 16:33                               ` Toke Høiland-Jørgensen
2021-03-15 19:58                             ` Pali Rohár
2021-03-16  9:25                               ` Pali Rohár
2021-03-18 22:43                                 ` Toke Høiland-Jørgensen
2021-03-18 23:16                                   ` Pali Rohár
2021-03-26 12:50                                     ` Pali Rohár
2021-03-26 15:25                                       ` Toke Høiland-Jørgensen
2021-03-26 15:34                                         ` Pali Rohár
2021-03-26 16:54                                           ` Toke Høiland-Jørgensen
2021-03-26 17:11                                             ` Pali Rohár
2021-03-26 17:51                                               ` Toke Høiland-Jørgensen
2021-03-29 17:09                                                 ` Pali Rohár
2021-03-31 14:02                                                   ` Toke Høiland-Jørgensen
2021-03-31 16:15                                                     ` Pali Rohár
2021-03-31 16:53                                                       ` Toke Høiland-Jørgensen
2020-10-29  1:21             ` Marek Behun
2020-10-29 15:12           ` Rob Herring
2020-10-27 18:03 ` Marek Behun
2020-10-27 19:00   ` Toke Høiland-Jørgensen
2020-10-27 20:19     ` Marek Behun
2020-10-27 20:49       ` Toke Høiland-Jørgensen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87imasof9v.fsf@toke.dk \
    --to=toke@redhat.com \
    --cc=helgaas@kernel.org \
    --cc=ilias.apalodimas@linaro.org \
    --cc=jason@lakedaemon.net \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=marek.behun@nic.cz \
    --cc=pali@kernel.org \
    --cc=robh@kernel.org \
    --cc=thomas.petazzoni@bootlin.com \
    --cc=vtolkm@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).