intel ax200 not properly reinitialized on reboot

intel ax200 not properly reinitialized on reboot

[Jason A. Donenfeld]

Hi,

I have a Thinkpad P1 gen 2 (same as X1 extreme gen 2), which has an
AX200 -- 52:00.0 0280: 8086:2723 (rev 1a). Things mostly work, but
I've noticed that after a reboot, the card stops working.
Specifically, wpa_supplicant can't talk to it anymore. Or, sometimes
it appears to drop off the bus completely. After doing a full
poweroff+poweron (not a reboot), it works again.

Here's what one of the failure modes looks like:

zx2c4@thinkpad ~ $ journalctl -b -1|egrep '(iwl|wpa)'
Dec 10 11:57:55 thinkpad systemd[1]: Created slice system-wpa_supplicant.slice.
Dec 10 11:57:55 thinkpad kernel: iwlwifi 0000:52:00.0: enabling device
(0000 -> 0002)
Dec 10 11:57:55 thinkpad kernel: iwlwifi 0000:52:00.0: Direct firmware
load for iwlwifi-cc-a0-52.ucode failed with error -2
Dec 10 11:57:55 thinkpad kernel: iwlwifi 0000:52:00.0: Direct firmware
load for iwlwifi-cc-a0-51.ucode failed with error -2
Dec 10 11:57:55 thinkpad kernel: iwlwifi 0000:52:00.0: Direct firmware
load for iwlwifi-cc-a0-50.ucode failed with error -2
Dec 10 11:57:55 thinkpad kernel: iwlwifi 0000:52:00.0: Direct firmware
load for iwlwifi-cc-a0-49.ucode failed with error -2
Dec 10 11:57:55 thinkpad kernel: iwlwifi 0000:52:00.0:
TLV_FW_FSEQ_VERSION: FSEQ Version: 43.2.23.17
Dec 10 11:57:55 thinkpad kernel: iwlwifi 0000:52:00.0: Found debug
destination: EXTERNAL_DRAM
Dec 10 11:57:55 thinkpad kernel: iwlwifi 0000:52:00.0: Found debug
configuration: 0
Dec 10 11:57:55 thinkpad kernel: iwlwifi 0000:52:00.0: loaded firmware
version 48.4fa0041f.0 op_mode iwlmvm
Dec 10 11:57:55 thinkpad kernel: iwlwifi 0000:52:00.0: Detected
Intel(R) Wi-Fi 6 AX200 160MHz, REV=0x340
Dec 10 11:57:55 thinkpad kernel: iwlwifi 0000:52:00.0: Applying debug
destination EXTERNAL_DRAM
Dec 10 11:57:55 thinkpad kernel: iwlwifi 0000:52:00.0: Allocated
0x00400000 bytes for firmware monitor.
Dec 10 11:57:55 thinkpad kernel: iwlwifi 0000:52:00.0: base HW
address: sanitized
Dec 10 11:57:56 thinkpad wpa_supplicant[747]: Successfully initialized
wpa_supplicant
Dec 10 11:57:56 thinkpad wpa_supplicant[747]: Could not set interface
wlan0 flags (UP): Connection timed out
Dec 10 11:57:56 thinkpad wpa_supplicant[747]: nl80211: Could not set
interface 'wlan0' UP
Dec 10 11:57:56 thinkpad wpa_supplicant[747]: nl80211: deinit
ifname=wlan0 disabled_11b_rates=0
Dec 10 11:57:56 thinkpad wpa_supplicant[747]: wlan0: Failed to
initialize driver interface
Dec 10 11:57:56 thinkpad systemd[1]: wpa_supplicant@wlan0.service:
Main process exited, code=exited, status=255/EXCEPTION
Dec 10 11:57:56 thinkpad systemd[1]: wpa_supplicant@wlan0.service:
Failed with result 'exit-code'.
Dec 10 11:58:10 thinkpad systemd[1]: Removed slice system-wpa_supplicant.slice.

I assume this is something to do with the bios not powercycling the
pcie bus during reboot, leaving the card in some kind of inconsistent
state. Perhaps it's possible to add a quirk to always powercycle it
before initializing and loading firmware?

Thanks,
Jason

Re: intel ax200 not properly reinitialized on reboot

[Johannes Berg]

On Tue, 2019-12-10 at 12:05 +0100, Jason A. Donenfeld wrote:
> Hi,
> 
> I have a Thinkpad P1 gen 2 (same as X1 extreme gen 2), which has an
> AX200 -- 52:00.0 0280: 8086:2723 (rev 1a). Things mostly work, but
> I've noticed that after a reboot, the card stops working.
> Specifically, wpa_supplicant can't talk to it anymore. Or, sometimes
> it appears to drop off the bus completely. After doing a full
> poweroff+poweron (not a reboot), it works again.

How frequent is this for you? We had a bug report like this on some
other platform where it happens like ... 2-3% of the time?

[snip]

> I assume this is something to do with the bios not powercycling the
> pcie bus during reboot, leaving the card in some kind of inconsistent
> state. Perhaps it's possible to add a quirk to always powercycle it
> before initializing and loading firmware?

Good question, yeah.

Can you stick this info into a bug at bugzilla.kernel.org and add 
linuxwifi@intel.com to it? Or even add me personally since I've dealt
with the other report before.

I can't check the progress of the other report right now, but then I can
track it.

johannes

Re: intel ax200 not properly reinitialized on reboot

[Jason A. Donenfeld]

Hi Johannes,

On Tue, Dec 10, 2019 at 9:34 PM Johannes Berg <johannes@sipsolutions.net> wrote:
> How frequent is this for you? We had a bug report like this on some
> other platform where it happens like ... 2-3% of the time?

It happens 100% of the time that I reboot instead of poweroff+poweron.
> Can you stick this info into a bug at bugzilla.kernel.org and add
> linuxwifi@intel.com to it? Or even add me personally since I've dealt
> with the other report before.

Sure, but I'm unable to change the owner. I filed it as
https://bugzilla.kernel.org/show_bug.cgi?id=205829 if you'd like to
tweak the fields.

Jason

Re: intel ax200 not properly reinitialized on reboot

[Johannes Berg]

Hi Jason,

> > How frequent is this for you? We had a bug report like this on some
> > other platform where it happens like ... 2-3% of the time?
> 
> It happens 100% of the time that I reboot instead of poweroff+poweron.

Oh, wow.

> > Can you stick this info into a bug at bugzilla.kernel.org and add
> > linuxwifi@intel.com to it? Or even add me personally since I've dealt
> > with the other report before.
> 
> Sure, but I'm unable to change the owner. I filed it as
> https://bugzilla.kernel.org/show_bug.cgi?id=205829 if you'd like to
> tweak the fields.

Yep, done, thanks.

I'll check the other bug tomorrow.

johannes