* intel AX200 crash on 5.2.7+
@ 2019-09-09 17:03 Ben Greear
2019-09-09 17:40 ` Johannes Berg
0 siblings, 1 reply; 4+ messages in thread
From: Ben Greear @ 2019-09-09 17:03 UTC (permalink / raw)
To: linux-wireless
Hello,
Looks like we managed to crash the AX200 firmware. This was running 5.2.7+ kernel
in an apu2 platform.
Is there a better place to report/discuss this?
[ 6065.589162] iwlwifi 0000:01:00.0: iwl_trans_send_cmd bad state = 0
[ 6065.594315] iwlwifi 0000:01:00.0: iwl_trans_send_cmd bad state = 0
[ 6066.028461] alloc_contig_range: 4 callbacks suppressed
[ 6066.028468] alloc_contig_range: [11cf00, 11cf08) PFNs busy
[ 6066.028891] alloc_contig_range: [11cf00, 11cf08) PFNs busy
[ 6066.029418] alloc_contig_range: [11cf00, 11cf08) PFNs busy
[ 6066.029738] alloc_contig_range: [11cf00, 11cf08) PFNs busy
[ 6066.029991] alloc_contig_range: [11cf20, 11cf28) PFNs busy
[ 6066.030359] alloc_contig_range: [11cf28, 11cf30) PFNs busy
[ 6066.030686] alloc_contig_range: [11cf00, 11cf08) PFNs busy
[ 6066.030879] alloc_contig_range: [11cf20, 11cf28) PFNs busy
[ 6066.031175] alloc_contig_range: [11cf28, 11cf30) PFNs busy
[ 6066.031622] alloc_contig_range: [11cf00, 11cf08) PFNs busy
[ 6066.159033] iwlwifi 0000:01:00.0: Microcode SW error detected. Restarting 0x0.
[ 6066.165395] iwlwifi 0000:01:00.0: Start IWL Error Log Dump:
[ 6066.170089] iwlwifi 0000:01:00.0: Status: 0x00000080, count: 6
[ 6066.175035] iwlwifi 0000:01:00.0: Loaded firmware version: 46.3cfab8da.0
[ 6066.180908] iwlwifi 0000:01:00.0: 0x00000942 | ADVANCED_SYSASSERT
[ 6066.186838] iwlwifi 0000:01:00.0: 0x000022F0 | trm_hw_status0
[ 6066.191483] iwlwifi 0000:01:00.0: 0x00000000 | trm_hw_status1
[ 6066.196141] iwlwifi 0000:01:00.0: 0x004FC308 | branchlink2
[ 6066.200492] iwlwifi 0000:01:00.0: 0x000013A6 | interruptlink1
[ 6066.205108] iwlwifi 0000:01:00.0: 0x000013A6 | interruptlink2
[ 6066.209728] iwlwifi 0000:01:00.0: 0x00030018 | data1
[ 6066.213631] iwlwifi 0000:01:00.0: 0x06000102 | data2
[ 6066.217421] iwlwifi 0000:01:00.0: 0x010C0605 | data3
[ 6066.221279] iwlwifi 0000:01:00.0: 0x00000000 | beacon time
[ 6066.225677] iwlwifi 0000:01:00.0: 0x00016A84 | tsf low
[ 6066.229754] iwlwifi 0000:01:00.0: 0x00000000 | tsf hi
[ 6066.233656] iwlwifi 0000:01:00.0: 0x00000000 | time gp1
[ 6066.237702] iwlwifi 0000:01:00.0: 0x00016A85 | time gp2
[ 6066.241771] iwlwifi 0000:01:00.0: 0x00000001 | uCode revision type
[ 6066.246840] iwlwifi 0000:01:00.0: 0x0000002E | uCode version major
[ 6066.251970] iwlwifi 0000:01:00.0: 0x3CFAB8DA | uCode version minor
[ 6066.256990] iwlwifi 0000:01:00.0: 0x00000340 | hw version
[ 6066.261300] iwlwifi 0000:01:00.0: 0x18C89000 | board version
[ 6066.265877] iwlwifi 0000:01:00.0: 0x8008FD0C | hcmd
[ 6066.269662] iwlwifi 0000:01:00.0: 0x82121000 | isr0
[ 6066.273458] iwlwifi 0000:01:00.0: 0x00004000 | isr1
[ 6066.277145] iwlwifi 0000:01:00.0: 0x08F00002 | isr2
[ 6066.280873] iwlwifi 0000:01:00.0: 0x04C0001C | isr3
[ 6066.284666] iwlwifi 0000:01:00.0: 0x00000000 | isr4
[ 6066.288464] iwlwifi 0000:01:00.0: 0x00000000 | last cmd Id
[ 6066.292860] iwlwifi 0000:01:00.0: 0x000151BE | wait_event
[ 6066.297132] iwlwifi 0000:01:00.0: 0x00000000 | l2p_control
[ 6066.301482] iwlwifi 0000:01:00.0: 0x00000000 | l2p_duration
[ 6066.305997] iwlwifi 0000:01:00.0: 0x00000000 | l2p_mhvalid
[ 6066.310347] iwlwifi 0000:01:00.0: 0x00000000 | l2p_addr_match
[ 6066.315031] iwlwifi 0000:01:00.0: 0x00000009 | lmpm_pmg_sel
[ 6066.319564] iwlwifi 0000:01:00.0: 0x16042142 | timestamp
[ 6066.323790] iwlwifi 0000:01:00.0: 0x00000024 | flow_handler
[ 6066.328256] iwlwifi 0000:01:00.0: Start IWL Error Log Dump:
[ 6066.332696] iwlwifi 0000:01:00.0: Status: 0x00000080, count: 7
[ 6066.337412] iwlwifi 0000:01:00.0: 0x20000070 | NMI_INTERRUPT_LMAC_FATAL
[ 6066.342996] iwlwifi 0000:01:00.0: 0x00000000 | umac branchlink1
[ 6066.347956] iwlwifi 0000:01:00.0: 0xC008B1C0 | umac branchlink2
[ 6066.352839] iwlwifi 0000:01:00.0: 0x80489F5E | umac interruptlink1
[ 6066.357873] iwlwifi 0000:01:00.0: 0x80489F5E | umac interruptlink2
[ 6066.362946] iwlwifi 0000:01:00.0: 0x00000400 | umac data1
[ 6066.367230] iwlwifi 0000:01:00.0: 0x80489F5E | umac data2
[ 6066.371543] iwlwifi 0000:01:00.0: 0xDEADBEEF | umac data3
[ 6066.375864] iwlwifi 0000:01:00.0: 0x0000002E | umac major
[ 6066.380130] iwlwifi 0000:01:00.0: 0x3CFAB8DA | umac minor
[ 6066.384454] iwlwifi 0000:01:00.0: 0x00016A96 | frame pointer
[ 6066.389009] iwlwifi 0000:01:00.0: 0xC088628C | stack pointer
[ 6066.393542] iwlwifi 0000:01:00.0: 0x00010C00 | last host cmd
[ 6066.398258] iwlwifi 0000:01:00.0: 0x00000000 | isr status reg
[ 6066.403123] iwlwifi 0000:01:00.0: Fseq Registers:
[ 6066.406667] iwlwifi 0000:01:00.0: 0xE0000000 | FSEQ_ERROR_CODE
[ 6066.411353] iwlwifi 0000:01:00.0: 0x80290001 | FSEQ_TOP_INIT_VERSION
[ 6066.416585] iwlwifi 0000:01:00.0: 0x80050008 | FSEQ_CNVIO_INIT_VERSION
[ 6066.421945] iwlwifi 0000:01:00.0: 0x0000A503 | FSEQ_OTP_VERSION
[ 6066.426745] iwlwifi 0000:01:00.0: 0x80000003 | FSEQ_TOP_CONTENT_VERSION
[ 6066.432205] iwlwifi 0000:01:00.0: 0x4552414E | FSEQ_ALIVE_TOKEN
[ 6066.436945] iwlwifi 0000:01:00.0: 0x00100530 | FSEQ_CNVI_ID
[ 6066.441305] iwlwifi 0000:01:00.0: 0x00000532 | FSEQ_CNVR_ID
[ 6066.445706] iwlwifi 0000:01:00.0: 0x00100530 | CNVI_AUX_MISC_CHIP
[ 6066.450683] iwlwifi 0000:01:00.0: 0x00000532 | CNVR_AUX_MISC_CHIP
[ 6066.455696] iwlwifi 0000:01:00.0: 0x05B0905B | CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[ 6066.462428] iwlwifi 0000:01:00.0: 0x0000025B | CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[ 6066.469601] iwlwifi 0000:01:00.0: Failed to start RT ucode: -5
[ 6066.469800] iwlwifi 0000:01:00.0: Collecting data: trigger 2 fired.
[ 6066.474667] ieee80211 wiphy0: Hardware restart was requested
[ 6066.474705] iwlwifi 0000:01:00.0: Firmware not running - cannot dump error
[ 6066.670312] iwlwifi 0000:01:00.0: Microcode SW error detected. Restarting 0x0.
[ 6066.676579] iwlwifi 0000:01:00.0: Start IWL Error Log Dump:
[ 6066.681073] iwlwifi 0000:01:00.0: Status: 0x00000090, count: 6
[ 6066.686028] iwlwifi 0000:01:00.0: Loaded firmware version: 46.3cfab8da.0
[ 6066.691805] iwlwifi 0000:01:00.0: 0x00000942 | ADVANCED_SYSASSERT
[ 6066.697807] iwlwifi 0000:01:00.0: 0x000022F0 | trm_hw_status0
[ 6066.702552] iwlwifi 0000:01:00.0: 0x00000000 | trm_hw_status1
[ 6066.707275] iwlwifi 0000:01:00.0: 0x004FC308 | branchlink2
[ 6066.711613] iwlwifi 0000:01:00.0: 0x000013A6 | interruptlink1
[ 6066.716231] iwlwifi 0000:01:00.0: 0x000013A6 | interruptlink2
[ 6066.720830] iwlwifi 0000:01:00.0: 0xFFF3001D | data1
[ 6066.724695] iwlwifi 0000:01:00.0: 0x06000201 | data2
[ 6066.728540] iwlwifi 0000:01:00.0: 0x01200603 | data3
[ 6066.732462] iwlwifi 0000:01:00.0: 0x00000000 | beacon time
[ 6066.736908] iwlwifi 0000:01:00.0: 0x00016CBA | tsf low
[ 6066.740883] iwlwifi 0000:01:00.0: 0x00000000 | tsf hi
[ 6066.744841] iwlwifi 0000:01:00.0: 0x00000000 | time gp1
[ 6066.748946] iwlwifi 0000:01:00.0: 0x00016CBA | time gp2
[ 6066.753177] iwlwifi 0000:01:00.0: 0x00000001 | uCode revision type
[ 6066.758309] iwlwifi 0000:01:00.0: 0x0000002E | uCode version major
[ 6066.763468] iwlwifi 0000:01:00.0: 0x3CFAB8DA | uCode version minor
[ 6066.768602] iwlwifi 0000:01:00.0: 0x00000340 | hw version
[ 6066.772947] iwlwifi 0000:01:00.0: 0x18C89000 | board version
[ 6066.777493] iwlwifi 0000:01:00.0: 0x8008FD0C | hcmd
[ 6066.781223] iwlwifi 0000:01:00.0: 0x82121000 | isr0
[ 6066.784900] iwlwifi 0000:01:00.0: 0x00000000 | isr1
[ 6066.788623] iwlwifi 0000:01:00.0: 0x08F00002 | isr2
[ 6066.792304] iwlwifi 0000:01:00.0: 0x04C0001C | isr3
[ 6066.796062] iwlwifi 0000:01:00.0: 0x00000000 | isr4
[ 6066.799853] iwlwifi 0000:01:00.0: 0x00000000 | last cmd Id
[ 6066.804214] iwlwifi 0000:01:00.0: 0x000151BE | wait_event
[ 6066.808507] iwlwifi 0000:01:00.0: 0x00000000 | l2p_control
[ 6066.812858] iwlwifi 0000:01:00.0: 0x00000000 | l2p_duration
[ 6066.817280] iwlwifi 0000:01:00.0: 0x00000000 | l2p_mhvalid
[ 6066.821654] iwlwifi 0000:01:00.0: 0x00000000 | l2p_addr_match
[ 6066.826302] iwlwifi 0000:01:00.0: 0x00000009 | lmpm_pmg_sel
[ 6066.830814] iwlwifi 0000:01:00.0: 0x16042142 | timestamp
[ 6066.835048] iwlwifi 0000:01:00.0: 0x00000024 | flow_handler
[ 6066.839597] iwlwifi 0000:01:00.0: Start IWL Error Log Dump:
[ 6066.843962] iwlwifi 0000:01:00.0: Status: 0x00000090, count: 7
[ 6066.848688] iwlwifi 0000:01:00.0: 0x20000070 | NMI_INTERRUPT_LMAC_FATAL
[ 6066.854249] iwlwifi 0000:01:00.0: 0x00000000 | umac branchlink1
[ 6066.859120] iwlwifi 0000:01:00.0: 0xC008B1C0 | umac branchlink2
[ 6066.863939] iwlwifi 0000:01:00.0: 0x80489F5E | umac interruptlink1
[ 6066.869063] iwlwifi 0000:01:00.0: 0x80489F5E | umac interruptlink2
[ 6066.874151] iwlwifi 0000:01:00.0: 0x00000400 | umac data1
[ 6066.878595] iwlwifi 0000:01:00.0: 0x80489F5E | umac data2
[ 6066.882851] iwlwifi 0000:01:00.0: 0xDEADBEEF | umac data3
[ 6066.887132] iwlwifi 0000:01:00.0: 0x0000002E | umac major
[ 6066.891368] iwlwifi 0000:01:00.0: 0x3CFAB8DA | umac minor
[ 6066.895677] iwlwifi 0000:01:00.0: 0x00016CCC | frame pointer
[ 6066.900213] iwlwifi 0000:01:00.0: 0xC088628C | stack pointer
[ 6066.904821] iwlwifi 0000:01:00.0: 0x00010C00 | last host cmd
[ 6066.909553] iwlwifi 0000:01:00.0: 0x00000000 | isr status reg
[ 6066.914192] iwlwifi 0000:01:00.0: Fseq Registers:
[ 6066.917799] iwlwifi 0000:01:00.0: 0xE0000000 | FSEQ_ERROR_CODE
[ 6066.922556] iwlwifi 0000:01:00.0: 0x80290001 | FSEQ_TOP_INIT_VERSION
[ 6066.927827] iwlwifi 0000:01:00.0: 0x80050008 | FSEQ_CNVIO_INIT_VERSION
[ 6066.933241] iwlwifi 0000:01:00.0: 0x0000A503 | FSEQ_OTP_VERSION
[ 6066.938043] iwlwifi 0000:01:00.0: 0x80000003 | FSEQ_TOP_CONTENT_VERSION
[ 6066.943689] iwlwifi 0000:01:00.0: 0x4552414E | FSEQ_ALIVE_TOKEN
[ 6066.948427] iwlwifi 0000:01:00.0: 0x00100530 | FSEQ_CNVI_ID
[ 6066.952870] iwlwifi 0000:01:00.0: 0x00000532 | FSEQ_CNVR_ID
[ 6066.957297] iwlwifi 0000:01:00.0: 0x00100530 | CNVI_AUX_MISC_CHIP
[ 6066.962267] iwlwifi 0000:01:00.0: 0x00000532 | CNVR_AUX_MISC_CHIP
[ 6066.967218] iwlwifi 0000:01:00.0: 0x05B0905B | CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[ 6066.973944] iwlwifi 0000:01:00.0: 0x0000025B | CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[ 6066.981216] iwlwifi 0000:01:00.0: Firmware error during reconfiguration - reprobe!
[ 6066.981231] iwlwifi 0000:01:00.0: Failed to start RT ucode: -5
[ 6066.988465] iwlwifi 0000:01:00.0: LED command failed: -5
[ 6066.992905] iwlwifi 0000:01:00.0: Collecting data: trigger 16 fired.
[ 6066.992941] iwlwifi 0000:01:00.0: LED command failed: -5
[ 6067.242271] iwlwifi 0000:01:00.0: Firmware not running - cannot dump error
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: intel AX200 crash on 5.2.7+
2019-09-09 17:03 intel AX200 crash on 5.2.7+ Ben Greear
@ 2019-09-09 17:40 ` Johannes Berg
2019-09-09 17:54 ` Ben Greear
0 siblings, 1 reply; 4+ messages in thread
From: Johannes Berg @ 2019-09-09 17:40 UTC (permalink / raw)
To: Ben Greear, linux-wireless
On Mon, 2019-09-09 at 10:03 -0700, Ben Greear wrote:
> Hello,
>
> Looks like we managed to crash the AX200 firmware. This was running 5.2.7+ kernel
> in an apu2 platform.
>
> Is there a better place to report/discuss this?
This is OK for first reports, but usually we'll ask to file a bug on
bugzilla.kernel.org (and assign to linuxwifi@intel.com if you can? Not
sure it's possible - or add that to CC at least)
> [ 6066.180908] iwlwifi 0000:01:00.0: 0x00000942 | ADVANCED_SYSASSERT
Hmm, that's a calibration failure.
Did you do anything special in that environment?
I guess filing a bug would be best, so we have a record for it and can
hand it off to the firmware team or similar.
johannes
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: intel AX200 crash on 5.2.7+
2019-09-09 17:40 ` Johannes Berg
@ 2019-09-09 17:54 ` Ben Greear
2019-10-03 17:51 ` Ben Greear
0 siblings, 1 reply; 4+ messages in thread
From: Ben Greear @ 2019-09-09 17:54 UTC (permalink / raw)
To: Johannes Berg, linux-wireless
On 9/9/19 10:40 AM, Johannes Berg wrote:
> On Mon, 2019-09-09 at 10:03 -0700, Ben Greear wrote:
>> Hello,
>>
>> Looks like we managed to crash the AX200 firmware. This was running 5.2.7+ kernel
>> in an apu2 platform.
>>
>> Is there a better place to report/discuss this?
>
> This is OK for first reports, but usually we'll ask to file a bug on
> bugzilla.kernel.org (and assign to linuxwifi@intel.com if you can? Not
> sure it's possible - or add that to CC at least)
>
>> [ 6066.180908] iwlwifi 0000:01:00.0: 0x00000942 | ADVANCED_SYSASSERT
>
> Hmm, that's a calibration failure.
>
> Did you do anything special in that environment?
Nothing that I'm aware of. The person who found the crash said they had
run some throughput tests, and then the radio went away.
We put one of the radios in a more powerful system, and will continue
testing.
>
> I guess filing a bug would be best, so we have a record for it and can
> hand it off to the firmware team or similar.
I was able to add them to CC I think, but don't think I can re-assign the bug.
https://bugzilla.kernel.org/show_bug.cgi?id=204801
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: intel AX200 crash on 5.2.7+
2019-09-09 17:54 ` Ben Greear
@ 2019-10-03 17:51 ` Ben Greear
0 siblings, 0 replies; 4+ messages in thread
From: Ben Greear @ 2019-10-03 17:51 UTC (permalink / raw)
To: Johannes Berg, linux-wireless, Luca Coelho
On 9/9/19 10:54 AM, Ben Greear wrote:
> On 9/9/19 10:40 AM, Johannes Berg wrote:
>> On Mon, 2019-09-09 at 10:03 -0700, Ben Greear wrote:
>>> Hello,
>>>
>>> Looks like we managed to crash the AX200 firmware. This was running 5.2.7+ kernel
>>> in an apu2 platform.
>>>
>>> Is there a better place to report/discuss this?
>>
>> This is OK for first reports, but usually we'll ask to file a bug on
>> bugzilla.kernel.org (and assign to linuxwifi@intel.com if you can? Not
>> sure it's possible - or add that to CC at least)
>>
>>> [ 6066.180908] iwlwifi 0000:01:00.0: 0x00000942 | ADVANCED_SYSASSERT
>>
>> Hmm, that's a calibration failure.
>>
>> Did you do anything special in that environment?
>
> Nothing that I'm aware of. The person who found the crash said they had
> run some throughput tests, and then the radio went away.
>
> We put one of the radios in a more powerful system, and will continue
> testing.
Hello,
We have been capturing the firmware crash binary files from the AX200, I guess
it uses the same API as ath10k, so something just worked for once.
They are about 5MB each. Is this something you can use to further debug
this? I'll be happy to send them to whoever can make use of them.
It seems pretty easy to reproduce these transmit bugs, at least...maybe you
can just as easily reproduce it yourself?
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-10-03 17:51 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-09 17:03 intel AX200 crash on 5.2.7+ Ben Greear
2019-09-09 17:40 ` Johannes Berg
2019-09-09 17:54 ` Ben Greear
2019-10-03 17:51 ` Ben Greear
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).