Re: [BUG] brcmfmac: brcmf_sdio_bus_rxctl: resumed on timeout (WiFi dies)

From: Arend van Spriel <arend.vanspriel@broadcom.com>
To: Dmitry Osipenko <digetx@gmail.com>,
	Franky Lin <franky.lin@broadcom.com>,
	Hante Meuleman <hante.meuleman@broadcom.com>,
	Chi-Hsien Lin <chi-hsien.lin@cypress.com>,
	Wright Feng <wright.feng@cypress.com>,
	Kalle Valo <kvalo@codeaurora.org>
Cc: "linux-wireless@vger.kernel.org" <linux-wireless@vger.kernel.org>,
	"brcm80211-dev-list.pdl@broadcom.com" 
	<brcm80211-dev-list.pdl@broadcom.com>,
	"brcm80211-dev-list@cypress.com" <brcm80211-dev-list@cypress.com>,
	netdev <netdev@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [BUG] brcmfmac: brcmf_sdio_bus_rxctl: resumed on timeout (WiFi dies)
Date: Thu, 27 May 2021 18:42:56 +0200	[thread overview]
Message-ID: <cc328771-0c1d-93e7-cec6-3f4fb7f64d02@broadcom.com> (raw)
In-Reply-To: <fcf95129-cba7-817d-4bfd-8efaf92f957f@gmail.com>

On 5/26/2021 5:10 PM, Dmitry Osipenko wrote:
> Hello,
> 
> After updating to Ubuntu 21.04 I found two problems related to the BRCMF_C_GET_ASSOCLIST using an older BCM4329 SDIO WiFi.
> 
> 1. The kernel is spammed with:
> 
>   ieee80211 phy0: brcmf_cfg80211_dump_station: BRCMF_C_GET_ASSOCLIST unsupported, err=-52
>   ieee80211 phy0: brcmf_cfg80211_dump_station: BRCMF_C_GET_ASSOCLIST unsupported, err=-52
>   ieee80211 phy0: brcmf_cfg80211_dump_station: BRCMF_C_GET_ASSOCLIST unsupported, err=-52
> 
> Which happens apparently due to a newer NetworkManager version that pokes dump_station() periodically. I sent [1] that fixes this noise.
> 
> [1] https://patchwork.kernel.org/project/linux-wireless/list/?series=480715

Right. I noticed this one and did not have anything to add to the 
review/suggestion.

> 2. The other much worse problem is that WiFi eventually dies now with these errors:
> 
> ...
>   ieee80211 phy0: brcmf_cfg80211_dump_station: BRCMF_C_GET_ASSOCLIST unsupported, err=-52
>   brcmfmac: brcmf_sdio_bus_rxctl: resumed on timeout
>   ieee80211 phy0: brcmf_cfg80211_dump_station: BRCMF_C_GET_ASSOCLIST unsupported, err=-110
>   ieee80211 phy0: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
> 
>  From this point all firmware calls start to fail with err=-110 and WiFi doesn't work anymore. This problem is reproducible with 5.13-rc and current -next, I haven't checked older kernel versions. Somehow it's worse using a recent -next, WiFi dies quicker.
> 
> What's interesting is that I see that there is always a pending signal in brcmf_sdio_dcmd_resp_wait() when timeout happens. It looks like the timeout happens when there is access to a swap partition, which stalls system for a second or two, but this is not 100%. Increasing DCMD_RESP_TIMEOUT doesn't help.

The timeout error (-110) can have two root causes that I am aware off. 
Either the firmware died or the SDIO layer has gone haywire. Not sure if 
that swap partition is on eMMC device, but if so it could be related. 
You could try generating device coredump. If that also gives -110 errors 
we know it is the SDIO layer.

> Please let me know if you have any ideas of how to fix this trouble properly or if you need need any more info.
> 
> Removing BRCMF_C_GET_ASSOCLIST firmware call entirely from the driver fixes the problem.

My guess is that reducing interaction with firmware is what is avoiding 
the issue and not so much this specific firmware command. As always it 
is good to know the conditions in which the issue occurs. What is the 
hardware platform you are running Ubuntu on? Stuff like that.

Regards,
Arend