Hangs with NetworkManager and rtw8723_de with kernel 6.1

Hangs with NetworkManager and rtw8723_de with kernel 6.1

[François Valenduc]

Good morning,

systemrescue recently switched from kernel 5.15 to kernel 6.1 and 
NetworkManager started to hangs at startup. This seems to be linked to 
my wifi card (RTL8723DE). I am also able to reproduce the problem under 
gentoo with the same versions for NetworkManager and linux-firmware than 
those used in systemrescue (1.42.2 and 20230210). Under gentoo, the 
problem appears with kernel 6.1.14 and 6.2.1. I am able to use 
NetworkManager and systemrescue without problem on another computer 
which has another wifi card.

Does anybody have a solution to this problem ?

Thanks in advance for your help,

Best regards,

François Valenduc

Re: Hangs with NetworkManager and rtw8723_de with kernel 6.1

[Larry Finger]

On 3/1/23 00:26, François Valenduc wrote:
> systemrescue recently switched from kernel 5.15 to kernel 6.1 and NetworkManager 
> started to hangs at startup. This seems to be linked to my wifi card 
> (RTL8723DE). I am also able to reproduce the problem under gentoo with the same 
> versions for NetworkManager and linux-firmware than those used in systemrescue 
> (1.42.2 and 20230210). Under gentoo, the problem appears with kernel 6.1.14 and 
> 6.2.1. I am able to use NetworkManager and systemrescue without problem on 
> another computer which has another wifi card.
> 
> Does anybody have a solution to this problem ?

François,

Without any log information regarding the specific hang, it will be difficult to 
debug this issue.

Have you installed the drivers from the rtw88 repo at GitHub.com? If so, they 
may be interfering with the drivers in the kernel. You can test to see if this 
is the problem by doing an edit of the command line while booting. When GRUB is 
showing the kernel you wish to boot, press "e". This should bring up a minimal 
editor. Scroll down until you find the stanza pertaining to the kernel you are 
booting. It will start with "menuentry 'xxxxx'", where the x's are replaced with 
the description of the kernel you want.

Once you find that entry, scroll down to the line that starts with "linuxefi" or 
"linux", then move to the end of that line, and add 
"rd.driver.blacklist=rtw_8723de" to blacklist the driver from GitHub, or
"rd.driver.blacklist=rtw88_8723de" to blacklist the kernel version. Once you 
have added the new info, press the F10 key yo continue the boot.

If one of those lets the computer boot, add that driver to a blacklist file in 
/usr/lib/modprobe.d/, or wherever your distro keeps the modprobe.d directory.

If neither work, then report back, and we will try to get a dump of the logs 
when the hand happens.

Larry

Re: Hangs with NetworkManager and rtw8723_de with kernel 6.1

[François Valenduc]

Le 1/03/23 à 17:40, Larry Finger a écrit :
> On 3/1/23 00:26, François Valenduc wrote:
>> systemrescue recently switched from kernel 5.15 to kernel 6.1 and 
>> NetworkManager started to hangs at startup. This seems to be linked 
>> to my wifi card (RTL8723DE). I am also able to reproduce the problem 
>> under gentoo with the same versions for NetworkManager and 
>> linux-firmware than those used in systemrescue (1.42.2 and 20230210). 
>> Under gentoo, the problem appears with kernel 6.1.14 and 6.2.1. I am 
>> able to use NetworkManager and systemrescue without problem on 
>> another computer which has another wifi card.
>>
>> Does anybody have a solution to this problem ?
>
> François,
>
> Without any log information regarding the specific hang, it will be 
> difficult to debug this issue.
>
> Have you installed the drivers from the rtw88 repo at GitHub.com? If 
> so, they may be interfering with the drivers in the kernel. You can 
> test to see if this is the problem by doing an edit of the command 
> line while booting. When GRUB is showing the kernel you wish to boot, 
> press "e". This should bring up a minimal editor. Scroll down until 
> you find the stanza pertaining to the kernel you are booting. It will 
> start with "menuentry 'xxxxx'", where the x's are replaced with the 
> description of the kernel you want.
>
> Once you find that entry, scroll down to the line that starts with 
> "linuxefi" or "linux", then move to the end of that line, and add 
> "rd.driver.blacklist=rtw_8723de" to blacklist the driver from GitHub, or
> "rd.driver.blacklist=rtw88_8723de" to blacklist the kernel version. 
> Once you have added the new info, press the F10 key yo continue the boot.
>
> If one of those lets the computer boot, add that driver to a blacklist 
> file in /usr/lib/modprobe.d/, or wherever your distro keeps the 
> modprobe.d directory.
>
> If neither work, then report back, and we will try to get a dump of 
> the logs when the hand happens.
>
> Larry
>
Good evening,

For sure, I am using the kernel driver in gentoo or in systemrescue. Can 
you explain how I could obtains the logs when the crash occurs ?

François

Re: Hangs with NetworkManager and rtw8723_de with kernel 6.1

[Larry Finger]

On 3/1/23 13:15, François Valenduc wrote:
> Good evening,
> 
> For sure, I am using the kernel driver in gentoo or in systemrescue. Can you 
> explain how I could obtains the logs when the crash occurs ?

François,

The best way is to use netconsole, which requires another Linux computer 
(target) connected to your network, and a wired connection on the computer under 
test. The documentation is at 
https://www.kernel.org/doc/html/latest/networking/netconsole.html.

That writeup is not easy to follow, but you need to do the following:
1. Install netconsole or nc depending on your distro on your target computer.
2. Run the command 'nc -u -l -p 6665' or 'netconsole -u -l -p 6665' depending on 
your distro.
3. Learn the IP address of the target computer using 'ip address'.
4. Know the network name of your wired connection on the test computer.
5. Create an options file in /usr/lib/modprobe.d/50-netconsole.conf. This line 
should contain the following line:
options netconsole netconsole="@/<name from step4>,@<address from step 3>/"
My computer contains options netconsole netconsole="@/enp0s25,@192.168.1.51/"
6. Force module netconsole to be loaded at boot. How to do that is distro 
dependent. You will need to explore that on your own.

Reboot the test computer in the configuration that fails. If setup correctly, 
the console dump will appear on the target computer.

Good luck.

Larry

RE: Hangs with NetworkManager and rtw8723_de with kernel 6.1

[Ping-Ke Shih]

> -----Original Message-----
> From: François Valenduc <francoisvalenduc@gmail.com>
> Sent: Wednesday, March 1, 2023 2:26 PM
> To: linux-wireless@vger.kernel.org
> Subject: Hangs with NetworkManager and rtw8723_de with kernel 6.1
> 
> Good morning,
> 
> systemrescue recently switched from kernel 5.15 to kernel 6.1 and
> NetworkManager started to hangs at startup. This seems to be linked to
> my wifi card (RTL8723DE). I am also able to reproduce the problem under
> gentoo with the same versions for NetworkManager and linux-firmware than
> those used in systemrescue (1.42.2 and 20230210). Under gentoo, the
> problem appears with kernel 6.1.14 and 6.2.1. I am able to use
> NetworkManager and systemrescue without problem on another computer
> which has another wifi card.
> 
> Does anybody have a solution to this problem ?
> 

I think this problem is the same as
https://bugzilla.kernel.org/show_bug.cgi?id=217016

Ping-Ke

Re: Hangs with NetworkManager and rtw8723_de with kernel 6.1

[François Valenduc]

Le 2/03/23 à 02:27, Ping-Ke Shih a écrit :
>
>> -----Original Message-----
>> From: François Valenduc <francoisvalenduc@gmail.com>
>> Sent: Wednesday, March 1, 2023 2:26 PM
>> To: linux-wireless@vger.kernel.org
>> Subject: Hangs with NetworkManager and rtw8723_de with kernel 6.1
>>
>> Good morning,
>>
>> systemrescue recently switched from kernel 5.15 to kernel 6.1 and
>> NetworkManager started to hangs at startup. This seems to be linked to
>> my wifi card (RTL8723DE). I am also able to reproduce the problem under
>> gentoo with the same versions for NetworkManager and linux-firmware than
>> those used in systemrescue (1.42.2 and 20230210). Under gentoo, the
>> problem appears with kernel 6.1.14 and 6.2.1. I am able to use
>> NetworkManager and systemrescue without problem on another computer
>> which has another wifi card.
>>
>> Does anybody have a solution to this problem ?
>>
> I think this problem is the same as
> https://bugzilla.kernel.org/show_bug.cgi?id=217016
>
> Ping-Ke
>
>
This is indeed the same bug. I applied this patch on top of kernel 
6.1.14 and the hang doesn't occur anymore, neither in Gentoo or in 
SystemRescue. I see this patch has been merged in mainline and marked 
for stable. However, it is not included in the current review for 
6.1.15. I hope it will make it to stable kernels soon.

Thanks for your help,

François Valenduc

Re: Hangs with NetworkManager and rtw8723_de with kernel 6.1

[Larry Finger]

On 3/2/23 00:24, François Valenduc wrote:
> 
> Le 2/03/23 à 02:27, Ping-Ke Shih a écrit :
>>
>>> -----Original Message-----
>>> From: François Valenduc <francoisvalenduc@gmail.com>
>>> Sent: Wednesday, March 1, 2023 2:26 PM
>>> To: linux-wireless@vger.kernel.org
>>> Subject: Hangs with NetworkManager and rtw8723_de with kernel 6.1
>>>
>>> Good morning,
>>>
>>> systemrescue recently switched from kernel 5.15 to kernel 6.1 and
>>> NetworkManager started to hangs at startup. This seems to be linked to
>>> my wifi card (RTL8723DE). I am also able to reproduce the problem under
>>> gentoo with the same versions for NetworkManager and linux-firmware than
>>> those used in systemrescue (1.42.2 and 20230210). Under gentoo, the
>>> problem appears with kernel 6.1.14 and 6.2.1. I am able to use
>>> NetworkManager and systemrescue without problem on another computer
>>> which has another wifi card.
>>>
>>> Does anybody have a solution to this problem ?
>>>
>> I think this problem is the same as
>> https://bugzilla.kernel.org/show_bug.cgi?id=217016
>>
>> Ping-Ke
>>
>>
> This is indeed the same bug. I applied this patch on top of kernel 6.1.14 and 
> the hang doesn't occur anymore, neither in Gentoo or in SystemRescue. I see this 
> patch has been merged in mainline and marked for stable. However, it is not 
> included in the current review for 6.1.15. I hope it will make it to stable 
> kernels soon.

Thanks Ping-Ke for the help here. I am glad the problem is solved.

For your information, that patch is included in the rtw88 repo at GitHub.com.

Larry