Re: MT7922 problem with "fix rx filter incorrect by drv/fw inconsistent"

Re: MT7922 problem with "fix rx filter incorrect by drv/fw inconsistent"

[Thorsten Leemhuis]

[CCing the wifi-driver and the net developers, as a "JFYI" to ensure
they are aware of this "newer kernel requires newer firmware"
regression, so they can jump in if they want]

On 22.05.23 16:12, Thorsten Leemhuis wrote:
> On 22.05.23 15:20, Andrey Rakhmatullin wrote:
>> On Mon, May 22, 2023 at 03:00:30PM +0200, Linux regression tracking #adding (Thorsten Leemhuis) wrote:
>>> On 18.05.23 16:39, Andrey Rakhmatullin wrote:
>>>> Hello. I have a "MEDIATEK Corp. MT7922 802.11ax PCI Express Wireless
>>>> Network Adapter" (14c3:0616) and when the commit c222f77fd4 ("wifi: mt76:
>>>> mt7921: fix rx filter incorrect by drv/fw inconsistent") is applied (found
>>>> by bisecting, checked by reverting it on v6.3) I have the following
>>>> problem on my machine: when I connect to my router no DHCPv4 exchange
>>>> happens, I don't see responses in tcpdump. My network setup is non-trivial
>>>> though, and it looks like the problem is specific to it, but I still
>>>> wonder if it's some bug in the aforementioned patch as my setup works with
>>>> all other devices and I would expect it to work as long as the network
>>>> packets sent by the device are the same.
>>>>
>>>> My setup is as follows: I have an ISP router which provides a 2.4GHz
>>>> network and another router (Xiaomi R4AC with OpenWRT) connected by
>>>> Ethernet to it that provides a 5GHz network and is configured as a "Relay
>>>> bridge" (using relayd) to forward packets to the ISP router and back. This
>>>> includes DHCPv4 packets, which are handled by the ISP router. tcpdump on
>>>> the machine with MT7922 shows that the DHCP requests are sent while the
>>>> responses are not received, while tcpdump on the bridge router shows both
>>>> requests and responses.
>>>>
>>>> I've tried connecting the machine to the ISP router network directly and
>>>> also to another AP (one on my phone) and those work correctly on all
>>>> kernels.
>>
>> Deren Wu asked me privately
>> if I'm using the latest firmware, and I
>> wasn't. I updated the firmware and now the problem doesn't happen.
>> The firmware where the problem happens is
>> mediatek/WIFI_RAM_CODE_MT7922_1.bin from the linux-firmware commit
>> e2d11744ef (file size 826740, md5sum 8ff1bdc0f54f255bb2a1d6825781506b),
>> the one where the problem doesn't happen is from the commit 6569484e6b
>> (file size 827124, md5sum 14c08c8298b639ee52409b5e9711a083).
> 
> FWIW, just checked: that commit is from 2023-05-15, so quite recent.
> 
>> I haven't
>> tried the version committed between these ones.
>> Not sure if this should be reported to regzbot and if there are any
>> further actions needed by the kernel maintainers.
> 
> Well, to quote the first sentence from
> Documentation/driver-api/firmware/firmware-usage-guidelines.rst
> 
> ```Users switching to a newer kernel should *not* have to install newer
> firmware files to keep their hardware working.```
> 
> IOW: the problem you ran into should not happen. This afaics makes it a
> regression that needs to be addressed -- at least if it's something that
> is likely to hit others users as well. But I'd guess that's the case.

Well, until now I didn't see any other report about a problem like this.
Maybe things work better for others with that hardware – in that case it
might be something not worth making a fuzz about. But I'll wait another
week or two before I remove this from the tracking.

Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
--
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
If I did something stupid, please tell me, as explained on that page.

Re: MT7922 problem with "fix rx filter incorrect by drv/fw inconsistent"

[Kalle Valo]

Thorsten Leemhuis <regressions@leemhuis.info> writes:

> [CCing the wifi-driver and the net developers, as a "JFYI" to ensure
> they are aware of this "newer kernel requires newer firmware"
> regression, so they can jump in if they want]
>
> On 22.05.23 16:12, Thorsten Leemhuis wrote:
>> On 22.05.23 15:20, Andrey Rakhmatullin wrote:
>>> On Mon, May 22, 2023 at 03:00:30PM +0200, Linux regression tracking
>>> #adding (Thorsten Leemhuis) wrote:
>>>> On 18.05.23 16:39, Andrey Rakhmatullin wrote:
>>>>> Hello. I have a "MEDIATEK Corp. MT7922 802.11ax PCI Express Wireless
>>>>> Network Adapter" (14c3:0616) and when the commit c222f77fd4 ("wifi: mt76:
>>>>> mt7921: fix rx filter incorrect by drv/fw inconsistent") is applied (found
>>>>> by bisecting, checked by reverting it on v6.3) I have the following
>>>>> problem on my machine: when I connect to my router no DHCPv4 exchange
>>>>> happens, I don't see responses in tcpdump. My network setup is non-trivial
>>>>> though, and it looks like the problem is specific to it, but I still
>>>>> wonder if it's some bug in the aforementioned patch as my setup works with
>>>>> all other devices and I would expect it to work as long as the network
>>>>> packets sent by the device are the same.
>>>>>
>>>>> My setup is as follows: I have an ISP router which provides a 2.4GHz
>>>>> network and another router (Xiaomi R4AC with OpenWRT) connected by
>>>>> Ethernet to it that provides a 5GHz network and is configured as a "Relay
>>>>> bridge" (using relayd) to forward packets to the ISP router and back. This
>>>>> includes DHCPv4 packets, which are handled by the ISP router. tcpdump on
>>>>> the machine with MT7922 shows that the DHCP requests are sent while the
>>>>> responses are not received, while tcpdump on the bridge router shows both
>>>>> requests and responses.
>>>>>
>>>>> I've tried connecting the machine to the ISP router network directly and
>>>>> also to another AP (one on my phone) and those work correctly on all
>>>>> kernels.
>>>
>>> Deren Wu asked me privately
>>> if I'm using the latest firmware, and I
>>> wasn't. I updated the firmware and now the problem doesn't happen.
>>> The firmware where the problem happens is
>>> mediatek/WIFI_RAM_CODE_MT7922_1.bin from the linux-firmware commit
>>> e2d11744ef (file size 826740, md5sum 8ff1bdc0f54f255bb2a1d6825781506b),
>>> the one where the problem doesn't happen is from the commit 6569484e6b
>>> (file size 827124, md5sum 14c08c8298b639ee52409b5e9711a083).
>> 
>> FWIW, just checked: that commit is from 2023-05-15, so quite recent.
>> 
>>> I haven't
>>> tried the version committed between these ones.
>>> Not sure if this should be reported to regzbot and if there are any
>>> further actions needed by the kernel maintainers.
>> 
>> Well, to quote the first sentence from
>> Documentation/driver-api/firmware/firmware-usage-guidelines.rst
>> 
>> ```Users switching to a newer kernel should *not* have to install newer
>> firmware files to keep their hardware working.```
>> 
>> IOW: the problem you ran into should not happen. This afaics makes it a
>> regression that needs to be addressed -- at least if it's something that
>> is likely to hit others users as well. But I'd guess that's the case.
>
> Well, until now I didn't see any other report about a problem like this.
> Maybe things work better for others with that hardware – in that case it
> might be something not worth making a fuzz about. But I'll wait another
> week or two before I remove this from the tracking.

Yeah, this is bad. mt76 (or any other wireless driver) must not require
a new firmware whenever upgrading the kernel. Instead the old and new
firmware should coexist (for example have firmware-2.bin for the new
version and firmware.bin for the old version). Then mt76 should first
try loading the new firmware (eg. firmware-2.bin) and then try the old
one (eg. firmware.bin).

Should we revert commit c222f77fd4 or how to solve this?

-- 
https://patchwork.kernel.org/project/linux-wireless/list/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

Re: MT7922 problem with "fix rx filter incorrect by drv/fw inconsistent"

[Lorenzo Bianconi]

[-- Attachment #1: Type: text/plain, Size: 4756 bytes --]

> Thorsten Leemhuis <regressions@leemhuis.info> writes:
> 
> > [CCing the wifi-driver and the net developers, as a "JFYI" to ensure
> > they are aware of this "newer kernel requires newer firmware"
> > regression, so they can jump in if they want]
> >
> > On 22.05.23 16:12, Thorsten Leemhuis wrote:
> >> On 22.05.23 15:20, Andrey Rakhmatullin wrote:
> >>> On Mon, May 22, 2023 at 03:00:30PM +0200, Linux regression tracking
> >>> #adding (Thorsten Leemhuis) wrote:
> >>>> On 18.05.23 16:39, Andrey Rakhmatullin wrote:
> >>>>> Hello. I have a "MEDIATEK Corp. MT7922 802.11ax PCI Express Wireless
> >>>>> Network Adapter" (14c3:0616) and when the commit c222f77fd4 ("wifi: mt76:
> >>>>> mt7921: fix rx filter incorrect by drv/fw inconsistent") is applied (found
> >>>>> by bisecting, checked by reverting it on v6.3) I have the following
> >>>>> problem on my machine: when I connect to my router no DHCPv4 exchange
> >>>>> happens, I don't see responses in tcpdump. My network setup is non-trivial
> >>>>> though, and it looks like the problem is specific to it, but I still
> >>>>> wonder if it's some bug in the aforementioned patch as my setup works with
> >>>>> all other devices and I would expect it to work as long as the network
> >>>>> packets sent by the device are the same.
> >>>>>
> >>>>> My setup is as follows: I have an ISP router which provides a 2.4GHz
> >>>>> network and another router (Xiaomi R4AC with OpenWRT) connected by
> >>>>> Ethernet to it that provides a 5GHz network and is configured as a "Relay
> >>>>> bridge" (using relayd) to forward packets to the ISP router and back. This
> >>>>> includes DHCPv4 packets, which are handled by the ISP router. tcpdump on
> >>>>> the machine with MT7922 shows that the DHCP requests are sent while the
> >>>>> responses are not received, while tcpdump on the bridge router shows both
> >>>>> requests and responses.
> >>>>>
> >>>>> I've tried connecting the machine to the ISP router network directly and
> >>>>> also to another AP (one on my phone) and those work correctly on all
> >>>>> kernels.

@Andrey: IIUC the issue, you do not receive any DHCP offer/reply when
you try to connect to the OpenWrt 5GHz AP, right? If so, are you able to
provide a traffic sniff obtained from a monitor interface running on a node
connected to the same SSID when the MT7922 client is trying to connect?
It would be very helpful if you can run this test with encryption enabled
and disabled. Thanks in advance.

Regards,
Lorenzo

> >>>
> >>> Deren Wu asked me privately
> >>> if I'm using the latest firmware, and I
> >>> wasn't. I updated the firmware and now the problem doesn't happen.
> >>> The firmware where the problem happens is
> >>> mediatek/WIFI_RAM_CODE_MT7922_1.bin from the linux-firmware commit
> >>> e2d11744ef (file size 826740, md5sum 8ff1bdc0f54f255bb2a1d6825781506b),
> >>> the one where the problem doesn't happen is from the commit 6569484e6b
> >>> (file size 827124, md5sum 14c08c8298b639ee52409b5e9711a083).
> >> 
> >> FWIW, just checked: that commit is from 2023-05-15, so quite recent.
> >> 
> >>> I haven't
> >>> tried the version committed between these ones.
> >>> Not sure if this should be reported to regzbot and if there are any
> >>> further actions needed by the kernel maintainers.
> >> 
> >> Well, to quote the first sentence from
> >> Documentation/driver-api/firmware/firmware-usage-guidelines.rst
> >> 
> >> ```Users switching to a newer kernel should *not* have to install newer
> >> firmware files to keep their hardware working.```
> >> 
> >> IOW: the problem you ran into should not happen. This afaics makes it a
> >> regression that needs to be addressed -- at least if it's something that
> >> is likely to hit others users as well. But I'd guess that's the case.
> >
> > Well, until now I didn't see any other report about a problem like this.
> > Maybe things work better for others with that hardware – in that case it
> > might be something not worth making a fuzz about. But I'll wait another
> > week or two before I remove this from the tracking.
> 
> Yeah, this is bad. mt76 (or any other wireless driver) must not require
> a new firmware whenever upgrading the kernel. Instead the old and new
> firmware should coexist (for example have firmware-2.bin for the new
> version and firmware.bin for the old version). Then mt76 should first
> try loading the new firmware (eg. firmware-2.bin) and then try the old
> one (eg. firmware.bin).
> 
> Should we revert commit c222f77fd4 or how to solve this?
> 
> -- 
> https://patchwork.kernel.org/project/linux-wireless/list/
> 
> https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: MT7922 problem with "fix rx filter incorrect by drv/fw inconsistent"

[Thorsten Leemhuis]

On 12.06.23 14:39, Kalle Valo wrote:
> Thorsten Leemhuis <regressions@leemhuis.info> writes:
>> On 22.05.23 16:12, Thorsten Leemhuis wrote:
>>> On 22.05.23 15:20, Andrey Rakhmatullin wrote:
>>>> On Mon, May 22, 2023 at 03:00:30PM +0200, Linux regression tracking
>>>> #adding (Thorsten Leemhuis) wrote:
>>>>> On 18.05.23 16:39, Andrey Rakhmatullin wrote:
>>>> I updated the firmware and now the problem doesn't happen.
>>>> The firmware where the problem happens is
>>>> mediatek/WIFI_RAM_CODE_MT7922_1.bin from the linux-firmware commit
>>>> e2d11744ef (file size 826740, md5sum 8ff1bdc0f54f255bb2a1d6825781506b),
>>>> the one where the problem doesn't happen is from the commit 6569484e6b
>>>> (file size 827124, md5sum 14c08c8298b639ee52409b5e9711a083).
>>> FWIW, just checked: that commit is from 2023-05-15, so quite recent.
>>>
>>>> I haven't
>>>> tried the version committed between these ones.
>>>> Not sure if this should be reported to regzbot and if there are any
>>>> further actions needed by the kernel maintainers.
>>>
>>> Well, to quote the first sentence from
>>> Documentation/driver-api/firmware/firmware-usage-guidelines.rst
>>>
>>> ```Users switching to a newer kernel should *not* have to install newer
>>> firmware files to keep their hardware working.```
>>>
>>> IOW: the problem you ran into should not happen. This afaics makes it a
>>> regression that needs to be addressed -- at least if it's something that
>>> is likely to hit others users as well. But I'd guess that's the case.
>>
>> Well, until now I didn't see any other report about a problem like this.
>> Maybe things work better for others with that hardware – in that case it
>> might be something not worth making a fuzz about. But I'll wait another
>> week or two before I remove this from the tracking.
> 
> Yeah, this is bad. mt76 (or any other wireless driver) must not require
> a new firmware whenever upgrading the kernel. Instead the old and new
> firmware should coexist (for example have firmware-2.bin for the new
> version and firmware.bin for the old version). Then mt76 should first
> try loading the new firmware (eg. firmware-2.bin) and then try the old
> one (eg. firmware.bin).
> 
> Should we revert commit c222f77fd4 or how to solve this?

I had hoped someone would rely with an opinion on this, but nobody did
(and the reporter didn't reply to Lorenzo inquiry). So if you asked for
mine:

Hmmm. Tricky. This was the only such report I noticed. Giving that and
the risk that a revert might cause regressions on its own, I guess it
might be better to leave everything as it is for now - and re-evaluate
the situation in case more problems show up.

Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
--
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
If I did something stupid, please tell me, as explained on that page.

Re: MT7922 problem with "fix rx filter incorrect by drv/fw inconsistent"

[Linux regression tracking (Thorsten Leemhuis)]

Hi, Thorsten here, the Linux kernel's regression tracker. Top-posting
for once, to make this easily accessible to everyone.

FWIW, I'm dropping this from the list of tracked regressions now. This
wasn't handled as it IMHO should be, but whatever, at this point it
afaics is best to leave things as they are, unless more reports of this
kind show up.

Thx everyone.

#regzbot inconclusive: not fixed, but fixing likely would cause more
trouble than it's worth, unless more people complain

Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
--
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
If I did something stupid, please tell me, as explained on that page.

On 19.06.23 14:48, Thorsten Leemhuis wrote:
> On 12.06.23 14:39, Kalle Valo wrote:
>> Thorsten Leemhuis <regressions@leemhuis.info> writes:
>>> On 22.05.23 16:12, Thorsten Leemhuis wrote:
>>>> On 22.05.23 15:20, Andrey Rakhmatullin wrote:
>>>>> On Mon, May 22, 2023 at 03:00:30PM +0200, Linux regression tracking
>>>>> #adding (Thorsten Leemhuis) wrote:
>>>>>> On 18.05.23 16:39, Andrey Rakhmatullin wrote:
>>>>> I updated the firmware and now the problem doesn't happen.
>>>>> The firmware where the problem happens is
>>>>> mediatek/WIFI_RAM_CODE_MT7922_1.bin from the linux-firmware commit
>>>>> e2d11744ef (file size 826740, md5sum 8ff1bdc0f54f255bb2a1d6825781506b),
>>>>> the one where the problem doesn't happen is from the commit 6569484e6b
>>>>> (file size 827124, md5sum 14c08c8298b639ee52409b5e9711a083).
>>>> FWIW, just checked: that commit is from 2023-05-15, so quite recent.
>>>>
>>>>> I haven't
>>>>> tried the version committed between these ones.
>>>>> Not sure if this should be reported to regzbot and if there are any
>>>>> further actions needed by the kernel maintainers.
>>>>
>>>> Well, to quote the first sentence from
>>>> Documentation/driver-api/firmware/firmware-usage-guidelines.rst
>>>>
>>>> ```Users switching to a newer kernel should *not* have to install newer
>>>> firmware files to keep their hardware working.```
>>>>
>>>> IOW: the problem you ran into should not happen. This afaics makes it a
>>>> regression that needs to be addressed -- at least if it's something that
>>>> is likely to hit others users as well. But I'd guess that's the case.
>>>
>>> Well, until now I didn't see any other report about a problem like this.
>>> Maybe things work better for others with that hardware – in that case it
>>> might be something not worth making a fuzz about. But I'll wait another
>>> week or two before I remove this from the tracking.
>>
>> Yeah, this is bad. mt76 (or any other wireless driver) must not require
>> a new firmware whenever upgrading the kernel. Instead the old and new
>> firmware should coexist (for example have firmware-2.bin for the new
>> version and firmware.bin for the old version). Then mt76 should first
>> try loading the new firmware (eg. firmware-2.bin) and then try the old
>> one (eg. firmware.bin).
>>
>> Should we revert commit c222f77fd4 or how to solve this?

> Hmmm. Tricky. This was the only such report I noticed. Giving that and
> the risk that a revert might cause regressions on its own, I guess it
> might be better to leave everything as it is for now - and re-evaluate
> the situation in case more problems show up.