[Regression 5.9][Bisected 1df2bdba528b] Wifi GTK rekeying fails: Sending of EAPol packages broken

[Regression 5.9][Bisected 1df2bdba528b] Wifi GTK rekeying fails: Sending of EAPol packages broken

[Thomas Deutschmann]

Hi,

after upgrading to linux-5.9.0 I noticed that my wifi got disassociated 
every 10 minutes when access point triggered rekeying for GTK.

This happened with iwd but not with wpa_supplicant. iwd was logging

> wlan0: disassociated from aa:bb:cc:dd:ap:01 (Reason: 2=PREV_AUTH_NOT_VALID)
> wlan0: authenticate with aa:bb:cc:dd:ap:01
> wlan0: send auth to aa:bb:cc:dd:ap:01 (try 1/3)
> wlan0: authenticated
> wlan0: associate with aa:bb:cc:dd:ap:01 (try 1/3)
> wlan0: RX AssocResp from aa:bb:cc:dd:ap:01 (capab=0x1511 status=0 aid=1)
> wlan0: associated

With the help of iwd developers (many thanks!) we noticed that EAPoL 
packets didn't reach access point. As workaround, using the legacy way 
to send EAPoL packets by setting

> [General]
> ControlPortOverNL80211=False

in iwd's main.conf, worked. So it became clear that this is a kernel 
problem.

I now finished bisecting the kernel and 
1df2bdba528b5a7a30f1b107b6924aa79af5e00e [1] is the first bad commit:

> commit 1df2bdba528b5a7a30f1b107b6924aa79af5e00e
> Author: Mathy Vanhoef
> Date:   Thu Jul 23 14:01:48 2020 +0400
> 
>     mac80211: never drop injected frames even if normally not allowed
> 
>     In ieee80211_tx_dequeue there is a check to see if the dequeued frame
>     is allowed in the current state. Injected frames that are normally
>     not allowed are being be dropped here. Fix this by checking if a
>     frame was injected and if so always allowing it.
> 
>     Signed-off-by: Mathy Vanhoef
>     Link: https://lore.kernel.org/r/20200723100153.31631-1-Mathy.Vanhoef@kuleuven.be
>     Signed-off-by: Johannes Berg
> 
>  net/mac80211/tx.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Complete bisect log:

> git bisect start
> # good: [665c6ff082e214537beef2e39ec366cddf446d52] Linux 5.8.15
> git bisect good 665c6ff082e214537beef2e39ec366cddf446d52
> # bad: [bbf5c979011a099af5dc76498918ed7df445635b] Linux 5.9
> git bisect bad bbf5c979011a099af5dc76498918ed7df445635b
> # good: [bcf876870b95592b52519ed4aafcf9d95999bc9c] Linux 5.8
> git bisect good bcf876870b95592b52519ed4aafcf9d95999bc9c
> # bad: [47ec5303d73ea344e84f46660fff693c57641386] Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next
> git bisect bad 47ec5303d73ea344e84f46660fff693c57641386
> # good: [8f7be6291529011a58856bf178f52ed5751c68ac] Merge tag 'mmc-v5.9' of git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc
> git bisect good 8f7be6291529011a58856bf178f52ed5751c68ac
> # bad: [76769c38b45d94f5492ff9be363ac7007fd8e58b] Merge tag 'mlx5-updates-2020-08-03' of git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux
> git bisect bad 76769c38b45d94f5492ff9be363ac7007fd8e58b
> # good: [94d9f78f4d64b967273a676167bd34ddad2f978c] docs: networking: timestamping: add section for stacked PHC devices
> git bisect good 94d9f78f4d64b967273a676167bd34ddad2f978c
> # good: [5ee30564c85c94b7dc78aa6cce09e9712b2ad70d] ice: update reporting of autoneg capabilities
> git bisect good 5ee30564c85c94b7dc78aa6cce09e9712b2ad70d
> # good: [bd69058f50d5ffa659423bcfa6fe6280ce9c760a] net: ll_temac: Use devm_platform_ioremap_resource_byname()
> git bisect good bd69058f50d5ffa659423bcfa6fe6280ce9c760a
> # skip: [7dbc63f0a5402293e887e89a7974c5e48405565d] ice: Misc minor fixes
> git bisect skip 7dbc63f0a5402293e887e89a7974c5e48405565d
> # good: [1303a51c24100b3b1915d6f9072fe5ae5bb4c5f6] cfg80211/mac80211: add connected to auth server to station info
> git bisect good 1303a51c24100b3b1915d6f9072fe5ae5bb4c5f6
> # skip: [f34f55557ac9a4dfbfbf36c70585d1648ab5cd90] ice: Allow 2 queue pairs per VF on SR-IOV initialization
> git bisect skip f34f55557ac9a4dfbfbf36c70585d1648ab5cd90
> # bad: [cc5d229a122106733a85c279d89d7703f21e4d4f] fsl/fman: check dereferencing null pointer
> git bisect bad cc5d229a122106733a85c279d89d7703f21e4d4f
> # good: [6fc8c827dd4fa615965c4eac9bbfd465f6eb8fb4] tcp: syncookies: create mptcp request socket for ACK cookies with MPTCP option
> git bisect good 6fc8c827dd4fa615965c4eac9bbfd465f6eb8fb4
> # bad: [b90a1269184a3ff374562d243419ad2fa9d3b1aa] Merge branch 'net-openvswitch-masks-cache-enhancements'
> git bisect bad b90a1269184a3ff374562d243419ad2fa9d3b1aa
> # skip: [829eb208e80d6db95c0201cb8fa00c2f9ad87faf] rtnetlink: add support for protodown reason
> git bisect skip 829eb208e80d6db95c0201cb8fa00c2f9ad87faf
> # bad: [0e8642cf369a37b718c15effa6ffd52c00fd7d15] tcp: fix build fong CONFIG_MPTCP=n
> git bisect bad 0e8642cf369a37b718c15effa6ffd52c00fd7d15
> # skip: [48040793fa6003d211f021c6ad273477bcd90d91] tcp: add earliest departure time to SCM_TIMESTAMPING_OPT_STATS
> git bisect skip 48040793fa6003d211f021c6ad273477bcd90d91
> # good: [bc5cbd73eb493944b8665dc517f684c40eb18a4a] iavf: use generic power management
> git bisect good bc5cbd73eb493944b8665dc517f684c40eb18a4a
> # skip: [8f3f330da28ede9d106cd9d5c5ccd6a3e7e9b50b] tun: add missing rcu annotation in tun_set_ebpf()
> git bisect skip 8f3f330da28ede9d106cd9d5c5ccd6a3e7e9b50b
> # skip: [9466a1ccebbe54ac57fb8a89c2b4b854826546a8] mptcp: enable JOIN requests even if cookies are in use
> git bisect skip 9466a1ccebbe54ac57fb8a89c2b4b854826546a8
> # good: [09a071f52bbedddef626e71c0fd210838532f347] Documentation: intel: Replace HTTP links with HTTPS ones
> git bisect good 09a071f52bbedddef626e71c0fd210838532f347
> # bad: [75e6b594bbaeeb3f8287a2e6eb8811384b8c7195] cfg80211: invert HE BSS color 'disabled' to 'enabled'
> git bisect bad 75e6b594bbaeeb3f8287a2e6eb8811384b8c7195
> # bad: [1df2bdba528b5a7a30f1b107b6924aa79af5e00e] mac80211: never drop injected frames even if normally not allowed
> git bisect bad 1df2bdba528b5a7a30f1b107b6924aa79af5e00e
> # good: [180ac48ee62f53c26787350a956c5ac371cbe0b7] mac80211: calculate skb hash early when using itxq
> git bisect good 180ac48ee62f53c26787350a956c5ac371cbe0b7
> # good: [322cd27c06450b2db2cb6bdc68f3814149baf767] cfg80211/mac80211: avoid bss color setting in non-HE modes
> git bisect good 322cd27c06450b2db2cb6bdc68f3814149baf767
> # good: [fd17dba1c860d39f655a3a08387c21e3ceca8c55] cfg80211: Add support to advertize OCV support
> git bisect good fd17dba1c860d39f655a3a08387c21e3ceca8c55
> # first bad commit: [1df2bdba528b5a7a30f1b107b6924aa79af5e00e] mac80211: never drop injected frames even if normally not allowed


See also:
=========
[1] 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1df2bdba528b5a7a30f1b107b6924aa79af5e00e


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5

Re: [Regression 5.9][Bisected 1df2bdba528b] Wifi GTK rekeying fails: Sending of EAPol packages broken

[Mathy Vanhoef]

Hello Thomas,

That's surprising regression, I'll try to reproduce it next week.

- Mathy

On 10/15/20 5:44 AM, Thomas Deutschmann wrote:
> Hi,
> 
> after upgrading to linux-5.9.0 I noticed that my wifi got disassociated
> every 10 minutes when access point triggered rekeying for GTK.
> 
> This happened with iwd but not with wpa_supplicant. iwd was logging
> 
>> wlan0: disassociated from aa:bb:cc:dd:ap:01 (Reason:
>> 2=PREV_AUTH_NOT_VALID)
>> wlan0: authenticate with aa:bb:cc:dd:ap:01
>> wlan0: send auth to aa:bb:cc:dd:ap:01 (try 1/3)
>> wlan0: authenticated
>> wlan0: associate with aa:bb:cc:dd:ap:01 (try 1/3)
>> wlan0: RX AssocResp from aa:bb:cc:dd:ap:01 (capab=0x1511 status=0 aid=1)
>> wlan0: associated
> 
> With the help of iwd developers (many thanks!) we noticed that EAPoL
> packets didn't reach access point. As workaround, using the legacy way
> to send EAPoL packets by setting
> 
>> [General]
>> ControlPortOverNL80211=False
> 
> in iwd's main.conf, worked. So it became clear that this is a kernel
> problem.
> 
> I now finished bisecting the kernel and
> 1df2bdba528b5a7a30f1b107b6924aa79af5e00e [1] is the first bad commit:
> 
>> commit 1df2bdba528b5a7a30f1b107b6924aa79af5e00e
>> Author: Mathy Vanhoef
>> Date:   Thu Jul 23 14:01:48 2020 +0400
>>
>>     mac80211: never drop injected frames even if normally not allowed
>>
>>     In ieee80211_tx_dequeue there is a check to see if the dequeued frame
>>     is allowed in the current state. Injected frames that are normally
>>     not allowed are being be dropped here. Fix this by checking if a
>>     frame was injected and if so always allowing it.
>>
>>     Signed-off-by: Mathy Vanhoef
>>     Link:
>> https://lore.kernel.org/r/20200723100153.31631-1-Mathy.Vanhoef@kuleuven.be
>>
>>     Signed-off-by: Johannes Berg
>>
>>  net/mac80211/tx.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> Complete bisect log:
> 
>> git bisect start
>> # good: [665c6ff082e214537beef2e39ec366cddf446d52] Linux 5.8.15
>> git bisect good 665c6ff082e214537beef2e39ec366cddf446d52
>> # bad: [bbf5c979011a099af5dc76498918ed7df445635b] Linux 5.9
>> git bisect bad bbf5c979011a099af5dc76498918ed7df445635b
>> # good: [bcf876870b95592b52519ed4aafcf9d95999bc9c] Linux 5.8
>> git bisect good bcf876870b95592b52519ed4aafcf9d95999bc9c
>> # bad: [47ec5303d73ea344e84f46660fff693c57641386] Merge
>> git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next
>> git bisect bad 47ec5303d73ea344e84f46660fff693c57641386
>> # good: [8f7be6291529011a58856bf178f52ed5751c68ac] Merge tag
>> 'mmc-v5.9' of git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc
>> git bisect good 8f7be6291529011a58856bf178f52ed5751c68ac
>> # bad: [76769c38b45d94f5492ff9be363ac7007fd8e58b] Merge tag
>> 'mlx5-updates-2020-08-03' of
>> git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux
>> git bisect bad 76769c38b45d94f5492ff9be363ac7007fd8e58b
>> # good: [94d9f78f4d64b967273a676167bd34ddad2f978c] docs: networking:
>> timestamping: add section for stacked PHC devices
>> git bisect good 94d9f78f4d64b967273a676167bd34ddad2f978c
>> # good: [5ee30564c85c94b7dc78aa6cce09e9712b2ad70d] ice: update
>> reporting of autoneg capabilities
>> git bisect good 5ee30564c85c94b7dc78aa6cce09e9712b2ad70d
>> # good: [bd69058f50d5ffa659423bcfa6fe6280ce9c760a] net: ll_temac: Use
>> devm_platform_ioremap_resource_byname()
>> git bisect good bd69058f50d5ffa659423bcfa6fe6280ce9c760a
>> # skip: [7dbc63f0a5402293e887e89a7974c5e48405565d] ice: Misc minor fixes
>> git bisect skip 7dbc63f0a5402293e887e89a7974c5e48405565d
>> # good: [1303a51c24100b3b1915d6f9072fe5ae5bb4c5f6] cfg80211/mac80211:
>> add connected to auth server to station info
>> git bisect good 1303a51c24100b3b1915d6f9072fe5ae5bb4c5f6
>> # skip: [f34f55557ac9a4dfbfbf36c70585d1648ab5cd90] ice: Allow 2 queue
>> pairs per VF on SR-IOV initialization
>> git bisect skip f34f55557ac9a4dfbfbf36c70585d1648ab5cd90
>> # bad: [cc5d229a122106733a85c279d89d7703f21e4d4f] fsl/fman: check
>> dereferencing null pointer
>> git bisect bad cc5d229a122106733a85c279d89d7703f21e4d4f
>> # good: [6fc8c827dd4fa615965c4eac9bbfd465f6eb8fb4] tcp: syncookies:
>> create mptcp request socket for ACK cookies with MPTCP option
>> git bisect good 6fc8c827dd4fa615965c4eac9bbfd465f6eb8fb4
>> # bad: [b90a1269184a3ff374562d243419ad2fa9d3b1aa] Merge branch
>> 'net-openvswitch-masks-cache-enhancements'
>> git bisect bad b90a1269184a3ff374562d243419ad2fa9d3b1aa
>> # skip: [829eb208e80d6db95c0201cb8fa00c2f9ad87faf] rtnetlink: add
>> support for protodown reason
>> git bisect skip 829eb208e80d6db95c0201cb8fa00c2f9ad87faf
>> # bad: [0e8642cf369a37b718c15effa6ffd52c00fd7d15] tcp: fix build fong
>> CONFIG_MPTCP=n
>> git bisect bad 0e8642cf369a37b718c15effa6ffd52c00fd7d15
>> # skip: [48040793fa6003d211f021c6ad273477bcd90d91] tcp: add earliest
>> departure time to SCM_TIMESTAMPING_OPT_STATS
>> git bisect skip 48040793fa6003d211f021c6ad273477bcd90d91
>> # good: [bc5cbd73eb493944b8665dc517f684c40eb18a4a] iavf: use generic
>> power management
>> git bisect good bc5cbd73eb493944b8665dc517f684c40eb18a4a
>> # skip: [8f3f330da28ede9d106cd9d5c5ccd6a3e7e9b50b] tun: add missing
>> rcu annotation in tun_set_ebpf()
>> git bisect skip 8f3f330da28ede9d106cd9d5c5ccd6a3e7e9b50b
>> # skip: [9466a1ccebbe54ac57fb8a89c2b4b854826546a8] mptcp: enable JOIN
>> requests even if cookies are in use
>> git bisect skip 9466a1ccebbe54ac57fb8a89c2b4b854826546a8
>> # good: [09a071f52bbedddef626e71c0fd210838532f347] Documentation:
>> intel: Replace HTTP links with HTTPS ones
>> git bisect good 09a071f52bbedddef626e71c0fd210838532f347
>> # bad: [75e6b594bbaeeb3f8287a2e6eb8811384b8c7195] cfg80211: invert HE
>> BSS color 'disabled' to 'enabled'
>> git bisect bad 75e6b594bbaeeb3f8287a2e6eb8811384b8c7195
>> # bad: [1df2bdba528b5a7a30f1b107b6924aa79af5e00e] mac80211: never drop
>> injected frames even if normally not allowed
>> git bisect bad 1df2bdba528b5a7a30f1b107b6924aa79af5e00e
>> # good: [180ac48ee62f53c26787350a956c5ac371cbe0b7] mac80211: calculate
>> skb hash early when using itxq
>> git bisect good 180ac48ee62f53c26787350a956c5ac371cbe0b7
>> # good: [322cd27c06450b2db2cb6bdc68f3814149baf767] cfg80211/mac80211:
>> avoid bss color setting in non-HE modes
>> git bisect good 322cd27c06450b2db2cb6bdc68f3814149baf767
>> # good: [fd17dba1c860d39f655a3a08387c21e3ceca8c55] cfg80211: Add
>> support to advertize OCV support
>> git bisect good fd17dba1c860d39f655a3a08387c21e3ceca8c55
>> # first bad commit: [1df2bdba528b5a7a30f1b107b6924aa79af5e00e]
>> mac80211: never drop injected frames even if normally not allowed
> 
> 
> See also:
> =========
> [1]
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1df2bdba528b5a7a30f1b107b6924aa79af5e00e
> 
> 
> 

Re: [Regression 5.9][Bisected 1df2bdba528b] Wifi GTK rekeying fails: Sending of EAPol packages broken

[Mathy Vanhoef]

I've managed to reproduce the issue, or at least a related issue. Can
you try the draft patch below and see if that fixes it?

[PATCH] mac80211: fix regression where EAPOL frames were sent in plaintext

When sending EAPOL frames via NL80211 they are treated as injected
frames in mac80211. Due to commit 1df2bdba528b ("mac80211: never drop
injected frames even if normally not allowed") these injected frames
were not assigned a sta context in the function ieee80211_tx_dequeue,
causing certain wireless network cards to always send EAPOL frames in
plaintext. This may cause compatibility issues with some clients or
APs, which for instance can cause the group key handshake to fail and
in turn would cause the station to get disconnected.

This commit fixes this regression by assigning a sta context in
ieee80211_tx_dequeue to injected frames as well.

Note that sending EAPOL frames in plaintext is not a security issue
since they contain their own encryption and authentication protection.

Fixes: 1df2bdba528b ("mac80211: never drop injected frames even if normally not allowed")
---
 net/mac80211/tx.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
index 8ba10a48d..55b41167a 100644
--- a/net/mac80211/tx.c
+++ b/net/mac80211/tx.c
@@ -3619,13 +3619,14 @@ struct sk_buff *ieee80211_tx_dequeue(struct ieee80211_hw *hw,
 	tx.skb = skb;
 	tx.sdata = vif_to_sdata(info->control.vif);
 
-	if (txq->sta && !(info->flags & IEEE80211_TX_CTL_INJECTED)) {
+	if (txq->sta) {
 		tx.sta = container_of(txq->sta, struct sta_info, sta);
 		/*
 		 * Drop unicast frames to unauthorised stations unless they are
-		 * EAPOL frames from the local station.
+		 * injected frames or EAPOL frames from the local station.
 		 */
-		if (unlikely(ieee80211_is_data(hdr->frame_control) &&
+		if (unlikely(!(info->flags & IEEE80211_TX_CTL_INJECTED) &&
+			     ieee80211_is_data(hdr->frame_control) &&
 			     !ieee80211_vif_is_mesh(&tx.sdata->vif) &&
 			     tx.sdata->vif.type != NL80211_IFTYPE_OCB &&
 			     !is_multicast_ether_addr(hdr->addr1) &&
-- 
2.28.0

Re: [Regression 5.9][Bisected 1df2bdba528b] Wifi GTK rekeying fails: Sending of EAPol packages broken

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 494 bytes --]

Mathy Vanhoef <Mathy.Vanhoef@kuleuven.be> on Sat, 2020/10/17 23:08:
> I've managed to reproduce the issue, or at least a related issue. Can
> you try the draft patch below and see if that fixes it?

This patch fixes the regression for me. Thanks a lot!
-- 
main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
"CX:;",b;for(a/*    Best regards             my address:    */=0;b=c[a++];)
putchar(b-1/(/*    Chris            cc -ox -xc - && ./x    */b/42*2-3)*42);}

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

RE: [Regression 5.9][Bisected 1df2bdba528b] Wifi GTK rekeying fails: Sending of EAPol packages broken

[Thomas Deutschmann]

[-- Attachment #1: Type: text/plain, Size: 188 bytes --]

Hi Mathy,

I can also confirm that the patch works for me, thank you!


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5

[-- Attachment #2: openpgp-digital-signature.asc --]
[-- Type: application/pgp-signature, Size: 619 bytes --]