regressions.lists.linux.dev archive mirror
 help / color / mirror / Atom feed
* Re: xfrm regression: TCP MSS calculation broken by commit b515d263, results in TCP stall
       [not found] <20220114173133.tzmdm2hy4flhblo3@dwarf.suse.cz>
@ 2022-01-16 10:18 ` Thorsten Leemhuis
  2022-02-04  8:59   ` xfrm regression: TCP MSS calculation broken by commit b515d263, results in TCP stall #forregzbot Thorsten Leemhuis
       [not found] ` <20220114174058.rqhtuwpfhq6czldn@dwarf.suse.cz>
  1 sibling, 1 reply; 4+ messages in thread
From: Thorsten Leemhuis @ 2022-01-16 10:18 UTC (permalink / raw)
  To: Jiri Bohac, Sabrina Dubroca, Steffen Klassert, Herbert Xu,
	David S. Miller
  Cc: Mike Maloney, Eric Dumazet, netdev, regressions

[TLDR: I'm adding this regression to regzbot, the Linux kernel
regression tracking bot; most text you find below is compiled from a few
templates paragraphs some of you might have seen already.]

Top-posting for once, to make this easy accessible to everyone.

On 14.01.22 18:31, Jiri Bohac wrote:
> 
> our customer found that commit
> b515d2637276a3810d6595e10ab02c13bfd0b63a ("xfrm: xfrm_state_mtu
> should return at least 1280 for ipv6") in v5.14 breaks the TCP
> MSS calculation in ipsec transport mode, resulting complete
> stalls of TCP connections. This happens when the (P)MTU is 1280
> or slighly larger.
> 
> The desired formula for the MSS is:
> 	MSS = (MTU - ESP_overhead) - IP header - TCP header
> 
> However, the above patch clamps the (MTU - ESP_overhead) to a
> minimum of 1280, turning the formula into
> 	MSS = max(MTU - ESP overhead, 1280) -  IP header - TCP header
> 
> With the (P)MTU near 1280, the calculated MSS is too large and
> the resulting TCP packets never make it to the destination
> because they are over the actual PMTU.
> 
> Trying to fix the exact same problem as the broken patch, which I
> was unaware of, I sent an alternative patch in this thread of
> April 2021:
> https://lore.kernel.org/netdev/20210429170254.5grfgsz2hgy2qjhk@dwarf.suse.cz/
> (note the v1 is broken and followed by v2!)
> 
> In that thread I also found other problems with
> b515d2637276a3810d6595e10ab02c13bfd0b63a - in tunnel mode it
> causes suboptimal double fragmentation:
> https://lore.kernel.org/netdev/20210429202529.codhwpc7w6kbudug@dwarf.suse.cz/
> 
> I therefore propose to revert
> b515d2637276a3810d6595e10ab02c13bfd0b63a and
> apply the v2 version of my patch, which I'll re-send in reply to
> this e-mail.

Thanks for the report.

To be sure this issue doesn't fall through the cracks unnoticed, I'm
adding it to regzbot, my Linux kernel regression tracking bot:

#regzbot ^introduced b515d2637276a3810d6595e10ab02c13bfd0b63a
#regzbot title xfrm: TCP MSS calculation broken by commit b515d263,
results in TCP stall
#regzbot ignore-activity

Reminder: when fixing the issue, please add a 'Link:' tag with the URL
to the report (the parent of this mail) using the kernel.org redirector,
as explained in 'Documentation/process/submitting-patches.rst'. Regzbot
then will automatically mark the regression as resolved once the fix
lands in the appropriate tree. For more details about regzbot see footer.

Sending this to everyone that got the initial report, to make all aware
of the tracking. I also hope that messages like this motivate people to
directly get at least the regression mailing list and ideally even
regzbot involved when dealing with regressions, as messages like this
wouldn't be needed then.

Don't worry, I'll send further messages wrt to this regression just to
the lists (with a tag in the subject so people can filter them away), as
long as they are intended just for regzbot. With a bit of luck no such
messages will be needed anyway.

Ciao, Thorsten (wearing his 'Linux kernel regression tracker' hat)

P.S.: As a Linux kernel regression tracker I'm getting a lot of reports
on my table. I can only look briefly into most of them. Unfortunately
therefore I sometimes will get things wrong or miss something important.
I hope that's not the case here; if you think it is, don't hesitate to
tell me about it in a public reply, that's in everyone's interest.

BTW, I have no personal interest in this issue, which is tracked using
regzbot, my Linux kernel regression tracking bot
(https://linux-regtracking.leemhuis.info/regzbot/). I'm only posting
this mail to get things rolling again and hence don't need to be CC on
all further activities wrt to this regression.

---
Additional information about regzbot:

If you want to know more about regzbot, check out its web-interface, the
getting start guide, and/or the references documentation:

https://linux-regtracking.leemhuis.info/regzbot/
https://gitlab.com/knurd42/regzbot/-/blob/main/docs/getting_started.md
https://gitlab.com/knurd42/regzbot/-/blob/main/docs/reference.md

The last two documents will explain how you can interact with regzbot
yourself if your want to.

Hint for reporters: when reporting a regression it's in your interest to
tell #regzbot about it in the report, as that will ensure the regression
gets on the radar of regzbot and the regression tracker. That's in your
interest, as they will make sure the report won't fall through the
cracks unnoticed.

Hint for developers: you normally don't need to care about regzbot once
it's involved. Fix the issue as you normally would, just remember to
include a 'Link:' tag to the report in the commit message, as explained
in Documentation/process/submitting-patches.rst
That aspect was recently was made more explicit in commit 1f57bd42b77c:
https://git.kernel.org/linus/1f57bd42b77c


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: xfrm regression: TCP MSS calculation broken by commit b515d263, results in TCP stall #forregzbot
  2022-01-16 10:18 ` xfrm regression: TCP MSS calculation broken by commit b515d263, results in TCP stall Thorsten Leemhuis
@ 2022-02-04  8:59   ` Thorsten Leemhuis
  0 siblings, 0 replies; 4+ messages in thread
From: Thorsten Leemhuis @ 2022-02-04  8:59 UTC (permalink / raw)
  To: regressions

For the record:

#regzbot fixed-by: a6d95c5a628a09be

https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?id=a6d95c5a628a09be129f25d5663a7e9db8261f51

> Revert "xfrm: xfrm_state_mtu should return at least 1280 for ipv6"
> This reverts commit b515d2637276a3810d6595e10ab02c13bfd0b63a.
> 
> Commit b515d2637276a3810d6595e10ab02c13bfd0b63a ("xfrm: xfrm_state_mtu
> should return at least 1280 for ipv6") in v5.14 breaks the TCP MSS
> calculation in ipsec transport mode, resulting complete stalls of TCP
> connections. This happens when the (P)MTU is 1280 or slighly larger.
> 
> The desired formula for the MSS is:
> MSS = (MTU - ESP_overhead) - IP header - TCP header
> 
> However, the above commit clamps the (MTU - ESP_overhead) to a
> minimum of 1280, turning the formula into
> MSS = max(MTU - ESP overhead, 1280) -  IP header - TCP header
> 
> With the (P)MTU near 1280, the calculated MSS is too large and the
> resulting TCP packets never make it to the destination because they
> are over the actual PMTU.
> 
> The above commit also causes suboptimal double fragmentation in
> xfrm tunnel mode, as described in
> https://lore.kernel.org/netdev/20210429202529.codhwpc7w6kbudug@dwarf.suse.cz/
> 
> The original problem the above commit was trying to fix is now fixed
> by commit 6596a0229541270fb8d38d989f91b78838e5e9da ("xfrm: fix MTU
> regression").
> 
> Signed-off-by: Jiri Bohac <jbohac@suse.cz>
> Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>

TWIMC: this mail is primarily send for documentation purposes and for
regzbot, my Linux kernel regression tracking bot. These mails usually
contain '#forregzbot' in the subject, to make them easy to spot and filter.

On 16.01.22 11:18, Thorsten Leemhuis wrote:
> [TLDR: I'm adding this regression to regzbot, the Linux kernel
> regression tracking bot; most text you find below is compiled from a few
> templates paragraphs some of you might have seen already.]
> 
> Top-posting for once, to make this easy accessible to everyone.
> 
> On 14.01.22 18:31, Jiri Bohac wrote:
>>
>> our customer found that commit
>> b515d2637276a3810d6595e10ab02c13bfd0b63a ("xfrm: xfrm_state_mtu
>> should return at least 1280 for ipv6") in v5.14 breaks the TCP
>> MSS calculation in ipsec transport mode, resulting complete
>> stalls of TCP connections. This happens when the (P)MTU is 1280
>> or slighly larger.
>>
>> The desired formula for the MSS is:
>> 	MSS = (MTU - ESP_overhead) - IP header - TCP header
>>
>> However, the above patch clamps the (MTU - ESP_overhead) to a
>> minimum of 1280, turning the formula into
>> 	MSS = max(MTU - ESP overhead, 1280) -  IP header - TCP header
>>
>> With the (P)MTU near 1280, the calculated MSS is too large and
>> the resulting TCP packets never make it to the destination
>> because they are over the actual PMTU.
>>
>> Trying to fix the exact same problem as the broken patch, which I
>> was unaware of, I sent an alternative patch in this thread of
>> April 2021:
>> https://lore.kernel.org/netdev/20210429170254.5grfgsz2hgy2qjhk@dwarf.suse.cz/
>> (note the v1 is broken and followed by v2!)
>>
>> In that thread I also found other problems with
>> b515d2637276a3810d6595e10ab02c13bfd0b63a - in tunnel mode it
>> causes suboptimal double fragmentation:
>> https://lore.kernel.org/netdev/20210429202529.codhwpc7w6kbudug@dwarf.suse.cz/
>>
>> I therefore propose to revert
>> b515d2637276a3810d6595e10ab02c13bfd0b63a and
>> apply the v2 version of my patch, which I'll re-send in reply to
>> this e-mail.
> 
> Thanks for the report.
> 
> To be sure this issue doesn't fall through the cracks unnoticed, I'm
> adding it to regzbot, my Linux kernel regression tracking bot:
> 
> #regzbot ^introduced b515d2637276a3810d6595e10ab02c13bfd0b63a
> #regzbot title xfrm: TCP MSS calculation broken by commit b515d263,
> results in TCP stall
> #regzbot ignore-activity
> 
> Reminder: when fixing the issue, please add a 'Link:' tag with the URL
> to the report (the parent of this mail) using the kernel.org redirector,
> as explained in 'Documentation/process/submitting-patches.rst'. Regzbot
> then will automatically mark the regression as resolved once the fix
> lands in the appropriate tree. For more details about regzbot see footer.
> 
> Sending this to everyone that got the initial report, to make all aware
> of the tracking. I also hope that messages like this motivate people to
> directly get at least the regression mailing list and ideally even
> regzbot involved when dealing with regressions, as messages like this
> wouldn't be needed then.
> 
> Don't worry, I'll send further messages wrt to this regression just to
> the lists (with a tag in the subject so people can filter them away), as
> long as they are intended just for regzbot. With a bit of luck no such
> messages will be needed anyway.
> 
> Ciao, Thorsten (wearing his 'Linux kernel regression tracker' hat)
> 
> P.S.: As a Linux kernel regression tracker I'm getting a lot of reports
> on my table. I can only look briefly into most of them. Unfortunately
> therefore I sometimes will get things wrong or miss something important.
> I hope that's not the case here; if you think it is, don't hesitate to
> tell me about it in a public reply, that's in everyone's interest.
> 
> BTW, I have no personal interest in this issue, which is tracked using
> regzbot, my Linux kernel regression tracking bot
> (https://linux-regtracking.leemhuis.info/regzbot/). I'm only posting
> this mail to get things rolling again and hence don't need to be CC on
> all further activities wrt to this regression.
> 
> ---
> Additional information about regzbot:
> 
> If you want to know more about regzbot, check out its web-interface, the
> getting start guide, and/or the references documentation:
> 
> https://linux-regtracking.leemhuis.info/regzbot/
> https://gitlab.com/knurd42/regzbot/-/blob/main/docs/getting_started.md
> https://gitlab.com/knurd42/regzbot/-/blob/main/docs/reference.md
> 
> The last two documents will explain how you can interact with regzbot
> yourself if your want to.
> 
> Hint for reporters: when reporting a regression it's in your interest to
> tell #regzbot about it in the report, as that will ensure the regression
> gets on the radar of regzbot and the regression tracker. That's in your
> interest, as they will make sure the report won't fall through the
> cracks unnoticed.
> 
> Hint for developers: you normally don't need to care about regzbot once
> it's involved. Fix the issue as you normally would, just remember to
> include a 'Link:' tag to the report in the commit message, as explained
> in Documentation/process/submitting-patches.rst
> That aspect was recently was made more explicit in commit 1f57bd42b77c:
> https://git.kernel.org/linus/1f57bd42b77c
> 

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] Revert "xfrm: xfrm_state_mtu should return at least 1280 for ipv6"
       [not found]               ` <20220201064639.GS1223722@gauss3.secunet.de>
@ 2022-02-15 14:59                 ` Thorsten Leemhuis
  2022-02-16 11:02                   ` Steffen Klassert
  0 siblings, 1 reply; 4+ messages in thread
From: Thorsten Leemhuis @ 2022-02-15 14:59 UTC (permalink / raw)
  To: Steffen Klassert, Jiri Bohac
  Cc: Sabrina Dubroca, Herbert Xu, David S. Miller, Mike Maloney,
	Eric Dumazet, netdev, Jakub Kicinski, regressions

Hi, this is your Linux kernel regression tracker speaking.

The patch discussed below is now in linux-next for about 11 days, but
not yet in the net tree afaics. Will it be merged this week? And
shouldn't this patch have these stables tags?

Cc: <stable@vger.kernel.org> # 5.14: 6596a0229541 xfrm: fix MTU regression
Cc: <stable@vger.kernel.org> # 5.14

And maybe a fixes tag, too?

Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)

P.S.: As the Linux kernel's regression tracker I'm getting a lot of
reports on my table. I can only look briefly into most of them and lack
knowledge about most of the areas they concern. I thus unfortunately
will sometimes get things wrong or miss something important. I hope
that's not the case here; if you think it is, don't hesitate to tell me
in a public reply, it's in everyone's interest to set the public record
straight.

#regzbot poke

On 01.02.22 07:46, Steffen Klassert wrote:
> On Wed, Jan 26, 2022 at 04:00:18PM +0100, Jiri Bohac wrote:
>> This reverts commit b515d2637276a3810d6595e10ab02c13bfd0b63a.
>>
>> Commit b515d2637276a3810d6595e10ab02c13bfd0b63a ("xfrm: xfrm_state_mtu
>> should return at least 1280 for ipv6") in v5.14 breaks the TCP MSS
>> calculation in ipsec transport mode, resulting complete stalls of TCP
>> connections. This happens when the (P)MTU is 1280 or slighly larger.
>>
>> The desired formula for the MSS is:
>> MSS = (MTU - ESP_overhead) - IP header - TCP header
>>
>> However, the above commit clamps the (MTU - ESP_overhead) to a
>> minimum of 1280, turning the formula into
>> MSS = max(MTU - ESP overhead, 1280) -  IP header - TCP header
>>
>> With the (P)MTU near 1280, the calculated MSS is too large and the
>> resulting TCP packets never make it to the destination because they
>> are over the actual PMTU.
>>
>> The above commit also causes suboptimal double fragmentation in
>> xfrm tunnel mode, as described in
>> https://lore.kernel.org/netdev/20210429202529.codhwpc7w6kbudug@dwarf.suse.cz/
>>
>> The original problem the above commit was trying to fix is now fixed
>> by commit 6596a0229541270fb8d38d989f91b78838e5e9da ("xfrm: fix MTU
>> regression").
>>
>> Signed-off-by: Jiri Bohac <jbohac@suse.cz>
> 
> Applied, thanks Jiri!

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] Revert "xfrm: xfrm_state_mtu should return at least 1280 for ipv6"
  2022-02-15 14:59                 ` [PATCH] Revert "xfrm: xfrm_state_mtu should return at least 1280 for ipv6" Thorsten Leemhuis
@ 2022-02-16 11:02                   ` Steffen Klassert
  0 siblings, 0 replies; 4+ messages in thread
From: Steffen Klassert @ 2022-02-16 11:02 UTC (permalink / raw)
  To: Thorsten Leemhuis
  Cc: Jiri Bohac, Sabrina Dubroca, Herbert Xu, David S. Miller,
	Mike Maloney, Eric Dumazet, netdev, Jakub Kicinski, regressions

On Tue, Feb 15, 2022 at 03:59:38PM +0100, Thorsten Leemhuis wrote:
> Hi, this is your Linux kernel regression tracker speaking.
> 
> The patch discussed below is now in linux-next for about 11 days, but
> not yet in the net tree afaics. Will it be merged this week? And

It will be merged with the next pull request for the ipsec tree
that will happen likely next week.

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2022-02-16 11:11 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <20220114173133.tzmdm2hy4flhblo3@dwarf.suse.cz>
2022-01-16 10:18 ` xfrm regression: TCP MSS calculation broken by commit b515d263, results in TCP stall Thorsten Leemhuis
2022-02-04  8:59   ` xfrm regression: TCP MSS calculation broken by commit b515d263, results in TCP stall #forregzbot Thorsten Leemhuis
     [not found] ` <20220114174058.rqhtuwpfhq6czldn@dwarf.suse.cz>
     [not found]   ` <20220119073519.GJ1223722@gauss3.secunet.de>
     [not found]     ` <20220119091233.pzqdlzpcyicjavk5@dwarf.suse.cz>
     [not found]       ` <20220124154531.GM1223722@gauss3.secunet.de>
     [not found]         ` <20220125094102.ju7bhuplcxnkyv4x@dwarf.suse.cz>
     [not found]           ` <20220126064214.GO1223722@gauss3.secunet.de>
     [not found]             ` <20220126150018.7cdfxtkq2nfkqj4j@dwarf.suse.cz>
     [not found]               ` <20220201064639.GS1223722@gauss3.secunet.de>
2022-02-15 14:59                 ` [PATCH] Revert "xfrm: xfrm_state_mtu should return at least 1280 for ipv6" Thorsten Leemhuis
2022-02-16 11:02                   ` Steffen Klassert

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).