* [MPTCP][PATCH mptcp-next] mptcp: don't fallback when data_fin enabled
@ 2021-05-07 7:58 Geliang Tang
2021-05-07 8:51 ` Matthieu Baerts
0 siblings, 1 reply; 6+ messages in thread
From: Geliang Tang @ 2021-05-07 7:58 UTC (permalink / raw)
To: mptcp; +Cc: Geliang Tang
This patch fixed this fallback error:
<7>[ 1477.059635] MPTCP: msk=000000006bf0cf61
<7>[ 1485.132216] MPTCP: subflow=000000002296fc22
<7>[ 1485.132233] MPTCP: subflow=000000002296fc22, family=2
<7>[ 1485.132242] MPTCP: subflow=000000002296fc22
<7>[ 1485.132411] MPTCP: msk=000000005b84960a
<7>[ 1485.132688] MPTCP: subflow=000000002388ad62
<7>[ 1485.132695] MPTCP: subflow=000000002388ad62, family=2
<7>[ 1485.132702] MPTCP: subflow=000000002388ad62
<7>[ 1485.132974] MPTCP: ssk=000000005ab5dbc8, local_key=9186175039591172243, token=2833190200, idsn=13562085501921405456
<7>[ 1485.133131] MPTCP: subflow=000000002296fc22
<7>[ 1485.133200] MPTCP: subflow_req=00000000ca26b42c, listener=000000002296fc22
<7>[ 1485.133209] MPTCP: MP_CAPABLE version=1, flags=1, optlen=4 sndr=582, rcvr=18446744072046692096 len=65535 csum=65535
<7>[ 1485.133227] MPTCP: req=00000000ca26b42c local_key=9900409276882973726, token=2796355457, idsn=5747750831347864753
<7>[ 1485.133288] MPTCP: subflow_req=00000000ca26b42c, local_key=9900409276882973726
<7>[ 1485.133384] MPTCP: subflow=000000002388ad62 synack seq=72748901
<7>[ 1485.133393] MPTCP: MP_CAPABLE version=1, flags=1, optlen=12 sndr=9900409276882973726, rcvr=18446633460193955073 len=0 csum=0
<7>[ 1485.133402] MPTCP: subflow=000000002388ad62, remote_key=9900409276882973726
<7>[ 1485.133409] MPTCP: msk=00000000875940e7, token=2833190200
<7>[ 1485.133416] MPTCP: msk=00000000875940e7, token=2833190200 side=0
<7>[ 1485.133457] MPTCP: subflow=000000002388ad62, local_key=9186175039591172243, remote_key=9900409276882973726 map_len=0
<7>[ 1485.133490] MPTCP: listener=000000002296fc22, req=00000000ca26b42c, conn=000000005b84960a
<7>[ 1485.133499] MPTCP: MP_CAPABLE version=1, flags=1, optlen=20 sndr=9186175039591172243, rcvr=9900409276882973726 len=0 csum=0
<7>[ 1485.133587] MPTCP: subflow=000000000587a2a5
<7>[ 1485.133625] MPTCP: msk=000000005dcb6e30, token=2796355457 side=1
<7>[ 1485.133658] MPTCP: MP_CAPABLE version=1, flags=1, optlen=20 sndr=9186175039591172243, rcvr=9900409276882973726 len=39782 csum=65535
<7>[ 1485.134972] MPTCP: msk=00000000875940e7 snd_data_fin_enable=0 shutdown=3 state=4 pending=0
<7>[ 1485.134987] MPTCP: msk=00000000875940e7 snd_data_fin_enable=1 pending=0 snd_nxt=13562085501921405457 write_seq=13562085501921405458
<7>[ 1485.134998] MPTCP: Sending DATA_FIN on subflow 000000005ab5dbc8
<7>[ 1485.135083] MPTCP: DSS
<7>[ 1485.135089] MPTCP: data_fin=1 dsn64=1 use_map=1 ack64=0 use_ack=1
<7>[ 1485.135097] MPTCP: data_ack=2683442354
<7>[ 1485.135102] MPTCP: data_seq=13562085501921405457 subflow_seq=0 data_len=1 csum=0:0
<7>[ 1485.135198] MPTCP: msk=000000005dcb6e30 snd_data_fin_enable=0 pending=0 snd_nxt=5747750831347864754 write_seq=5747750831347864754
<7>[ 1485.135233] MPTCP: msk=00000000875940e7 state=4
<7>[ 1485.135297] MPTCP: msk=000000005b84960a state=7
<7>[ 1485.135302] MPTCP: msk=000000005b84960a
<7>[ 1485.135387] MPTCP: msk=000000005dcb6e30
<7>[ 1485.135436] MPTCP: subflow_req=00000000ca26b42c
<7>[ 1485.135472] MPTCP: check_fully_established:fallback to TCP (msk=00000000875940e7)
<7>[ 1485.135628] MPTCP: msk=000000005b84960a
<7>[ 1485.339305] MPTCP: msk=00000000875940e7 snd_data_fin_enable=1 pending=0 snd_nxt=13562085501921405458 write_seq=13562085501921405458
<7>[ 1485.339329] MPTCP: msk=00000000875940e7
Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/176
Signed-off-by: Geliang Tang <geliangtang@gmail.com>
---
net/mptcp/options.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/mptcp/options.c b/net/mptcp/options.c
index f3e379a0f08d..940b32511443 100644
--- a/net/mptcp/options.c
+++ b/net/mptcp/options.c
@@ -901,7 +901,7 @@ static bool check_fully_established(struct mptcp_sock *msk, struct sock *ssk,
* MP_JOIN subflows.
*/
if (!mp_opt->mp_capable) {
- if (subflow->mp_join)
+ if (subflow->mp_join || mptcp_data_fin_enabled(msk))
goto reset;
subflow->mp_capable = 0;
pr_fallback(msk);
--
2.31.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [MPTCP][PATCH mptcp-next] mptcp: don't fallback when data_fin enabled
2021-05-07 7:58 [MPTCP][PATCH mptcp-next] mptcp: don't fallback when data_fin enabled Geliang Tang
@ 2021-05-07 8:51 ` Matthieu Baerts
2021-05-07 9:08 ` Geliang Tang
0 siblings, 1 reply; 6+ messages in thread
From: Matthieu Baerts @ 2021-05-07 8:51 UTC (permalink / raw)
To: Geliang Tang; +Cc: mptcp
Hi Geliang,
On 07/05/2021 09:58, Geliang Tang wrote:
> This patch fixed this fallback error:
Thank you for the patch!
> diff --git a/net/mptcp/options.c b/net/mptcp/options.c
> index f3e379a0f08d..940b32511443 100644
> --- a/net/mptcp/options.c
> +++ b/net/mptcp/options.c
> @@ -901,7 +901,7 @@ static bool check_fully_established(struct mptcp_sock *msk, struct sock *ssk,
> * MP_JOIN subflows.
> */
> if (!mp_opt->mp_capable) {
> - if (subflow->mp_join)
> + if (subflow->mp_join || mptcp_data_fin_enabled(msk))> goto reset;
I don't think we should send a reset here. For me this behaviour is normal.
When looking at the debug trace:
> <7>[ 1485.133416] MPTCP: msk=00000000875940e7, token=2833190200 side=0
> <7>[ 1485.134972] MPTCP: msk=00000000875940e7 snd_data_fin_enable=0
shutdown=3 state=4 pending=0
It looks like the userspace has closed the connection:
mptcp_close() → __mptcp_wr_shutdown()
There we do:
/* will be ignored by fallback sockets */
WRITE_ONCE(msk->write_seq, msk->write_seq + 1);
WRITE_ONCE(msk->snd_data_fin_enable, 1);
> <7>[ 1485.134987] MPTCP: msk=00000000875940e7 snd_data_fin_enable=1
pending=0 snd_nxt=13562085501921405457
From:
__mptcp_check_send_data_fin(sk);
> <7>[ 1485.135233] MPTCP: msk=00000000875940e7 state=4
From:
mptcp_close()
> <7>[ 1485.135472] MPTCP: check_fully_established:fallback to TCP
(msk=00000000875940e7)
It means we certainly got a FIN back:
mptcp_incoming_options() → check_fully_established()
> <7>[ 1485.339305] MPTCP: msk=00000000875940e7 snd_data_fin_enable=1
pending=0 snd_nxt=13562085501921405458 write_seq=13562085501921405458
From:
__mptcp_check_send_data_fin()
But here we will not send the DATA_FIN because we did a fallback to TCP.
But we change the state to properly close the socket.
> <7>[ 1485.339329] MPTCP: msk=00000000875940e7
By chance, do you have a packet trace to confirm that?
It's unclear to me if we send a DATA_FIN in this case. I guess we should
not if we are not in fully_established and I think that's what we do but
I'm unsure now.
Cheers,
Matt
--
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [MPTCP][PATCH mptcp-next] mptcp: don't fallback when data_fin enabled
2021-05-07 8:51 ` Matthieu Baerts
@ 2021-05-07 9:08 ` Geliang Tang
2021-05-07 12:19 ` Matthieu Baerts
0 siblings, 1 reply; 6+ messages in thread
From: Geliang Tang @ 2021-05-07 9:08 UTC (permalink / raw)
To: Matthieu Baerts; +Cc: mptcp
Hi Matt,
Thanks for your review.
Matthieu Baerts <matthieu.baerts@tessares.net> 于2021年5月7日周五 下午4:51写道:
>
> Hi Geliang,
>
> On 07/05/2021 09:58, Geliang Tang wrote:
> > This patch fixed this fallback error:
>
> Thank you for the patch!
>
> > diff --git a/net/mptcp/options.c b/net/mptcp/options.c
> > index f3e379a0f08d..940b32511443 100644
> > --- a/net/mptcp/options.c
> > +++ b/net/mptcp/options.c
> > @@ -901,7 +901,7 @@ static bool check_fully_established(struct mptcp_sock *msk, struct sock *ssk,
> > * MP_JOIN subflows.
> > */
> > if (!mp_opt->mp_capable) {
> > - if (subflow->mp_join)
> > + if (subflow->mp_join || mptcp_data_fin_enabled(msk))> goto reset;
>
> I don't think we should send a reset here. For me this behaviour is normal.
Yes, we don't need to send a reset here. What about just return here?
Something like this:
diff --git a/net/mptcp/options.c b/net/mptcp/options.c
index 3428c163299b..bd62dadfefdb 100644
--- a/net/mptcp/options.c
+++ b/net/mptcp/options.c
@@ -905,6 +905,8 @@ static bool check_fully_established(struct
mptcp_sock *msk, struct sock *ssk,
if (!mp_opt->mp_capable) {
if (subflow->mp_join)
goto reset;
+ if (mptcp_data_fin_enabled(msk))
+ return false;
subflow->mp_capable = 0;
pr_fallback(msk);
__mptcp_do_fallback(msk);
-Geliang
>
> When looking at the debug trace:
>
> > <7>[ 1485.133416] MPTCP: msk=00000000875940e7, token=2833190200 side=0
> > <7>[ 1485.134972] MPTCP: msk=00000000875940e7 snd_data_fin_enable=0
> shutdown=3 state=4 pending=0
>
> It looks like the userspace has closed the connection:
>
> mptcp_close() → __mptcp_wr_shutdown()
>
> There we do:
>
> /* will be ignored by fallback sockets */
> WRITE_ONCE(msk->write_seq, msk->write_seq + 1);
> WRITE_ONCE(msk->snd_data_fin_enable, 1);
>
> > <7>[ 1485.134987] MPTCP: msk=00000000875940e7 snd_data_fin_enable=1
> pending=0 snd_nxt=13562085501921405457
>
> From:
>
> __mptcp_check_send_data_fin(sk);
>
> > <7>[ 1485.135233] MPTCP: msk=00000000875940e7 state=4
>
> From:
>
> mptcp_close()
>
> > <7>[ 1485.135472] MPTCP: check_fully_established:fallback to TCP
> (msk=00000000875940e7)
>
> It means we certainly got a FIN back:
>
> mptcp_incoming_options() → check_fully_established()
>
> > <7>[ 1485.339305] MPTCP: msk=00000000875940e7 snd_data_fin_enable=1
> pending=0 snd_nxt=13562085501921405458 write_seq=13562085501921405458
>
> From:
>
> __mptcp_check_send_data_fin()
>
> But here we will not send the DATA_FIN because we did a fallback to TCP.
> But we change the state to properly close the socket.
>
> > <7>[ 1485.339329] MPTCP: msk=00000000875940e7
>
> By chance, do you have a packet trace to confirm that?
>
> It's unclear to me if we send a DATA_FIN in this case. I guess we should
> not if we are not in fully_established and I think that's what we do but
> I'm unsure now.
>
> Cheers,
> Matt
> --
> Tessares | Belgium | Hybrid Access Solutions
> www.tessares.net
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [MPTCP][PATCH mptcp-next] mptcp: don't fallback when data_fin enabled
2021-05-07 9:08 ` Geliang Tang
@ 2021-05-07 12:19 ` Matthieu Baerts
2021-05-07 15:23 ` Matthieu Baerts
0 siblings, 1 reply; 6+ messages in thread
From: Matthieu Baerts @ 2021-05-07 12:19 UTC (permalink / raw)
To: Geliang Tang; +Cc: mptcp
Hi Geliang,
On 07/05/2021 11:08, Geliang Tang wrote:
> Hi Matt,
>
> Thanks for your review.
>
> Matthieu Baerts <matthieu.baerts@tessares.net> 于2021年5月7日周五 下午4:51写道:
>>
>> Hi Geliang,
>>
>> On 07/05/2021 09:58, Geliang Tang wrote:
>>> This patch fixed this fallback error:
>>
>> Thank you for the patch!
>>
>>> diff --git a/net/mptcp/options.c b/net/mptcp/options.c
>>> index f3e379a0f08d..940b32511443 100644
>>> --- a/net/mptcp/options.c
>>> +++ b/net/mptcp/options.c
>>> @@ -901,7 +901,7 @@ static bool check_fully_established(struct mptcp_sock *msk, struct sock *ssk,
>>> * MP_JOIN subflows.
>>> */
>>> if (!mp_opt->mp_capable) {
>>> - if (subflow->mp_join)
>>> + if (subflow->mp_join || mptcp_data_fin_enabled(msk))> goto reset;
>>
>> I don't think we should send a reset here. For me this behaviour is normal.
>
> Yes, we don't need to send a reset here. What about just return here?
> Something like this:
>
> diff --git a/net/mptcp/options.c b/net/mptcp/options.c
> index 3428c163299b..bd62dadfefdb 100644
> --- a/net/mptcp/options.c
> +++ b/net/mptcp/options.c
> @@ -905,6 +905,8 @@ static bool check_fully_established(struct
> mptcp_sock *msk, struct sock *ssk,
> if (!mp_opt->mp_capable) {
> if (subflow->mp_join)
> goto reset;
> + if (mptcp_data_fin_enabled(msk))
> + return false;
> subflow->mp_capable = 0;
> pr_fallback(msk);
> __mptcp_do_fallback(msk);
I think we still need to do a fallback here not to expect a DATA_ACK.
For example in __mptcp_check_send_data_fin(), we change the state
because we will not get a DATA_ACK.
So I think everything is OK there.
Just one thing: should we send a DATA_FIN if we were not "fully
established" when closing the socket?
Cheers,
Matt
--
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [MPTCP][PATCH mptcp-next] mptcp: don't fallback when data_fin enabled
2021-05-07 12:19 ` Matthieu Baerts
@ 2021-05-07 15:23 ` Matthieu Baerts
2021-05-08 2:58 ` Geliang Tang
0 siblings, 1 reply; 6+ messages in thread
From: Matthieu Baerts @ 2021-05-07 15:23 UTC (permalink / raw)
To: Geliang Tang; +Cc: mptcp
On 07/05/2021 14:19, Matthieu Baerts wrote:
> Hi Geliang,
>
> On 07/05/2021 11:08, Geliang Tang wrote:
>> Hi Matt,
>>
>> Thanks for your review.
>>
>> Matthieu Baerts <matthieu.baerts@tessares.net> 于2021年5月7日周五 下午4:51写道:
>>>
>>> Hi Geliang,
>>>
>>> On 07/05/2021 09:58, Geliang Tang wrote:
>>>> This patch fixed this fallback error:
>>>
>>> Thank you for the patch!
>>>
>>>> diff --git a/net/mptcp/options.c b/net/mptcp/options.c
>>>> index f3e379a0f08d..940b32511443 100644
>>>> --- a/net/mptcp/options.c
>>>> +++ b/net/mptcp/options.c
>>>> @@ -901,7 +901,7 @@ static bool check_fully_established(struct mptcp_sock *msk, struct sock *ssk,
>>>> * MP_JOIN subflows.
>>>> */
>>>> if (!mp_opt->mp_capable) {
>>>> - if (subflow->mp_join)
>>>> + if (subflow->mp_join || mptcp_data_fin_enabled(msk))> goto reset;
>>>
>>> I don't think we should send a reset here. For me this behaviour is normal.
>>
>> Yes, we don't need to send a reset here. What about just return here?
>> Something like this:
>>
>> diff --git a/net/mptcp/options.c b/net/mptcp/options.c
>> index 3428c163299b..bd62dadfefdb 100644
>> --- a/net/mptcp/options.c
>> +++ b/net/mptcp/options.c
>> @@ -905,6 +905,8 @@ static bool check_fully_established(struct
>> mptcp_sock *msk, struct sock *ssk,
>> if (!mp_opt->mp_capable) {
>> if (subflow->mp_join)
>> goto reset;
>> + if (mptcp_data_fin_enabled(msk))
>> + return false;
>> subflow->mp_capable = 0;
>> pr_fallback(msk);
>> __mptcp_do_fallback(msk);
>
> I think we still need to do a fallback here not to expect a DATA_ACK.
> For example in __mptcp_check_send_data_fin(), we change the state
> because we will not get a DATA_ACK.
>
> So I think everything is OK there.
>
> Just one thing: should we send a DATA_FIN if we were not "fully
> established" when closing the socket?
Paolo shared this answer on IRC:
I would say yes. The server side could have already allocated the msk
socket (and the 4th ack could have been lost)
Which makes sense.
So I think we should not do anything here. Except maybe a comment here
in the code? e.g.
diff --git a/net/mptcp/options.c b/net/mptcp/options.c
index 3428c163299b..43d55039743a 100644
--- a/net/mptcp/options.c
+++ b/net/mptcp/options.c
@@ -905,6 +905,14 @@ static bool check_fully_established(struct
mptcp_sock *msk, struct sock *ssk,
if (!mp_opt->mp_capable) {
if (subflow->mp_join)
goto reset;
+
+ /* We also want to fallback if the connection is closed
while we
+ * were not in fully established mode.
+ * But we still want to send the DATA_FIN if the
userspace has
+ * closed the connection at this stage, just in case the
server
+ * side has already allocated the msk socket or if the
4th ACK
+ * has been lost.
+ */
subflow->mp_capable = 0;
pr_fallback(msk);
__mptcp_do_fallback(msk);
Cheers,
Matt
--
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [MPTCP][PATCH mptcp-next] mptcp: don't fallback when data_fin enabled
2021-05-07 15:23 ` Matthieu Baerts
@ 2021-05-08 2:58 ` Geliang Tang
0 siblings, 0 replies; 6+ messages in thread
From: Geliang Tang @ 2021-05-08 2:58 UTC (permalink / raw)
To: Matthieu Baerts; +Cc: mptcp
Hi Matt, Paolo,
Matthieu Baerts <matthieu.baerts@tessares.net> 于2021年5月7日周五 下午11:23写道:
>
> On 07/05/2021 14:19, Matthieu Baerts wrote:
> > Hi Geliang,
> >
> > On 07/05/2021 11:08, Geliang Tang wrote:
> >> Hi Matt,
> >>
> >> Thanks for your review.
> >>
> >> Matthieu Baerts <matthieu.baerts@tessares.net> 于2021年5月7日周五 下午4:51写道:
> >>>
> >>> Hi Geliang,
> >>>
> >>> On 07/05/2021 09:58, Geliang Tang wrote:
> >>>> This patch fixed this fallback error:
> >>>
> >>> Thank you for the patch!
> >>>
> >>>> diff --git a/net/mptcp/options.c b/net/mptcp/options.c
> >>>> index f3e379a0f08d..940b32511443 100644
> >>>> --- a/net/mptcp/options.c
> >>>> +++ b/net/mptcp/options.c
> >>>> @@ -901,7 +901,7 @@ static bool check_fully_established(struct mptcp_sock *msk, struct sock *ssk,
> >>>> * MP_JOIN subflows.
> >>>> */
> >>>> if (!mp_opt->mp_capable) {
> >>>> - if (subflow->mp_join)
> >>>> + if (subflow->mp_join || mptcp_data_fin_enabled(msk))> goto reset;
> >>>
> >>> I don't think we should send a reset here. For me this behaviour is normal.
> >>
> >> Yes, we don't need to send a reset here. What about just return here?
> >> Something like this:
> >>
> >> diff --git a/net/mptcp/options.c b/net/mptcp/options.c
> >> index 3428c163299b..bd62dadfefdb 100644
> >> --- a/net/mptcp/options.c
> >> +++ b/net/mptcp/options.c
> >> @@ -905,6 +905,8 @@ static bool check_fully_established(struct
> >> mptcp_sock *msk, struct sock *ssk,
> >> if (!mp_opt->mp_capable) {
> >> if (subflow->mp_join)
> >> goto reset;
> >> + if (mptcp_data_fin_enabled(msk))
> >> + return false;
> >> subflow->mp_capable = 0;
> >> pr_fallback(msk);
> >> __mptcp_do_fallback(msk);
> >
> > I think we still need to do a fallback here not to expect a DATA_ACK.
> > For example in __mptcp_check_send_data_fin(), we change the state
> > because we will not get a DATA_ACK.
> >
> > So I think everything is OK there.
> >
> > Just one thing: should we send a DATA_FIN if we were not "fully
> > established" when closing the socket?
>
> Paolo shared this answer on IRC:
>
> I would say yes. The server side could have already allocated the msk
> socket (and the 4th ack could have been lost)
>
> Which makes sense.
>
> So I think we should not do anything here. Except maybe a comment here
> in the code? e.g.
>
I think it's a little strange to include only a comment in a patch. Let's
just drop this patch (mptcp: don't fallback when data_fin enabled), and
close the ticket #176.
Thanks.
-Geliang
>
> diff --git a/net/mptcp/options.c b/net/mptcp/options.c
> index 3428c163299b..43d55039743a 100644
> --- a/net/mptcp/options.c
> +++ b/net/mptcp/options.c
> @@ -905,6 +905,14 @@ static bool check_fully_established(struct
> mptcp_sock *msk, struct sock *ssk,
> if (!mp_opt->mp_capable) {
> if (subflow->mp_join)
> goto reset;
> +
> + /* We also want to fallback if the connection is closed
> while we
> + * were not in fully established mode.
> + * But we still want to send the DATA_FIN if the
> userspace has
> + * closed the connection at this stage, just in case the
> server
> + * side has already allocated the msk socket or if the
> 4th ACK
> + * has been lost.
> + */
> subflow->mp_capable = 0;
> pr_fallback(msk);
> __mptcp_do_fallback(msk);
>
>
> Cheers,
> Matt
>
> --
> Tessares | Belgium | Hybrid Access Solutions
> www.tessares.net
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-05-08 2:58 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-07 7:58 [MPTCP][PATCH mptcp-next] mptcp: don't fallback when data_fin enabled Geliang Tang
2021-05-07 8:51 ` Matthieu Baerts
2021-05-07 9:08 ` Geliang Tang
2021-05-07 12:19 ` Matthieu Baerts
2021-05-07 15:23 ` Matthieu Baerts
2021-05-08 2:58 ` Geliang Tang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).