MPTCP checksum interop

MPTCP checksum interop

[Mat Martineau]

I did some tests with connections with checksums enabled, between the 
export branch and the multipath-tcp.org mptcp_trunk branch (v1 mode).

When the multipath-tcp.org kernel was listening, the connection would 
always fall back to TCP when the first data was sent by the upstream 
kernel. The multipath-tcp.org kernel was the first to fall back and stop 
sending MPTCP headers.

When the upstream kernel was listening, the multipath-tcp.org kernel would 
send corrupt TCP options with the first data packet (the first time a DSS 
option was sent). The upstream kernel would then send TCP RST.

The initial impression is that there are some issues with mptcp_trunk and 
checksums - I did not have time today to look at packet captures between 
two mptcp_trunk kernels.

These results were the same when only one side of the connection had 
checksums enabled, or if both kernels did.

I haven't narrowed down the cause, but will do some more experiments on 
Monday.

--
Mat Martineau
Intel

Re: MPTCP checksum interop

[Paolo Abeni]

Hello

On Fri, 2021-06-11 at 20:57 -0700, Mat Martineau wrote:
> I did some tests with connections with checksums enabled, between the 
> export branch and the multipath-tcp.org mptcp_trunk branch (v1 mode).

Thank you for checking!

> When the multipath-tcp.org kernel was listening, the connection would 
> always fall back to TCP when the first data was sent by the upstream 
> kernel. The multipath-tcp.org kernel was the first to fall back and stop 
> sending MPTCP headers.

This sounds like a csum error detected ??! What does wireshark say
about the MPTCP csum value?

> When the upstream kernel was listening, the multipath-tcp.org kernel would 
> send corrupt TCP options with the first data packet (the first time a DSS 
> option was sent). The upstream kernel would then send TCP RST.

How does TCP option look like? Could you please share a short pcap
snippet?

Thanks!

Paolo

Re: MPTCP checksum interop

[Mat Martineau]

On Mon, 14 Jun 2021, Paolo Abeni wrote:

> Hello
>
> On Fri, 2021-06-11 at 20:57 -0700, Mat Martineau wrote:
>> I did some tests with connections with checksums enabled, between the
>> export branch and the multipath-tcp.org mptcp_trunk branch (v1 mode).
>
> Thank you for checking!
>
>> When the multipath-tcp.org kernel was listening, the connection would
>> always fall back to TCP when the first data was sent by the upstream
>> kernel. The multipath-tcp.org kernel was the first to fall back and stop
>> sending MPTCP headers.
>
> This sounds like a csum error detected ??! What does wireshark say
> about the MPTCP csum value?

Wireshark doesn't complain about the checksum, and the MPTCP option parses 
cleanly.

Fallback appears to have more to do with mptcp_trunk's handling for 
MP_CAPABLE + data. I turned on debug output on the mptcp_trunk side and 
saw:

mptcp_prevalidate_skb: mptcp_prevalidate_skb 0xc20cfa4 will fallback - pi 1 from tcp_data_queue+0x444/0x600, seq 2884338481 mptcp-flags 0x0

This might be the problem in mptcp.h:

#define MPTCPV1_SUB_LEN_CAPABLE_DATA		22
#define MPTCPV1_SUB_LEN_CAPABLE_DATA_CSUM	22   // <---- should be 24


>
>> When the upstream kernel was listening, the multipath-tcp.org kernel would
>> send corrupt TCP options with the first data packet (the first time a DSS
>> option was sent). The upstream kernel would then send TCP RST.
>
> How does TCP option look like? Could you please share a short pcap
> snippet?

The MPTCP option is not of the correct length to contain a checksum, and 
the bytes after the MPTCP option are getting interpreted by Wireshark as a 
separate (broken) option. That broken option looks like random bytes: 
invalid option number and way-too-large option length. The 
multipath-tcp.org kernel is also sending MP_CAPABLE + data here. The above 
definitions look suspicous for this too.

I'll add a github issue and upload the .pcap

--
Mat Martineau
Intel