[RFC PATCH mptcp-next v13 0/7] mptcp: Fast Open Mechanism

[RFC PATCH mptcp-next v13 0/7] mptcp: Fast Open Mechanism

[Dmytro Shytyi]

These patches focus on the Initiator and partially on Listener side.
The next options in userspace are available:
a) sendto(..., ..., ..., MSG_FASTOPEN, ..., ...);
b) setsockopt(..., SOL_TCP, TCP_FASTOPEN, ..., ...);

These patches implement Appendix-B of RFC8684 (MPTFO).

We would like to credit Paulo Abeni, Mat Martineau, Matthieu Baerts and
Benjamin Hesmans for advices and ideas that improved these patches.

Dmytro Shytyi (7):
  mptcp: add __mptcp_pre_connect() helper
  mptcp: add __mptcp_stream_connect() helper
  mptcp: introduce MSG_FASTOPEN flag.
  mptcp: implement delayed seq generation for passive fastopen
  mptcp: add subflow_v(4,6)_send_synack()
  mptcp: add TCP_FASTOPEN sock option
  selftests: mptfo initiator/listener

 net/mptcp/Makefile                            |   2 +-
 net/mptcp/fastopen.c                          |  96 +++++++++++++++++
 net/mptcp/options.c                           |   5 +
 net/mptcp/protocol.c                          |  64 +++++++----
 net/mptcp/protocol.h                          |  13 +++
 net/mptcp/sockopt.c                           |   5 +-
 net/mptcp/subflow.c                           |  42 ++++++++
 tools/testing/selftests/net/mptcp/mptfo.sh    |  13 +++
 .../selftests/net/mptcp/mptfo_initiator.c     |  43 ++++++++
 .../selftests/net/mptcp/mptfo_listener.c      | 100 ++++++++++++++++++
 10 files changed, 360 insertions(+), 23 deletions(-)
 create mode 100644 net/mptcp/fastopen.c
 create mode 100644 tools/testing/selftests/net/mptcp/mptfo.sh
 create mode 100644 tools/testing/selftests/net/mptcp/mptfo_initiator.c
 create mode 100644 tools/testing/selftests/net/mptcp/mptfo_listener.c

-- 
2.34.1

[RFC PATCH mptcp-next v13 1/7] mptcp: add __mptcp_pre_connect() helper

[Dmytro Shytyi]

Set the fastopen_req inside the __mptcp_pre_connect() helper
and call it in the mptcp_sendmsg(). Used together with the
next patch.

Signed-off-by: Dmytro Shytyi <dmytro@shytyi.net>
---
 net/mptcp/Makefile   |  2 +-
 net/mptcp/fastopen.c | 27 +++++++++++++++++++++++++++
 net/mptcp/protocol.c |  2 ++
 net/mptcp/protocol.h |  5 +++++
 4 files changed, 35 insertions(+), 1 deletion(-)
 create mode 100644 net/mptcp/fastopen.c

diff --git a/net/mptcp/Makefile b/net/mptcp/Makefile
index 8a7f68efa35f..c42ad8609876 100644
--- a/net/mptcp/Makefile
+++ b/net/mptcp/Makefile
@@ -2,7 +2,7 @@
 obj-$(CONFIG_MPTCP) += mptcp.o
 
 mptcp-y := protocol.o subflow.o options.o token.o crypto.o ctrl.o pm.o diag.o \
-	   mib.o pm_netlink.o sockopt.o pm_userspace.o sched.o
+	   mib.o pm_netlink.o sockopt.o pm_userspace.o sched.o fastopen.o
 
 obj-$(CONFIG_SYN_COOKIES) += syncookies.o
 obj-$(CONFIG_INET_MPTCP_DIAG) += mptcp_diag.o
diff --git a/net/mptcp/fastopen.c b/net/mptcp/fastopen.c
new file mode 100644
index 000000000000..65a6a87c0079
--- /dev/null
+++ b/net/mptcp/fastopen.c
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0
+ * MPTCP Fast Open Mechanism. Copyright (c) 2021-2022, Dmytro SHYTYI
+ */
+
+#include "protocol.h"
+
+void __mptcp_pre_connect(struct mptcp_sock *msk, struct sock *ssk,
+			 struct msghdr *msg, size_t size)
+{
+	struct tcp_sock *tp;
+	struct sk_buff *skb;
+	struct ubuf_info *uarg;
+
+	lock_sock(ssk);
+
+	tp = tcp_sk(ssk);
+
+	skb = tcp_write_queue_tail(ssk);
+	uarg = msg_zerocopy_realloc(ssk, size, skb_zcopy(skb));
+	tp->fastopen_req = kzalloc(sizeof(*tp->fastopen_req),
+				   ssk->sk_allocation);
+	tp->fastopen_req->data = msg;
+	tp->fastopen_req->size = size;
+	tp->fastopen_req->uarg = uarg;
+
+	release_sock(ssk);
+}
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 8feb684408f7..b080237d9d7e 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -1688,6 +1688,8 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		int copied_syn = 0;
 
 		lock_sock(ssk);
+		if (msg->msg_flags & MSG_FASTOPEN && sk->sk_state == TCP_CLOSE)
+			__mptcp_pre_connect(msk, ssk, msg, len);
 
 		ret = tcp_sendmsg_fastopen(ssk, msg, &copied_syn, len, NULL);
 		copied += copied_syn;
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index 93c535440a5c..30c03c70fdeb 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -838,6 +838,11 @@ void mptcp_event_addr_announced(const struct sock *ssk, const struct mptcp_addr_
 void mptcp_event_addr_removed(const struct mptcp_sock *msk, u8 id);
 bool mptcp_userspace_pm_active(const struct mptcp_sock *msk);
 
+// Fast Open Mechanism functions begin
+void __mptcp_pre_connect(struct mptcp_sock *msk, struct sock *ssk,
+			 struct msghdr *msg, size_t len);
+// Fast Open Mechanism functions end
+
 static inline bool mptcp_pm_should_add_signal(struct mptcp_sock *msk)
 {
 	return READ_ONCE(msk->pm.addr_signal) &
-- 
2.34.1

[RFC PATCH mptcp-next v13 2/7] mptcp: add __mptcp_stream_connect() helper

[Dmytro Shytyi]

This helper is exctract from mptcp_stream_connect() without locks.
Used in the mptcp_sendmsg() in conjunction with __mptcp_pre_connect().

Signed-off-by: Dmytro Shytyi <dmytro@shytyi.net>
---
 net/mptcp/protocol.c | 53 +++++++++++++++++++++++++++++++-------------
 net/mptcp/protocol.h |  2 ++
 2 files changed, 40 insertions(+), 15 deletions(-)

diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index b080237d9d7e..f499e3e5a06b 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -1687,22 +1687,35 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		struct sock *ssk = ssock->sk;
 		int copied_syn = 0;
 
-		lock_sock(ssk);
-		if (msg->msg_flags & MSG_FASTOPEN && sk->sk_state == TCP_CLOSE)
+		if (msg->msg_flags & MSG_FASTOPEN  && sk->sk_state == TCP_CLOSE) {
 			__mptcp_pre_connect(msk, ssk, msg, len);
+			/* we take nmpc_socket (first subflow) inside of __mptcp_stream_connect() */
+			/* avoid lock_sock(ssk) as it seems leading to deadlock */
+			ret = __mptcp_stream_connect(sk->sk_socket, msg->msg_name,
+						     msg->msg_namelen, msg->msg_flags);
+
+			if (ret >= 0)
+				copied += len;
+			else
+				goto do_error;
+		} else {
 
-		ret = tcp_sendmsg_fastopen(ssk, msg, &copied_syn, len, NULL);
-		copied += copied_syn;
-		if (ret == -EINPROGRESS && copied_syn > 0) {
-			/* reflect the new state on the MPTCP socket */
-			inet_sk_state_store(sk, inet_sk_state_load(ssk));
-			release_sock(ssk);
-			goto out;
-		} else if (ret) {
+			lock_sock(ssk);
+
+			ret = tcp_sendmsg_fastopen(ssk, msg, &copied_syn, len, NULL);
+
+			copied += copied_syn;
+			if (ret == -EINPROGRESS && copied_syn > 0) {
+				/* reflect the new state on the MPTCP socket */
+				inet_sk_state_store(sk, inet_sk_state_load(ssk));
+				release_sock(ssk);
+				goto out;
+			} else if (ret) {
+				release_sock(ssk);
+				goto do_error;
+			}
 			release_sock(ssk);
-			goto do_error;
 		}
-		release_sock(ssk);
 	}
 
 	timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT);
@@ -3570,15 +3583,14 @@ static void mptcp_subflow_early_fallback(struct mptcp_sock *msk,
 	__mptcp_do_fallback(msk);
 }
 
-static int mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,
-				int addr_len, int flags)
+int __mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,
+			   int addr_len, int flags)
 {
 	struct mptcp_sock *msk = mptcp_sk(sock->sk);
 	struct mptcp_subflow_context *subflow;
 	struct socket *ssock;
 	int err = -EINVAL;
 
-	lock_sock(sock->sk);
 	if (uaddr) {
 		if (addr_len < sizeof(uaddr->sa_family))
 			goto unlock;
@@ -3633,7 +3645,18 @@ static int mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,
 		inet_sk_state_store(sock->sk, inet_sk_state_load(ssock->sk));
 
 unlock:
+	return err;
+}
+
+static int mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,
+				int addr_len, int flags)
+{
+	int err = -EINVAL;
+
+	lock_sock(sock->sk);
+	err = __mptcp_stream_connect(sock, uaddr, addr_len, flags);
 	release_sock(sock->sk);
+
 	return err;
 }
 
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index 30c03c70fdeb..e596f45763fd 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -837,6 +837,8 @@ void mptcp_event(enum mptcp_event_type type, const struct mptcp_sock *msk,
 void mptcp_event_addr_announced(const struct sock *ssk, const struct mptcp_addr_info *info);
 void mptcp_event_addr_removed(const struct mptcp_sock *msk, u8 id);
 bool mptcp_userspace_pm_active(const struct mptcp_sock *msk);
+int __mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,
+			   int addr_len, int flags);
 
 // Fast Open Mechanism functions begin
 void __mptcp_pre_connect(struct mptcp_sock *msk, struct sock *ssk,
-- 
2.34.1

[RFC PATCH mptcp-next v13 3/7] mptcp: introduce MSG_FASTOPEN flag.

[Dmytro Shytyi]

In the following patches we will analyse the MSG_FASTOPEN flag
in the mptcp_sendmsg() and invoke the MPTFO.

Signed-of-by: Benjamin Hesmans <benjamin.hesmans@tessares.net>
Signed-off-by: Dmytro Shytyi <dmytro@shytyi.net>
---
 net/mptcp/protocol.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index f499e3e5a06b..0d8602ddf146 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -1673,17 +1673,14 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 	int ret = 0;
 	long timeo;
 
-	/* we don't support FASTOPEN yet */
-	if (msg->msg_flags & MSG_FASTOPEN)
-		return -EOPNOTSUPP;
-
 	/* silently ignore everything else */
-	msg->msg_flags &= MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL;
+	msg->msg_flags &= MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_FASTOPEN;
 
 	lock_sock(sk);
 
 	ssock = __mptcp_nmpc_socket(msk);
-	if (unlikely(ssock && inet_sk(ssock->sk)->defer_connect)) {
+	if (unlikely(ssock && (inet_sk(ssock->sk)->defer_connect ||
+			       msg->msg_flags & MSG_FASTOPEN))) {
 		struct sock *ssk = ssock->sk;
 		int copied_syn = 0;
 
-- 
2.34.1

[RFC PATCH mptcp-next v13 4/7] mptcp: implement delayed seq generation for passive fastopen

[Dmytro Shytyi]

With fastopen in place the first subflow socket is created
before the MPC handshake completes, and we need to properly initialize
the sequence numbers at MPC ACK reception.

Signed-off-by: Dmytro Shytyi <dmytro@shytyi.net>
---
 net/mptcp/fastopen.c | 14 ++++++++++++++
 net/mptcp/options.c  |  5 +++++
 net/mptcp/protocol.h |  3 +++
 3 files changed, 22 insertions(+)

diff --git a/net/mptcp/fastopen.c b/net/mptcp/fastopen.c
index 65a6a87c0079..dca736c61815 100644
--- a/net/mptcp/fastopen.c
+++ b/net/mptcp/fastopen.c
@@ -25,3 +25,17 @@ void __mptcp_pre_connect(struct mptcp_sock *msk, struct sock *ssk,
 
 	release_sock(ssk);
 }
+
+void mptcp_gen_msk_ackseq_fastopen(struct mptcp_sock *msk, struct mptcp_subflow_context *subflow,
+				   struct mptcp_options_received mp_opt)
+{
+	u64 ack_seq;
+
+	WRITE_ONCE(msk->can_ack, true);
+	WRITE_ONCE(msk->remote_key, mp_opt.sndr_key);
+	mptcp_crypto_key_sha(msk->remote_key, NULL, &ack_seq);
+	ack_seq++;
+	WRITE_ONCE(msk->ack_seq, ack_seq);
+	pr_debug("ack_seq=%llu sndr_key=%llu", msk->ack_seq, mp_opt.sndr_key);
+	atomic64_set(&msk->rcv_wnd_sent, ack_seq);
+}
diff --git a/net/mptcp/options.c b/net/mptcp/options.c
index 30d289044e71..0b6c4535750c 100644
--- a/net/mptcp/options.c
+++ b/net/mptcp/options.c
@@ -1208,6 +1208,11 @@ bool mptcp_incoming_options(struct sock *sk, struct sk_buff *skb)
 			mpext->dsn64 = 1;
 			mpext->mpc_map = 1;
 			mpext->data_fin = 0;
+
+			if (msk->is_mptfo) {
+				mptcp_gen_msk_ackseq_fastopen(msk, subflow, mp_opt);
+				mpext->data_seq = READ_ONCE(msk->ack_seq);
+			}
 		} else {
 			mpext->data_seq = mp_opt.data_seq;
 			mpext->subflow_seq = mp_opt.subflow_seq;
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index e596f45763fd..c356c993597c 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -282,6 +282,7 @@ struct mptcp_sock {
 	bool		use_64bit_ack; /* Set when we received a 64-bit DSN */
 	bool		csum_enabled;
 	bool		allow_infinite_fallback;
+	bool		is_mptfo;
 	u8		mpc_endpoint_id;
 	u8		recvmsg_inq:1,
 			cork:1,
@@ -843,6 +844,8 @@ int __mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,
 // Fast Open Mechanism functions begin
 void __mptcp_pre_connect(struct mptcp_sock *msk, struct sock *ssk,
 			 struct msghdr *msg, size_t len);
+void mptcp_gen_msk_ackseq_fastopen(struct mptcp_sock *msk, struct mptcp_subflow_context *subflow,
+				   struct mptcp_options_received mp_opt);
 // Fast Open Mechanism functions end
 
 static inline bool mptcp_pm_should_add_signal(struct mptcp_sock *msk)
-- 
2.34.1

[RFC PATCH mptcp-next v13 5/7] mptcp: add subflow_v(4,6)_send_synack()

[Dmytro Shytyi]

In this patch we add skb to the msk, dequeue it from sk, remove TSs and
do skb mapping.

Signed-off-by: Dmytro Shytyi <dmytro@shytyi.net>
---
 net/mptcp/fastopen.c | 55 ++++++++++++++++++++++++++++++++++++++++++++
 net/mptcp/protocol.c |  2 +-
 net/mptcp/protocol.h |  3 +++
 net/mptcp/subflow.c  | 42 +++++++++++++++++++++++++++++++++
 4 files changed, 101 insertions(+), 1 deletion(-)

diff --git a/net/mptcp/fastopen.c b/net/mptcp/fastopen.c
index dca736c61815..0193ed455a73 100644
--- a/net/mptcp/fastopen.c
+++ b/net/mptcp/fastopen.c
@@ -4,6 +4,61 @@
 
 #include "protocol.h"
 
+struct mptcp_skb_cb {
+	u64 map_seq;
+	u64 end_seq;
+	u32 offset;
+	u8  has_rxtstamp:1;
+};
+
+#define MPTCP_SKB_CB(__skb)	((struct mptcp_skb_cb *)&((__skb)->cb[0]))
+
+void subflow_fastopen_send_synack_set_params(struct mptcp_subflow_context *subflow,
+					     struct request_sock *treq)
+{
+	struct tcp_request_sock *tcp_r_sock = tcp_rsk(treq);
+	struct sock *ssk = subflow->tcp_sock;
+	struct sock *sk = subflow->conn;
+	struct mptcp_sock *msk;
+	struct sk_buff *skb;
+	struct tcp_sock *tp;
+	u32 offset;
+
+	msk = mptcp_sk(sk);
+	tp = tcp_sk(ssk);
+
+	/* mark subflow/msk as "mptfo" */
+	msk->is_mptfo = 1;
+
+	skb = skb_peek(&ssk->sk_receive_queue);
+
+	/* dequeue the skb from sk receive queue */
+	__skb_unlink(skb, &ssk->sk_receive_queue);
+	skb_ext_reset(skb);
+	skb_orphan(skb);
+
+	/* set the skb mapping */
+	tp->copied_seq += tp->rcv_nxt - tcp_r_sock->rcv_isn - 1;
+	subflow->map_seq = mptcp_subflow_get_mapped_dsn(subflow);
+	subflow->ssn_offset = tp->copied_seq - 1;
+
+	/* innitialize MPTCP_CB */
+	offset = tp->copied_seq - TCP_SKB_CB(skb)->seq;
+	MPTCP_SKB_CB(skb)->map_seq = mptcp_subflow_get_mapped_dsn(subflow);
+	MPTCP_SKB_CB(skb)->end_seq = msk->ack_seq;
+	MPTCP_SKB_CB(skb)->offset = offset;
+	MPTCP_SKB_CB(skb)->has_rxtstamp = TCP_SKB_CB(skb)->has_rxtstamp;
+
+	mptcp_data_lock(sk);
+
+	mptcp_set_owner_r(skb, sk);
+	__skb_queue_tail(&msk->receive_queue, skb);
+
+	(sk)->sk_data_ready(sk);
+
+	mptcp_data_unlock(sk);
+}
+
 void __mptcp_pre_connect(struct mptcp_sock *msk, struct sock *ssk,
 			 struct msghdr *msg, size_t size)
 {
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 0d8602ddf146..7f6de5028778 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -200,7 +200,7 @@ static void mptcp_rfree(struct sk_buff *skb)
 	mptcp_rmem_uncharge(sk, len);
 }
 
-static void mptcp_set_owner_r(struct sk_buff *skb, struct sock *sk)
+void mptcp_set_owner_r(struct sk_buff *skb, struct sock *sk)
 {
 	skb_orphan(skb);
 	skb->sk = sk;
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index c356c993597c..4d550241dbb2 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -846,6 +846,9 @@ void __mptcp_pre_connect(struct mptcp_sock *msk, struct sock *ssk,
 			 struct msghdr *msg, size_t len);
 void mptcp_gen_msk_ackseq_fastopen(struct mptcp_sock *msk, struct mptcp_subflow_context *subflow,
 				   struct mptcp_options_received mp_opt);
+void mptcp_set_owner_r(struct sk_buff *skb, struct sock *sk);
+void subflow_fastopen_send_synack_set_params(struct mptcp_subflow_context *subflow,
+					     struct request_sock *req);
 // Fast Open Mechanism functions end
 
 static inline bool mptcp_pm_should_add_signal(struct mptcp_sock *msk)
diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c
index 07dd23d0fe04..3c1f971f1d9e 100644
--- a/net/mptcp/subflow.c
+++ b/net/mptcp/subflow.c
@@ -307,6 +307,46 @@ static struct dst_entry *subflow_v4_route_req(const struct sock *sk,
 	return NULL;
 }
 
+static int subflow_v4_send_synack(const struct sock *sk, struct dst_entry *dst,
+				  struct flowi *fl,
+				  struct request_sock *req,
+				  struct tcp_fastopen_cookie *foc,
+				  enum tcp_synack_type synack_type,
+				  struct sk_buff *syn_skb)
+{
+	struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
+	struct inet_request_sock *ireq = inet_rsk(req);
+
+	/* clear tstamp_ok, as needed depending on cookie */
+	if (foc && foc->len > -1)
+		ireq->tstamp_ok = 0;
+
+	if (synack_type == TCP_SYNACK_FASTOPEN)
+		subflow_fastopen_send_synack_set_params(subflow, req);
+
+	return tcp_request_sock_ipv4_ops.send_synack(sk, dst, fl, req, foc, synack_type, syn_skb);
+}
+
+static int subflow_v6_send_synack(const struct sock *sk, struct dst_entry *dst,
+				  struct flowi *fl,
+				  struct request_sock *req,
+				  struct tcp_fastopen_cookie *foc,
+				  enum tcp_synack_type synack_type,
+				  struct sk_buff *syn_skb)
+{
+	struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
+	struct inet_request_sock *ireq = inet_rsk(req);
+
+	/* clear tstamp_ok, as needed depending on cookie */
+	if (foc && foc->len > -1)
+		ireq->tstamp_ok = 0;
+
+	if (synack_type == TCP_SYNACK_FASTOPEN)
+		subflow_fastopen_send_synack_set_params(subflow, req);
+
+	return tcp_request_sock_ipv6_ops.send_synack(sk, dst, fl, req, foc, synack_type, syn_skb);
+}
+
 #if IS_ENABLED(CONFIG_MPTCP_IPV6)
 static struct dst_entry *subflow_v6_route_req(const struct sock *sk,
 					      struct sk_buff *skb,
@@ -1920,6 +1960,7 @@ void __init mptcp_subflow_init(void)
 
 	subflow_request_sock_ipv4_ops = tcp_request_sock_ipv4_ops;
 	subflow_request_sock_ipv4_ops.route_req = subflow_v4_route_req;
+	subflow_request_sock_ipv4_ops.send_synack = subflow_v4_send_synack;
 
 	subflow_specific = ipv4_specific;
 	subflow_specific.conn_request = subflow_v4_conn_request;
@@ -1933,6 +1974,7 @@ void __init mptcp_subflow_init(void)
 #if IS_ENABLED(CONFIG_MPTCP_IPV6)
 	subflow_request_sock_ipv6_ops = tcp_request_sock_ipv6_ops;
 	subflow_request_sock_ipv6_ops.route_req = subflow_v6_route_req;
+	subflow_request_sock_ipv6_ops.send_synack = subflow_v6_send_synack;
 
 	subflow_v6_specific = ipv6_specific;
 	subflow_v6_specific.conn_request = subflow_v6_conn_request;
-- 
2.34.1

[RFC PATCH mptcp-next v13 6/7] mptcp: add TCP_FASTOPEN sock option

[Dmytro Shytyi]

We add the TCP_FASTOPEN socket option in this patch that is going to
 be set by the listener side.

Signed-off-by: Dmytro Shytyi <dmytro@shytyi.net>
---
 net/mptcp/sockopt.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/net/mptcp/sockopt.c b/net/mptcp/sockopt.c
index f85e9bbfe86f..7f6977295256 100644
--- a/net/mptcp/sockopt.c
+++ b/net/mptcp/sockopt.c
@@ -559,6 +559,7 @@ static bool mptcp_supported_sockopt(int level, int optname)
 		case TCP_NOTSENT_LOWAT:
 		case TCP_TX_DELAY:
 		case TCP_INQ:
+		case TCP_FASTOPEN:
 		case TCP_FASTOPEN_CONNECT:
 		case TCP_FASTOPEN_NO_COOKIE:
 			return true;
@@ -569,7 +570,7 @@ static bool mptcp_supported_sockopt(int level, int optname)
 		/* TCP_REPAIR, TCP_REPAIR_QUEUE, TCP_QUEUE_SEQ, TCP_REPAIR_OPTIONS,
 		 * TCP_REPAIR_WINDOW are not supported, better avoid this mess
 		 */
-		/* TCP_FASTOPEN_KEY, TCP_FASTOPEN are not supported because
+		/* TCP_FASTOPEN_KEY is not supported because
 		 * fastopen for the listener side is currently unsupported
 		 */
 	}
@@ -801,6 +802,7 @@ static int mptcp_setsockopt_sol_tcp(struct mptcp_sock *msk, int optname,
 		/* See tcp.c: TCP_DEFER_ACCEPT does not fail */
 		mptcp_setsockopt_first_sf_only(msk, SOL_TCP, optname, optval, optlen);
 		return 0;
+	case TCP_FASTOPEN:
 	case TCP_FASTOPEN_CONNECT:
 	case TCP_FASTOPEN_NO_COOKIE:
 		return mptcp_setsockopt_first_sf_only(msk, SOL_TCP, optname,
@@ -1166,6 +1168,7 @@ static int mptcp_getsockopt_sol_tcp(struct mptcp_sock *msk, int optname,
 	case TCP_INFO:
 	case TCP_CC_INFO:
 	case TCP_DEFER_ACCEPT:
+	case TCP_FASTOPEN:
 	case TCP_FASTOPEN_CONNECT:
 	case TCP_FASTOPEN_NO_COOKIE:
 		return mptcp_getsockopt_first_sf_only(msk, SOL_TCP, optname,
-- 
2.34.1

[RFC PATCH mptcp-next v13 7/7] selftests: mptfo initiator/listener

[Dmytro Shytyi]

MPTFO tests: these are examples of initiator (sendto) and listener,
probably are going to be integrated to the mptcp_connect.* selftests

Signed-off-by: Dmytro Shytyi <dmytro@shytyi.net>
---
 tools/testing/selftests/net/mptcp/mptfo.sh    |  13 +++
 .../selftests/net/mptcp/mptfo_initiator.c     |  43 ++++++++
 .../selftests/net/mptcp/mptfo_listener.c      | 100 ++++++++++++++++++
 3 files changed, 156 insertions(+)
 create mode 100644 tools/testing/selftests/net/mptcp/mptfo.sh
 create mode 100644 tools/testing/selftests/net/mptcp/mptfo_initiator.c
 create mode 100644 tools/testing/selftests/net/mptcp/mptfo_listener.c

diff --git a/tools/testing/selftests/net/mptcp/mptfo.sh b/tools/testing/selftests/net/mptcp/mptfo.sh
new file mode 100644
index 000000000000..9ed0cb281094
--- /dev/null
+++ b/tools/testing/selftests/net/mptcp/mptfo.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+#This is an example of environmen that was used to generate wireshark
+sudo ip netns add server
+sudo ip netns add client
+sudo ip link add veth0 type veth peer name veth1
+sudo ip link set veth1 netns server
+sudo ip link set veth0 netns client
+sudo ip netns exec client ip a a 10.10.0.1/24 dev veth0
+sudo ip netns exec server ip a a 10.10.0.2/24 dev veth1
+sudo ip netns exec client ip link set dev  veth0 up
+sudo ip netns exec server ip link set dev  veth1 up
+sudo ip netns exec server bash -c "echo 2 > /proc/sys/net/ipv4/tcp_fastopen"
+sudo ip netns exec client bash -c "echo 1 > /proc/sys/net/ipv4/tcp_fastopen"
diff --git a/tools/testing/selftests/net/mptcp/mptfo_initiator.c b/tools/testing/selftests/net/mptcp/mptfo_initiator.c
new file mode 100644
index 000000000000..05e6a3d62bb8
--- /dev/null
+++ b/tools/testing/selftests/net/mptcp/mptfo_initiator.c
@@ -0,0 +1,43 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <arpa/inet.h>
+#include <netinet/in.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/socket.h>
+#include <unistd.h>
+#include <netinet/tcp.h>
+#include <string.h>
+#include <signal.h>
+
+#define SERVER_PORT 7003
+
+int main(int argc, char *argv[])
+{
+	unsigned char valsyn[3] = "abc";
+	struct sockaddr_in daddr;
+	char *valend = "fff";
+	char *val1 = "zz1";
+	char *val2 = "zz2";
+	char *val3 = "zz3";
+	int sock_fd = -1;
+	int ret;
+
+	memset(&daddr, 0, sizeof(daddr));
+	inet_pton(AF_INET, "10.10.0.2", &daddr.sin_addr);
+	daddr.sin_family = AF_INET;
+	daddr.sin_port = htons(SERVER_PORT);
+
+	sock_fd = socket(AF_INET, SOCK_STREAM, IPPROTO_MPTCP);
+
+	ret = sendto(sock_fd, valsyn, 3, MSG_FASTOPEN, (struct sockaddr *) &daddr, sizeof(daddr));
+	ret = write(sock_fd, val1, 3);
+	ret = write(sock_fd, val2, 3);
+	ret = write(sock_fd, val2, 3);
+	ret = write(sock_fd, val2, 3);
+	ret = write(sock_fd, val3, 3);
+	ret = write(sock_fd, valend, 3);
+
+	close(sock_fd);
+	return EXIT_SUCCESS;
+}
diff --git a/tools/testing/selftests/net/mptcp/mptfo_listener.c b/tools/testing/selftests/net/mptcp/mptfo_listener.c
new file mode 100644
index 000000000000..6890349b14bb
--- /dev/null
+++ b/tools/testing/selftests/net/mptcp/mptfo_listener.c
@@ -0,0 +1,100 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <arpa/inet.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/socket.h>
+#include <sys/types.h>
+#include <unistd.h>
+#include <netinet/in.h>
+#include <linux/in.h>
+#include <netinet/tcp.h>
+
+#define CLIENT_QUEUE_LEN 10
+#define SERVER_PORT 7003
+
+int main(void)
+{
+	int listen_sock_fd = -1, client_sock_fd = -1;
+	char str_addr[INET6_ADDRSTRLEN];
+	struct sockaddr_in server_addr;
+	int ret, flag;
+	int qlen = 5;
+	char ch;
+
+	server_addr.sin_family = AF_INET;
+	inet_pton(AF_INET, "10.10.0.2", &server_addr.sin_addr);
+	server_addr.sin_port = htons(SERVER_PORT);
+
+	/* Create socket for listening (client requests) */
+	listen_sock_fd = socket(AF_INET, SOCK_STREAM, IPPROTO_MPTCP);
+	if (listen_sock_fd == -1) {
+		perror("socket()server");
+		return EXIT_FAILURE;
+	}
+
+	/* Set socket to reuse address */
+	flag = 1;
+	ret = setsockopt(listen_sock_fd, SOL_SOCKET, SO_REUSEADDR, &flag, sizeof(flag));
+	if (ret == -1) {
+		perror("setsockopt()");
+		return EXIT_FAILURE;
+	}
+
+	ret = setsockopt(listen_sock_fd, SOL_TCP, TCP_FASTOPEN, &qlen, sizeof(qlen));
+	if (ret == -1) {
+		perror("setsockopt()TCP_FASTOPEN");
+		return EXIT_FAILURE;
+	}
+
+	/* Bind address and socket together */
+	ret = bind(listen_sock_fd, (struct sockaddr *)&server_addr, sizeof(server_addr));
+	if (ret == -1) {
+		perror("bind()");
+		close(listen_sock_fd);
+		return EXIT_FAILURE;
+	}
+
+	/* Create listening queue (client requests) */
+	ret = listen(listen_sock_fd, CLIENT_QUEUE_LEN);
+	if (ret == -1) {
+		perror("listen()");
+		close(listen_sock_fd);
+		return EXIT_FAILURE;
+	}
+	perror("Server listening");
+	while (1) {
+		/* Do TCP handshake with client */
+		client_sock_fd = accept(listen_sock_fd,
+				NULL,
+				0);
+		if (client_sock_fd == -1) {
+			perror("accept()");
+			close(listen_sock_fd);
+			return EXIT_FAILURE;
+		} else {
+			perror("ACCEPT_SUCCESS");
+		}
+
+		char rb[1024];
+
+		while (1) {
+			ret = read(client_sock_fd, rb, 3);
+
+			if (ret == -1) {
+				perror("SERVVERread()");
+				close(client_sock_fd);
+				break;
+			} else {
+				fprintf(stderr, "received %c%c%c from client", rb[0], rb[1], rb[2]);
+			}
+			if (rb[0] == 'f' && rb[1] == 'f' && rb[2] == 'f') {
+				close(client_sock_fd);
+				break;
+			}
+
+		}
+	}
+
+	return EXIT_SUCCESS;
+}
-- 
2.34.1

Re: selftests: mptfo initiator/listener: Build Failure

[MPTCP CI]

Hi Dmytro,

Thank you for your modifications, that's great!

But sadly, our CI spotted some issues with it when trying to build it.

You can find more details there:

  https://patchwork.kernel.org/project/mptcp/patch/20221001031502.29152-8-dmytro@shytyi.net/
  https://github.com/multipath-tcp/mptcp_net-next/actions/runs/3197013753

Status: failure
Initiator: MPTCPimporter
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/7c22bca6e0ab

Feel free to reply to this email if you cannot access logs, if you need
some support to fix the error, if this doesn't seem to be caused by your
modifications or if the error is a false positive one.

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: selftests: mptfo initiator/listener: Tests Results

[MPTCP CI]

Hi Dmytro,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal:
  - Unstable: 2 failed test(s): selftest_mptcp_join selftest_mptfo 🔴:
  - Task: https://cirrus-ci.com/task/6306020468195328
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/6306020468195328/summary/summary.txt

- KVM Validation: debug:
  - Unstable: 2 failed test(s): packetdrill_add_addr selftest_mptfo 🔴:
  - Task: https://cirrus-ci.com/task/4898645584642048
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/4898645584642048/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/7c22bca6e0ab


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: selftests: mptfo initiator/listener: Build Failure

[MPTCP CI]

Hi Dmytro,

Thank you for your modifications, that's great!

But sadly, our CI spotted some issues with it when trying to build it.

You can find more details there:

  https://patchwork.kernel.org/project/mptcp/patch/20220927225341.14165-8-dmytro@shytyi.net/
  https://github.com/multipath-tcp/mptcp_net-next/actions/runs/3139594634

Status: failure
Initiator: MPTCPimporter
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/4285eae5e20b

Feel free to reply to this email if you cannot access logs, if you need
some support to fix the error, if this doesn't seem to be caused by your
modifications or if the error is a false positive one.

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: selftests: mptfo initiator/listener: Build Failure

[MPTCP CI]

Hi Dmytro,

Thank you for your modifications, that's great!

But sadly, our CI spotted some issues with it when trying to build it.

You can find more details there:

  https://patchwork.kernel.org/project/mptcp/patch/20220921125558.19483-7-dmytro@shytyi.net/
  https://github.com/multipath-tcp/mptcp_net-next/actions/runs/3098139757

Status: failure
Initiator: MPTCPimporter
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/5eb61bb88e80

Feel free to reply to this email if you cannot access logs, if you need
some support to fix the error, if this doesn't seem to be caused by your
modifications or if the error is a false positive one.

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: selftests: mptfo initiator/listener: Build Failure

[MPTCP CI]

Hi Dmytro,

Thank you for your modifications, that's great!

But sadly, our CI spotted some issues with it when trying to build it.

You can find more details there:

  https://patchwork.kernel.org/project/mptcp/patch/20220920125243.2880-8-dmytro@shytyi.net/
  https://github.com/multipath-tcp/mptcp_net-next/actions/runs/3090294936

Status: failure
Initiator: MPTCPimporter
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/413c19c617ef

Feel free to reply to this email if you cannot access logs, if you need
some support to fix the error, if this doesn't seem to be caused by your
modifications or if the error is a false positive one.

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: selftests: mptfo initiator/listener: Build Failure

[MPTCP CI]

Hi Dmytro,

Thank you for your modifications, that's great!

But sadly, our CI spotted some issues with it when trying to build it.

You can find more details there:

  https://patchwork.kernel.org/project/mptcp/patch/20220915235604.26018-10-dmytro@shytyi.net/
  https://github.com/multipath-tcp/mptcp_net-next/actions/runs/3064785105

Status: failure
Initiator: MPTCPimporter
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/a04a47ac6e30

Feel free to reply to this email if you cannot access logs, if you need
some support to fix the error, if this doesn't seem to be caused by your
modifications or if the error is a false positive one.

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)