[MPTCP][PATCH v2 mptcp-next 00/10] fullmesh path manager support

[MPTCP][PATCH v2 mptcp-next 00/10] fullmesh path manager support

[Geliang Tang]

Implement the in-kernel fullmesh path manager like on the mptcp.org
kernel.

v2:
 - Implement the fullmesh mode as an extension to the netlink PM, not a
   standalone PM as Paolo suggested.
 - drop duplicate code.
 - add a new per endpoint flag MPTCP_PM_ADDR_FLAG_FULLMESH.

Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/193

Geliang Tang (10):
  mptcp: add a new sysctl fullmesh_enabled
  mptcp: register ipv4 addr notifier
  mptcp: register ipv6 addr notifier
  mptcp: add netdev up event handler
  mptcp: invoke mptcp_nl_remove_subflow_and_signal_addr in rcu_work
  mptcp: add netdev down event handler
  mptcp: add proc file local_addr_list
  selftests: mptcp: print the fullmesh flag
  selftests: mptcp: add fullmesh testcases
  selftests: mptcp: del uncontinuous removing ids

 Documentation/networking/mptcp-sysctl.rst     |   8 ++
 include/uapi/linux/mptcp.h                    |   1 +
 net/mptcp/Kconfig                             |  10 ++
 net/mptcp/Makefile                            |   1 +
 net/mptcp/ctrl.c                              |  16 +++
 net/mptcp/pm.c                                |   3 +
 net/mptcp/pm_fullmesh.c                       |  94 ++++++++++++++
 net/mptcp/pm_netlink.c                        | 116 +++++++++++++++++-
 net/mptcp/protocol.h                          |   6 +
 .../testing/selftests/net/mptcp/mptcp_join.sh |  91 +++++++++++++-
 tools/testing/selftests/net/mptcp/pm_nl_ctl.c |   7 ++
 11 files changed, 342 insertions(+), 11 deletions(-)
 create mode 100644 net/mptcp/pm_fullmesh.c

-- 
2.31.1

[MPTCP][PATCH v2 mptcp-next 01/10] mptcp: add a new sysctl fullmesh_enabled

[Geliang Tang]

This patch added a new sysctl, named fullmesh_enabled, to control whether
the fullmesh path manager mode can be enabled.

Signed-off-by: Geliang Tang <geliangtang@gmail.com>
---
 Documentation/networking/mptcp-sysctl.rst |  8 ++++++++
 net/mptcp/ctrl.c                          | 16 ++++++++++++++++
 net/mptcp/protocol.h                      |  1 +
 3 files changed, 25 insertions(+)

diff --git a/Documentation/networking/mptcp-sysctl.rst b/Documentation/networking/mptcp-sysctl.rst
index b0d4da71e68e..7e8350cbb042 100644
--- a/Documentation/networking/mptcp-sysctl.rst
+++ b/Documentation/networking/mptcp-sysctl.rst
@@ -57,3 +57,11 @@ stale_loss_cnt - INTEGER
 	This is a per-namespace sysctl.
 
 	Default: 4
+
+fullmesh_enabled - BOOLEAN
+	Control whether the fullmesh path manager mode can be enabled.
+
+	The fullmesh path manager mode can be enabled if the value it nonzero.
+	This is a per-namespace sysctl.
+
+	Default: 0
diff --git a/net/mptcp/ctrl.c b/net/mptcp/ctrl.c
index 8b235468c88f..432a859e9153 100644
--- a/net/mptcp/ctrl.c
+++ b/net/mptcp/ctrl.c
@@ -26,6 +26,7 @@ struct mptcp_pernet {
 	u8 mptcp_enabled;
 	u8 checksum_enabled;
 	u8 allow_join_initial_addr_port;
+	u8 fullmesh_enabled;
 };
 
 static struct mptcp_pernet *mptcp_get_pernet(const struct net *net)
@@ -58,6 +59,11 @@ unsigned int mptcp_stale_loss_cnt(const struct net *net)
 	return mptcp_get_pernet(net)->stale_loss_cnt;
 }
 
+int mptcp_is_fullmesh_enabled(struct net *net)
+{
+	return mptcp_get_pernet(net)->fullmesh_enabled;
+}
+
 static void mptcp_pernet_set_defaults(struct mptcp_pernet *pernet)
 {
 	pernet->mptcp_enabled = 1;
@@ -65,6 +71,7 @@ static void mptcp_pernet_set_defaults(struct mptcp_pernet *pernet)
 	pernet->checksum_enabled = 0;
 	pernet->allow_join_initial_addr_port = 1;
 	pernet->stale_loss_cnt = 4;
+	pernet->fullmesh_enabled = 0;
 }
 
 #ifdef CONFIG_SYSCTL
@@ -108,6 +115,14 @@ static struct ctl_table mptcp_sysctl_table[] = {
 		.mode = 0644,
 		.proc_handler = proc_douintvec_minmax,
 	},
+	{
+		.procname	= "fullmesh_enabled",
+		.maxlen		= sizeof(u8),
+		.mode		= 0644,
+		.proc_handler	= proc_dou8vec_minmax,
+		.extra1		= SYSCTL_ZERO,
+		.extra2		= SYSCTL_ONE
+	},
 	{}
 };
 
@@ -128,6 +143,7 @@ static int mptcp_pernet_new_table(struct net *net, struct mptcp_pernet *pernet)
 	table[2].data = &pernet->checksum_enabled;
 	table[3].data = &pernet->allow_join_initial_addr_port;
 	table[4].data = &pernet->stale_loss_cnt;
+	table[5].data = &pernet->fullmesh_enabled;
 
 	hdr = register_net_sysctl(net, MPTCP_SYSCTL_PATH, table);
 	if (!hdr)
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index 8bdd038def38..ce972edc7bbe 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -562,6 +562,7 @@ unsigned int mptcp_get_add_addr_timeout(const struct net *net);
 int mptcp_is_checksum_enabled(const struct net *net);
 int mptcp_allow_join_id0(const struct net *net);
 unsigned int mptcp_stale_loss_cnt(const struct net *net);
+int mptcp_is_fullmesh_enabled(struct net *net);
 void mptcp_subflow_fully_established(struct mptcp_subflow_context *subflow,
 				     struct mptcp_options_received *mp_opt);
 bool __mptcp_retransmit_pending_data(struct sock *sk);
-- 
2.31.1

[MPTCP][PATCH v2 mptcp-next 02/10] mptcp: register ipv4 addr notifier

[Geliang Tang]

This patch added a new file pm_fullmesh.c, and modify Makefile and
Kconfig to support it.

Implemented a new function mptcp_pm_fm_init(). In it registered a ipv4
addr notifier, named mptcp_pm_addr4_notifier, to deal with the events
of net device UP, DOWN and CHANGE, and skip the loopback device.

Save the ipv4 address, and pass it to the event handler.

Signed-off-by: Geliang Tang <geliangtang@gmail.com>
---
 net/mptcp/Kconfig       | 10 +++++++++
 net/mptcp/Makefile      |  1 +
 net/mptcp/pm.c          |  3 +++
 net/mptcp/pm_fullmesh.c | 48 +++++++++++++++++++++++++++++++++++++++++
 net/mptcp/protocol.h    |  3 +++
 5 files changed, 65 insertions(+)
 create mode 100644 net/mptcp/pm_fullmesh.c

diff --git a/net/mptcp/Kconfig b/net/mptcp/Kconfig
index 10c97e19a7da..fd7f02ec6442 100644
--- a/net/mptcp/Kconfig
+++ b/net/mptcp/Kconfig
@@ -37,4 +37,14 @@ config MPTCP_KUNIT_TEST
 
 	  If unsure, say N.
 
+config MPTCP_FULLMESH
+	bool "The fullmesh path manager mode"
+	default n
+	help
+	  Use the fullmesh path manager mode. In this mode, all the net device
+	  addresses will use to create the subflows automatically. When the
+	  addresses are deleted, the subflows will be closed automatically.
+
+	  If unsure, say N.
+
 endif
diff --git a/net/mptcp/Makefile b/net/mptcp/Makefile
index 0a0608b6b4b4..83c59c8fdbbf 100644
--- a/net/mptcp/Makefile
+++ b/net/mptcp/Makefile
@@ -13,3 +13,4 @@ mptcp_token_test-objs := token_test.o
 obj-$(CONFIG_MPTCP_KUNIT_TEST) += mptcp_crypto_test.o mptcp_token_test.o
 
 obj-$(CONFIG_BPF) += bpf.o
+obj-$(CONFIG_MPTCP_FULLMESH) += pm_fullmesh.o
diff --git a/net/mptcp/pm.c b/net/mptcp/pm.c
index 0ed3e565f8f8..57c69e8d4bed 100644
--- a/net/mptcp/pm.c
+++ b/net/mptcp/pm.c
@@ -353,4 +353,7 @@ void mptcp_pm_data_init(struct mptcp_sock *msk)
 void __init mptcp_pm_init(void)
 {
 	mptcp_pm_nl_init();
+#if IS_ENABLED(CONFIG_MPTCP_FULLMESH)
+	mptcp_pm_fm_init();
+#endif
 }
diff --git a/net/mptcp/pm_fullmesh.c b/net/mptcp/pm_fullmesh.c
new file mode 100644
index 000000000000..38a44ce8a5fa
--- /dev/null
+++ b/net/mptcp/pm_fullmesh.c
@@ -0,0 +1,48 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#define pr_fmt(fmt) "MPTCP: " fmt
+
+#include <linux/kernel.h>
+#include <net/mptcp.h>
+
+#include "protocol.h"
+
+static void addr_event_handler(unsigned long event, struct net *net,
+			       struct mptcp_addr_info *addr)
+{
+	if (!mptcp_is_fullmesh_enabled(net))
+		return;
+}
+
+static int mptcp_pm_addr4_event(struct notifier_block *this,
+				unsigned long event, void *ptr)
+{
+	const struct in_ifaddr *ifa = (struct in_ifaddr *)ptr;
+	struct net *net = dev_net(ifa->ifa_dev->dev);
+	struct mptcp_addr_info addr = { 0 };
+
+	if (!(event == NETDEV_UP || event == NETDEV_DOWN || event == NETDEV_CHANGE))
+		goto out;
+
+	if (ifa->ifa_scope > RT_SCOPE_LINK ||
+	    ipv4_is_loopback(ifa->ifa_local))
+		goto out;
+
+	addr.family = AF_INET;
+	addr.addr.s_addr = ifa->ifa_local;
+
+	addr_event_handler(event, net, &addr);
+
+out:
+	return NOTIFY_DONE;
+}
+
+static struct notifier_block mptcp_pm_addr4_notifier = {
+	.notifier_call = mptcp_pm_addr4_event,
+};
+
+void __init mptcp_pm_fm_init(void)
+{
+	if (register_inetaddr_notifier(&mptcp_pm_addr4_notifier))
+		return;
+}
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index ce972edc7bbe..2c3a4d507454 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -810,6 +810,9 @@ unsigned int mptcp_pm_get_add_addr_signal_max(struct mptcp_sock *msk);
 unsigned int mptcp_pm_get_add_addr_accept_max(struct mptcp_sock *msk);
 unsigned int mptcp_pm_get_subflows_max(struct mptcp_sock *msk);
 unsigned int mptcp_pm_get_local_addr_max(struct mptcp_sock *msk);
+#if IS_ENABLED(CONFIG_MPTCP_FULLMESH)
+void __init mptcp_pm_fm_init(void);
+#endif
 
 void mptcp_sockopt_sync(struct mptcp_sock *msk, struct sock *ssk);
 void mptcp_sockopt_sync_all(struct mptcp_sock *msk);
-- 
2.31.1

[MPTCP][PATCH v2 mptcp-next 03/10] mptcp: register ipv6 addr notifier

[Geliang Tang]

This patch registered a ipv6 addr notifier, named mptcp_pm_addr6_notifier,
to deal with the events of net device UP, DOWN and CHANGE, and skip the
loopback device.

Save the ipv6 address, and pass it to the event handler.

Signed-off-by: Geliang Tang <geliangtang@gmail.com>
---
 net/mptcp/pm_fullmesh.c | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/net/mptcp/pm_fullmesh.c b/net/mptcp/pm_fullmesh.c
index 38a44ce8a5fa..bac06ddc658b 100644
--- a/net/mptcp/pm_fullmesh.c
+++ b/net/mptcp/pm_fullmesh.c
@@ -3,6 +3,8 @@
 #define pr_fmt(fmt) "MPTCP: " fmt
 
 #include <linux/kernel.h>
+#include <linux/inetdevice.h>
+#include <net/addrconf.h>
 #include <net/mptcp.h>
 
 #include "protocol.h"
@@ -41,8 +43,47 @@ static struct notifier_block mptcp_pm_addr4_notifier = {
 	.notifier_call = mptcp_pm_addr4_event,
 };
 
+#if IS_ENABLED(CONFIG_MPTCP_IPV6)
+
+static int mptcp_pm_addr6_event(struct notifier_block *this,
+				unsigned long event, void *ptr)
+{
+	const struct inet6_ifaddr *ifa6 = (struct inet6_ifaddr *)ptr;
+	struct net *net = dev_net(ifa6->idev->dev);
+	int addr_type = ipv6_addr_type(&ifa6->addr);
+	struct mptcp_addr_info addr = { 0 };
+
+	if (!(event == NETDEV_UP || event == NETDEV_DOWN || event == NETDEV_CHANGE))
+		goto out;
+
+	if (ifa6->scope > RT_SCOPE_LINK ||
+	    addr_type == IPV6_ADDR_ANY ||
+	    (addr_type & IPV6_ADDR_LOOPBACK) ||
+	    (addr_type & IPV6_ADDR_LINKLOCAL))
+		goto out;
+
+	addr.family = AF_INET6;
+	addr.addr6 = ifa6->addr;
+
+	addr_event_handler(event, net, &addr);
+
+out:
+	return NOTIFY_DONE;
+}
+
+static struct notifier_block mptcp_pm_addr6_notifier = {
+	.notifier_call = mptcp_pm_addr6_event,
+};
+
+#endif
+
 void __init mptcp_pm_fm_init(void)
 {
 	if (register_inetaddr_notifier(&mptcp_pm_addr4_notifier))
 		return;
+
+#if IS_ENABLED(CONFIG_MPTCP_IPV6)
+	if (register_inet6addr_notifier(&mptcp_pm_addr6_notifier))
+		unregister_inetaddr_notifier(&mptcp_pm_addr4_notifier);
+#endif
 }
-- 
2.31.1

[MPTCP][PATCH v2 mptcp-next 04/10] mptcp: add netdev up event handler

[Geliang Tang]

This patch added the net device UP event handler function named
mptcp_fm_cmd_add_addr. In it, alloc an address entry, populate it, and
append this entry to the local address list. Then invoke
mptcp_nl_add_subflow_or_signal_addr to create the new subflows.

Signed-off-by: Geliang Tang <geliangtang@gmail.com>
---
 include/uapi/linux/mptcp.h |  1 +
 net/mptcp/pm_fullmesh.c    |  3 +++
 net/mptcp/pm_netlink.c     | 31 +++++++++++++++++++++++++++++--
 net/mptcp/protocol.h       |  1 +
 4 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/include/uapi/linux/mptcp.h b/include/uapi/linux/mptcp.h
index 7b05f7102321..f66038b9551f 100644
--- a/include/uapi/linux/mptcp.h
+++ b/include/uapi/linux/mptcp.h
@@ -73,6 +73,7 @@ enum {
 #define MPTCP_PM_ADDR_FLAG_SIGNAL			(1 << 0)
 #define MPTCP_PM_ADDR_FLAG_SUBFLOW			(1 << 1)
 #define MPTCP_PM_ADDR_FLAG_BACKUP			(1 << 2)
+#define MPTCP_PM_ADDR_FLAG_FULLMESH			(1 << 3)
 
 enum {
 	MPTCP_PM_CMD_UNSPEC,
diff --git a/net/mptcp/pm_fullmesh.c b/net/mptcp/pm_fullmesh.c
index bac06ddc658b..6c52a64657f4 100644
--- a/net/mptcp/pm_fullmesh.c
+++ b/net/mptcp/pm_fullmesh.c
@@ -14,6 +14,9 @@ static void addr_event_handler(unsigned long event, struct net *net,
 {
 	if (!mptcp_is_fullmesh_enabled(net))
 		return;
+
+	if (event == NETDEV_UP)
+		mptcp_fm_cmd_add_addr(net, addr);
 }
 
 static int mptcp_pm_addr4_event(struct notifier_block *this,
diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c
index ac0aa6faacfa..3437de74f003 100644
--- a/net/mptcp/pm_netlink.c
+++ b/net/mptcp/pm_netlink.c
@@ -170,7 +170,8 @@ select_local_address(const struct pm_nl_pernet *pernet,
 	rcu_read_lock();
 	__mptcp_flush_join_list(msk);
 	list_for_each_entry_rcu(entry, &pernet->local_addr_list, list) {
-		if (!(entry->flags & MPTCP_PM_ADDR_FLAG_SUBFLOW))
+		if (!(entry->flags & MPTCP_PM_ADDR_FLAG_SUBFLOW) &&
+		    !(entry->flags & MPTCP_PM_ADDR_FLAG_FULLMESH))
 			continue;
 
 		if (entry->addr.family != sk->sk_family) {
@@ -745,7 +746,8 @@ static int mptcp_pm_nl_append_new_local_addr(struct pm_nl_pernet *pernet,
 		addr_max = pernet->add_addr_signal_max;
 		WRITE_ONCE(pernet->add_addr_signal_max, addr_max + 1);
 	}
-	if (entry->flags & MPTCP_PM_ADDR_FLAG_SUBFLOW) {
+	if (entry->flags & MPTCP_PM_ADDR_FLAG_SUBFLOW ||
+	    entry->flags & MPTCP_PM_ADDR_FLAG_FULLMESH) {
 		addr_max = pernet->local_addr_max;
 		WRITE_ONCE(pernet->local_addr_max, addr_max + 1);
 	}
@@ -1954,6 +1956,31 @@ static struct genl_family mptcp_genl_family __ro_after_init = {
 	.n_mcgrps	= ARRAY_SIZE(mptcp_pm_mcgrps),
 };
 
+#if IS_ENABLED(CONFIG_MPTCP_FULLMESH)
+
+int mptcp_fm_cmd_add_addr(struct net *net, const struct mptcp_addr_info *addr)
+{
+	struct pm_nl_pernet *pernet = net_generic(net, pm_nl_pernet_id);
+	struct mptcp_pm_addr_entry *entry;
+
+	entry = kzalloc(sizeof(*entry), GFP_ATOMIC);
+	if (!entry)
+		return -ENOMEM;
+
+	entry->addr = *addr;
+	entry->addr.id = 0;
+	entry->addr.port = 0;
+	entry->flags |= MPTCP_PM_ADDR_FLAG_FULLMESH;
+	entry->ifindex = 0;
+
+	mptcp_pm_nl_append_new_local_addr(pernet, entry);
+	mptcp_nl_add_subflow_or_signal_addr(net);
+
+	return 0;
+}
+
+#endif
+
 static int __net_init pm_nl_init_net(struct net *net)
 {
 	struct pm_nl_pernet *pernet = net_generic(net, pm_nl_pernet_id);
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index 2c3a4d507454..78a9cd90b0f9 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -812,6 +812,7 @@ unsigned int mptcp_pm_get_subflows_max(struct mptcp_sock *msk);
 unsigned int mptcp_pm_get_local_addr_max(struct mptcp_sock *msk);
 #if IS_ENABLED(CONFIG_MPTCP_FULLMESH)
 void __init mptcp_pm_fm_init(void);
+int mptcp_fm_cmd_add_addr(struct net *net, const struct mptcp_addr_info *addr);
 #endif
 
 void mptcp_sockopt_sync(struct mptcp_sock *msk, struct sock *ssk);
-- 
2.31.1

[MPTCP][PATCH v2 mptcp-next 05/10] mptcp: invoke mptcp_nl_remove_subflow_and_signal_addr in rcu_work

[Geliang Tang]

mptcp_nl_remove_subflow_and_signal_addr will be invoked in the atomic
context in the next patch. So we need to move it into the rcu_work to
remove the subflow and signal the RM_ADDR suboption.

Signed-off-by: Geliang Tang <geliangtang@gmail.com>
---
 net/mptcp/pm_netlink.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c
index 3437de74f003..533818b4dba2 100644
--- a/net/mptcp/pm_netlink.c
+++ b/net/mptcp/pm_netlink.c
@@ -1177,6 +1177,7 @@ static int mptcp_nl_remove_subflow_and_signal_addr(struct net *net,
 
 struct addr_entry_release_work {
 	struct rcu_work	rwork;
+	struct net *net;
 	struct mptcp_pm_addr_entry *entry;
 };
 
@@ -1184,10 +1185,14 @@ static void mptcp_pm_release_addr_entry(struct work_struct *work)
 {
 	struct addr_entry_release_work *w;
 	struct mptcp_pm_addr_entry *entry;
+	struct net *net;
 
 	w = container_of(to_rcu_work(work), struct addr_entry_release_work, rwork);
+	net = w->net;
 	entry = w->entry;
 	if (entry) {
+		if (net)
+			mptcp_nl_remove_subflow_and_signal_addr(net, &entry->addr);
 		if (entry->lsk)
 			sock_release(entry->lsk);
 		kfree(entry);
@@ -1195,13 +1200,14 @@ static void mptcp_pm_release_addr_entry(struct work_struct *work)
 	kfree(w);
 }
 
-static void mptcp_pm_free_addr_entry(struct mptcp_pm_addr_entry *entry)
+static void mptcp_pm_free_addr_entry(struct net *net, struct mptcp_pm_addr_entry *entry)
 {
 	struct addr_entry_release_work *w;
 
 	w = kmalloc(sizeof(*w), GFP_ATOMIC);
 	if (w) {
 		INIT_RCU_WORK(&w->rwork, mptcp_pm_release_addr_entry);
+		w->net = net;
 		w->entry = entry;
 		queue_rcu_work(system_wq, &w->rwork);
 	}
@@ -1283,8 +1289,7 @@ static int mptcp_nl_cmd_del_addr(struct sk_buff *skb, struct genl_info *info)
 	__clear_bit(entry->addr.id, pernet->id_bitmap);
 	spin_unlock_bh(&pernet->lock);
 
-	mptcp_nl_remove_subflow_and_signal_addr(sock_net(skb->sk), &entry->addr);
-	mptcp_pm_free_addr_entry(entry);
+	mptcp_pm_free_addr_entry(sock_net(skb->sk), entry);
 
 	return ret;
 }
@@ -1345,7 +1350,7 @@ static void __flush_addrs(struct list_head *list)
 		cur = list_entry(list->next,
 				 struct mptcp_pm_addr_entry, list);
 		list_del_rcu(&cur->list);
-		mptcp_pm_free_addr_entry(cur);
+		mptcp_pm_free_addr_entry(NULL, cur);
 	}
 }
 
-- 
2.31.1

[MPTCP][PATCH v2 mptcp-next 06/10] mptcp: add netdev down event handler

[Geliang Tang]

This patch added the net device DOWN event handler function named
mptcp_fm_cmd_del_addr. In it, traverse the local address list to find
the deleting address entry, pass this entry to mptcp_pm_free_addr_entry,
then start the rcu_work to remove the subflow and signal the RM_ADDR
suboption.

Signed-off-by: Geliang Tang <geliangtang@gmail.com>
---
 net/mptcp/pm_fullmesh.c |  2 ++
 net/mptcp/pm_netlink.c  | 20 ++++++++++++++++++++
 net/mptcp/protocol.h    |  1 +
 3 files changed, 23 insertions(+)

diff --git a/net/mptcp/pm_fullmesh.c b/net/mptcp/pm_fullmesh.c
index 6c52a64657f4..beb6e4c7cf46 100644
--- a/net/mptcp/pm_fullmesh.c
+++ b/net/mptcp/pm_fullmesh.c
@@ -17,6 +17,8 @@ static void addr_event_handler(unsigned long event, struct net *net,
 
 	if (event == NETDEV_UP)
 		mptcp_fm_cmd_add_addr(net, addr);
+	else if (event == NETDEV_DOWN)
+		mptcp_fm_cmd_del_addr(net, addr);
 }
 
 static int mptcp_pm_addr4_event(struct notifier_block *this,
diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c
index 533818b4dba2..765d59dd2a5d 100644
--- a/net/mptcp/pm_netlink.c
+++ b/net/mptcp/pm_netlink.c
@@ -1984,6 +1984,26 @@ int mptcp_fm_cmd_add_addr(struct net *net, const struct mptcp_addr_info *addr)
 	return 0;
 }
 
+int mptcp_fm_cmd_del_addr(struct net *net, struct mptcp_addr_info *addr)
+{
+	struct pm_nl_pernet *pernet = net_generic(net, pm_nl_pernet_id);
+	struct mptcp_pm_addr_entry *entry, *tmp;
+
+	spin_lock_bh(&pernet->lock);
+	list_for_each_entry_safe(entry, tmp, &pernet->local_addr_list, list) {
+		if (addresses_equal(&entry->addr, addr, false)) {
+			list_del_rcu(&entry->list);
+			spin_unlock_bh(&pernet->lock);
+			mptcp_pm_free_addr_entry(net, entry);
+
+			return 0;
+		}
+	}
+	spin_unlock_bh(&pernet->lock);
+
+	return 0;
+}
+
 #endif
 
 static int __net_init pm_nl_init_net(struct net *net)
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index 78a9cd90b0f9..1dbd8de17a22 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -813,6 +813,7 @@ unsigned int mptcp_pm_get_local_addr_max(struct mptcp_sock *msk);
 #if IS_ENABLED(CONFIG_MPTCP_FULLMESH)
 void __init mptcp_pm_fm_init(void);
 int mptcp_fm_cmd_add_addr(struct net *net, const struct mptcp_addr_info *addr);
+int mptcp_fm_cmd_del_addr(struct net *net, struct mptcp_addr_info *addr);
 #endif
 
 void mptcp_sockopt_sync(struct mptcp_sock *msk, struct sock *ssk);
-- 
2.31.1

[MPTCP][PATCH v2 mptcp-next 07/10] mptcp: add proc file local_addr_list

[Geliang Tang]

This patch added a proc file /proc/net/local_addr_list to show all
the addresses on the local address list.

Signed-off-by: Geliang Tang <geliangtang@gmail.com>
---
 net/mptcp/pm_netlink.c | 52 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c
index 765d59dd2a5d..3bf7467af74b 100644
--- a/net/mptcp/pm_netlink.c
+++ b/net/mptcp/pm_netlink.c
@@ -8,6 +8,9 @@
 
 #include <linux/inet.h>
 #include <linux/kernel.h>
+#ifdef CONFIG_PROC_FS
+#include <linux/proc_fs.h>
+#endif
 #include <net/tcp.h>
 #include <net/netns/generic.h>
 #include <net/mptcp.h>
@@ -2006,6 +2009,46 @@ int mptcp_fm_cmd_del_addr(struct net *net, struct mptcp_addr_info *addr)
 
 #endif
 
+#ifdef CONFIG_PROC_FS
+
+/* Output /proc/net/local_addr_list */
+static int mptcp_addr_list_seq_show(struct seq_file *seq, void *v)
+{
+	const struct net *net = seq->private;
+	struct mptcp_pm_addr_entry *cur;
+	struct pm_nl_pernet *pernet;
+
+	pernet = net_generic(net, pm_nl_pernet_id);
+
+	seq_puts(seq, "ID, Family, Address, Flags\n");
+
+	spin_lock_bh(&pernet->lock);
+
+	list_for_each_entry(cur, &pernet->local_addr_list, list) {
+		seq_printf(seq, "%u, ", cur->addr.id);
+		if (cur->addr.family == AF_INET)
+			seq_printf(seq, "4 %pI4, ", &cur->addr.addr);
+		else if (cur->addr.family == AF_INET6)
+			seq_printf(seq, "6 %pI6, ", &cur->addr.addr6);
+
+		if (cur->flags & MPTCP_PM_ADDR_FLAG_SUBFLOW)
+			seq_puts(seq, "subflow ");
+		if (cur->flags & MPTCP_PM_ADDR_FLAG_SIGNAL)
+			seq_puts(seq, "signal ");
+		if (cur->flags & MPTCP_PM_ADDR_FLAG_BACKUP)
+			seq_puts(seq, "backup ");
+		if (cur->flags & MPTCP_PM_ADDR_FLAG_FULLMESH)
+			seq_puts(seq, "fullmesh ");
+		seq_puts(seq, "\n");
+	}
+
+	spin_unlock_bh(&pernet->lock);
+
+	return 0;
+}
+
+#endif
+
 static int __net_init pm_nl_init_net(struct net *net)
 {
 	struct pm_nl_pernet *pernet = net_generic(net, pm_nl_pernet_id);
@@ -2015,6 +2058,12 @@ static int __net_init pm_nl_init_net(struct net *net)
 	pernet->stale_loss_cnt = 4;
 	spin_lock_init(&pernet->lock);
 
+#ifdef CONFIG_PROC_FS
+	if (!proc_create_net_single("local_addr_list", 0444, net->proc_net,
+				    mptcp_addr_list_seq_show, NULL))
+		return -EINVAL;
+#endif
+
 	/* No need to initialize other pernet fields, the struct is zeroed at
 	 * allocation time.
 	 */
@@ -2033,6 +2082,9 @@ static void __net_exit pm_nl_exit_net(struct list_head *net_list)
 		 * other modifiers
 		 */
 		__flush_addrs(&pernet->local_addr_list);
+#ifdef CONFIG_PROC_FS
+		remove_proc_entry("local_addr_list", net->proc_net);
+#endif
 	}
 }
 
-- 
2.31.1

[MPTCP][PATCH v2 mptcp-next 08/10] selftests: mptcp: print the fullmesh flag

[Geliang Tang]

This patch dealt with the MPTCP_PM_ADDR_FLAG_FULLMESH flag in
print_addr() to print out the fullmesh flag.

Signed-off-by: Geliang Tang <geliangtang@gmail.com>
---
 tools/testing/selftests/net/mptcp/pm_nl_ctl.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tools/testing/selftests/net/mptcp/pm_nl_ctl.c b/tools/testing/selftests/net/mptcp/pm_nl_ctl.c
index 115decfdc1ef..78e0d37d008e 100644
--- a/tools/testing/selftests/net/mptcp/pm_nl_ctl.c
+++ b/tools/testing/selftests/net/mptcp/pm_nl_ctl.c
@@ -422,6 +422,13 @@ static void print_addr(struct rtattr *attrs, int len)
 					printf(",");
 			}
 
+			if (flags & MPTCP_PM_ADDR_FLAG_FULLMESH) {
+				printf("fullmesh");
+				flags &= ~MPTCP_PM_ADDR_FLAG_FULLMESH;
+				if (flags)
+					printf(",");
+			}
+
 			/* bump unknown flags, if any */
 			if (flags)
 				printf("0x%x", flags);
-- 
2.31.1

[MPTCP][PATCH v2 mptcp-next 09/10] selftests: mptcp: add fullmesh testcases

[Geliang Tang]

This patch added the net device UP and DOWN testcases for the fullmesh
path manager.

Signed-off-by: Geliang Tang <geliangtang@gmail.com>
---
 .../testing/selftests/net/mptcp/mptcp_join.sh | 81 ++++++++++++++++++-
 1 file changed, 80 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh
index 220154cb92a7..823300986459 100755
--- a/tools/testing/selftests/net/mptcp/mptcp_join.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh
@@ -160,6 +160,27 @@ reset_with_allow_join_id0()
 	ip netns exec $ns2 sysctl -q net.mptcp.allow_join_initial_addr_port=$ns2_enable
 }
 
+reset_fullmesh()
+{
+	reset
+
+	ip netns exec $ns2 sysctl -q net.mptcp.fullmesh_enabled=1
+
+	for i in `seq 5 8`; do
+		ip link add ns1eth$i netns "$ns1" type veth peer name ns2eth$i netns "$ns2"
+		ip -net "$ns1" addr add 10.0.$i.1/24 dev ns1eth$i
+		ip -net "$ns1" addr add dead:beef:$i::1/64 dev ns1eth$i nodad
+		ip -net "$ns1" link set ns1eth$i up
+
+		ip -net "$ns2" addr add 10.0.$i.2/24 dev ns2eth$i
+		ip -net "$ns2" addr add dead:beef:$i::2/64 dev ns2eth$i nodad
+		ip -net "$ns2" link set ns2eth$i up
+
+		# let $ns2 reach any $ns1 address from any interface
+		ip -net "$ns2" route add default via 10.0.$i.1 dev ns2eth$i metric 10$i
+	done
+}
+
 ip -Version > /dev/null 2>&1
 if [ $? -ne 0 ];then
 	echo "SKIP: Could not run test without ip tool"
@@ -364,6 +385,16 @@ do_transfer()
 		elif [ $rm_nr_ns1 -eq 9 ]; then
 			sleep 1
 			ip netns exec ${listener_ns} ./pm_nl_ctl del 0 ${connect_addr}
+		elif [ $rm_nr_ns1 -eq 10 ]; then
+			local addr
+			local i=5
+			if is_v6 "${connect_addr}"; then
+				addr="dead:beef:$i::1/64"
+			else
+				addr="10.0.$i.1/24"
+			fi
+			sleep 2
+			ip -net "${listener_ns}" addr del $addr dev ns1eth$i
 		fi
 	fi
 
@@ -412,6 +443,16 @@ do_transfer()
 			fi
 			sleep 1
 			ip netns exec ${connector_ns} ./pm_nl_ctl del 0 $addr
+		elif [ $rm_nr_ns2 -eq 10 ]; then
+			local addr
+			local i=5
+			if is_v6 "${connect_addr}"; then
+				addr="dead:beef:$i::2/64"
+			else
+				addr="10.0.$i.2/24"
+			fi
+			sleep 2
+			ip -net "${connector_ns}" addr del $addr dev ns2eth$i
 		fi
 	fi
 
@@ -1683,6 +1724,39 @@ deny_join_id0_tests()
 	chk_join_nr "subflow and address allow join id0 2" 1 1 1
 }
 
+fullmesh_tests()
+{
+	# fullmesh add
+	reset_fullmesh
+	ip netns exec $ns1 ./pm_nl_ctl limits 8 8
+	ip netns exec $ns2 ./pm_nl_ctl limits 8 8
+	run_tests $ns1 $ns2 10.0.1.1 0 0 0 slow
+	chk_join_nr "fullmesh add" 4 4 4
+
+	# fullmesh add IPv6
+	reset_fullmesh
+	ip netns exec $ns1 ./pm_nl_ctl limits 8 8
+	ip netns exec $ns2 ./pm_nl_ctl limits 8 8
+	run_tests $ns1 $ns2 dead:beef:1::1 0 0 0 slow
+	chk_join_nr "fullmesh add IPv6" 4 4 4
+
+	# fullmesh del
+	reset_fullmesh
+	ip netns exec $ns1 ./pm_nl_ctl limits 8 8
+	ip netns exec $ns2 ./pm_nl_ctl limits 8 8
+	run_tests $ns1 $ns2 10.0.1.1 0 0 -10 slow
+	chk_join_nr "fullmesh del" 4 4 4
+	chk_rm_nr 1 1
+
+	# fullmesh del IPv6
+	reset_fullmesh
+	ip netns exec $ns1 ./pm_nl_ctl limits 8 8
+	ip netns exec $ns2 ./pm_nl_ctl limits 8 8
+	run_tests $ns1 $ns2 dead:beef:1::1 0 0 -10 slow
+	chk_join_nr "fullmesh del IPv6" 4 4 4
+	chk_rm_nr 1 1
+}
+
 all_tests()
 {
 	subflows_tests
@@ -1698,6 +1772,7 @@ all_tests()
 	syncookies_tests
 	checksum_tests
 	deny_join_id0_tests
+	fullmesh_tests
 }
 
 usage()
@@ -1716,6 +1791,7 @@ usage()
 	echo "  -k syncookies_tests"
 	echo "  -S checksum_tests"
 	echo "  -d deny_join_id0_tests"
+	echo "  -m fullmesh_tests"
 	echo "  -c capture pcap files"
 	echo "  -C enable data checksum"
 	echo "  -h help"
@@ -1751,7 +1827,7 @@ if [ $do_all_tests -eq 1 ]; then
 	exit $ret
 fi
 
-while getopts 'fsltra64bpkdchCS' opt; do
+while getopts 'fsltra64bpkdmchCS' opt; do
 	case $opt in
 		f)
 			subflows_tests
@@ -1792,6 +1868,9 @@ while getopts 'fsltra64bpkdchCS' opt; do
 		d)
 			deny_join_id0_tests
 			;;
+		m)
+			fullmesh_tests
+			;;
 		c)
 			;;
 		C)
-- 
2.31.1

[MPTCP][PATCH v2 mptcp-next 10/10] selftests: mptcp: del uncontinuous removing ids

[Geliang Tang]

This patch added the uncontinuous removing ids support in the removing
address testcases.

Signed-off-by: Geliang Tang <geliangtang@gmail.com>
---
 tools/testing/selftests/net/mptcp/mptcp_join.sh | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh
index 823300986459..a1efba165e80 100755
--- a/tools/testing/selftests/net/mptcp/mptcp_join.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh
@@ -366,17 +366,18 @@ do_transfer()
 		let rm_nr_ns1=-addr_nr_ns1
 		if [ $rm_nr_ns1 -lt 8 ]; then
 			counter=1
+			pos=1
 			dump=(`ip netns exec ${listener_ns} ./pm_nl_ctl dump`)
 			if [ ${#dump[@]} -gt 0 ]; then
-				id=${dump[1]}
 				sleep 1
 
 				while [ $counter -le $rm_nr_ns1 ]
 				do
+					id=${dump[$pos]}
 					ip netns exec ${listener_ns} ./pm_nl_ctl del $id
 					sleep 1
 					let counter+=1
-					let id+=1
+					let pos+=5
 				done
 			fi
 		elif [ $rm_nr_ns1 -eq 8 ]; then
@@ -418,17 +419,18 @@ do_transfer()
 		let rm_nr_ns2=-addr_nr_ns2
 		if [ $rm_nr_ns2 -lt 8 ]; then
 			counter=1
+			pos=1
 			dump=(`ip netns exec ${connector_ns} ./pm_nl_ctl dump`)
 			if [ ${#dump[@]} -gt 0 ]; then
-				id=${dump[1]}
 				sleep 1
 
 				while [ $counter -le $rm_nr_ns2 ]
 				do
+					id=${dump[$pos]}
 					ip netns exec ${connector_ns} ./pm_nl_ctl del $id
 					sleep 1
 					let counter+=1
-					let id+=1
+					let pos+=5
 				done
 			fi
 		elif [ $rm_nr_ns2 -eq 8 ]; then
-- 
2.31.1

Re: [MPTCP][PATCH v2 mptcp-next 02/10] mptcp: register ipv4 addr notifier

[Paolo Abeni]

On Wed, 2021-07-21 at 22:31 +0800, Geliang Tang wrote:
> This patch added a new file pm_fullmesh.c, and modify Makefile and
> Kconfig to support it.
> 
> Implemented a new function mptcp_pm_fm_init(). In it registered a ipv4
> addr notifier, named mptcp_pm_addr4_notifier, to deal with the events
> of net device UP, DOWN and CHANGE, and skip the loopback device.

Do we absolutely need these notifiers? can we use instead e.g.
NetworkManager scrips to create/delete endpoints as needed ?!?

/P

Re: [MPTCP][PATCH v2 mptcp-next 01/10] mptcp: add a new sysctl fullmesh_enabled

[Matthieu Baerts]

Hi Geliang,

Thank you for sharing this v2!

On 21/07/2021 16:31, Geliang Tang wrote:
> This patch added a new sysctl, named fullmesh_enabled, to control whether
> the fullmesh path manager mode can be enabled.

Should we not fully control the PM via Netlink?

I know it is easy to add sysctl knobs but mixing places for the
configuration might be confusing :-/

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net

Re: [MPTCP][PATCH v2 mptcp-next 02/10] mptcp: register ipv4 addr notifier

[Matthieu Baerts]

Hi Geliang, Paolo,

On 21/07/2021 19:08, Paolo Abeni wrote:
> On Wed, 2021-07-21 at 22:31 +0800, Geliang Tang wrote:
>> This patch added a new file pm_fullmesh.c, and modify Makefile and
>> Kconfig to support it.
>>
>> Implemented a new function mptcp_pm_fm_init(). In it registered a ipv4
>> addr notifier, named mptcp_pm_addr4_notifier, to deal with the events
>> of net device UP, DOWN and CHANGE, and skip the loopback device.
> 
> Do we absolutely need these notifiers? can we use instead e.g.
> NetworkManager scrips to create/delete endpoints as needed ?!?

I understand it is convenient to have this code in the kernel but I
don't think it will be easy to manage all different cases: ignoring some
interfaces, specific IPs, all v6 or v4, sometimes loopback/link-local
are interesting to use, etc.

It looks like an easy job for a userspace daemon managing the
connections, no?
Is it not possible to have external hooks (shell scripts? an app with
extra rights?) with the daemon managing connections on Android?

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net

Re: [MPTCP][PATCH v2 mptcp-next 04/10] mptcp: add netdev up event handler

[Matthieu Baerts]

Hi Geliang,

On 21/07/2021 16:31, Geliang Tang wrote:
> This patch added the net device UP event handler function named
> mptcp_fm_cmd_add_addr. In it, alloc an address entry, populate it, and
> append this entry to the local address list. Then invoke
> mptcp_nl_add_subflow_or_signal_addr to create the new subflows.

(...)

> diff --git a/include/uapi/linux/mptcp.h b/include/uapi/linux/mptcp.h
> index 7b05f7102321..f66038b9551f 100644
> --- a/include/uapi/linux/mptcp.h
> +++ b/include/uapi/linux/mptcp.h
> @@ -73,6 +73,7 @@ enum {
>  #define MPTCP_PM_ADDR_FLAG_SIGNAL			(1 << 0)
>  #define MPTCP_PM_ADDR_FLAG_SUBFLOW			(1 << 1)
>  #define MPTCP_PM_ADDR_FLAG_BACKUP			(1 << 2)
> +#define MPTCP_PM_ADDR_FLAG_FULLMESH			(1 << 3)

Adding a new flag for the Netlink API seems to be the good direction to
take I think. But I think we should only set this flag using Netlink.

> diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c
> index ac0aa6faacfa..3437de74f003 100644
> --- a/net/mptcp/pm_netlink.c
> +++ b/net/mptcp/pm_netlink.c

(...)

> @@ -1954,6 +1956,31 @@ static struct genl_family mptcp_genl_family __ro_after_init = {
>  	.n_mcgrps	= ARRAY_SIZE(mptcp_pm_mcgrps),
>  };
>  
> +#if IS_ENABLED(CONFIG_MPTCP_FULLMESH)
> +
> +int mptcp_fm_cmd_add_addr(struct net *net, const struct mptcp_addr_info *addr)

Should it not be declared in fullmesh.c file? It is specific to FM.

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net

Re: [MPTCP][PATCH v2 mptcp-next 07/10] mptcp: add proc file local_addr_list

[Matthieu Baerts]

Hi Geliang,

On 21/07/2021 16:31, Geliang Tang wrote:
> This patch added a proc file /proc/net/local_addr_list to show all
> the addresses on the local address list.

This looks like a very useful info and handy to read but I don't think
upstream -net maintainers will accept new entries in /proc/net.
Instead, a Netlink API should be used if I'm not mistaken.

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net

Re: [MPTCP][PATCH v2 mptcp-next 10/10] selftests: mptcp: del uncontinuous removing ids

[Matthieu Baerts]

Hi Geliang,

On 21/07/2021 16:31, Geliang Tang wrote:
> This patch added the uncontinuous removing ids support in the removing
> address testcases.

Is it a fix for a previous modification? If yes, may you add a "Fixes"
tag please?
Or is it only visible with this series? What are the consequences
without this patch?

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net

Re: [MPTCP][PATCH v2 mptcp-next 00/10] fullmesh path manager support

[Matthieu Baerts]

Hi Geliang,

On 21/07/2021 16:31, Geliang Tang wrote:
> Implement the in-kernel fullmesh path manager like on the mptcp.org
> kernel.

Thank you for the series!

Do not hesitate to share architecture design if you are unsure before
writing a lot of code ;-)
(But if it is easier for you to share code, that's not an issue for me :) )

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net