[RFC ipsec-next v2 0/8] Add IP-TFS mode to xfrm

[RFC ipsec-next v2 0/8] Add IP-TFS mode to xfrm

[Christian Hopps]

From: Christian Hopps <chopps@labn.net>

This patchset adds a new xfrm mode implementing on-demand IP-TFS. IP-TFS
(AggFrag encapsulation) has been standardized in RFC9347.

Link: https://www.rfc-editor.org/rfc/rfc9347.txt

This feature supports demand driven (i.e., non-constant send rate) IP-TFS to
take advantage of the AGGFRAG ESP payload encapsulation. This payload type
supports aggregation and fragmentation of the inner IP packet stream which in
turn yields higher small-packet bandwidth as well as reducing MTU/PMTU issues.
Congestion control is unimplementated as the send rate is demand driven rather
than constant.

In order to allow loading this fucntionality as a module a set of callbacks
xfrm_mode_cbs has been added to xfrm as well.

[RFC ipsec-next v2 1/8] iptfs: config: add CONFIG_XFRM_IPTFS

[Christian Hopps]

From: Christian Hopps <chopps@labn.net>

Add new Kconfig option to enable IP-TFS (RFC9347) functionality.

Signed-off-by: Christian Hopps <chopps@labn.net>
---
 net/xfrm/Kconfig | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/net/xfrm/Kconfig b/net/xfrm/Kconfig
index 3adf31a83a79..7a41d36bb080 100644
--- a/net/xfrm/Kconfig
+++ b/net/xfrm/Kconfig
@@ -134,6 +134,22 @@ config NET_KEY_MIGRATE
 
 	  If unsure, say N.
 
+config XFRM_IPTFS
+	bool "IPsec IP-TFS/AGGFRAG (RFC 9347) encapsulation support"
+	depends on XFRM
+	help
+	  Information on the IP-TFS/AGGFRAG encapsulation can be found
+	  in RFC 9347. This feature supports demand driven (i.e.,
+	  non-constant send rate) IP-TFS to take advantage of the
+	  AGGFRAG ESP payload encapsulation. This payload type
+	  supports aggregation and fragmentation of the inner IP
+	  packet stream which in turn yields higher small-packet
+	  bandwidth as well as reducing MTU/PMTU issues. Congestion
+	  control is unimplementated as the send rate is demand driven
+	  rather than constant.
+
+	  If unsure, say N.
+
 config XFRM_ESPINTCP
 	bool
 
-- 
2.42.0

[RFC ipsec-next v2 2/8] iptfs: uapi: ip: add ip_tfs_*_hdr packet formats

[Christian Hopps]

From: Christian Hopps <chopps@labn.net>

Add the on-wire basic and congestion-control IP-TFS packet headers.

Signed-off-by: Christian Hopps <chopps@labn.net>
---
 include/uapi/linux/ip.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/include/uapi/linux/ip.h b/include/uapi/linux/ip.h
index 283dec7e3645..cc83878ecf08 100644
--- a/include/uapi/linux/ip.h
+++ b/include/uapi/linux/ip.h
@@ -137,6 +137,23 @@ struct ip_beet_phdr {
 	__u8 reserved;
 };
 
+struct ip_iptfs_hdr {
+	__u8 subtype;		/* 0*: basic, 1: CC */
+	__u8 flags;
+	__be16 block_offset;
+};
+
+struct ip_iptfs_cc_hdr {
+	__u8 subtype;		/* 0: basic, 1*: CC */
+	__u8 flags;
+	__be16 block_offset;
+	__be32 loss_rate;
+	__u8 rtt_and_adelay1[4];
+	__u8 adelay2_and_xdelay[4];
+	__be32 tval;
+	__be32 techo;
+};
+
 /* index values for the variables in ipv4_devconf */
 enum
 {
-- 
2.42.0

[RFC ipsec-next v2 3/8] iptfs: uapi: IPPROTO_AGGFRAG AGGFRAG in ESP

[Christian Hopps]

From: Christian Hopps <chopps@labn.net>

Add the RFC assigned IP protocol number for AGGFRAG.

Signed-off-by: Christian Hopps <chopps@labn.net>
---
 include/uapi/linux/in.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/uapi/linux/in.h b/include/uapi/linux/in.h
index e682ab628dfa..e6a1f3e4c58c 100644
--- a/include/uapi/linux/in.h
+++ b/include/uapi/linux/in.h
@@ -79,6 +79,8 @@ enum {
 #define IPPROTO_MPLS		IPPROTO_MPLS
   IPPROTO_ETHERNET = 143,	/* Ethernet-within-IPv6 Encapsulation	*/
 #define IPPROTO_ETHERNET	IPPROTO_ETHERNET
+  IPPROTO_AGGFRAG = 144,	/* AGGFRAG in ESP (RFC 9347)		*/
+#define IPPROTO_AGGFRAG		IPPROTO_AGGFRAG
   IPPROTO_RAW = 255,		/* Raw IP packets			*/
 #define IPPROTO_RAW		IPPROTO_RAW
   IPPROTO_MPTCP = 262,		/* Multipath TCP connection		*/
-- 
2.42.0

[RFC ipsec-next v2 4/8] iptfs: sysctl: allow configuration of global default values

[Christian Hopps]

From: Christian Hopps <chopps@labn.net>

Add sysctls for the changing the IPTFS default SA values.

Signed-off-by: Christian Hopps <chopps@labn.net>
---
 Documentation/networking/xfrm_sysctl.rst | 29 ++++++++++++++++++
 include/net/netns/xfrm.h                 |  6 ++++
 include/net/xfrm.h                       |  7 +++++
 net/xfrm/xfrm_sysctl.c                   | 38 ++++++++++++++++++++++++
 4 files changed, 80 insertions(+)

diff --git a/Documentation/networking/xfrm_sysctl.rst b/Documentation/networking/xfrm_sysctl.rst
index 47b9bbdd0179..9e628806c110 100644
--- a/Documentation/networking/xfrm_sysctl.rst
+++ b/Documentation/networking/xfrm_sysctl.rst
@@ -9,3 +9,32 @@ XFRM Syscall
 
 xfrm_acq_expires - INTEGER
 	default 30 - hard timeout in seconds for acquire requests
+
+xfrm_iptfs_maxqsize - UNSIGNED INTEGER
+        The default IPTFS max output queue size in octets. The output queue is
+        where received packets destined for output over an IPTFS tunnel are
+        stored prior to being output in aggregated/fragmented form over the
+        IPTFS tunnel.
+
+        Default 1M.
+
+xfrm_iptfs_drptime - UNSIGNED INTEGER
+        The default IPTFS drop time in microseconds. The drop time is the amount
+        of time before a missing out-of-order IPTFS tunnel packet is considered
+        lost. See also the reorder window.
+
+        Default 1s (1000000).
+
+xfrm_iptfs_idelay - UNSIGNED INTEGER
+        The default IPTFS initial output delay in microseconds. The initial
+        output delay is the amount of time prior to servicing the output queue
+        after queueing the first packet on said queue.
+
+        Default 0.
+
+xfrm_iptfs_rewin - UNSIGNED INTEGER
+        The default IPTFS reorder window size. The reorder window size dictates
+        the maximum number of IPTFS tunnel packets in a sequence that may arrive
+        out of order.
+
+        Default 3.
diff --git a/include/net/netns/xfrm.h b/include/net/netns/xfrm.h
index bd7c3be4af5d..d5ad2155d0bb 100644
--- a/include/net/netns/xfrm.h
+++ b/include/net/netns/xfrm.h
@@ -65,6 +65,12 @@ struct netns_xfrm {
 	u32			sysctl_aevent_rseqth;
 	int			sysctl_larval_drop;
 	u32			sysctl_acq_expires;
+#if IS_ENABLED(CONFIG_XFRM_IPTFS)
+	u32			sysctl_iptfs_drptime;
+	u32			sysctl_iptfs_idelay;
+	u32			sysctl_iptfs_maxqsize;
+	u32			sysctl_iptfs_rewin;
+#endif
 
 	u8			policy_default[XFRM_POLICY_MAX];
 
diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index c9bb0f892f55..d2e87344d175 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -2190,4 +2190,11 @@ static inline int register_xfrm_interface_bpf(void)
 
 #endif
 
+#if IS_ENABLED(CONFIG_XFRM_IPTFS)
+#define XFRM_IPTFS_DEFAULT_MAX_QUEUE_SIZE (1024 * 1024)
+#define XFRM_IPTFS_DEFAULT_INIT_DELAY_USECS (0)
+#define XFRM_IPTFS_DEFAULT_DROP_TIME_USECS (1000000)
+#define XFRM_IPTFS_DEFAULT_REORDER_WINDOW (3)
+#endif
+
 #endif	/* _NET_XFRM_H */
diff --git a/net/xfrm/xfrm_sysctl.c b/net/xfrm/xfrm_sysctl.c
index 7fdeafc838a7..bf8e73a6c38e 100644
--- a/net/xfrm/xfrm_sysctl.c
+++ b/net/xfrm/xfrm_sysctl.c
@@ -10,6 +10,12 @@ static void __net_init __xfrm_sysctl_init(struct net *net)
 	net->xfrm.sysctl_aevent_rseqth = XFRM_AE_SEQT_SIZE;
 	net->xfrm.sysctl_larval_drop = 1;
 	net->xfrm.sysctl_acq_expires = 30;
+#if IS_ENABLED(CONFIG_XFRM_IPTFS)
+	net->xfrm.sysctl_iptfs_maxqsize = XFRM_IPTFS_DEFAULT_MAX_QUEUE_SIZE;
+	net->xfrm.sysctl_iptfs_drptime = XFRM_IPTFS_DEFAULT_DROP_TIME_USECS;
+	net->xfrm.sysctl_iptfs_idelay = XFRM_IPTFS_DEFAULT_INIT_DELAY_USECS;
+	net->xfrm.sysctl_iptfs_rewin = XFRM_IPTFS_DEFAULT_REORDER_WINDOW;
+#endif
 }
 
 #ifdef CONFIG_SYSCTL
@@ -38,6 +44,32 @@ static struct ctl_table xfrm_table[] = {
 		.mode		= 0644,
 		.proc_handler	= proc_dointvec
 	},
+#if IS_ENABLED(CONFIG_XFRM_IPTFS)
+	{
+		.procname	= "xfrm_iptfs_drptime",
+		.maxlen		= sizeof(uint),
+		.mode		= 0644,
+		.proc_handler	= proc_douintvec
+	},
+	{
+		.procname	= "xfrm_iptfs_idelay",
+		.maxlen		= sizeof(uint),
+		.mode		= 0644,
+		.proc_handler	= proc_douintvec
+	},
+	{
+		.procname	= "xfrm_iptfs_maxqsize",
+		.maxlen		= sizeof(uint),
+		.mode		= 0644,
+		.proc_handler	= proc_douintvec
+	},
+	{
+		.procname	= "xfrm_iptfs_rewin",
+		.maxlen		= sizeof(uint),
+		.mode		= 0644,
+		.proc_handler	= proc_douintvec
+	},
+#endif
 	{}
 };
 
@@ -55,6 +87,12 @@ int __net_init xfrm_sysctl_init(struct net *net)
 	table[1].data = &net->xfrm.sysctl_aevent_rseqth;
 	table[2].data = &net->xfrm.sysctl_larval_drop;
 	table[3].data = &net->xfrm.sysctl_acq_expires;
+#if IS_ENABLED(CONFIG_XFRM_IPTFS)
+	table[4].data = &net->xfrm.sysctl_iptfs_drptime;
+	table[5].data = &net->xfrm.sysctl_iptfs_idelay;
+	table[6].data = &net->xfrm.sysctl_iptfs_maxqsize;
+	table[7].data = &net->xfrm.sysctl_iptfs_rewin;
+#endif
 
 	/* Don't export sysctls to unprivileged users */
 	if (net->user_ns != &init_user_ns) {
-- 
2.42.0

[RFC ipsec-next v2 5/8] iptfs: netlink: add config (netlink) options

[Christian Hopps]

From: Christian Hopps <chopps@labn.net>

Add netlink options for configuring IP-TFS SAs.

Signed-off-by: Christian Hopps <chopps@labn.net>
---
 include/uapi/linux/xfrm.h |  9 ++++++++-
 net/xfrm/xfrm_compat.c    | 10 ++++++++--
 net/xfrm/xfrm_user.c      | 16 ++++++++++++++++
 3 files changed, 32 insertions(+), 3 deletions(-)

diff --git a/include/uapi/linux/xfrm.h b/include/uapi/linux/xfrm.h
index 6a77328be114..3e98d45d70a6 100644
--- a/include/uapi/linux/xfrm.h
+++ b/include/uapi/linux/xfrm.h
@@ -153,7 +153,8 @@ enum {
 #define XFRM_MODE_ROUTEOPTIMIZATION 2
 #define XFRM_MODE_IN_TRIGGER 3
 #define XFRM_MODE_BEET 4
-#define XFRM_MODE_MAX 5
+#define XFRM_MODE_IPTFS 5
+#define XFRM_MODE_MAX 6
 
 /* Netlink configuration messages.  */
 enum {
@@ -315,6 +316,12 @@ enum xfrm_attr_type_t {
 	XFRMA_SET_MARK_MASK,	/* __u32 */
 	XFRMA_IF_ID,		/* __u32 */
 	XFRMA_MTIMER_THRESH,	/* __u32 in seconds for input SA */
+	XFRMA_IPTFS_PKT_SIZE,	/* __u32 Size of outer packet, 0 for PMTU */
+	XFRMA_IPTFS_MAX_QSIZE,	/* __u32 max ingress queue size */
+	XFRMA_IPTFS_DONT_FRAG,	/* don't use fragmentation */
+	XFRMA_IPTFS_DROP_TIME,	/* __u32 usec to wait for next seq */
+	XFRMA_IPTFS_REORD_WIN,	/* __u16 reorder window size */
+	XFRMA_IPTFS_INIT_DELAY,	/* __u32 initial packet wait delay (usec) */
 	__XFRMA_MAX
 
 #define XFRMA_OUTPUT_MARK XFRMA_SET_MARK	/* Compatibility */
diff --git a/net/xfrm/xfrm_compat.c b/net/xfrm/xfrm_compat.c
index 655fe4ff8621..35b2abf33e54 100644
--- a/net/xfrm/xfrm_compat.c
+++ b/net/xfrm/xfrm_compat.c
@@ -277,9 +277,15 @@ static int xfrm_xlate64_attr(struct sk_buff *dst, const struct nlattr *src)
 	case XFRMA_SET_MARK_MASK:
 	case XFRMA_IF_ID:
 	case XFRMA_MTIMER_THRESH:
+	case XFRMA_IPTFS_PKT_SIZE:
+	case XFRMA_IPTFS_MAX_QSIZE:
+	case XFRMA_IPTFS_DONT_FRAG:
+	case XFRMA_IPTFS_DROP_TIME:
+	case XFRMA_IPTFS_REORD_WIN:
+	case XFRMA_IPTFS_INIT_DELAY:
 		return xfrm_nla_cpy(dst, src, nla_len(src));
 	default:
-		BUILD_BUG_ON(XFRMA_MAX != XFRMA_MTIMER_THRESH);
+		BUILD_BUG_ON(XFRMA_MAX != XFRMA_IPTFS_INIT_DELAY);
 		pr_warn_once("unsupported nla_type %d\n", src->nla_type);
 		return -EOPNOTSUPP;
 	}
@@ -434,7 +440,7 @@ static int xfrm_xlate32_attr(void *dst, const struct nlattr *nla,
 	int err;
 
 	if (type > XFRMA_MAX) {
-		BUILD_BUG_ON(XFRMA_MAX != XFRMA_MTIMER_THRESH);
+		BUILD_BUG_ON(XFRMA_MAX != XFRMA_IPTFS_INIT_DELAY);
 		NL_SET_ERR_MSG(extack, "Bad attribute");
 		return -EOPNOTSUPP;
 	}
diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c
index ad01997c3aa9..fa2059de51f5 100644
--- a/net/xfrm/xfrm_user.c
+++ b/net/xfrm/xfrm_user.c
@@ -272,6 +272,16 @@ static int verify_newsa_info(struct xfrm_usersa_info *p,
 			NL_SET_ERR_MSG(extack, "TFC padding can only be used in tunnel mode");
 			goto out;
 		}
+		if ((attrs[XFRMA_IPTFS_PKT_SIZE] ||
+		     attrs[XFRMA_IPTFS_MAX_QSIZE] ||
+		     attrs[XFRMA_IPTFS_DONT_FRAG] ||
+		     attrs[XFRMA_IPTFS_DROP_TIME] ||
+		     attrs[XFRMA_IPTFS_REORD_WIN] ||
+		     attrs[XFRMA_IPTFS_INIT_DELAY]) &&
+		    p->mode != XFRM_MODE_IPTFS) {
+			NL_SET_ERR_MSG(extack, "IPTFS options can only be used in IPTFS mode");
+			goto out;
+		}
 		break;
 
 	case IPPROTO_COMP:
@@ -3046,6 +3056,12 @@ const struct nla_policy xfrma_policy[XFRMA_MAX+1] = {
 	[XFRMA_SET_MARK_MASK]	= { .type = NLA_U32 },
 	[XFRMA_IF_ID]		= { .type = NLA_U32 },
 	[XFRMA_MTIMER_THRESH]   = { .type = NLA_U32 },
+	[XFRMA_IPTFS_PKT_SIZE]	= { .type = NLA_U32 },
+	[XFRMA_IPTFS_MAX_QSIZE]	= { .type = NLA_U32 },
+	[XFRMA_IPTFS_DONT_FRAG]	= { .type = NLA_FLAG },
+	[XFRMA_IPTFS_DROP_TIME]	= { .type = NLA_U32 },
+	[XFRMA_IPTFS_REORD_WIN]	= { .type = NLA_U16 },
+	[XFRMA_IPTFS_INIT_DELAY] = { .type = NLA_U32 },
 };
 EXPORT_SYMBOL_GPL(xfrma_policy);
 
-- 
2.42.0

[RFC ipsec-next v2 6/8] iptfs: xfrm: Add mode_cbs module functionality

[Christian Hopps]

From: Christian Hopps <chopps@labn.net>

Add a set of callbacks xfrm_mode_cbs to xfrm_state. These callbacks
enable the addition of new xfrm modes, such as IP-TFS to be defined
in modules.

Signed-off-by: Christian Hopps <chopps@labn.net>
---
 include/net/xfrm.h     | 36 ++++++++++++++++++++++++++++++++++++
 net/xfrm/xfrm_device.c |  3 ++-
 net/xfrm/xfrm_input.c  | 14 ++++++++++++--
 net/xfrm/xfrm_output.c |  9 +++++++--
 net/xfrm/xfrm_policy.c | 18 +++++++++++-------
 net/xfrm/xfrm_state.c  | 41 +++++++++++++++++++++++++++++++++++++++++
 net/xfrm/xfrm_user.c   | 10 ++++++++++
 7 files changed, 119 insertions(+), 12 deletions(-)

diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index d2e87344d175..aeeadadc9545 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -204,6 +204,7 @@ struct xfrm_state {
 		u16		family;
 		xfrm_address_t	saddr;
 		int		header_len;
+		int		enc_hdr_len;
 		int		trailer_len;
 		u32		extra_flags;
 		struct xfrm_mark	smark;
@@ -289,6 +290,9 @@ struct xfrm_state {
 	/* Private data of this transformer, format is opaque,
 	 * interpreted by xfrm_type methods. */
 	void			*data;
+
+	const struct xfrm_mode_cbs	*mode_cbs;
+	void				*mode_data;
 };
 
 static inline struct net *xs_net(struct xfrm_state *x)
@@ -441,6 +445,38 @@ struct xfrm_type_offload {
 int xfrm_register_type_offload(const struct xfrm_type_offload *type, unsigned short family);
 void xfrm_unregister_type_offload(const struct xfrm_type_offload *type, unsigned short family);
 
+struct xfrm_mode_cbs {
+	struct module	*owner;
+	/* Add/delete state in the new xfrm_state in `x`. */
+	int	(*create_state)(struct xfrm_state *x);
+	void	(*delete_state)(struct xfrm_state *x);
+
+	/* Called while handling the user netlink options. */
+	int	(*user_init)(struct net *net, struct xfrm_state *x,
+			     struct nlattr **attrs);
+	int	(*copy_to_user)(struct xfrm_state *x, struct sk_buff *skb);
+
+	u32	(*get_inner_mtu)(struct xfrm_state *x, int outer_mtu);
+
+	/* Called to handle received xfrm (egress) packets. */
+	int	(*input)(struct xfrm_state *x, struct sk_buff *skb);
+
+	/* Placed in dst_output of the dst when an xfrm_state is bound. */
+	int	(*output)(struct net *net, struct sock *sk, struct sk_buff *skb);
+
+	/**
+	 * Prepare the skb for output for the given mode. Returns:
+	 *    Error value, if 0 then skb values should be as follows:
+	 *    transport_header should point at ESP header
+	 *    network_header should point at Outer IP header
+	 *    mac_header should point at protocol/nexthdr of the outer IP
+	 */
+	int	(*prepare_output)(struct xfrm_state *x, struct sk_buff *skb);
+};
+
+int xfrm_register_mode_cbs(u8 mode, const struct xfrm_mode_cbs *mode_cbs);
+void xfrm_unregister_mode_cbs(u8 mode);
+
 static inline int xfrm_af2proto(unsigned int family)
 {
 	switch(family) {
diff --git a/net/xfrm/xfrm_device.c b/net/xfrm/xfrm_device.c
index 3784534c9185..8b848540ea47 100644
--- a/net/xfrm/xfrm_device.c
+++ b/net/xfrm/xfrm_device.c
@@ -42,7 +42,8 @@ static void __xfrm_mode_tunnel_prep(struct xfrm_state *x, struct sk_buff *skb,
 		skb->transport_header = skb->network_header + hsize;
 
 	skb_reset_mac_len(skb);
-	pskb_pull(skb, skb->mac_len + x->props.header_len);
+	pskb_pull(skb,
+		  skb->mac_len + x->props.header_len - x->props.enc_hdr_len);
 }
 
 static void __xfrm_mode_beet_prep(struct xfrm_state *x, struct sk_buff *skb,
diff --git a/net/xfrm/xfrm_input.c b/net/xfrm/xfrm_input.c
index bd4ce21d76d7..824f7b7f90e0 100644
--- a/net/xfrm/xfrm_input.c
+++ b/net/xfrm/xfrm_input.c
@@ -437,6 +437,9 @@ static int xfrm_inner_mode_input(struct xfrm_state *x,
 		WARN_ON_ONCE(1);
 		break;
 	default:
+		if (x->mode_cbs && x->mode_cbs->input)
+			return x->mode_cbs->input(x, skb);
+
 		WARN_ON_ONCE(1);
 		break;
 	}
@@ -479,6 +482,10 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
 
 		family = x->props.family;
 
+		/* An encap_type of -3 indicates reconstructed inner packet */
+		if (encap_type == -3)
+			goto resume_decapped;
+
 		/* An encap_type of -1 indicates async resumption. */
 		if (encap_type == -1) {
 			async = 1;
@@ -660,11 +667,14 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
 
 		XFRM_MODE_SKB_CB(skb)->protocol = nexthdr;
 
-		if (xfrm_inner_mode_input(x, skb)) {
+		err = xfrm_inner_mode_input(x, skb);
+		if (err == -EINPROGRESS)
+			return 0;
+		else if (err) {
 			XFRM_INC_STATS(net, LINUX_MIB_XFRMINSTATEMODEERROR);
 			goto drop;
 		}
-
+resume_decapped:
 		if (x->outer_mode.flags & XFRM_MODE_FLAG_TUNNEL) {
 			decaps = 1;
 			break;
diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
index 662c83beb345..4390c111410d 100644
--- a/net/xfrm/xfrm_output.c
+++ b/net/xfrm/xfrm_output.c
@@ -280,7 +280,9 @@ static int xfrm4_tunnel_encap_add(struct xfrm_state *x, struct sk_buff *skb)
 	skb_set_inner_network_header(skb, skb_network_offset(skb));
 	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
 
-	skb_set_network_header(skb, -x->props.header_len);
+	/* backup to add space for the outer encap */
+	skb_set_network_header(skb,
+			       -x->props.header_len + x->props.enc_hdr_len);
 	skb->mac_header = skb->network_header +
 			  offsetof(struct iphdr, protocol);
 	skb->transport_header = skb->network_header + sizeof(*top_iph);
@@ -325,7 +327,8 @@ static int xfrm6_tunnel_encap_add(struct xfrm_state *x, struct sk_buff *skb)
 	skb_set_inner_network_header(skb, skb_network_offset(skb));
 	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
 
-	skb_set_network_header(skb, -x->props.header_len);
+	skb_set_network_header(skb,
+			       -x->props.header_len + x->props.enc_hdr_len);
 	skb->mac_header = skb->network_header +
 			  offsetof(struct ipv6hdr, nexthdr);
 	skb->transport_header = skb->network_header + sizeof(*top_iph);
@@ -472,6 +475,8 @@ static int xfrm_outer_mode_output(struct xfrm_state *x, struct sk_buff *skb)
 		WARN_ON_ONCE(1);
 		break;
 	default:
+		if (x->mode_cbs->prepare_output)
+			return x->mode_cbs->prepare_output(x, skb);
 		WARN_ON_ONCE(1);
 		break;
 	}
diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
index d2dddc570f4f..3220b01121f3 100644
--- a/net/xfrm/xfrm_policy.c
+++ b/net/xfrm/xfrm_policy.c
@@ -2707,13 +2707,17 @@ static struct dst_entry *xfrm_bundle_create(struct xfrm_policy *policy,
 
 		dst1->input = dst_discard;
 
-		rcu_read_lock();
-		afinfo = xfrm_state_afinfo_get_rcu(inner_mode->family);
-		if (likely(afinfo))
-			dst1->output = afinfo->output;
-		else
-			dst1->output = dst_discard_out;
-		rcu_read_unlock();
+		if (xfrm[i]->mode_cbs && xfrm[i]->mode_cbs->output) {
+			dst1->output = xfrm[i]->mode_cbs->output;
+		} else {
+			rcu_read_lock();
+			afinfo = xfrm_state_afinfo_get_rcu(inner_mode->family);
+			if (likely(afinfo))
+				dst1->output = afinfo->output;
+			else
+				dst1->output = dst_discard_out;
+			rcu_read_unlock();
+		}
 
 		xdst_prev = xdst;
 
diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
index bda5327bf34d..f5e1a17ebf74 100644
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -513,6 +513,36 @@ static const struct xfrm_mode *xfrm_get_mode(unsigned int encap, int family)
 	return NULL;
 }
 
+static struct xfrm_mode_cbs xfrm_mode_cbs_map[XFRM_MODE_MAX];
+
+int xfrm_register_mode_cbs(u8 mode, const struct xfrm_mode_cbs *mode_cbs)
+{
+	if (mode >= XFRM_MODE_MAX)
+		return -EINVAL;
+
+	xfrm_mode_cbs_map[mode] = *mode_cbs;
+	return 0;
+}
+EXPORT_SYMBOL(xfrm_register_mode_cbs);
+
+void xfrm_unregister_mode_cbs(u8 mode)
+{
+	if (mode >= XFRM_MODE_MAX)
+		return;
+
+	memset(&xfrm_mode_cbs_map[mode], 0, sizeof(xfrm_mode_cbs_map[mode]));
+}
+EXPORT_SYMBOL(xfrm_unregister_mode_cbs);
+
+static const struct xfrm_mode_cbs *xfrm_get_mode_cbs(u8 mode)
+{
+	if (mode >= XFRM_MODE_MAX)
+		return NULL;
+	if (mode == XFRM_MODE_IPTFS && !xfrm_mode_cbs_map[mode].create_state)
+		request_module("xfrm-iptfs");
+	return &xfrm_mode_cbs_map[mode];
+}
+
 void xfrm_state_free(struct xfrm_state *x)
 {
 	kmem_cache_free(xfrm_state_cache, x);
@@ -521,6 +551,8 @@ EXPORT_SYMBOL(xfrm_state_free);
 
 static void ___xfrm_state_destroy(struct xfrm_state *x)
 {
+	if (x->mode_cbs && x->mode_cbs->delete_state)
+		x->mode_cbs->delete_state(x);
 	hrtimer_cancel(&x->mtimer);
 	del_timer_sync(&x->rtimer);
 	kfree(x->aead);
@@ -2765,6 +2797,9 @@ u32 xfrm_state_mtu(struct xfrm_state *x, int mtu)
 	case XFRM_MODE_TUNNEL:
 		break;
 	default:
+		if (x->mode_cbs && x->mode_cbs->get_inner_mtu)
+			return x->mode_cbs->get_inner_mtu(x, mtu);
+
 		WARN_ON_ONCE(1);
 		break;
 	}
@@ -2850,6 +2885,12 @@ int __xfrm_init_state(struct xfrm_state *x, bool init_replay, bool offload,
 			goto error;
 	}
 
+	x->mode_cbs = xfrm_get_mode_cbs(x->props.mode);
+	if (x->mode_cbs && x->mode_cbs->create_state) {
+		err = x->mode_cbs->create_state(x);
+		if (err)
+			goto error;
+	}
 error:
 	return err;
 }
diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c
index fa2059de51f5..795da945fbc2 100644
--- a/net/xfrm/xfrm_user.c
+++ b/net/xfrm/xfrm_user.c
@@ -779,6 +779,12 @@ static struct xfrm_state *xfrm_state_construct(struct net *net,
 			goto error;
 	}
 
+	if (x->mode_cbs && x->mode_cbs->user_init) {
+		err = x->mode_cbs->user_init(net, x, attrs);
+		if (err)
+			goto error;
+	}
+
 	return x;
 
 error:
@@ -1192,6 +1198,10 @@ static int copy_to_user_state_extra(struct xfrm_state *x,
 		if (ret)
 			goto out;
 	}
+	if (x->mode_cbs && x->mode_cbs->copy_to_user)
+		ret = x->mode_cbs->copy_to_user(x, skb);
+	if (ret)
+		goto out;
 	if (x->mapping_maxage)
 		ret = nla_put_u32(skb, XFRMA_MTIMER_THRESH, x->mapping_maxage);
 out:
-- 
2.42.0

[RFC ipsec-next v2 7/8] iptfs: xfrm: add generic iptfs defines and functionality

[Christian Hopps]

From: Christian Hopps <chopps@labn.net>

Define `XFRM_MODE_IPTFS` and `IPSEC_MODE_IPTFS` constants, and add these to
switch case and conditionals adjacent with the existing TUNNEL modes.

Signed-off-by: Christian Hopps <chopps@labn.net>
---
 include/net/xfrm.h         |  1 +
 include/uapi/linux/ipsec.h |  3 ++-
 include/uapi/linux/snmp.h  |  2 ++
 net/ipv4/esp4.c            |  3 ++-
 net/ipv6/esp6.c            |  3 ++-
 net/netfilter/nft_xfrm.c   |  3 ++-
 net/xfrm/xfrm_device.c     |  1 +
 net/xfrm/xfrm_output.c     |  4 ++++
 net/xfrm/xfrm_policy.c     |  8 ++++++--
 net/xfrm/xfrm_proc.c       |  2 ++
 net/xfrm/xfrm_state.c      | 12 ++++++++++++
 net/xfrm/xfrm_user.c       |  3 +++
 12 files changed, 39 insertions(+), 6 deletions(-)

diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index aeeadadc9545..a6e0e848918d 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -37,6 +37,7 @@
 #define XFRM_PROTO_COMP		108
 #define XFRM_PROTO_IPIP		4
 #define XFRM_PROTO_IPV6		41
+#define XFRM_PROTO_IPTFS	IPPROTO_AGGFRAG
 #define XFRM_PROTO_ROUTING	IPPROTO_ROUTING
 #define XFRM_PROTO_DSTOPTS	IPPROTO_DSTOPTS
 
diff --git a/include/uapi/linux/ipsec.h b/include/uapi/linux/ipsec.h
index 50d8ee1791e2..696b790f4346 100644
--- a/include/uapi/linux/ipsec.h
+++ b/include/uapi/linux/ipsec.h
@@ -14,7 +14,8 @@ enum {
 	IPSEC_MODE_ANY		= 0,	/* We do not support this for SA */
 	IPSEC_MODE_TRANSPORT	= 1,
 	IPSEC_MODE_TUNNEL	= 2,
-	IPSEC_MODE_BEET         = 3
+	IPSEC_MODE_BEET         = 3,
+	IPSEC_MODE_IPTFS        = 4
 };
 
 enum {
diff --git a/include/uapi/linux/snmp.h b/include/uapi/linux/snmp.h
index 26f33a4c253d..d0b45f4c22c7 100644
--- a/include/uapi/linux/snmp.h
+++ b/include/uapi/linux/snmp.h
@@ -331,6 +331,8 @@ enum
 	LINUX_MIB_XFRMFWDHDRERROR,		/* XfrmFwdHdrError*/
 	LINUX_MIB_XFRMOUTSTATEINVALID,		/* XfrmOutStateInvalid */
 	LINUX_MIB_XFRMACQUIREERROR,		/* XfrmAcquireError */
+	LINUX_MIB_XFRMINIPTFSERROR,		/* XfrmInIptfsError */
+	LINUX_MIB_XFRMOUTNOQSPACE,		/* XfrmOutNoQueueSpace */
 	__LINUX_MIB_XFRMMAX
 };
 
diff --git a/net/ipv4/esp4.c b/net/ipv4/esp4.c
index 2be2d4922557..b7047c0dd7ea 100644
--- a/net/ipv4/esp4.c
+++ b/net/ipv4/esp4.c
@@ -816,7 +816,8 @@ int esp_input_done2(struct sk_buff *skb, int err)
 	}
 
 	skb_pull_rcsum(skb, hlen);
-	if (x->props.mode == XFRM_MODE_TUNNEL)
+	if (x->props.mode == XFRM_MODE_TUNNEL ||
+	    x->props.mode == XFRM_MODE_IPTFS)
 		skb_reset_transport_header(skb);
 	else
 		skb_set_transport_header(skb, -ihl);
diff --git a/net/ipv6/esp6.c b/net/ipv6/esp6.c
index fddd0cbdede1..10f2190207a8 100644
--- a/net/ipv6/esp6.c
+++ b/net/ipv6/esp6.c
@@ -865,7 +865,8 @@ int esp6_input_done2(struct sk_buff *skb, int err)
 	skb_postpull_rcsum(skb, skb_network_header(skb),
 			   skb_network_header_len(skb));
 	skb_pull_rcsum(skb, hlen);
-	if (x->props.mode == XFRM_MODE_TUNNEL)
+	if (x->props.mode == XFRM_MODE_TUNNEL ||
+	    x->props.mode == XFRM_MODE_IPTFS)
 		skb_reset_transport_header(skb);
 	else
 		skb_set_transport_header(skb, -hdr_len);
diff --git a/net/netfilter/nft_xfrm.c b/net/netfilter/nft_xfrm.c
index 452f8587adda..291b029391cd 100644
--- a/net/netfilter/nft_xfrm.c
+++ b/net/netfilter/nft_xfrm.c
@@ -112,7 +112,8 @@ static bool xfrm_state_addr_ok(enum nft_xfrm_keys k, u8 family, u8 mode)
 		return true;
 	}
 
-	return mode == XFRM_MODE_BEET || mode == XFRM_MODE_TUNNEL;
+	return mode == XFRM_MODE_BEET || mode == XFRM_MODE_TUNNEL ||
+	       mode == XFRM_MODE_IPTFS;
 }
 
 static void nft_xfrm_state_get_key(const struct nft_xfrm *priv,
diff --git a/net/xfrm/xfrm_device.c b/net/xfrm/xfrm_device.c
index 8b848540ea47..a40f5e09829e 100644
--- a/net/xfrm/xfrm_device.c
+++ b/net/xfrm/xfrm_device.c
@@ -69,6 +69,7 @@ static void __xfrm_mode_beet_prep(struct xfrm_state *x, struct sk_buff *skb,
 static void xfrm_outer_mode_prep(struct xfrm_state *x, struct sk_buff *skb)
 {
 	switch (x->outer_mode.encap) {
+	case XFRM_MODE_IPTFS:
 	case XFRM_MODE_TUNNEL:
 		if (x->outer_mode.family == AF_INET)
 			return __xfrm_mode_tunnel_prep(x, skb,
diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
index 4390c111410d..16c981ca61ca 100644
--- a/net/xfrm/xfrm_output.c
+++ b/net/xfrm/xfrm_output.c
@@ -680,6 +680,10 @@ static void xfrm_get_inner_ipproto(struct sk_buff *skb, struct xfrm_state *x)
 
 		return;
 	}
+	if (x->outer_mode.encap == XFRM_MODE_IPTFS) {
+		xo->inner_ipproto = IPPROTO_AGGFRAG;
+		return;
+	}
 
 	/* non-Tunnel Mode */
 	if (!skb->encapsulation)
diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
index 3220b01121f3..94e5889a77d6 100644
--- a/net/xfrm/xfrm_policy.c
+++ b/net/xfrm/xfrm_policy.c
@@ -2468,6 +2468,7 @@ xfrm_tmpl_resolve_one(struct xfrm_policy *policy, const struct flowi *fl,
 		struct xfrm_tmpl *tmpl = &policy->xfrm_vec[i];
 
 		if (tmpl->mode == XFRM_MODE_TUNNEL ||
+		    tmpl->mode == XFRM_MODE_IPTFS ||
 		    tmpl->mode == XFRM_MODE_BEET) {
 			remote = &tmpl->id.daddr;
 			local = &tmpl->saddr;
@@ -3252,7 +3253,8 @@ struct dst_entry *xfrm_lookup_with_ifid(struct net *net,
 ok:
 	xfrm_pols_put(pols, drop_pols);
 	if (dst && dst->xfrm &&
-	    dst->xfrm->props.mode == XFRM_MODE_TUNNEL)
+	    (dst->xfrm->props.mode == XFRM_MODE_TUNNEL ||
+	     dst->xfrm->props.mode == XFRM_MODE_IPTFS))
 		dst->flags |= DST_XFRM_TUNNEL;
 	return dst;
 
@@ -4353,6 +4355,7 @@ static int migrate_tmpl_match(const struct xfrm_migrate *m, const struct xfrm_tm
 		switch (t->mode) {
 		case XFRM_MODE_TUNNEL:
 		case XFRM_MODE_BEET:
+		case XFRM_MODE_IPTFS:
 			if (xfrm_addr_equal(&t->id.daddr, &m->old_daddr,
 					    m->old_family) &&
 			    xfrm_addr_equal(&t->saddr, &m->old_saddr,
@@ -4395,7 +4398,8 @@ static int xfrm_policy_migrate(struct xfrm_policy *pol,
 				continue;
 			n++;
 			if (pol->xfrm_vec[i].mode != XFRM_MODE_TUNNEL &&
-			    pol->xfrm_vec[i].mode != XFRM_MODE_BEET)
+			    pol->xfrm_vec[i].mode != XFRM_MODE_BEET &&
+			    pol->xfrm_vec[i].mode != XFRM_MODE_IPTFS)
 				continue;
 			/* update endpoints */
 			memcpy(&pol->xfrm_vec[i].id.daddr, &mp->new_daddr,
diff --git a/net/xfrm/xfrm_proc.c b/net/xfrm/xfrm_proc.c
index fee9b5cf37a7..d92b1b760749 100644
--- a/net/xfrm/xfrm_proc.c
+++ b/net/xfrm/xfrm_proc.c
@@ -41,6 +41,8 @@ static const struct snmp_mib xfrm_mib_list[] = {
 	SNMP_MIB_ITEM("XfrmFwdHdrError", LINUX_MIB_XFRMFWDHDRERROR),
 	SNMP_MIB_ITEM("XfrmOutStateInvalid", LINUX_MIB_XFRMOUTSTATEINVALID),
 	SNMP_MIB_ITEM("XfrmAcquireError", LINUX_MIB_XFRMACQUIREERROR),
+	SNMP_MIB_ITEM("XfrmInIptfsError", LINUX_MIB_XFRMINIPTFSERROR),
+	SNMP_MIB_ITEM("XfrmOutNoQueueSpace", LINUX_MIB_XFRMOUTNOQSPACE),
 	SNMP_MIB_SENTINEL
 };
 
diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
index f5e1a17ebf74..786f3fc0d428 100644
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -465,6 +465,11 @@ static const struct xfrm_mode xfrm4_mode_map[XFRM_MODE_MAX] = {
 		.flags = XFRM_MODE_FLAG_TUNNEL,
 		.family = AF_INET,
 	},
+	[XFRM_MODE_IPTFS] = {
+		.encap = XFRM_MODE_IPTFS,
+		.flags = XFRM_MODE_FLAG_TUNNEL,
+		.family = AF_INET,
+	},
 };
 
 static const struct xfrm_mode xfrm6_mode_map[XFRM_MODE_MAX] = {
@@ -486,6 +491,11 @@ static const struct xfrm_mode xfrm6_mode_map[XFRM_MODE_MAX] = {
 		.flags = XFRM_MODE_FLAG_TUNNEL,
 		.family = AF_INET6,
 	},
+	[XFRM_MODE_IPTFS] = {
+		.encap = XFRM_MODE_IPTFS,
+		.flags = XFRM_MODE_FLAG_TUNNEL,
+		.family = AF_INET6,
+	},
 };
 
 static const struct xfrm_mode *xfrm_get_mode(unsigned int encap, int family)
@@ -2083,6 +2093,7 @@ static int __xfrm6_state_sort_cmp(const void *p)
 #endif
 	case XFRM_MODE_TUNNEL:
 	case XFRM_MODE_BEET:
+	case XFRM_MODE_IPTFS:
 		return 4;
 	}
 	return 5;
@@ -2109,6 +2120,7 @@ static int __xfrm6_tmpl_sort_cmp(const void *p)
 #endif
 	case XFRM_MODE_TUNNEL:
 	case XFRM_MODE_BEET:
+	case XFRM_MODE_IPTFS:
 		return 3;
 	}
 	return 4;
diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c
index 795da945fbc2..4b0e462d5e31 100644
--- a/net/xfrm/xfrm_user.c
+++ b/net/xfrm/xfrm_user.c
@@ -353,6 +353,7 @@ static int verify_newsa_info(struct xfrm_usersa_info *p,
 	case XFRM_MODE_TUNNEL:
 	case XFRM_MODE_ROUTEOPTIMIZATION:
 	case XFRM_MODE_BEET:
+	case XFRM_MODE_IPTFS:
 		break;
 
 	default:
@@ -1830,6 +1831,8 @@ static int validate_tmpl(int nr, struct xfrm_user_tmpl *ut, u16 family,
 				return -EINVAL;
 			}
 			break;
+		case XFRM_MODE_IPTFS:
+			break;
 		default:
 			if (ut[i].family != prev_family) {
 				NL_SET_ERR_MSG(extack, "Mode in template doesn't support a family change");
-- 
2.42.0

[RFC ipsec-next v2 8/8] iptfs: impl: add new iptfs xfrm mode impl

[Christian Hopps]

From: Christian Hopps <chopps@labn.net>

Add a new xfrm mode implementing AggFrag/IP-TFS from RFC9347.

This utilizes the new xfrm_mode_cbs to implement demand-driven IP-TFS
functionality. This functionality can be used to increase bandwidth
utilization through small packet aggregation, as well as help solve PMTU
issues through it's efficient use of fragmentation.

Link: https://www.rfc-editor.org/rfc/rfc9347.txt

Signed-off-by: Christian Hopps <chopps@labn.net>
---
 include/net/iptfs.h    |   18 +
 net/xfrm/Makefile      |    1 +
 net/xfrm/trace_iptfs.h |  224 ++++
 net/xfrm/xfrm_iptfs.c  | 2696 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 2939 insertions(+)
 create mode 100644 include/net/iptfs.h
 create mode 100644 net/xfrm/trace_iptfs.h
 create mode 100644 net/xfrm/xfrm_iptfs.c

diff --git a/include/net/iptfs.h b/include/net/iptfs.h
new file mode 100644
index 000000000000..d8f2e494f251
--- /dev/null
+++ b/include/net/iptfs.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _NET_IPTFS_H
+#define _NET_IPTFS_H
+
+#include <linux/types.h>
+#include <linux/ip.h>
+
+#define IPTFS_SUBTYPE_BASIC 0
+#define IPTFS_SUBTYPE_CC 1
+#define IPTFS_SUBTYPE_LAST IPTFS_SUBTYPE_CC
+
+#define IPTFS_CC_FLAGS_ECN_CE 0x1
+#define IPTFS_CC_FLAGS_PLMTUD 0x2
+
+extern void xfrm_iptfs_get_rtt_and_delays(struct ip_iptfs_cc_hdr *cch, u32 *rtt,
+					  u32 *actual_delay, u32 *xmit_delay);
+
+#endif /* _NET_IPTFS_H */
diff --git a/net/xfrm/Makefile b/net/xfrm/Makefile
index cd47f88921f5..9b870a3274a7 100644
--- a/net/xfrm/Makefile
+++ b/net/xfrm/Makefile
@@ -20,4 +20,5 @@ obj-$(CONFIG_XFRM_USER) += xfrm_user.o
 obj-$(CONFIG_XFRM_USER_COMPAT) += xfrm_compat.o
 obj-$(CONFIG_XFRM_IPCOMP) += xfrm_ipcomp.o
 obj-$(CONFIG_XFRM_INTERFACE) += xfrm_interface.o
+obj-$(CONFIG_XFRM_IPTFS) += xfrm_iptfs.o
 obj-$(CONFIG_XFRM_ESPINTCP) += espintcp.o
diff --git a/net/xfrm/trace_iptfs.h b/net/xfrm/trace_iptfs.h
new file mode 100644
index 000000000000..994b6dafd1ef
--- /dev/null
+++ b/net/xfrm/trace_iptfs.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* xfrm_trace_iptfs.h
+ *
+ * August 12 2023, Christian Hopps <chopps@labn.net>
+ *
+ * Copyright (c) 2023, LabN Consulting, L.L.C.
+ */
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM iptfs
+
+#if !defined(_TRACE_IPTFS_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_IPTFS_H
+
+#include <linux/kernel.h>
+#include <linux/skbuff.h>
+#include <linux/tracepoint.h>
+#include <net/ip.h>
+
+struct xfrm_iptfs_data;
+
+TRACE_EVENT(iptfs_egress_recv,
+	    TP_PROTO(struct sk_buff *skb, struct xfrm_iptfs_data *xtfs, u16 blkoff),
+	    TP_ARGS(skb, xtfs, blkoff),
+	    TP_STRUCT__entry(
+		    __field(struct sk_buff *, skb)
+		    __field(void *, head)
+		    __field(void *, head_pg_addr)
+		    __field(void *, pg0addr)
+		    __field(u32, skb_len)
+		    __field(u32, data_len)
+		    __field(u32, headroom)
+		    __field(u32, tailroom)
+		    __field(u32, tail)
+		    __field(u32, end)
+		    __field(u32, pg0off)
+		    __field(u8, head_frag)
+		    __field(u8, frag_list)
+		    __field(u8, nr_frags)
+		    __field(u16, blkoff)),
+	    TP_fast_assign(
+		    __entry->skb = skb;
+		    __entry->head = skb->head;
+		    __entry->skb_len = skb->len;
+		    __entry->data_len = skb->data_len;
+		    __entry->headroom = skb_headroom(skb);
+		    __entry->tailroom = skb_tailroom(skb);
+		    __entry->tail = skb->tail;
+		    __entry->end = skb->end;
+		    __entry->head_frag = skb->head_frag;
+		    __entry->frag_list = (bool)skb_shinfo(skb)->frag_list;
+		    __entry->nr_frags = skb_shinfo(skb)->nr_frags;
+		    __entry->blkoff = blkoff;
+		    __entry->head_pg_addr = page_address(virt_to_head_page(skb->head));
+		    __entry->pg0addr = (__entry->nr_frags
+					? page_address(skb_shinfo(skb)->frags[0].bv_page)
+					: 0);
+		    __entry->pg0off = (__entry->nr_frags ? skb_shinfo(skb)->frags[0].bv_offset : 0);
+
+		    ),
+	    TP_printk("EGRESS: skb=%p len=%u data_len=%u headroom=%u head_frag=%u frag_list=%u nr_frags=%u blkoff=%u\n\t\ttailroom=%u tail=%u end=%u head=%p hdpgaddr=%p pg0->addr=%p pg0->data=%p pg0->off=%u",
+		      __entry->skb, __entry->skb_len, __entry->data_len, __entry->headroom,
+		      __entry->head_frag, __entry->frag_list, __entry->nr_frags, __entry->blkoff,
+		      __entry->tailroom, __entry->tail, __entry->end, __entry->head,
+		      __entry->head_pg_addr, __entry->pg0addr, __entry->pg0addr + __entry->pg0off,
+		      __entry->pg0off)
+	)
+
+DECLARE_EVENT_CLASS(iptfs_ingress_preq_event,
+		    TP_PROTO(struct sk_buff *skb, struct xfrm_iptfs_data *xtfs,
+			     u32 pmtu, u8 was_gso),
+		    TP_ARGS(skb, xtfs, pmtu, was_gso),
+		    TP_STRUCT__entry(
+			__field(struct sk_buff *, skb)
+			__field(u32, skb_len)
+			__field(u32, data_len)
+			__field(u32, pmtu)
+			__field(u32, queue_size)
+			__field(u32, proto_seq)
+			__field(u8, proto)
+			__field(u8, was_gso)
+			    ),
+		    TP_fast_assign(
+			    __entry->skb = skb;
+			    __entry->skb_len = skb->len;
+			    __entry->data_len = skb->data_len;
+			    __entry->queue_size = xtfs->cfg.max_queue_size - xtfs->queue_size;
+			    __entry->proto = __trace_ip_proto(ip_hdr(skb));
+			    __entry->proto_seq = __trace_ip_proto_seq(ip_hdr(skb));
+			    __entry->pmtu = pmtu;
+			__entry->was_gso = was_gso;
+			    ),
+		    TP_printk("INGRPREQ: skb=%p len=%u data_len=%u qsize=%u proto=%u proto_seq=%u pmtu=%u was_gso=%u",
+			      __entry->skb, __entry->skb_len, __entry->data_len,
+			      __entry->queue_size, __entry->proto, __entry->proto_seq,
+			      __entry->pmtu, __entry->was_gso));
+
+DEFINE_EVENT(iptfs_ingress_preq_event, iptfs_enqueue,
+	     TP_PROTO(struct sk_buff *skb, struct xfrm_iptfs_data *xtfs, u32 pmtu, u8 was_gso),
+	     TP_ARGS(skb, xtfs, pmtu, was_gso));
+
+DEFINE_EVENT(iptfs_ingress_preq_event, iptfs_no_queue_space,
+	     TP_PROTO(struct sk_buff *skb, struct xfrm_iptfs_data *xtfs, u32 pmtu, u8 was_gso),
+	     TP_ARGS(skb, xtfs, pmtu, was_gso));
+
+DEFINE_EVENT(iptfs_ingress_preq_event, iptfs_too_big,
+	     TP_PROTO(struct sk_buff *skb, struct xfrm_iptfs_data *xtfs, u32 pmtu, u8 was_gso),
+	     TP_ARGS(skb, xtfs, pmtu, was_gso));
+
+DECLARE_EVENT_CLASS(
+	iptfs_ingress_postq_event,
+	TP_PROTO(struct sk_buff *skb, u32 mtu, u16 blkoff, struct iphdr *iph),
+	TP_ARGS(skb, mtu, blkoff, iph),
+	TP_STRUCT__entry(__field(struct sk_buff *, skb)
+			 __field(u32, skb_len)
+			 __field(u32, data_len)
+			 __field(u32, mtu)
+			 __field(u32, proto_seq)
+			 __field(u16, blkoff)
+			 __field(u8, proto)),
+	TP_fast_assign(__entry->skb = skb;
+		       __entry->skb_len = skb->len;
+		       __entry->data_len = skb->data_len;
+		       __entry->mtu = mtu;
+		       __entry->blkoff = blkoff;
+		       __entry->proto = iph ? __trace_ip_proto(iph) : 0;
+		       __entry->proto_seq = iph ? __trace_ip_proto_seq(iph) : 0;
+		),
+	TP_printk(
+		"INGRPSTQ: skb=%p len=%u data_len=%u mtu=%u blkoff=%u proto=%u proto_seq=%u",
+		__entry->skb, __entry->skb_len, __entry->data_len, __entry->mtu,
+		__entry->blkoff, __entry->proto, __entry->proto_seq));
+
+DEFINE_EVENT(iptfs_ingress_postq_event, iptfs_first_dequeue,
+	     TP_PROTO(struct sk_buff *skb, u32 mtu, u16 blkoff,
+		      struct iphdr *iph),
+	     TP_ARGS(skb, mtu, blkoff, iph));
+
+DEFINE_EVENT(iptfs_ingress_postq_event, iptfs_first_fragmenting,
+	     TP_PROTO(struct sk_buff *skb, u32 mtu, u16 blkoff,
+		      struct iphdr *iph),
+	     TP_ARGS(skb, mtu, blkoff, iph));
+
+DEFINE_EVENT(iptfs_ingress_postq_event, iptfs_first_final_fragment,
+	     TP_PROTO(struct sk_buff *skb, u32 mtu, u16 blkoff,
+		      struct iphdr *iph),
+	     TP_ARGS(skb, mtu, blkoff, iph));
+
+DEFINE_EVENT(iptfs_ingress_postq_event, iptfs_first_toobig,
+	     TP_PROTO(struct sk_buff *skb, u32 mtu, u16 blkoff,
+		      struct iphdr *iph),
+	     TP_ARGS(skb, mtu, blkoff, iph));
+
+TRACE_EVENT(iptfs_ingress_nth_peek,
+	    TP_PROTO(struct sk_buff *skb, u32 remaining),
+	    TP_ARGS(skb, remaining),
+	    TP_STRUCT__entry(__field(struct sk_buff *, skb)
+			     __field(u32, skb_len)
+			     __field(u32, remaining)),
+	    TP_fast_assign(__entry->skb = skb;
+			   __entry->skb_len = skb->len;
+			   __entry->remaining = remaining;
+		    ),
+	    TP_printk("INGRPSTQ: NTHPEEK: skb=%p len=%u remaining=%u",
+		      __entry->skb, __entry->skb_len, __entry->remaining));
+
+TRACE_EVENT(iptfs_ingress_nth_add, TP_PROTO(struct sk_buff *skb, u8 share_ok),
+	    TP_ARGS(skb, share_ok),
+	    TP_STRUCT__entry(__field(struct sk_buff *, skb)
+			     __field(u32, skb_len)
+			     __field(u32, data_len)
+			     __field(u8, share_ok)
+			     __field(u8, head_frag)
+			     __field(u8, pp_recycle)
+			     __field(u8, cloned)
+			     __field(u8, shared)
+			     __field(u8, nr_frags)
+			     __field(u8, frag_list)
+		    ),
+	    TP_fast_assign(__entry->skb = skb;
+			   __entry->skb_len = skb->len;
+			   __entry->data_len = skb->data_len;
+			   __entry->share_ok = share_ok;
+			   __entry->head_frag = skb->head_frag;
+			   __entry->pp_recycle = skb->pp_recycle;
+			   __entry->cloned = skb_cloned(skb);
+			   __entry->shared = skb_shared(skb);
+			   __entry->nr_frags = skb_shinfo(skb)->nr_frags;
+			   __entry->frag_list = (bool)skb_shinfo(skb)->frag_list;
+		    ),
+	    TP_printk("INGRPSTQ: NTHADD: skb=%p len=%u data_len=%u share_ok=%u head_frag=%u pp_recycle=%u cloned=%u shared=%u nr_frags=%u frag_list=%u",
+		      __entry->skb, __entry->skb_len, __entry->data_len, __entry->share_ok,
+		      __entry->head_frag, __entry->pp_recycle, __entry->cloned, __entry->shared,
+		      __entry->nr_frags, __entry->frag_list));
+
+DECLARE_EVENT_CLASS(iptfs_timer_event,
+		    TP_PROTO(struct xfrm_iptfs_data *xtfs, u64 time_val),
+		    TP_ARGS(xtfs, time_val),
+		    TP_STRUCT__entry(
+			__field(u64, time_val)
+			__field(u64, set_time)),
+		    TP_fast_assign(
+			__entry->time_val = time_val;
+			__entry->set_time = xtfs->iptfs_settime;
+		    ),
+		    TP_printk("TIMER: set_time=%llu time_val=%llu",
+			      __entry->set_time, __entry->time_val));
+
+DEFINE_EVENT(iptfs_timer_event, iptfs_timer_start,
+	     TP_PROTO(struct xfrm_iptfs_data *xtfs, u64 time_val),
+	     TP_ARGS(xtfs, time_val));
+
+DEFINE_EVENT(iptfs_timer_event, iptfs_timer_expire,
+	     TP_PROTO(struct xfrm_iptfs_data *xtfs, u64 time_val),
+	     TP_ARGS(xtfs, time_val));
+
+#endif /* _TRACE_IPTFS_H */
+
+/* This part must be outside protection */
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH ../../net/xfrm
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE trace_iptfs
+#include <trace/define_trace.h>
diff --git a/net/xfrm/xfrm_iptfs.c b/net/xfrm/xfrm_iptfs.c
new file mode 100644
index 000000000000..65f7acdbe6a8
--- /dev/null
+++ b/net/xfrm/xfrm_iptfs.c
@@ -0,0 +1,2696 @@
+// SPDX-License-Identifier: GPL-2.0
+/* xfrm_iptfs: IPTFS encapsulation support
+ *
+ * April 21 2022, Christian Hopps <chopps@labn.net>
+ *
+ * Copyright (c) 2022, LabN Consulting, L.L.C.
+ *
+ */
+
+#include <linux/kernel.h>
+#include <linux/icmpv6.h>
+#include <net/gro.h>
+#include <net/icmp.h>
+#include <net/ip6_route.h>
+#include <net/inet_ecn.h>
+#include <net/iptfs.h>
+#include <net/xfrm.h>
+
+#include <crypto/aead.h>
+
+#include "xfrm_inout.h"
+#include "trace_iptfs.h"
+
+/* 1) skb->head should be cache aligned.
+ * 2) when resv is for L2 headers (i.e., ethernet) we want the cacheline to
+ * start -16 from data.
+ * 3) when resv is for L3+L2 headers IOW skb->data points at the IPTFS payload
+ * we want data to be cache line aligned so all the pushed headers will be in
+ * another cacheline.
+ */
+#define XFRM_IPTFS_MIN_L3HEADROOM 128
+#define XFRM_IPTFS_MIN_L2HEADROOM (64 + 16)
+#define IPTFS_FRAG_COPY_MAX 256	/* max for copying to create iptfs frags */
+#define IPTFS_PKT_SHARE_MIN 129	/* min to try to share vs copy pkt data */
+#define NSECS_IN_USEC 1000
+
+#define IPTFS_TYPE_NOCC 0
+#define IPTFS_TYPE_CC 1
+
+#define IPTFS_HRTIMER_MODE HRTIMER_MODE_REL_SOFT
+
+struct skb_wseq {
+	struct sk_buff *skb;
+	u64 drop_time;
+};
+
+struct xfrm_iptfs_config {
+	bool dont_frag : 1;
+	u16 reorder_win_size;
+	u32 pkt_size;	    /* outer_packet_size or 0 */
+	u32 max_queue_size; /* octets */
+	u64 init_delay_us;  /* microseconds */
+	u32 drop_time_us;   /* microseconds */
+};
+
+struct xfrm_iptfs_data {
+	struct xfrm_iptfs_config cfg;
+
+	/* Ingress User Input */
+	struct xfrm_state *x;	    /* owning state */
+	struct sk_buff_head queue;  /* output queue */
+	u32 queue_size;		    /* octets */
+	u32 ecn_queue_size;	    /* octets above which ECN mark */
+	u64 init_delay_ns;	    /* nanoseconds */
+	struct hrtimer iptfs_timer; /* output timer */
+	time64_t iptfs_settime;	    /* time timer was set */
+	u32 payload_mtu;	    /* max payload size */
+
+	/* Tunnel input reordering */
+	bool w_seq_set;		  /* true after first seq received */
+	u64 w_wantseq;		  /* expected next sequence */
+	struct skb_wseq *w_saved; /* the saved buf array */
+	u32 w_savedlen;		  /* the saved len (not size) */
+	spinlock_t drop_lock;
+	struct hrtimer drop_timer;
+	u64 drop_time_ns;
+
+	/* Tunnel input reassembly */
+	struct sk_buff *ra_newskb; /* new pkt being reassembled */
+	u64 ra_wantseq;		   /* expected next sequence */
+	u8 ra_runt[6];		   /* last pkt bytes from last skb */
+	u8 ra_runtlen;		   /* count of ra_runt */
+};
+
+static u32 __iptfs_get_inner_mtu(struct xfrm_state *x, int outer_mtu);
+static enum hrtimer_restart iptfs_delay_timer(struct hrtimer *me);
+static enum hrtimer_restart iptfs_drop_timer(struct hrtimer *me);
+
+/* ================= */
+/* Utility Functions */
+/* ================= */
+
+static u32 __trace_ip_proto(struct iphdr *iph)
+{
+	if (iph->version == 4)
+		return iph->protocol;
+	return ((struct ipv6hdr *)iph)->nexthdr;
+}
+
+static u32 __trace_ip_proto_seq(struct iphdr *iph)
+{
+	void *nexthdr;
+	u32 protocol = 0;
+
+	if (iph->version == 4) {
+		nexthdr = (void *)(iph + 1);
+		protocol = iph->protocol;
+	} else if (iph->version == 6) {
+		nexthdr = (void *)(((struct ipv6hdr *)(iph)) + 1);
+		protocol = ((struct ipv6hdr *)(iph))->nexthdr;
+	}
+	switch (protocol) {
+	case IPPROTO_ICMP:
+		return ntohs(((struct icmphdr *)nexthdr)->un.echo.sequence);
+	case IPPROTO_ICMPV6:
+		return ntohs(((struct icmp6hdr *)nexthdr)->icmp6_sequence);
+	case IPPROTO_TCP:
+		return ntohl(((struct tcphdr *)nexthdr)->seq);
+	case IPPROTO_UDP:
+		return ntohs(((struct udphdr *)nexthdr)->source);
+	default:
+		return 0;
+	}
+}
+
+static u64 __esp_seq(struct sk_buff *skb)
+{
+	u64 seq = ntohl(XFRM_SKB_CB(skb)->seq.input.low);
+
+	return seq | (u64)ntohl(XFRM_SKB_CB(skb)->seq.input.hi) << 32;
+}
+
+/* ================= */
+/* SK_BUFF Functions */
+/* ================= */
+
+/**
+ * iptfs_alloc_skb() - Allocate a new `skb` using a meta-data template.
+ * @tpl: the template to copy the new `skb`s meta-data from.
+ * @len: the linear length of the head data, zero is fine.
+ * @l3resv: true if reserve needs to support pushing L3 headers
+ *
+ * A new `skb` is allocated and it's meta-data is initialized from `tpl`, the
+ * head data is sized to `len` + reserved space set according to the @l3resv
+ * boolean. When @l3resv is false, resv is XFRM_IPTFS_MIN_L2HEADROOM which
+ * arranges for `skb->data - 16` (etherhdr space) to be the start of a cacheline.
+ * Otherwise, @l3resv is true and resv is either the size of headroom from `tpl` or
+ * XFRM_IPTFS_MIN_L3HEADROOM whichever is greater, which tries to align
+ * skb->data to a cacheline as all headers will be pushed on the previous
+ * cacheline bytes.
+ *
+ * When copying meta-data from the @tpl, the sk_buff->headers are not copied.
+ *
+ * Zero length skbs are allocated when we only need a head skb to hold new
+ * packet headers (basically the mac header) that sit on top of existing shared
+ * packet data.
+ *
+ * Return: the new skb or NULL.
+ */
+static struct sk_buff *iptfs_alloc_skb(struct sk_buff *tpl, u32 len,
+				       bool l3resv)
+{
+	struct sk_buff *skb;
+	u32 resv;
+
+	if (!l3resv) {
+		resv = XFRM_IPTFS_MIN_L2HEADROOM;
+	} else {
+		resv = skb_headroom(tpl);
+		if (resv < XFRM_IPTFS_MIN_L3HEADROOM)
+			resv = XFRM_IPTFS_MIN_L3HEADROOM;
+	}
+
+	skb = alloc_skb(len + resv, GFP_ATOMIC);
+	if (!skb) {
+		XFRM_INC_STATS(dev_net(tpl->dev), LINUX_MIB_XFRMINERROR);
+		return NULL;
+	}
+
+	skb_reserve(skb, resv);
+
+	/* Code from __copy_skb_header() -- we do not want any of the
+	 * tpl->headers copied over, so we aren't using `skb_copy_header()`.
+	 */
+	skb->tstamp = tpl->tstamp;
+	skb->dev = tpl->dev;
+	memcpy(skb->cb, tpl->cb, sizeof(skb->cb));
+	skb_dst_copy(skb, tpl);
+	__skb_ext_copy(skb, tpl);
+	__nf_copy(skb, tpl, false);
+
+	return skb;
+}
+
+/**
+ * skb_head_to_frag() - initialize a skb_frag_t based on skb head data
+ */
+static void skb_head_to_frag(const struct sk_buff *skb, skb_frag_t *frag)
+{
+	struct page *page = virt_to_head_page(skb->data);
+	unsigned char *addr = (unsigned char *)page_address(page);
+
+	BUG_ON(!skb->head_frag);
+	skb_frag_fill_page_desc(frag, page, skb->data - addr, skb_headlen(skb));
+}
+
+/**
+ * struct skb_frag_walk - use to track a walk through fragments
+ * @fragi: current fragment index
+ * @past: length of data in fragments before @fragi
+ * @total: length of data in all fragments
+ * @nr_frags: number of fragments present in array
+ * @initial_offset: the value passed in to skb_prepare_frag_walk()
+ * @pp_recycle: copy of skb->pp_recycle
+ * @frags: the page fragments inc. room for head page
+ */
+struct skb_frag_walk {
+	u32 fragi;
+	u32 past;
+	u32 total;
+	u32 nr_frags;
+	u32 initial_offset;
+	bool pp_recycle;
+	skb_frag_t frags[MAX_SKB_FRAGS + 1];
+};
+
+/**
+ * skb_prepare_frag_walk() - initialize a frag walk over an skb.
+ * @skb: the skb to walk.
+ * @initial_offset: start the walk @initial_offset into the skb.
+ * @walk: the walk to initialize
+ *
+ * Future calls to skb_add_frags() will expect the @offset value to be at
+ * least @initial_offset large.
+ */
+static void skb_prepare_frag_walk(struct sk_buff *skb, u32 initial_offset,
+				  struct skb_frag_walk *walk)
+{
+	struct skb_shared_info *shinfo = skb_shinfo(skb);
+	skb_frag_t *frag, *from;
+	u32 i;
+
+	walk->initial_offset = initial_offset;
+	walk->fragi = 0;
+	walk->past = 0;
+	walk->total = 0;
+	walk->nr_frags = 0;
+	walk->pp_recycle = skb->pp_recycle;
+
+	if (skb->head_frag) {
+		if (initial_offset >= skb_headlen(skb)) {
+			initial_offset -= skb_headlen(skb);
+		} else {
+			frag = &walk->frags[walk->nr_frags++];
+			skb_head_to_frag(skb, frag);
+			frag->bv_offset += initial_offset;
+			frag->bv_len -= initial_offset;
+			walk->total += frag->bv_len;
+			initial_offset = 0;
+		}
+	} else {
+		BUG_ON(skb_headlen(skb) > initial_offset);
+		initial_offset -= skb_headlen(skb);
+	}
+
+	for (i = 0; i < shinfo->nr_frags; i++) {
+		from = &shinfo->frags[i];
+		if (initial_offset >= from->bv_len) {
+			initial_offset -= from->bv_len;
+			continue;
+		}
+		frag = &walk->frags[walk->nr_frags++];
+		*frag = *from;
+		if (initial_offset) {
+			frag->bv_offset += initial_offset;
+			frag->bv_len -= initial_offset;
+			initial_offset = 0;
+		}
+		walk->total += frag->bv_len;
+	}
+	BUG_ON(initial_offset != 0);
+}
+
+static u32 __skb_reset_frag_walk(struct skb_frag_walk *walk, u32 offset)
+{
+	/* Adjust offset to refer to internal walk values */
+	BUG_ON(offset < walk->initial_offset);
+	offset -= walk->initial_offset;
+
+	/* Get to the correct fragment for offset */
+	while (offset < walk->past) {
+		walk->past -= walk->frags[--walk->fragi].bv_len;
+		if (offset >= walk->past)
+			break;
+		BUG_ON(walk->fragi == 0);
+	}
+	while (offset >= walk->past + walk->frags[walk->fragi].bv_len)
+		walk->past += walk->frags[walk->fragi++].bv_len;
+
+	/* offset now relative to this current frag */
+	offset -= walk->past;
+	return offset;
+}
+
+/**
+ * skb_can_add_frags() - check if ok to add frags from walk to skb
+ * @skb: skb to check for adding frags to
+ * @walk: the walk that will be used as source for frags.
+ * @offset: offset from beginning of original skb to start from.
+ * @len: amount of data to add frag references to in @skb.
+ */
+static bool skb_can_add_frags(const struct sk_buff *skb,
+			      struct skb_frag_walk *walk, u32 offset, u32 len)
+{
+	struct skb_shared_info *shinfo = skb_shinfo(skb);
+	u32 fragi, nr_frags, fraglen;
+
+	if (skb_has_frag_list(skb) || skb->pp_recycle != walk->pp_recycle)
+		return false;
+
+	/* Make offset relative to current frag after setting that */
+	offset = __skb_reset_frag_walk(walk, offset);
+
+	/* Verify we have array space for the fragments we need to add */
+	fragi = walk->fragi;
+	nr_frags  = shinfo->nr_frags;
+	while (len && fragi < walk->nr_frags) {
+		skb_frag_t *frag = &walk->frags[fragi];
+
+		fraglen = frag->bv_len;
+		if (offset) {
+			fraglen -= offset;
+			offset = 0;
+		}
+		if (++nr_frags > MAX_SKB_FRAGS)
+			return false;
+		if (len <= fraglen)
+			return true;
+		len -= fraglen;
+		fragi++;
+	}
+	/* We may not copy all @len but what we have will fit. */
+	return true;
+}
+
+/**
+ * skb_add_frags() - add a range of fragment references into an skb
+ * @skb: skb to add references into
+ * @walk: the walk to add referenced fragments from.
+ * @offset: offset from beginning of original skb to start from.
+ * @len: amount of data to add frag references to in @skb.
+ *
+ * skb_can_add_frags() should be called before this function to verify that the
+ * destination @skb is compatible with the walk and has space in the array for
+ * the to be added frag refrences.
+ *
+ * Return: The number of bytes not added to @skb b/c we reached the end of the
+ * walk before adding all of @len.
+ */
+static int skb_add_frags(struct sk_buff *skb, struct skb_frag_walk *walk,
+			 u32 offset, u32 len)
+{
+	struct skb_shared_info *shinfo = skb_shinfo(skb);
+	u32 fraglen;
+
+	BUG_ON(skb->pp_recycle != walk->pp_recycle);
+	if (!walk->nr_frags || offset >= walk->total + walk->initial_offset)
+		return len;
+
+	/* make offset relative to current frag after setting that */
+	offset = __skb_reset_frag_walk(walk, offset);
+	BUG_ON(shinfo->nr_frags >= MAX_SKB_FRAGS);
+
+	while (len && walk->fragi < walk->nr_frags) {
+		skb_frag_t *frag = &walk->frags[walk->fragi];
+		skb_frag_t *tofrag = &shinfo->frags[shinfo->nr_frags];
+
+		*tofrag = *frag;
+		if (offset) {
+			tofrag->bv_offset += offset;
+			tofrag->bv_len -= offset;
+			offset = 0;
+		}
+		__skb_frag_ref(tofrag);
+		shinfo->nr_frags++;
+		BUG_ON(shinfo->nr_frags > MAX_SKB_FRAGS);
+
+		/* see if we are done */
+		fraglen = tofrag->bv_len;
+		if (len < fraglen) {
+			tofrag->bv_len = len;
+			skb->len += len;
+			skb->data_len += len;
+			return 0;
+		}
+		/* advance to next source fragment */
+		len -= fraglen;		  /* careful, use dst bv_len */
+		skb->len += fraglen;	  /* careful, "   "    "     */
+		skb->data_len += fraglen; /* careful, "   "    "     */
+		walk->past +=
+			frag->bv_len; /* careful, use src bv_len */
+		walk->fragi++;
+	}
+	return len;
+}
+
+/**
+ * skb_copy_bits_seq - copy bits from a skb_seq_state to kernel buffer
+ * @st: source skb_seq_state
+ * @offset: offset in source
+ * @to: destination buffer
+ * @len: number of bytes to copy
+ *
+ * Copy @len bytes from @offset bytes into the source @st to the destination
+ * buffer @to. `offset` should increase (or be unchanged) with each subsequent
+ * call to this function. If offset needs to decrease from the previous use `st`
+ * should be reset first.
+ */
+static int skb_copy_bits_seq(struct skb_seq_state *st, int offset, void *to, int len)
+{
+	const u8 *data;
+	u32 sqlen;
+
+	for (;;) {
+		sqlen = skb_seq_read(offset, &data, st);
+		if (sqlen == 0)
+			return -ENOMEM;
+		if (sqlen >= len) {
+			memcpy(to, data, len);
+			return 0;
+		}
+		memcpy(to, data, sqlen);
+		to += sqlen;
+		offset += sqlen;
+		len -= sqlen;
+	}
+}
+
+/* ================================== */
+/* IPTFS Trace Event Definitions      */
+/* ================================== */
+
+#define CREATE_TRACE_POINTS
+#include "trace_iptfs.h"
+
+/* ================================== */
+/* IPTFS Receiving (egress) Functions */
+/* ================================== */
+
+/**
+ * iptfs_pskb_add_frags() - Create and add frags into a new sk_buff.
+ * @tpl: template to create new skb from.
+ * @walk: The source for fragments to add.
+ * @off: The offset into @walk to add frags from, also used with @st and
+ *       @copy_len.
+ * @len: The length of data to add covering frags from @walk into @skb.
+ *       This must be <= @skblen.
+ * @st: The sequence state to copy from into the new head skb.
+ * @copy_len: Copy @copy_len bytes from @st at offset @off into the new skb
+ *            linear space.
+ *
+ * Create a new sk_buff `skb` using the template @tpl. Copy @copy_len bytes from
+ * @st into the new skb linear space, and then add shared fragments from the
+ * frag walk for the remaining @len of data (i.e., @len - @copy_len bytes).
+ *
+ * Return: The newly allocated sk_buff `skb` or NULL if an error occurs.
+ */
+struct sk_buff *iptfs_pskb_add_frags(struct sk_buff *tpl,
+				     struct skb_frag_walk *walk, u32 off,
+				     u32 len, struct skb_seq_state *st,
+				     u32 copy_len)
+{
+	struct sk_buff *skb;
+
+	skb = iptfs_alloc_skb(tpl, copy_len, false);
+	if (!skb)
+		return NULL;
+
+	/* this should not normally be happening */
+	if (!skb_can_add_frags(skb, walk, off + copy_len, len - copy_len)) {
+		kfree_skb(skb);
+		return NULL;
+	}
+
+	if (copy_len &&
+	    skb_copy_bits_seq(st, off, skb_put(skb, copy_len), copy_len)) {
+		XFRM_INC_STATS(dev_net(st->root_skb->dev),
+			       LINUX_MIB_XFRMINERROR);
+		kfree_skb(skb);
+		return NULL;
+	}
+
+	skb_add_frags(skb, walk, off + copy_len, len - copy_len);
+	return skb;
+}
+
+/**
+ * iptfs_pskb_extract_seq() - Create and load data into a new sk_buff.
+ * @skblen: the total data size for `skb`.
+ * @st: The source for the rest of the data to copy into `skb`.
+ * @off: The offset into @st to copy data from.
+ * @len: The length of data to copy from @st into `skb`. This must be <=
+ *       @skblen.
+ *
+ * Create a new sk_buff `skb` with @skblen of packet data space. If non-zero,
+ * copy @rlen bytes of @runt into `skb`. Then using seq functions copy @len
+ * bytes from @st into `skb` starting from @off.
+ *
+ * It is an error for @len to be greater than the amount of data left in @st.
+ *
+ * Return: The newly allocated sk_buff `skb` or NULL if an error occurs.
+ */
+static struct sk_buff *
+iptfs_pskb_extract_seq(u32 skblen, struct skb_seq_state *st, u32 off, int len)
+{
+	struct sk_buff *skb = iptfs_alloc_skb(st->root_skb, skblen, false);
+
+	if (!skb)
+		return NULL;
+	if (skb_copy_bits_seq(st, off, skb_put(skb, len), len)) {
+		XFRM_INC_STATS(dev_net(st->root_skb->dev),
+			       LINUX_MIB_XFRMINERROR);
+		kfree_skb(skb);
+		return NULL;
+	}
+	return skb;
+}
+
+/**
+ * iptfs_input_save_runt() - save data in xtfs runt space.
+ *
+ * Save the small (`len`) start of a fragmented packet in `buf` in the xtfs data
+ * runt space.
+ */
+static void iptfs_input_save_runt(struct xfrm_iptfs_data *xtfs, u64 seq,
+				  u8 *buf, int len)
+{
+	BUG_ON(xtfs->ra_newskb); /* we won't have a new SKB yet */
+
+	memcpy(xtfs->ra_runt, buf, len);
+
+	xtfs->ra_runtlen = len;
+	xtfs->ra_wantseq = seq + 1;
+}
+
+/**
+ * __iptfs_iphlen() - return the v4/v6 header length using packet data.
+ *
+ * The version data is expected to be valid (i.e., either 4 or 6).
+ */
+static u32 __iptfs_iphlen(u8 *data)
+{
+	struct iphdr *iph = (struct iphdr *)data;
+
+	if (iph->version == 0x4)
+		return sizeof(*iph);
+	BUG_ON(iph->version != 0x6);
+	return sizeof(struct ipv6hdr);
+}
+
+/**
+ * __iptfs_iplen() - return the v4/v6 length using packet data.
+ *
+ * Grab the IPv4 or IPv6 length value in the start of the inner packet header
+ * pointed to by `data`. Assumes data len is enough for the length field only.
+ *
+ * The version data is expected to be valid (i.e., either 4 or 6).
+ */
+static u32 __iptfs_iplen(u8 *data)
+{
+	struct iphdr *iph = (struct iphdr *)data;
+
+	if (iph->version == 0x4)
+		return ntohs(iph->tot_len);
+	BUG_ON(iph->version != 0x6);
+	return ntohs(((struct ipv6hdr *)iph)->payload_len) +
+	       sizeof(struct ipv6hdr);
+}
+
+/**
+ * iptfs_complete_inner_skb() - finish preparing the inner packet for gro recv.
+ *
+ * Finish the standard xfrm processing on the inner packet prior to sending back
+ * through gro_cells_receive. We do this separately b/c we are building a list
+ * of packets in the hopes that one day a list will be taken by
+ * xfrm_input.
+ */
+static void iptfs_complete_inner_skb(struct xfrm_state *x, struct sk_buff *skb)
+{
+	skb_reset_network_header(skb);
+
+	/* The packet is going back through gro_cells_receive no need to
+	 * set this.
+	 */
+	skb_reset_transport_header(skb);
+
+	/* Packet already has checksum value set. */
+	skb->ip_summed = CHECKSUM_NONE;
+
+	/* Our skb will contain the header data copied when this outer packet
+	 * which contained the start of this inner packet. This is true
+	 * when we allocate a new skb as well as when we reuse the existing skb.
+	 */
+	if (ip_hdr(skb)->version == 0x4) {
+		struct iphdr *iph = ip_hdr(skb);
+
+		if (x->props.flags & XFRM_STATE_DECAP_DSCP)
+			ipv4_copy_dscp(XFRM_MODE_SKB_CB(skb)->tos, iph);
+		if (!(x->props.flags & XFRM_STATE_NOECN))
+			if (INET_ECN_is_ce(XFRM_MODE_SKB_CB(skb)->tos))
+				IP_ECN_set_ce(iph);
+
+		skb->protocol = htons(ETH_P_IP);
+	} else {
+		struct ipv6hdr *iph = ipv6_hdr(skb);
+
+		if (x->props.flags & XFRM_STATE_DECAP_DSCP)
+			ipv6_copy_dscp(XFRM_MODE_SKB_CB(skb)->tos, iph);
+		if (!(x->props.flags & XFRM_STATE_NOECN))
+			if (INET_ECN_is_ce(XFRM_MODE_SKB_CB(skb)->tos))
+				IP6_ECN_set_ce(skb, iph);
+
+		skb->protocol = htons(ETH_P_IPV6);
+	}
+}
+
+static void __iptfs_reassem_done(struct xfrm_iptfs_data *xtfs, bool free)
+{
+	int ret;
+
+	assert_spin_locked(&xtfs->drop_lock);
+
+	/* We don't care if it works locking takes care of things */
+	ret = hrtimer_try_to_cancel(&xtfs->drop_timer);
+	if (free)
+		kfree_skb(xtfs->ra_newskb);
+	xtfs->ra_newskb = NULL;
+}
+
+/**
+ * iptfs_reassem_done() - In-progress packet is aborted free the state.
+ */
+static void iptfs_reassem_abort(struct xfrm_iptfs_data *xtfs)
+{
+	__iptfs_reassem_done(xtfs, true);
+}
+
+/**
+ * iptfs_reassem_done() - In-progress packet is complete, clear the state.
+ */
+static void iptfs_reassem_done(struct xfrm_iptfs_data *xtfs)
+{
+	__iptfs_reassem_done(xtfs, false);
+}
+
+/**
+ * iptfs_reassem_cont() - Continue the reassembly of an inner packets.
+ *
+ * Process an IPTFS payload that has a non-zero `blkoff` or when we are
+ * expecting the continuation b/c we have a runt or in-progress packet.
+ */
+static u32 iptfs_reassem_cont(struct xfrm_iptfs_data *xtfs, u64 seq,
+			      struct skb_seq_state *st, struct sk_buff *skb,
+			      u32 data, u32 blkoff, struct list_head *list)
+{
+	struct skb_frag_walk _fragwalk;
+	struct skb_frag_walk *fragwalk = NULL;
+	struct sk_buff *newskb = xtfs->ra_newskb;
+	u32 remaining = skb->len - data;
+	u32 runtlen = xtfs->ra_runtlen;
+	u32 copylen, fraglen, ipremain, iphlen, iphremain, rrem;
+
+	/* Handle packet fragment we aren't expecting */
+	if (!runtlen && !xtfs->ra_newskb)
+		return data + min(blkoff, remaining);
+
+	/* Important to remember that input to this function is an ordered
+	 * packet stream (unless the user disabled the reorder window). Thus if
+	 * we are waiting for, and expecting the next packet so we can continue
+	 * assembly, a newer sequence number indicates older ones are not coming
+	 * (or if they do should be ignored). Technically we can receive older
+	 * ones when the reorder window is disabled; however, the user should
+	 * have disabled fragmentation in this case, and regardless we don't
+	 * deal with it.
+	 *
+	 * blkoff could be zero if the stream is messed up (or it's an all pad
+	 * insertion) be careful to handle that case in each of the below
+	 */
+
+	/* Too old case: This can happen when the reorder window is disabled so
+	 * ordering isn't actually guaranteed.
+	 */
+	if (seq < xtfs->ra_wantseq)
+		return data + remaining;
+
+	/* Too new case: We missed what we wanted cleanup. */
+	if (seq > xtfs->ra_wantseq) {
+		XFRM_INC_STATS(dev_net(skb->dev),
+			       LINUX_MIB_XFRMINIPTFSERROR);
+		goto abandon;
+	}
+
+	if (blkoff == 0) {
+		if ((*skb->data & 0xF0) != 0) {
+			XFRM_INC_STATS(dev_net(skb->dev),
+				       LINUX_MIB_XFRMINIPTFSERROR);
+			goto abandon;
+		}
+		/* Handle all pad case, advance expected sequence number.
+		 * (RFC 9347 S2.2.3)
+		 */
+		xtfs->ra_wantseq++;
+		/* will end parsing */
+		return data + remaining;
+	}
+
+	if (runtlen) {
+		BUG_ON(xtfs->ra_newskb);
+
+		/* Regardless of what happens we're done with the runt */
+		xtfs->ra_runtlen = 0;
+
+		/* The start of this inner packet was at the very end of the last
+		 * iptfs payload which didn't include enough for the ip header
+		 * length field. We must have *at least* that now.
+		 */
+		rrem = sizeof(xtfs->ra_runt) - runtlen;
+		if (remaining < rrem || blkoff < rrem) {
+			XFRM_INC_STATS(dev_net(skb->dev),
+				       LINUX_MIB_XFRMINIPTFSERROR);
+			goto abandon;
+		}
+
+		/* fill in the runt data */
+		if (skb_copy_bits_seq(st, data, &xtfs->ra_runt[runtlen],
+				      rrem)) {
+			XFRM_INC_STATS(dev_net(skb->dev),
+				       LINUX_MIB_XFRMINBUFFERERROR);
+			goto abandon;
+		}
+
+		/* We have enough data to get the ip length value now,
+		 * allocate an in progress skb
+		 */
+		ipremain = __iptfs_iplen(xtfs->ra_runt);
+		if (ipremain < sizeof(xtfs->ra_runt)) {
+			/* length has to be at least runtsize large */
+			XFRM_INC_STATS(dev_net(skb->dev),
+				       LINUX_MIB_XFRMINIPTFSERROR);
+			goto abandon;
+		}
+
+		/* For the runt case we don't attempt sharing currently. NOTE:
+		 * Currently, this IPTFS implementation will not create runts.
+		 */
+
+		newskb = iptfs_alloc_skb(skb, ipremain, false);
+		if (!newskb) {
+			XFRM_INC_STATS(dev_net(skb->dev),
+				       LINUX_MIB_XFRMINERROR);
+			goto abandon;
+		}
+		xtfs->ra_newskb = newskb;
+
+		/* Copy the runt data into the buffer, but leave data
+		 * pointers the same as normal non-runt case. The extra `rrem`
+		 * recopied bytes are basically cacheline free. Allows using
+		 * same logic below to complete.
+		 */
+		memcpy(skb_put(newskb, runtlen), xtfs->ra_runt,
+		       sizeof(xtfs->ra_runt));
+	}
+
+	/* Continue reassembling the packet */
+	ipremain = __iptfs_iplen(newskb->data);
+	iphlen = __iptfs_iphlen(newskb->data);
+
+	/* Sanity check, we created the newskb knowing the IP length so the IP
+	 * length can't now be shorter.
+	 */
+	BUG_ON(newskb->len > ipremain);
+
+	ipremain -= newskb->len;
+	if (blkoff < ipremain) {
+		/* Corrupt data, we don't have enough to complete the packet */
+		XFRM_INC_STATS(dev_net(skb->dev), LINUX_MIB_XFRMINIPTFSERROR);
+		goto abandon;
+	}
+
+	/* We want the IP header in linear space */
+	if (newskb->len < iphlen)  {
+		iphremain = iphlen - newskb->len;
+		if (blkoff < iphremain) {
+			XFRM_INC_STATS(dev_net(skb->dev),
+				       LINUX_MIB_XFRMINIPTFSERROR);
+			goto abandon;
+		}
+		fraglen = min(blkoff, remaining);
+		copylen = min(fraglen, iphremain);
+		BUG_ON(skb_tailroom(newskb) < copylen);
+		if (skb_copy_bits_seq(st, data, skb_put(newskb, copylen), copylen)) {
+			XFRM_INC_STATS(dev_net(skb->dev),
+				       LINUX_MIB_XFRMINBUFFERERROR);
+			goto abandon;
+		}
+		/* this is a silly condition that might occur anyway */
+		if (copylen < iphremain) {
+			xtfs->ra_wantseq++;
+			return data + fraglen;
+		}
+		/* update data and things derived from it */
+		data += copylen;
+		blkoff -= copylen;
+		remaining -= copylen;
+		ipremain -= copylen;
+	}
+
+	fraglen = min(blkoff, remaining);
+	copylen = min(fraglen, ipremain);
+
+	/* If we may have the opportunity to share prepare a fragwalk. */
+	if (!skb_has_frag_list(skb) && !skb_has_frag_list(newskb) &&
+	    (skb->head_frag || skb->len == skb->data_len) &&
+	    skb->pp_recycle == newskb->pp_recycle) {
+		fragwalk = &_fragwalk;
+		skb_prepare_frag_walk(skb, data, fragwalk);
+	}
+
+	/* Try share then copy. */
+	if (fragwalk && skb_can_add_frags(newskb, fragwalk, data, copylen)) {
+		u32 leftover;
+
+		leftover = skb_add_frags(newskb, fragwalk, data, copylen);
+		BUG_ON(leftover != 0);
+	} else {
+		/* We verified this was true in the main receive routine */
+		BUG_ON(skb_tailroom(newskb) < copylen);
+
+		/* copy fragment data into newskb */
+		if (skb_copy_bits_seq(st, data, skb_put(newskb, copylen), copylen)) {
+			XFRM_INC_STATS(dev_net(skb->dev),
+				       LINUX_MIB_XFRMINBUFFERERROR);
+			goto abandon;
+		}
+	}
+
+	if (copylen < ipremain) {
+		xtfs->ra_wantseq++;
+	} else {
+		/* We are done with packet reassembly! */
+		BUG_ON(copylen != ipremain);
+		iptfs_reassem_done(xtfs);
+		iptfs_complete_inner_skb(xtfs->x, newskb);
+		list_add_tail(&newskb->list, list);
+	}
+
+	/* will continue on to new data block or end */
+	return data + fraglen;
+
+abandon:
+	if (xtfs->ra_newskb) {
+		iptfs_reassem_abort(xtfs);
+	} else {
+		xtfs->ra_runtlen = 0;
+		xtfs->ra_wantseq = 0;
+	}
+	/* skip past fragment, maybe to end */
+	return data + min(blkoff, remaining);
+}
+
+/**
+ * iptfs_input_ordered() - handle next in order IPTFS payload.
+ *
+ * Process the IPTFS payload in `skb` and consume it afterwards.
+ */
+static int iptfs_input_ordered(struct xfrm_state *x, struct sk_buff *skb)
+{
+	u8 hbytes[sizeof(struct ipv6hdr)];
+	struct ip_iptfs_cc_hdr iptcch;
+	struct skb_seq_state skbseq;
+	struct skb_frag_walk _fragwalk;
+	struct skb_frag_walk *fragwalk = NULL;
+	struct list_head sublist; /* rename this it's just a list */
+	struct sk_buff *first_skb, *defer, *next;
+	const unsigned char *old_mac;
+	struct xfrm_iptfs_data *xtfs;
+	struct ip_iptfs_hdr *ipth;
+	struct iphdr *iph;
+	struct net *net;
+	u32 remaining, first_iplen, iplen, iphlen, data, tail;
+	u32 blkoff, capturelen;
+	u64 seq;
+
+	xtfs = x->mode_data;
+	net = dev_net(skb->dev);
+	first_skb = NULL;
+	defer = NULL;
+
+	seq = __esp_seq(skb);
+
+	/* Large enough to hold both types of header */
+	ipth = (struct ip_iptfs_hdr *)&iptcch;
+
+	/* Save the old mac header if set */
+	old_mac = skb_mac_header_was_set(skb) ? skb_mac_header(skb) : NULL;
+
+	skb_prepare_seq_read(skb, 0, skb->len, &skbseq);
+
+	/* Get the IPTFS header and validate it */
+
+	if (skb_copy_bits_seq(&skbseq, 0, ipth, sizeof(*ipth))) {
+		XFRM_INC_STATS(net, LINUX_MIB_XFRMINBUFFERERROR);
+		goto done;
+	}
+	data = sizeof(*ipth);
+
+	trace_iptfs_egress_recv(skb, xtfs, htons(ipth->block_offset));
+
+	/* Set data past the basic header */
+	if (ipth->subtype == IPTFS_SUBTYPE_CC) {
+		/* Copy the rest of the CC header */
+		remaining = sizeof(iptcch) - sizeof(*ipth);
+		if (skb_copy_bits_seq(&skbseq, data, ipth + 1, remaining)) {
+			XFRM_INC_STATS(net, LINUX_MIB_XFRMINBUFFERERROR);
+			goto done;
+		}
+		data += remaining;
+	} else if (ipth->subtype != IPTFS_SUBTYPE_BASIC) {
+		XFRM_INC_STATS(net, LINUX_MIB_XFRMINHDRERROR);
+		goto done;
+	}
+
+	if (ipth->flags != 0) {
+		XFRM_INC_STATS(net, LINUX_MIB_XFRMINHDRERROR);
+		goto done;
+	}
+
+	INIT_LIST_HEAD(&sublist);
+
+	/* Handle fragment at start of payload, and/or waiting reassembly. */
+
+	blkoff = ntohs(ipth->block_offset);
+	/* check before locking i.e., maybe */
+	if (blkoff || xtfs->ra_runtlen || xtfs->ra_newskb) {
+		spin_lock(&xtfs->drop_lock);
+
+		/* check again after lock */
+		if (blkoff || xtfs->ra_runtlen || xtfs->ra_newskb) {
+			data = iptfs_reassem_cont(xtfs, seq, &skbseq, skb, data,
+						  blkoff, &sublist);
+		}
+
+		spin_unlock(&xtfs->drop_lock);
+	}
+
+	/* New packets */
+
+	tail = skb->len;
+	BUG_ON(xtfs->ra_newskb && data < tail);
+
+	while (data < tail) {
+		u32 protocol = 0;
+
+		/* Gather information on the next data block.
+		 * `data` points to the start of the data block.
+		 */
+		remaining = tail - data;
+
+		/* try and copy enough bytes to read length from ipv4/ipv6 */
+		iphlen = min_t(u32, remaining, 6);
+		if (skb_copy_bits_seq(&skbseq, data, hbytes, iphlen)) {
+			XFRM_INC_STATS(net, LINUX_MIB_XFRMINBUFFERERROR);
+			goto done;
+		}
+
+		iph = (struct iphdr *)hbytes;
+		if (iph->version == 0x4) {
+			/* must have at least tot_len field present */
+			if (remaining < 4) {
+				/* save the bytes we have, advance data and exit */
+				iptfs_input_save_runt(xtfs, seq, hbytes,
+						      remaining);
+				data += remaining;
+				break;
+			}
+
+			iplen = htons(iph->tot_len);
+			iphlen = iph->ihl << 2;
+			protocol = htons(ETH_P_IP);
+			XFRM_MODE_SKB_CB(skbseq.root_skb)->tos = iph->tos;
+		} else if (iph->version == 0x6) {
+			/* must have at least payload_len field present */
+			if (remaining < 6) {
+				/* save the bytes we have, advance data and exit */
+				iptfs_input_save_runt(xtfs, seq, hbytes,
+						      remaining);
+				data += remaining;
+				break;
+			}
+
+			iplen = htons(((struct ipv6hdr *)hbytes)->payload_len);
+			iplen += sizeof(struct ipv6hdr);
+			iphlen = sizeof(struct ipv6hdr);
+			protocol = htons(ETH_P_IPV6);
+			XFRM_MODE_SKB_CB(skbseq.root_skb)->tos =
+				ipv6_get_dsfield((struct ipv6hdr *)iph);
+		} else if (iph->version == 0x0) {
+			/* pad */
+			data = tail;
+			break;
+		} else {
+			XFRM_INC_STATS(net, LINUX_MIB_XFRMINBUFFERERROR);
+			goto done;
+		}
+
+		if (unlikely(skbseq.stepped_offset)) {
+			/* We need to reset our seq read, it can't backup at
+			 * this point.
+			 */
+			struct sk_buff *save = skbseq.root_skb;
+
+			skb_abort_seq_read(&skbseq);
+			skb_prepare_seq_read(save, data, tail, &skbseq);
+		}
+
+		if (first_skb) {
+			skb = NULL;
+		} else {
+			first_skb = skb;
+			first_iplen = iplen;
+			fragwalk = NULL;
+
+			/* We are going to skip over `data` bytes to reach the
+			 * start of the IP header of `iphlen` len for `iplen`
+			 * inner packet.
+			 */
+
+			if (skb_has_frag_list(skb)) {
+				defer = skb;
+				skb = NULL;
+			} else if (data + iphlen <= skb_headlen(skb) &&
+				   /* make sure our header is 32-bit aligned? */
+				   /* ((uintptr_t)(skb->data + data) & 0x3) == 0 && */
+				   skb_tailroom(skb) + tail - data >= iplen) {
+				/* Reuse the received skb.
+				 *
+				 * We have enough headlen to pull past any
+				 * initial fragment data, leaving at least the
+				 * IP header in the linear buffer space.
+				 *
+				 * For linear buffer space we only require that
+				 * linear buffer space is large enough to
+				 * eventually hold the entire reassembled
+				 * packet (by including tailroom in the check).
+				 *
+				 * For non-linear tailroom is 0 and so we only
+				 * re-use if the entire packet is present
+				 * already.
+				 *
+				 * NOTE: there are many more options for
+				 * sharing, KISS for now. Also, this can produce
+				 * skb's with the IP header unaligned to 32
+				 * bits. If that ends up being a problem then a
+				 * check should be added to the conditional
+				 * above that the header lies on a 32-bit
+				 * boundary as well.
+				 */
+				skb_pull(skb, data);
+
+				/* our range just changed */
+				data = 0;
+				tail = skb->len;
+				remaining = skb->len;
+
+				skb->protocol = protocol;
+				skb_mac_header_rebuild(skb);
+				if (skb->mac_len)
+					eth_hdr(skb)->h_proto = skb->protocol;
+
+				/* all pointers could be changed now reset walk */
+				skb_abort_seq_read(&skbseq);
+				skb_prepare_seq_read(skb, data, tail, &skbseq);
+			} else if (skb->head_frag &&
+				   /* We have the IP header right now */
+				   remaining >= iphlen) {
+				fragwalk = &_fragwalk;
+				skb_prepare_frag_walk(skb, data, fragwalk);
+				defer = skb;
+				skb = NULL;
+			} else {
+				/* We couldn't reuse the input skb so allocate a
+				 * new one.
+				 */
+				defer = skb;
+				skb = NULL;
+			}
+
+			/* Don't trim `first_skb` until the end as we are
+			 * walking that data now.
+			 */
+		}
+
+		capturelen = min(iplen, remaining);
+		if (!skb) {
+			if (!fragwalk ||
+			    /* Large enough to be worth sharing */
+			    iplen < IPTFS_PKT_SHARE_MIN ||
+			    /* Have IP header + some data to share. */
+			    capturelen <= iphlen ||
+			    /* Try creating skb and adding frags */
+			    !(skb = iptfs_pskb_add_frags(first_skb, fragwalk,
+							 data, capturelen,
+							 &skbseq, iphlen))) {
+				skb = iptfs_pskb_extract_seq(iplen, &skbseq,
+							     data, capturelen);
+			}
+			if (!skb) {
+				/* skip to next packet or done */
+				data += capturelen;
+				continue;
+			}
+			BUG_ON(skb->len != capturelen);
+
+			skb->protocol = protocol;
+			if (old_mac) {
+				/* rebuild the mac header */
+				skb_set_mac_header(skb, -first_skb->mac_len);
+				memcpy(skb_mac_header(skb), old_mac,
+				       first_skb->mac_len);
+				eth_hdr(skb)->h_proto = skb->protocol;
+			}
+		}
+
+		data += capturelen;
+
+		if (skb->len < iplen) {
+			BUG_ON(data != tail);
+			BUG_ON(xtfs->ra_newskb);
+
+			/* Start reassembly */
+			spin_lock(&xtfs->drop_lock);
+
+			xtfs->ra_newskb = skb;
+			xtfs->ra_wantseq = seq + 1;
+			if (!hrtimer_is_queued(&xtfs->drop_timer)) {
+				/* softirq blocked lest the timer fire and interrupt us */
+				BUG_ON(!in_interrupt());
+				hrtimer_start(&xtfs->drop_timer,
+					      xtfs->drop_time_ns,
+					      IPTFS_HRTIMER_MODE);
+			}
+
+			spin_unlock(&xtfs->drop_lock);
+
+			break;
+		}
+
+		iptfs_complete_inner_skb(x, skb);
+		list_add_tail(&skb->list, &sublist);
+	}
+
+	if (data != tail)
+		/* this should not happen from the above code */
+		XFRM_INC_STATS(dev_net(skb->dev), LINUX_MIB_XFRMINIPTFSERROR);
+
+	if (first_skb && first_iplen && !defer && first_skb != xtfs->ra_newskb) {
+		/* first_skb is queued b/c !defer and not partial */
+		if (pskb_trim(first_skb, first_iplen)) {
+			/* error trimming */
+			list_del(&first_skb->list);
+			defer = first_skb;
+		}
+		first_skb->ip_summed = CHECKSUM_NONE;
+	}
+
+	/* Send the packets! */
+	list_for_each_entry_safe(skb, next, &sublist, list) {
+		BUG_ON(skb == defer);
+		skb_list_del_init(skb);
+		if (xfrm_input(skb, 0, 0, -3))
+			kfree_skb(skb);
+	}
+
+done:
+	skb = skbseq.root_skb;
+	skb_abort_seq_read(&skbseq);
+
+	if (defer) {
+		consume_skb(defer);
+	} else if (!first_skb) {
+		/* skb is the original passed in skb, but we didn't get far
+		 * enough to process it as the first_skb, if we had it would
+		 * either be save in ra_newskb, trimmed and sent on as an skb or
+		 * placed in defer to be freed.
+		 */
+		BUG_ON(!skb);
+		kfree_skb(skb);
+	}
+
+	return 0;
+}
+
+/* ------------------------------- */
+/* Input (Egress) Re-ordering Code */
+/* ------------------------------- */
+
+static void __vec_shift(struct xfrm_iptfs_data *xtfs, u32 shift)
+{
+	u32 savedlen = xtfs->w_savedlen;
+
+	if (shift > savedlen)
+		shift = savedlen;
+	if (shift != savedlen)
+		memcpy(xtfs->w_saved, xtfs->w_saved + shift,
+		       (savedlen - shift) * sizeof(*xtfs->w_saved));
+	memset(xtfs->w_saved + savedlen - shift, 0,
+	       shift * sizeof(*xtfs->w_saved));
+	xtfs->w_savedlen -= shift;
+}
+
+static int __reorder_past(struct xfrm_iptfs_data *xtfs, struct sk_buff *inskb,
+			  struct list_head *freelist, u32 *fcount)
+{
+	list_add_tail(&inskb->list, freelist);
+	(*fcount)++;
+	return 0;
+}
+
+static u32 __reorder_drop(struct xfrm_iptfs_data *xtfs, struct list_head *list)
+
+{
+	struct skb_wseq *s, *se;
+	const u32 savedlen = xtfs->w_savedlen;
+	u64 wantseq = xtfs->w_wantseq;
+	time64_t now = ktime_get_raw_fast_ns();
+	u32 count = 0;
+	u32 scount = 0;
+
+	BUG_ON(!savedlen);
+	if (xtfs->w_saved[0].drop_time > now)
+		goto set_timer;
+
+	wantseq = ++xtfs->w_wantseq;
+
+	/* Keep flushing packets until we reach a drop time greater than now. */
+	s = xtfs->w_saved;
+	se = s + savedlen;
+	do {
+		/* Walking past empty slots until we reach a packet */
+		for (; s < se && !s->skb; s++)
+			if (s->drop_time > now)
+				goto outerdone;
+		/* Sending packets until we hit another empty slot. */
+		for (; s < se && s->skb; scount++, s++)
+			list_add_tail(&s->skb->list, list);
+	} while (s < se);
+outerdone:
+
+	count = s - xtfs->w_saved;
+	if (count) {
+		xtfs->w_wantseq += count;
+
+		/* Shift handled slots plus final empty slot into slot 0. */
+		__vec_shift(xtfs, count);
+	}
+
+	if (xtfs->w_savedlen) {
+set_timer:
+		/* Drifting is OK */
+		hrtimer_start(&xtfs->drop_timer,
+			      xtfs->w_saved[0].drop_time - now,
+			      IPTFS_HRTIMER_MODE);
+	}
+	return scount;
+}
+
+static u32 __reorder_this(struct xfrm_iptfs_data *xtfs, struct sk_buff *inskb,
+			  struct list_head *list)
+
+{
+	struct skb_wseq *s, *se;
+	const u32 savedlen = xtfs->w_savedlen;
+	u64 wantseq = xtfs->w_wantseq;
+	u32 count = 0;
+
+	/* Got what we wanted. */
+	list_add_tail(&inskb->list, list);
+	wantseq = ++xtfs->w_wantseq;
+	if (!savedlen)
+		return 1;
+
+	/* Flush remaining consecutive packets. */
+
+	/* Keep sending until we hit another missed pkt. */
+	for (s = xtfs->w_saved, se = s + savedlen; s < se && s->skb; s++)
+		list_add_tail(&s->skb->list, list);
+	count = s - xtfs->w_saved;
+	if (count)
+		xtfs->w_wantseq += count;
+
+	/* Shift handled slots plus final empty slot into slot 0. */
+	__vec_shift(xtfs, count + 1);
+
+	return count + 1;
+}
+
+/* Set the slot's drop time and all the empty slots below it until reaching a
+ * filled slot which will already be set.
+ */
+static void iptfs_set_window_drop_times(struct xfrm_iptfs_data *xtfs, int index)
+{
+	const u32 savedlen = xtfs->w_savedlen;
+	struct skb_wseq *s = xtfs->w_saved;
+	time64_t drop_time;
+
+	assert_spin_locked(&xtfs->drop_lock);
+
+	if (savedlen > index + 1) {
+		/* we are below another, our drop time and the timer are already set */
+		BUG_ON(xtfs->w_saved[index + 1].drop_time !=
+		       xtfs->w_saved[index].drop_time);
+		return;
+	}
+	/* we are the most future so get a new drop time. */
+	drop_time = ktime_get_raw_fast_ns();
+	drop_time += xtfs->drop_time_ns;
+
+	/* Walk back through the array setting drop times as we go */
+	s[index].drop_time = drop_time;
+	while (index-- > 0 && !s[index].skb)
+		s[index].drop_time = drop_time;
+
+	/* If we walked all the way back, schedule the drop timer if needed */
+	if (index == -1 && !hrtimer_is_queued(&xtfs->drop_timer))
+		hrtimer_start(&xtfs->drop_timer, xtfs->drop_time_ns,
+			      IPTFS_HRTIMER_MODE);
+}
+
+static u32 __reorder_future_fits(struct xfrm_iptfs_data *xtfs,
+				 struct sk_buff *inskb,
+				 struct list_head *freelist, u32 *fcount)
+{
+	const u32 nslots = xtfs->cfg.reorder_win_size + 1;
+	const u64 inseq = __esp_seq(inskb);
+	const u64 wantseq = xtfs->w_wantseq;
+	const u64 distance = inseq - wantseq;
+	const u32 savedlen = xtfs->w_savedlen;
+	const u32 index = distance - 1;
+
+	BUG_ON(distance >= nslots);
+
+	/* Handle future sequence number received which fits in the window.
+	 *
+	 * We know we don't have the seq we want so we won't be able to flush
+	 * anything.
+	 */
+
+	/* slot count is 4, saved size is 3 savedlen is 2
+	 *
+	 * "window boundary" is based on the fixed window size
+	 * distance is also slot number
+	 * index is an array index (i.e., - 1 of slot)
+	 * : : - implicit NULL after array len
+	 *
+	 *          +--------- used length (savedlen == 2)
+	 *          |   +----- array size (nslots - 1 == 3)
+	 *          |   |   + window boundary (nslots == 4)
+	 *          V   V | V
+	 *                |
+	 *  0   1   2   3 |   slot number
+	 * ---  0   1   2 |   array index
+	 *     [-] [b] : :|   array
+	 *
+	 * "2" "3" "4" *5*|   seq numbers
+	 *
+	 * We receive seq number 5
+	 * distance == 3 [inseq(5) - w_wantseq(2)]
+	 * index == 2 [distance(6) - 1]
+	 */
+
+	if (xtfs->w_saved[index].skb) {
+		/* a dup of a future */
+		list_add_tail(&inskb->list, freelist);
+		(*fcount)++;
+		return 0;
+	}
+
+	xtfs->w_saved[index].skb = inskb;
+	xtfs->w_savedlen = max(savedlen, index + 1);
+	iptfs_set_window_drop_times(xtfs, index);
+
+	return 0;
+}
+
+static u32 __reorder_future_shifts(struct xfrm_iptfs_data *xtfs,
+				   struct sk_buff *inskb,
+				   struct list_head *list,
+				   struct list_head *freelist, u32 *fcount)
+{
+	const u32 nslots = xtfs->cfg.reorder_win_size + 1;
+	const u64 inseq = __esp_seq(inskb);
+	u32 savedlen = xtfs->w_savedlen;
+	u64 wantseq = xtfs->w_wantseq;
+	struct sk_buff *slot0 = NULL;
+	u64 last_drop_seq = xtfs->w_wantseq;
+	u64 distance, extra_drops, missed, s0seq;
+	u32 count = 0;
+	struct skb_wseq *wnext;
+	u32 beyond, shifting, slot;
+
+	BUG_ON(inseq <= wantseq);
+	distance = inseq - wantseq;
+	BUG_ON(distance <= nslots - 1);
+	beyond = distance - (nslots - 1);
+	missed = 0;
+
+	/* Handle future sequence number received.
+	 *
+	 * IMPORTANT: we are at least advancing w_wantseq (i.e., wantseq) by 1
+	 * b/c we are beyond the window boundary.
+	 *
+	 * We know we don't have the wantseq so that counts as a drop.
+	 */
+
+	/* ex: slot count is 4, array size is 3 savedlen is 2, slot 0 is the
+	 * missing sequence number.
+	 *
+	 * the final slot at savedlen (index savedlen - 1) is always occupied.
+	 *
+	 * beyond is "beyond array size" not savedlen.
+	 *
+	 *          +--------- array length (savedlen == 2)
+	 *          |   +----- array size (nslots - 1 == 3)
+	 *          |   |   +- window boundary (nslots == 4)
+	 *          V   V | V
+	 *                |
+	 *  0   1   2   3 |   slot number
+	 * ---  0   1   2 |   array index
+	 *     [b] [c] : :|   array
+	 *                |
+	 * "2" "3" "4" "5"|*6*  seq numbers
+	 *
+	 * We receive seq number 6
+	 * distance == 4 [inseq(6) - w_wantseq(2)]
+	 * newslot == distance
+	 * index == 3 [distance(4) - 1]
+	 * beyond == 1 [newslot(4) - lastslot((nslots(4) - 1))]
+	 * shifting == 1 [min(savedlen(2), beyond(1)]
+	 * slot0_skb == [b], and should match w_wantseq
+	 *
+	 *                +--- window boundary (nslots == 4)
+	 *  0   1   2   3 | 4   slot number
+	 * ---  0   1   2 | 3   array index
+	 *     [b] : : : :|     array
+	 * "2" "3" "4" "5" *6*  seq numbers
+	 *
+	 * We receive seq number 6
+	 * distance == 4 [inseq(6) - w_wantseq(2)]
+	 * newslot == distance
+	 * index == 3 [distance(4) - 1]
+	 * beyond == 1 [newslot(4) - lastslot((nslots(4) - 1))]
+	 * shifting == 1 [min(savedlen(1), beyond(1)]
+	 * slot0_skb == [b] and should match w_wantseq
+	 *
+	 *                +-- window boundary (nslots == 4)
+	 *  0   1   2   3 | 4   5   6   slot number
+	 * ---  0   1   2 | 3   4   5   array index
+	 *     [-] [c] : :|             array
+	 * "2" "3" "4" "5" "6" "7" *8*  seq numbers
+	 *
+	 * savedlen = 2, beyond = 3
+	 * iter 1: slot0 == NULL, missed++, lastdrop = 2 (2+1-1), slot0 = [-]
+	 * iter 2: slot0 == NULL, missed++, lastdrop = 3 (2+2-1), slot0 = [c]
+	 * 2 < 3, extra = 1 (3-2), missed += extra, lastdrop = 4 (2+2+1-1)
+	 *
+	 * We receive seq number 8
+	 * distance == 6 [inseq(8) - w_wantseq(2)]
+	 * newslot == distance
+	 * index == 5 [distance(6) - 1]
+	 * beyond == 3 [newslot(6) - lastslot((nslots(4) - 1))]
+	 * shifting == 2 [min(savedlen(2), beyond(3)]
+	 *
+	 * slot0_skb == NULL changed from [b] when "savedlen < beyond" is true.
+	 */
+
+	/* Now send any packets that are being shifted out of saved, and account
+	 * for missing packets that are exiting the window as we shift it.
+	 */
+
+	/* If savedlen > beyond we are shifting some, else all. */
+	shifting = min(savedlen, beyond);
+
+	/* slot0 is the buf that just shifted out and into slot0 */
+	slot0 = NULL;
+	s0seq = wantseq;
+	last_drop_seq = s0seq;
+	wnext = xtfs->w_saved;
+	for (slot = 1; slot <= shifting; slot++, wnext++) {
+		/* handle what was in slot0 before we occupy it */
+		if (!slot0) {
+			last_drop_seq = s0seq;
+			missed++;
+		} else {
+			list_add_tail(&slot0->list, list);
+			count++;
+		}
+		s0seq++;
+		slot0 = wnext->skb;
+		wnext->skb = NULL;
+	}
+
+	/* slot0 is now either NULL (in which case it's what we now are waiting
+	 * for, or a buf in which case we need to handle it like we received it;
+	 * however, we may be advancing past that buffer as well..
+	 */
+
+	/* Handle case where we need to shift more than we had saved, slot0 will
+	 * be NULL iff savedlen is 0, otherwise slot0 will always be
+	 * non-NULL b/c we shifted the final element, which is always set if
+	 * there is any saved, into slot0.
+	 */
+	if (savedlen < beyond) {
+		extra_drops = beyond - savedlen;
+		if (savedlen == 0) {
+			BUG_ON(slot0);
+			s0seq += extra_drops;
+			last_drop_seq = s0seq - 1;
+		} else {
+			extra_drops--; /* we aren't dropping what's in slot0 */
+			BUG_ON(!slot0);
+			list_add_tail(&slot0->list, list);
+			/* if extra_drops then we are going past this slot0
+			 * so we can safely advance last_drop_seq
+			 */
+			if (extra_drops)
+				last_drop_seq = s0seq + extra_drops;
+			s0seq += extra_drops + 1;
+			count++;
+		}
+		missed += extra_drops;
+		slot0 = NULL;
+		/* slot0 has had an empty slot pushed into it */
+	}
+
+	/* Remove the entries */
+	__vec_shift(xtfs, beyond);
+
+	/* Advance want seq */
+	xtfs->w_wantseq += beyond;
+
+	/* Process drops here when implementing congestion control */
+
+	/* We've shifted. plug the packet in at the end. */
+	xtfs->w_savedlen = nslots - 1;
+	xtfs->w_saved[xtfs->w_savedlen - 1].skb = inskb;
+	iptfs_set_window_drop_times(xtfs, xtfs->w_savedlen - 1);
+
+	/* if we don't have a slot0 then we must wait for it */
+	if (!slot0)
+		return count;
+
+	/* If slot0, seq must match new want seq */
+	BUG_ON(xtfs->w_wantseq != __esp_seq(slot0));
+
+	/* slot0 is valid, treat like we received expected. */
+	count += __reorder_this(xtfs, slot0, list);
+	return count;
+}
+
+/* Receive a new packet into the reorder window. Return a list of ordered
+ * packets from the window.
+ */
+static u32 iptfs_input_reorder(struct xfrm_iptfs_data *xtfs,
+			       struct sk_buff *inskb, struct list_head *list,
+			       struct list_head *freelist, u32 *fcount)
+{
+	const u32 nslots = xtfs->cfg.reorder_win_size + 1;
+	u64 inseq = __esp_seq(inskb);
+	u64 wantseq;
+
+	assert_spin_locked(&xtfs->drop_lock);
+
+	if (unlikely(!xtfs->w_seq_set)) {
+		xtfs->w_seq_set = true;
+		xtfs->w_wantseq = inseq;
+	}
+	wantseq = xtfs->w_wantseq;
+
+	if (likely(inseq == wantseq))
+		return __reorder_this(xtfs, inskb, list);
+	else if (inseq < wantseq)
+		return __reorder_past(xtfs, inskb, freelist, fcount);
+	else if ((inseq - wantseq) < nslots)
+		return __reorder_future_fits(xtfs, inskb, freelist, fcount);
+	else
+		return __reorder_future_shifts(xtfs, inskb, list, freelist,
+					       fcount);
+}
+
+/**
+ * iptfs_drop_timer() - Handle drop timer expiry.
+ *
+ * This is similar to our input function.
+ *
+ * The drop timer is set when we start an in progress reassembly, and also when
+ * we save a future packet in the window saved array.
+ *
+ * NOTE packets in the save window are always newer WRT drop times as
+ * they get further in the future. i.e. for:
+ *
+ *    if slots (S0, S1, ... Sn) and `Dn` is the drop time for slot `Sn`,
+ *    then D(n-1) <= D(n).
+ *
+ * So, regardless of why the timer is firing we can always discard any inprogress
+ * fragment; either it's the reassembly timer, or slot 0 is going to be
+ * dropped as S0 must have the most recent drop time, and slot 0 holds the
+ * continuation fragment of the in progress packet.
+ */
+static enum hrtimer_restart iptfs_drop_timer(struct hrtimer *me)
+{
+	struct sk_buff *skb, *next;
+	struct list_head freelist, list;
+	struct xfrm_iptfs_data *xtfs;
+	struct xfrm_state *x;
+	u32 count, fcount;
+
+	xtfs = container_of(me, typeof(*xtfs), drop_timer);
+	x = xtfs->x;
+
+	spin_lock(&xtfs->drop_lock);
+
+	INIT_LIST_HEAD(&list);
+	INIT_LIST_HEAD(&freelist);
+	fcount = 0;
+
+	/* Drop any in progress packet */
+
+	if (xtfs->ra_newskb) {
+		kfree_skb(xtfs->ra_newskb);
+		xtfs->ra_newskb = NULL;
+	}
+
+	/* Now drop as many packets as we should from the reordering window
+	 * saved array
+	 */
+	count = xtfs->w_savedlen ? __reorder_drop(xtfs, &list) : 0;
+
+	spin_unlock(&xtfs->drop_lock);
+
+	if (count) {
+		list_for_each_entry_safe(skb, next, &list, list) {
+			skb_list_del_init(skb);
+			(void)iptfs_input_ordered(x, skb);
+		}
+	}
+	return HRTIMER_NORESTART;
+}
+
+/**
+ * iptfs_input() - handle receipt of iptfs payload
+ * @x: xfrm state.
+ * @skb: the packet.
+ *
+ * We have an IPTFS payload order it if needed, then process newly in order
+ * packetsA.
+ */
+static int iptfs_input(struct xfrm_state *x, struct sk_buff *skb)
+{
+	struct list_head freelist, list;
+	struct xfrm_iptfs_data *xtfs = x->mode_data;
+	struct sk_buff *next;
+	u32 count, fcount;
+
+	/* Fast path for no reorder window. */
+	if (xtfs->cfg.reorder_win_size == 0) {
+		iptfs_input_ordered(x, skb);
+		goto done;
+	}
+
+	/* Fetch list of in-order packets from the reordering window as well as
+	 * a list of buffers we need to now free.
+	 */
+	INIT_LIST_HEAD(&list);
+	INIT_LIST_HEAD(&freelist);
+	fcount = 0;
+
+	spin_lock(&xtfs->drop_lock);
+	count = iptfs_input_reorder(xtfs, skb, &list, &freelist, &fcount);
+	spin_unlock(&xtfs->drop_lock);
+
+	if (count) {
+		list_for_each_entry_safe(skb, next, &list, list) {
+			skb_list_del_init(skb);
+			(void)iptfs_input_ordered(x, skb);
+		}
+	}
+
+	if (fcount) {
+		list_for_each_entry_safe(skb, next, &freelist, list) {
+			skb_list_del_init(skb);
+			kfree_skb(skb);
+		}
+	}
+done:
+	/* We always have dealt with the input SKB, either we are re-using it,
+	 * or we have freed it. Return EINPROGRESS so that xfrm_input stops
+	 * processing it.
+	 */
+	return -EINPROGRESS;
+}
+
+/* ================================= */
+/* IPTFS Sending (ingress) Functions */
+/* ================================= */
+
+/* ------------------------- */
+/* Enqueue to send functions */
+/* ------------------------- */
+
+/**
+ * iptfs_enqueue() - enqueue packet if ok to send.
+ * @xtfs: xtfs state
+ * @skb: the packet
+ *
+ * Return: true if packet enqueued.
+ */
+static bool iptfs_enqueue(struct xfrm_iptfs_data *xtfs, struct sk_buff *skb)
+{
+	u64 newsz = xtfs->queue_size + skb->len;
+	struct iphdr *iph;
+
+	assert_spin_locked(&xtfs->x->lock);
+
+	if (newsz > xtfs->cfg.max_queue_size)
+		return false;
+
+	/* Set ECN CE if we are above our ECN queue threshold */
+	if (newsz > xtfs->ecn_queue_size) {
+		iph = ip_hdr(skb);
+		if (iph->version == 4)
+			IP_ECN_set_ce(iph);
+		else if (iph->version == 6)
+			IP6_ECN_set_ce(skb, ipv6_hdr(skb));
+	}
+
+	__skb_queue_tail(&xtfs->queue, skb);
+	xtfs->queue_size += skb->len;
+	return true;
+}
+
+static int iptfs_get_cur_pmtu(struct xfrm_state *x, struct xfrm_iptfs_data *xtfs,
+			      struct sk_buff *skb)
+{
+	struct xfrm_dst *xdst = (struct xfrm_dst *)skb_dst(skb);
+	u32 payload_mtu = xtfs->payload_mtu;
+	u32 pmtu = __iptfs_get_inner_mtu(x, xdst->child_mtu_cached);
+
+	if (payload_mtu && payload_mtu < pmtu)
+		pmtu = payload_mtu;
+
+	return pmtu;
+}
+
+static int iptfs_is_too_big(struct sock *sk, struct sk_buff *skb, u32 pmtu)
+{
+	if (skb->len <= pmtu)
+		return 0;
+
+	/* We only send ICMP too big if the user has configured us as
+	 * dont-fragment.
+	 */
+	XFRM_INC_STATS(dev_net(skb->dev), LINUX_MIB_XFRMOUTERROR);
+
+	if (sk) {
+		xfrm_local_error(skb, pmtu);
+	} else if (ip_hdr(skb)->version == 4) {
+		icmp_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED,
+			  htonl(pmtu));
+	} else {
+		WARN_ON_ONCE(ip_hdr(skb)->version != 6);
+		icmpv6_send(skb, ICMPV6_PKT_TOOBIG, 0, pmtu);
+	}
+	return 1;
+}
+
+/* IPv4/IPv6 packet ingress to IPTFS tunnel, arrange to send in IPTFS payload
+ * (i.e., aggregating or fragmenting as appropriate).
+ * This is set in dst->output for an SA.
+ */
+static int iptfs_output_collect(struct net *net, struct sock *sk,
+				struct sk_buff *skb)
+{
+	struct dst_entry *dst = skb_dst(skb);
+	struct xfrm_state *x = dst->xfrm;
+	struct xfrm_iptfs_data *xtfs = x->mode_data;
+	struct sk_buff *segs, *nskb;
+	u32 count, qcount;
+	u32 pmtu = 0;
+	bool ok = true;
+	bool was_gso;
+
+	/* We have hooked into dst_entry->output which means we have skipped the
+	 * protocol specific netfilter (see xfrm4_output, xfrm6_output).
+	 * when our timer runs we will end up calling xfrm_output directly on
+	 * the encapsulated traffic.
+	 *
+	 * For both cases this is the NF_INET_POST_ROUTING hook which allows
+	 * changing the skb->dst entry which then may not be xfrm based anymore
+	 * in which case a REROUTED flag is set. and dst_output is called.
+	 *
+	 * For IPv6 we are also skipping fragmentation handling for local
+	 * sockets, which may or may not be good depending on our tunnel DF
+	 * setting. Normally with fragmentation supported we want to skip this
+	 * fragmentation.
+	 */
+
+	BUG_ON(!xtfs);
+
+	if (xtfs->cfg.dont_frag)
+		pmtu = iptfs_get_cur_pmtu(x, xtfs, skb);
+
+	/* Break apart GSO skbs. If the queue is nearing full then we want the
+	 * accounting and queuing to be based on the individual packets not on the
+	 * aggregate GSO buffer.
+	 */
+	was_gso = skb_is_gso(skb);
+	if (!was_gso) {
+		segs = skb;
+	} else {
+		segs = skb_gso_segment(skb, 0);
+		if (IS_ERR_OR_NULL(segs)) {
+			XFRM_INC_STATS(net, LINUX_MIB_XFRMOUTERROR);
+			kfree_skb(skb);
+			return PTR_ERR(segs);
+		}
+		consume_skb(skb);
+		skb = NULL;
+	}
+
+	count = 0;
+	qcount = 0;
+
+	/* We can be running on multiple cores and from the network softirq or
+	 * from user context depending on where the packet is coming from.
+	 */
+	spin_lock_bh(&x->lock);
+
+	skb_list_walk_safe(segs, skb, nskb) {
+		skb_mark_not_on_list(skb);
+		count++;
+
+		/* Once we drop due to no queue space we continue to drop the
+		 * rest of the packets from that GRO.
+		 */
+		if (!ok) {
+nospace:
+			trace_iptfs_no_queue_space(skb, xtfs, pmtu, was_gso);
+			XFRM_INC_STATS(dev_net(skb->dev), LINUX_MIB_XFRMOUTNOQSPACE);
+			kfree_skb_reason(skb, SKB_DROP_REASON_FULL_RING);
+			continue;
+		}
+
+		/* If the user indicated no iptfs fragmenting check before
+		 * enqueue.
+		 */
+		if (xtfs->cfg.dont_frag && iptfs_is_too_big(sk, skb, pmtu)) {
+			trace_iptfs_too_big(skb, xtfs, pmtu, was_gso);
+			kfree_skb_reason(skb, SKB_DROP_REASON_PKT_TOO_BIG);
+			continue;
+		}
+
+		/* Enqueue to send in tunnel */
+
+		ok = iptfs_enqueue(xtfs, skb);
+		if (!ok)
+			goto nospace;
+
+		trace_iptfs_enqueue(skb, xtfs, pmtu, was_gso);
+		qcount++;
+	}
+
+	/* Start a delay timer if we don't have one yet */
+	if (!hrtimer_is_queued(&xtfs->iptfs_timer)) {
+		/* softirq blocked lest the timer fire and interrupt us */
+		BUG_ON(!in_interrupt());
+		hrtimer_start(&xtfs->iptfs_timer, xtfs->init_delay_ns,
+			      IPTFS_HRTIMER_MODE);
+
+		xtfs->iptfs_settime = ktime_get_raw_fast_ns();
+		trace_iptfs_timer_start(xtfs, xtfs->init_delay_ns);
+	}
+
+	spin_unlock_bh(&x->lock);
+	return 0;
+}
+
+/* -------------------------- */
+/* Dequeue and send functions */
+/* -------------------------- */
+
+static void iptfs_output_prepare_skb(struct sk_buff *skb, u32 blkoff)
+{
+	struct ip_iptfs_hdr *h;
+	size_t hsz = sizeof(*h);
+
+	/* now reset values to be pointing at the rest of the packets */
+	h = skb_push(skb, hsz);
+	memset(h, 0, hsz);
+	if (blkoff)
+		h->block_offset = htons(blkoff);
+
+	/* network_header current points at the inner IP packet
+	 * move it to the iptfs header
+	 */
+	skb->transport_header = skb->network_header;
+	skb->network_header -= hsz;
+
+	IPCB(skb)->flags |= IPSKB_XFRM_TUNNEL_SIZE;
+
+	/* xfrm[46]_prepare_output sets skb->protocol here, but the resulting
+	 * called ip[6]_output functions also set this value as appropriate so
+	 * seems unnecessary
+	 *
+	 * skb->protocol = htons(ETH_P_IP) or htons(ETH_P_IPV6);
+	 */
+}
+
+/**
+ * iptfs_copy_create_frag() - create an inner fragment skb.
+ * @st: The source packet data.
+ * @offset: offset in @st of the new fragment data.
+ * @copy_len: the amount of data to copy from @st.
+ *
+ * Create a new skb holding a single IPTFS inner packet fragment. @copy_len must
+ * not be greater than the max fragment size.
+ *
+ * Return: the new fragment skb or an ERR_PTR().
+ */
+static struct sk_buff *iptfs_copy_create_frag(struct skb_seq_state *st,
+					      u32 offset, u32 copy_len)
+{
+	struct sk_buff *src = st->root_skb;
+	struct sk_buff *skb;
+	int err;
+
+	skb = iptfs_alloc_skb(src, copy_len, true);
+	if (!skb)
+		return ERR_PTR(-ENOMEM);
+
+	/* Now copy `copy_len` data from src */
+	err = skb_copy_bits_seq(st, offset, skb_put(skb, copy_len), copy_len);
+	if (err) {
+		XFRM_INC_STATS(dev_net(src->dev), LINUX_MIB_XFRMOUTERROR);
+		kfree_skb(skb);
+		return ERR_PTR(err);
+	}
+
+	return skb;
+}
+
+/**
+ * iptfs_copy_create_frags() - create and send N-1 fragments of a larger skb.
+ * @skbp: the source packet skb (IN), skb holding the last fragment in
+ *        the fragment stream (OUT).
+ * @xtfs: IPTFS SA state.
+ * @mtu: the max IPTFS fragment size.
+ *
+ * This function is responsible for fragmenting a larger inner packet into a
+ * sequence of IPTFS payload packets. The last fragment is returned rather than
+ * being sent so that the caller can append more inner packets (aggregation) if
+ * there is room.
+ */
+static int iptfs_copy_create_frags(struct sk_buff **skbp,
+				   struct xfrm_iptfs_data *xtfs, u32 mtu)
+{
+	struct skb_seq_state skbseq;
+	struct list_head sublist;
+	struct sk_buff *skb = *skbp;
+	struct sk_buff *nskb = *skbp;
+	u32 copy_len, offset;
+	u32 to_copy = skb->len - mtu;
+	u32 blkoff = 0;
+	int err = 0;
+
+	INIT_LIST_HEAD(&sublist);
+
+	BUG_ON(skb->len <= mtu);
+	skb_prepare_seq_read(skb, 0, skb->len, &skbseq);
+
+	/* A trimmed `skb` will be sent as the first fragment, later. */
+	offset = mtu;
+	to_copy = skb->len - offset;
+	while (to_copy) {
+		/* Send all but last fragment to allow agg. append */
+		trace_iptfs_first_fragmenting(nskb, mtu, to_copy, NULL);
+		list_add_tail(&nskb->list, &sublist);
+
+		/* FUTURE: if the packet has an odd/non-aligning length we could
+		 * send less data in the penultimate fragment so that the last
+		 * fragment then ends on an aligned boundary.
+		 */
+		copy_len = to_copy <= mtu ? to_copy : mtu;
+		nskb = iptfs_copy_create_frag(&skbseq, offset, copy_len);
+		if (IS_ERR(nskb)) {
+			XFRM_INC_STATS(dev_net(skb->dev),
+				       LINUX_MIB_XFRMOUTERROR);
+			skb_abort_seq_read(&skbseq);
+			err = PTR_ERR(nskb);
+			nskb = NULL;
+			break;
+		}
+		iptfs_output_prepare_skb(nskb, to_copy);
+		offset += copy_len;
+		to_copy -= copy_len;
+		blkoff = to_copy;
+	}
+	skb_abort_seq_read(&skbseq);
+
+	/* return last fragment that will be unsent (or NULL) */
+	*skbp = nskb;
+	if (nskb)
+		trace_iptfs_first_final_fragment(nskb, mtu, blkoff, NULL);
+
+	/* trim the original skb to MTU */
+	if (!err)
+		err = pskb_trim(skb, mtu);
+
+	if (err) {
+		/* Free all frags. Don't bother sending a partial packet we will
+		 * never complete.
+		 */
+		kfree_skb(nskb);
+		list_for_each_entry_safe(skb, nskb, &sublist, list) {
+			skb_list_del_init(skb);
+			kfree_skb(skb);
+		}
+		return err;
+	}
+
+	/* prepare the initial fragment with an iptfs header */
+	iptfs_output_prepare_skb(skb, 0);
+
+	/* Send all but last fragment. */
+	list_for_each_entry_safe(skb, nskb, &sublist, list) {
+		skb_list_del_init(skb);
+		xfrm_output(NULL, skb);
+	}
+
+	return 0;
+}
+
+/**
+ * iptfs_first_should_copy() - determine if we should copy packet data.
+ * @first_skb: the first skb in the packet
+ * @mtu: the MTU.
+ *
+ * Determine if we should create subsequent skbs to hold the remaining data from
+ * a large inner packet by copying the packet data, or cloning the original skb
+ * and adjusting the offsets.
+ */
+static bool iptfs_first_should_copy(struct sk_buff *first_skb, u32 mtu)
+{
+	u32 frag_copy_max;
+
+	/* If we have less than frag_copy_max for remaining packet we copy
+	 * those tail bytes as it is more efficient.
+	 */
+	frag_copy_max = mtu <= IPTFS_FRAG_COPY_MAX ? mtu : IPTFS_FRAG_COPY_MAX;
+	if ((int)first_skb->len - (int)mtu < (int)frag_copy_max)
+		return true;
+
+	/* If we have non-linear skb just use copy */
+	if (skb_is_nonlinear(first_skb))
+		return true;
+
+	/* So we have a simple linear skb, easy to clone and share */
+	return false;
+}
+
+/**
+ * iptfs_first_skb() - handle the first dequeued inner packet for output
+ * @skbp: the source packet skb (IN), skb holding the last fragment in
+ *        the fragment stream (OUT).
+ * @xtfs: IPTFS SA state.
+ * @mtu: the max IPTFS fragment size.
+ *
+ * This function is responsible for fragmenting a larger inner packet into a
+ * sequence of IPTFS payload packets. If it needs to fragment into subsequent
+ * skb's, it will either do so by copying or cloning.
+ *
+ * The last fragment is returned rather than being sent so that the caller can
+ * append more inner packets (aggregation) if there is room.
+ *
+ */
+static int iptfs_first_skb(struct sk_buff **skbp, struct xfrm_iptfs_data *xtfs,
+			   u32 mtu)
+{
+	struct sk_buff *skb = *skbp;
+	int err;
+
+	/* Classic ESP skips the don't fragment ICMP error if DF is clear on
+	 * the inner packet or ignore_df is set. Otherwise it will send an ICMP
+	 * or local error if the inner packet won't fit it's MTU.
+	 *
+	 * With IPTFS we do not care about the inner packet DF bit. If the
+	 * tunnel is configured to "don't fragment" we error back if things
+	 * don't fit in our max packet size. Otherwise we iptfs-fragment as
+	 * normal.
+	 */
+
+	/* The opportunity for HW offload has ended */
+	if (skb->ip_summed == CHECKSUM_PARTIAL) {
+		err = skb_checksum_help(skb);
+		if (err)
+			return err;
+	}
+
+	/* We've split these up before queuing */
+	BUG_ON(skb_is_gso(skb));
+
+	trace_iptfs_first_dequeue(skb, mtu, 0, ip_hdr(skb));
+
+	/* Simple case -- it fits. `mtu` accounted for all the overhead
+	 * including the basic IPTFS header.
+	 */
+	if (skb->len <= mtu) {
+		iptfs_output_prepare_skb(skb, 0);
+		return 0;
+	}
+
+	BUG_ON(xtfs->cfg.dont_frag);
+
+	if (iptfs_first_should_copy(skb, mtu))
+		return iptfs_copy_create_frags(skbp, xtfs, mtu);
+
+	/* For now we always copy */
+	return iptfs_copy_create_frags(skbp, xtfs, mtu);
+}
+
+static struct sk_buff **iptfs_rehome_fraglist(struct sk_buff **nextp,
+					      struct sk_buff *child)
+{
+	u32 fllen = 0;
+
+	BUG_ON(!skb_has_frag_list(child));
+
+	/* It might be possible to account for a frag list in addition to page
+	 * fragment if it's a valid state to be in. The page fragments size
+	 * should be kept as data_len so only the frag_list size is removed,
+	 * this must be done above as well took
+	 */
+	BUG_ON(skb_shinfo(child)->nr_frags);
+	*nextp = skb_shinfo(child)->frag_list;
+	while (*nextp) {
+		fllen += (*nextp)->len;
+		nextp = &(*nextp)->next;
+	}
+	skb_frag_list_init(child);
+	BUG_ON(fllen > child->data_len);
+	child->len -= fllen;
+	child->data_len -= fllen;
+
+	return nextp;
+}
+
+static void iptfs_consume_frags(struct sk_buff *to, struct sk_buff *from)
+{
+	struct skb_shared_info *fromi = skb_shinfo(from);
+	struct skb_shared_info *toi = skb_shinfo(to);
+	unsigned int new_truesize;
+
+	/* If we have data in a head page, grab it */
+	if (!skb_headlen(from)) {
+		new_truesize = SKB_TRUESIZE(skb_end_offset(from));
+	} else {
+		skb_head_to_frag(from, &toi->frags[toi->nr_frags]);
+		skb_frag_ref(to, toi->nr_frags++);
+		new_truesize = SKB_DATA_ALIGN(sizeof(struct sk_buff));
+	}
+
+	/* Move any other page fragments rather than copy */
+	memcpy(&toi->frags[toi->nr_frags], fromi->frags,
+	       sizeof(fromi->frags[0]) * fromi->nr_frags);
+	toi->nr_frags += fromi->nr_frags;
+	fromi->nr_frags = 0;
+	from->data_len = 0;
+	from->len = 0;
+	to->truesize += from->truesize - new_truesize;
+	from->truesize = new_truesize;
+
+	/* We are done with this SKB */
+	consume_skb(from);
+}
+
+static void iptfs_output_queued(struct xfrm_state *x, struct sk_buff_head *list)
+{
+	struct xfrm_iptfs_data *xtfs = x->mode_data;
+	u32 payload_mtu = xtfs->payload_mtu;
+	struct sk_buff *skb, *skb2, **nextp;
+	struct skb_shared_info *shi, *shi2;
+
+	/* For now we are just outputting packets as fast as we can, so if we
+	 * are fragmenting we will do so until the last inner packet has been
+	 * consumed.
+	 *
+	 * When we are fragmenting we need to output all outer packets that
+	 * contain the fragments of a single inner packet, consecutively (ESP
+	 * seq-wise). So we need a lock to keep another CPU from sending the
+	 * next batch of packets (it's `list`) and trying to output those, while
+	 * we output our `list` resuling with interleaved non-spec-client inner
+	 * packet streams. Thus we need to lock the IPTFS output on a per SA
+	 * basis while we process this list.
+	 */
+
+	/* NOTE: for the future, for timed packet sends, if our queue is not
+	 * growing longer (i.e., we are keeping up) and a packet we are about to
+	 * fragment will not fragment in then next outer packet, we might consider
+	 * holding on to it to send whole in the next slot. The question then is
+	 * does this introduce a continuous delay in the inner packet stream
+	 * with certain packet rates and sizes?
+	 */
+
+	/* and send them on their way */
+
+	while ((skb = __skb_dequeue(list))) {
+		struct xfrm_dst *xdst = (struct xfrm_dst *)skb_dst(skb);
+		u32 mtu = __iptfs_get_inner_mtu(x, xdst->child_mtu_cached);
+		bool share_ok = true;
+		int remaining;
+
+		/* protocol comes to us cleared sometimes */
+		skb->protocol = x->outer_mode.family == AF_INET ?
+					htons(ETH_P_IP) :
+					htons(ETH_P_IPV6);
+
+		if (payload_mtu && payload_mtu < mtu)
+			mtu = payload_mtu;
+
+		if (skb->len > mtu && xtfs->cfg.dont_frag) {
+			/* We handle this case before enqueueing so we are only
+			 * here b/c MTU changed after we enqueued before we
+			 * dequeued, just drop these.
+			 */
+			XFRM_INC_STATS(dev_net(skb->dev),
+				       LINUX_MIB_XFRMOUTERROR);
+
+			trace_iptfs_first_toobig(skb, mtu, 0, ip_hdr(skb));
+			kfree_skb_reason(skb, SKB_DROP_REASON_PKT_TOO_BIG);
+			continue;
+		}
+
+		/* iptfs_first_skb will free skb on error as well */
+		if (iptfs_first_skb(&skb, xtfs, mtu))
+			continue;
+
+		/* The returned skb is the last IPTFS fragment, it has it's
+		 * IPTFS header included and it's blkoff set just past the end
+		 * fragment data if needed. The space remaining to send more
+		 * inner packet data is `mtu` - (skb->len - sizeof iptfs
+		 * header). This is b/c the `mtu` value has the basic IPTFS
+		 * header len accounted for, and we added that header to the skb
+		 * so it is a part of skb->len, thus we subtract it from the skb
+		 * length.
+		 */
+		remaining = mtu - (skb->len - sizeof(struct ip_iptfs_hdr));
+
+		/* Re-home nested fragment lists. */
+		shi = skb_shinfo(skb);
+		nextp = &shi->frag_list;
+		while (*nextp) {
+			if (skb_has_frag_list(*nextp))
+				nextp = iptfs_rehome_fraglist(&(*nextp)->next,
+							      *nextp);
+			else
+				nextp = &(*nextp)->next;
+		}
+
+		if (shi->frag_list || skb_cloned(skb) || skb_shared(skb))
+			share_ok = false;
+
+		/* See if we have enough space to simply append.
+		 *
+		 * NOTE: Maybe do not append if we will be mis-aligned,
+		 * SW-based endpoints will probably have to copy in this
+		 * case.
+		 */
+		while ((skb2 = skb_peek(list))) {
+			trace_iptfs_ingress_nth_peek(skb2, remaining);
+			if (skb2->len > remaining)
+				break;
+
+			__skb_unlink(skb2, list);
+
+			/* The opportunity for HW offload has ended, if we
+			 * don't have a cksum in the packet we need to add one
+			 * before encap and transmit.
+			 */
+			if (skb2->ip_summed == CHECKSUM_PARTIAL) {
+				if (skb_checksum_help(skb2)) {
+					XFRM_INC_STATS(dev_net(skb_dst(skb2)->dev),
+						       LINUX_MIB_XFRMOUTERROR);
+					kfree_skb(skb2);
+					continue;
+				}
+			}
+
+			/* skb->pp_recycle is passed to __skb_flag_unref for all
+			 * frag pages so we can only share pages with skb's who
+			 * match ourselves.
+			 */
+			shi2 = skb_shinfo(skb2);
+			if (share_ok &&
+			    (shi2->frag_list ||
+			     (!skb2->head_frag && skb_headlen(skb)) ||
+			     skb->pp_recycle != skb2->pp_recycle ||
+			     skb_zcopy(skb2) ||
+			     (shi->nr_frags + shi2->nr_frags + 1 > MAX_SKB_FRAGS)))
+				share_ok = false;
+
+			/* do acct so we can free skb2 in share case */
+			skb->data_len += skb2->len;
+			skb->len += skb2->len;
+			remaining -= skb2->len;
+
+			trace_iptfs_ingress_nth_add(skb2, share_ok);
+
+			if (share_ok) {
+				iptfs_consume_frags(skb, skb2);
+			} else {
+				/* link on the frag_list */
+				*nextp = skb2;
+				nextp = &skb2->next;
+				BUG_ON(*nextp);
+				if (skb_has_frag_list(skb2))
+					nextp = iptfs_rehome_fraglist(nextp,
+								      skb2);
+				skb->truesize += skb2->truesize;
+			}
+		}
+
+		/* Consider fragmenting this skb2 that didn't fit. For demand
+		 * driven variable sized IPTFS pkts, though this isn't buying
+		 * a whole lot, especially if we are doing a copy which waiting
+		 * to send in a new pkt would not.
+		 */
+
+		xfrm_output(NULL, skb);
+	}
+}
+
+static enum hrtimer_restart iptfs_delay_timer(struct hrtimer *me)
+{
+	struct sk_buff_head list;
+	struct xfrm_iptfs_data *xtfs;
+	struct xfrm_state *x;
+	time64_t settime;
+	size_t osize;
+
+	xtfs = container_of(me, typeof(*xtfs), iptfs_timer);
+	x = xtfs->x;
+
+	/* Process all the queued packets
+	 *
+	 * softirq execution order: timer > tasklet > hrtimer
+	 *
+	 * Network rx will have run before us giving one last chance to queue
+	 * ingress packets for us to process and transmit.
+	 */
+
+	spin_lock(&x->lock);
+	__skb_queue_head_init(&list);
+	skb_queue_splice_init(&xtfs->queue, &list);
+	osize = xtfs->queue_size;
+	xtfs->queue_size = 0;
+	settime = xtfs->iptfs_settime;
+	spin_unlock(&x->lock);
+
+	/* After the above unlock, packets can begin queuing again, and the
+	 * timer can be set again, from another CPU either in softirq or user
+	 * context (not from this one since we are running at softirq level
+	 * already).
+	 */
+
+	trace_iptfs_timer_expire(xtfs,
+				 (unsigned long long)(ktime_get_raw_fast_ns() - settime));
+
+	iptfs_output_queued(x, &list);
+
+	return HRTIMER_NORESTART;
+}
+
+/**
+ * iptfs_encap_add_ipv4() - add outer encaps
+ *
+ * This was originally taken from xfrm4_tunnel_encap_add. The reason for the
+ * copy is that IP-TFS/AGGFRAG can have different functionality for how to set
+ * the TOS/DSCP bits. Sets the protocol to a different value and doesn't do
+ * anything with inner headers as they aren't pointing into a normal IP
+ * singleton inner packet.
+ */
+static int iptfs_encap_add_ipv4(struct xfrm_state *x, struct sk_buff *skb)
+{
+	struct dst_entry *dst = skb_dst(skb);
+	struct iphdr *top_iph;
+	int flags;
+
+	skb_reset_inner_network_header(skb);
+	skb_reset_inner_transport_header(skb);
+
+	skb_set_network_header(skb, -(x->props.header_len - x->props.enc_hdr_len));
+	skb->mac_header = skb->network_header + offsetof(struct iphdr, protocol);
+	skb->transport_header = skb->network_header + sizeof(*top_iph);
+
+	top_iph = ip_hdr(skb);
+	top_iph->ihl = 5;
+	top_iph->version = 4;
+	top_iph->protocol = IPPROTO_AGGFRAG;
+
+	/* As we have 0, fractional, 1 or N inner packets there's no obviously
+	 * correct DSCP mapping to inherit. ECN should be cleared per RFC9347
+	 * 3.1.
+	 */
+	top_iph->tos = 0;
+
+	flags = x->props.flags;
+	top_iph->frag_off = htons(IP_DF);
+	top_iph->ttl = ip4_dst_hoplimit(xfrm_dst_child(dst));
+	top_iph->saddr = x->props.saddr.a4;
+	top_iph->daddr = x->id.daddr.a4;
+	ip_select_ident(dev_net(dst->dev), skb, NULL);
+
+	return 0;
+}
+
+/**
+ * iptfs_encap_add_ipv6() - add outer encaps
+ *
+ * This was originally taken from xfrm6_tunnel_encap_add. The reason for the
+ * copy is that IP-TFS/AGGFRAG can have different functionality for how to set
+ * the flow label and TOS/DSCP bits. It also sets the protocol to a different
+ * value and doesn't do anything with inner headers as they aren't pointing into
+ * a normal IP singleton inner packet.
+ */
+static int iptfs_encap_add_ipv6(struct xfrm_state *x, struct sk_buff *skb)
+{
+	struct dst_entry *dst = skb_dst(skb);
+	struct ipv6hdr *top_iph;
+	int dsfield;
+
+	skb_reset_inner_network_header(skb);
+	skb_reset_inner_transport_header(skb);
+
+	skb_set_network_header(skb,
+			       -x->props.header_len + x->props.enc_hdr_len);
+	skb->mac_header = skb->network_header +
+		offsetof(struct ipv6hdr, nexthdr);
+	skb->transport_header = skb->network_header + sizeof(*top_iph);
+
+	top_iph = ipv6_hdr(skb);
+	top_iph->version = 6;
+	top_iph->priority = 0;
+	memset(top_iph->flow_lbl, 0, sizeof(top_iph->flow_lbl));
+	top_iph->nexthdr = IPPROTO_AGGFRAG;
+
+	/* As we have 0, fractional, 1 or N inner packets there's no obviously
+	 * correct DSCP mapping to inherit. ECN should be cleared per RFC9347
+	 * 3.1.
+	 */
+	dsfield = 0;
+	ipv6_change_dsfield(top_iph, 0, dsfield);
+
+	top_iph->hop_limit = ip6_dst_hoplimit(xfrm_dst_child(dst));
+	top_iph->saddr = *(struct in6_addr *)&x->props.saddr;
+	top_iph->daddr = *(struct in6_addr *)&x->id.daddr;
+
+	return 0;
+}
+
+/**
+ * iptfs_prepare_output() -  prepare the skb for output
+ *
+ * Return: Error value, if 0 then skb values should be as follows:
+ *    - transport_header should point at ESP header
+ *    - network_header should point at Outer IP header
+ *    - mac_header should point at protocol/nexthdr of the outer IP
+ */
+static int iptfs_prepare_output(struct xfrm_state *x, struct sk_buff *skb)
+{
+	if (x->outer_mode.family == AF_INET)
+		return iptfs_encap_add_ipv4(x, skb);
+	if (x->outer_mode.family == AF_INET6) {
+#if IS_ENABLED(CONFIG_IPV6)
+		return iptfs_encap_add_ipv6(x, skb);
+#else
+		WARN_ON_ONCE(1);
+		return -EAFNOSUPPORT;
+#endif
+	}
+	WARN_ON_ONCE(1);
+	return -EOPNOTSUPP;
+}
+
+/* ========================== */
+/* State Management Functions */
+/* ========================== */
+
+/**
+ * __iptfs_get_inner_mtu() - return inner MTU with no fragmentation.
+ */
+static u32 __iptfs_get_inner_mtu(struct xfrm_state *x, int outer_mtu)
+{
+	struct crypto_aead *aead;
+	u32 blksize;
+
+	aead = x->data;
+	blksize = ALIGN(crypto_aead_blocksize(aead), 4);
+	return ((outer_mtu - x->props.header_len - crypto_aead_authsize(aead)) &
+		~(blksize - 1)) - 2;
+}
+
+/**
+ * iptfs_get_mtu() - return the inner MTU for an IPTFS xfrm.
+ * @x:   XFRM state.
+ * @outer_mtu: Outer MTU for the encapsulated packet.
+ *
+ * Return: Correct MTU taking in to account the encap overhead.
+ */
+static u32 iptfs_get_inner_mtu(struct xfrm_state *x, int outer_mtu)
+{
+	struct xfrm_iptfs_data *xtfs = x->mode_data;
+
+	/* If not dont-frag we have no MTU */
+	if (!xtfs->cfg.dont_frag)
+		return x->outer_mode.family == AF_INET ? IP_MAX_MTU :
+							 IP6_MAX_MTU;
+	return __iptfs_get_inner_mtu(x, outer_mtu);
+}
+
+/**
+ * iptfs_user_init() - initialize the SA with IPTFS options from netlink.
+ */
+static int iptfs_user_init(struct net *net, struct xfrm_state *x,
+			   struct nlattr **attrs)
+{
+	struct xfrm_iptfs_data *xtfs = x->mode_data;
+	struct xfrm_iptfs_config *xc;
+
+	if (x->props.mode != XFRM_MODE_IPTFS)
+		return -EINVAL;
+
+	xc = &xtfs->cfg;
+	xc->reorder_win_size = net->xfrm.sysctl_iptfs_rewin;
+	xc->max_queue_size = net->xfrm.sysctl_iptfs_maxqsize;
+	xc->init_delay_us = net->xfrm.sysctl_iptfs_idelay;
+	xc->drop_time_us = net->xfrm.sysctl_iptfs_drptime;
+
+	if (attrs[XFRMA_IPTFS_DONT_FRAG])
+		xc->dont_frag = true;
+	if (attrs[XFRMA_IPTFS_REORD_WIN])
+		xc->reorder_win_size =
+			nla_get_u16(attrs[XFRMA_IPTFS_REORD_WIN]);
+	/* saved array is for saving 1..N seq nums from wantseq */
+	if (xc->reorder_win_size)
+		xtfs->w_saved = kcalloc(xc->reorder_win_size,
+					sizeof(*xtfs->w_saved), GFP_KERNEL);
+	if (attrs[XFRMA_IPTFS_PKT_SIZE]) {
+		xc->pkt_size = nla_get_u32(attrs[XFRMA_IPTFS_PKT_SIZE]);
+		if (!xc->pkt_size)
+			xtfs->payload_mtu = 0;
+		else if (xc->pkt_size > x->props.header_len)
+			xtfs->payload_mtu = xc->pkt_size - x->props.header_len;
+		else
+			return -EINVAL;
+	}
+	if (attrs[XFRMA_IPTFS_MAX_QSIZE])
+		xc->max_queue_size = nla_get_u32(attrs[XFRMA_IPTFS_MAX_QSIZE]);
+	if (attrs[XFRMA_IPTFS_DROP_TIME])
+		xc->drop_time_us = nla_get_u32(attrs[XFRMA_IPTFS_DROP_TIME]);
+	if (attrs[XFRMA_IPTFS_INIT_DELAY])
+		xc->init_delay_us = nla_get_u32(attrs[XFRMA_IPTFS_INIT_DELAY]);
+
+	xtfs->ecn_queue_size = (u64)xc->max_queue_size * 95 / 100;
+	xtfs->drop_time_ns = xc->drop_time_us * NSECS_IN_USEC;
+	xtfs->init_delay_ns = xc->init_delay_us * NSECS_IN_USEC;
+
+	return 0;
+}
+
+static int iptfs_copy_to_user(struct xfrm_state *x, struct sk_buff *skb)
+{
+	struct xfrm_iptfs_data *xtfs = x->mode_data;
+	struct xfrm_iptfs_config *xc = &xtfs->cfg;
+	int ret;
+
+	if (xc->dont_frag) {
+		ret = nla_put_flag(skb, XFRMA_IPTFS_DONT_FRAG);
+		if (ret)
+			return ret;
+	}
+	ret = nla_put_u16(skb, XFRMA_IPTFS_REORD_WIN, xc->reorder_win_size);
+	if (ret)
+		return ret;
+	ret = nla_put_u32(skb, XFRMA_IPTFS_PKT_SIZE, xc->pkt_size);
+	if (ret)
+		return ret;
+	ret = nla_put_u32(skb, XFRMA_IPTFS_MAX_QSIZE, xc->max_queue_size);
+	if (ret)
+		return ret;
+	ret = nla_put_u32(skb, XFRMA_IPTFS_DROP_TIME, xc->drop_time_us);
+	if (ret)
+		return ret;
+	ret = nla_put_u32(skb, XFRMA_IPTFS_INIT_DELAY, xc->init_delay_us);
+	return ret;
+}
+
+static int iptfs_create_state(struct xfrm_state *x)
+{
+	struct xfrm_iptfs_data *xtfs;
+
+	xtfs = kzalloc(sizeof(*xtfs), GFP_KERNEL);
+	if (!xtfs)
+		return -ENOMEM;
+	x->mode_data = xtfs;
+
+	xtfs->x = x;
+
+	__skb_queue_head_init(&xtfs->queue);
+	xtfs->init_delay_ns = xtfs->cfg.init_delay_us * NSECS_IN_USEC;
+	hrtimer_init(&xtfs->iptfs_timer, CLOCK_MONOTONIC, IPTFS_HRTIMER_MODE);
+	xtfs->iptfs_timer.function = iptfs_delay_timer;
+
+	xtfs->drop_time_ns = xtfs->cfg.drop_time_us * NSECS_IN_USEC;
+	spin_lock_init(&xtfs->drop_lock);
+	hrtimer_init(&xtfs->drop_timer, CLOCK_MONOTONIC, IPTFS_HRTIMER_MODE);
+	xtfs->drop_timer.function = iptfs_drop_timer;
+
+	/* Modify type (esp) adjustment values */
+
+	if (x->props.family == AF_INET)
+		x->props.header_len += sizeof(struct iphdr) + sizeof(struct ip_iptfs_hdr);
+	else if (x->props.family == AF_INET6)
+		x->props.header_len += sizeof(struct ipv6hdr) + sizeof(struct ip_iptfs_hdr);
+	x->props.enc_hdr_len = sizeof(struct ip_iptfs_hdr);
+
+	return 0;
+}
+
+static void iptfs_delete_state(struct xfrm_state *x)
+{
+	struct xfrm_iptfs_data *xtfs = x->mode_data;
+
+	if (IS_ERR_OR_NULL(xtfs))
+		return;
+
+	spin_lock(&xtfs->drop_lock);
+	hrtimer_cancel(&xtfs->iptfs_timer);
+	hrtimer_cancel(&xtfs->drop_timer);
+	spin_unlock(&xtfs->drop_lock);
+
+	kfree_sensitive(xtfs->w_saved);
+	kfree_sensitive(xtfs);
+}
+
+static const struct xfrm_mode_cbs iptfs_mode_cbs = {
+	.owner = THIS_MODULE,
+	.create_state = iptfs_create_state,
+	.delete_state = iptfs_delete_state,
+	.user_init = iptfs_user_init,
+	.copy_to_user = iptfs_copy_to_user,
+	.get_inner_mtu = iptfs_get_inner_mtu,
+	.input = iptfs_input,
+	.output = iptfs_output_collect,
+	.prepare_output = iptfs_prepare_output,
+};
+
+static int __init xfrm_iptfs_init(void)
+{
+	int err;
+
+	pr_info("xfrm_iptfs: IPsec IP-TFS tunnel mode module\n");
+
+	err = xfrm_register_mode_cbs(XFRM_MODE_IPTFS, &iptfs_mode_cbs);
+	if (err < 0)
+		pr_info("%s: can't register IP-TFS\n", __func__);
+
+	return err;
+}
+
+static void __exit xfrm_iptfs_fini(void)
+{
+	xfrm_unregister_mode_cbs(XFRM_MODE_IPTFS);
+}
+
+module_init(xfrm_iptfs_init);
+module_exit(xfrm_iptfs_fini);
+MODULE_LICENSE("GPL");
-- 
2.42.0

Re: [RFC ipsec-next v2 0/8] Add IP-TFS mode to xfrm

[Steffen Klassert]

On Sun, Nov 12, 2023 at 10:52:11PM -0500, Christian Hopps wrote:
> From: Christian Hopps <chopps@labn.net>
> 
> This patchset adds a new xfrm mode implementing on-demand IP-TFS. IP-TFS
> (AggFrag encapsulation) has been standardized in RFC9347.
> 
> Link: https://www.rfc-editor.org/rfc/rfc9347.txt
> 
> This feature supports demand driven (i.e., non-constant send rate) IP-TFS to
> take advantage of the AGGFRAG ESP payload encapsulation. This payload type
> supports aggregation and fragmentation of the inner IP packet stream which in
> turn yields higher small-packet bandwidth as well as reducing MTU/PMTU issues.
> Congestion control is unimplementated as the send rate is demand driven rather
> than constant.
> 
> In order to allow loading this fucntionality as a module a set of callbacks
> xfrm_mode_cbs has been added to xfrm as well.

I did a multiple days peer review with Chris on this pachset. So my
concerns are already addressed.

Further reviews are welcome! This is a bigger change and it would
be nice if more people could look at it.

Thanks!

Re: [RFC ipsec-next v2 8/8] iptfs: impl: add new iptfs xfrm mode impl

[kernel test robot]

Hi Christian,

[This is a private test report for your RFC patch.]
kernel test robot noticed the following build warnings:

[auto build test WARNING on klassert-ipsec-next/master]
[also build test WARNING on klassert-ipsec/master netfilter-nf/main linus/master v6.7-rc1 next-20231113]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Christian-Hopps/iptfs-config-add-CONFIG_XFRM_IPTFS/20231113-115642
base:   https://git.kernel.org/pub/scm/linux/kernel/git/klassert/ipsec-next.git master
patch link:    https://lore.kernel.org/r/20231113035219.920136-9-chopps%40chopps.org
patch subject: [RFC ipsec-next v2 8/8] iptfs: impl: add new iptfs xfrm mode impl
config: nios2-allmodconfig (https://download.01.org/0day-ci/archive/20231113/202311131710.pFEZgGgP-lkp@intel.com/config)
compiler: nios2-linux-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231113/202311131710.pFEZgGgP-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202311131710.pFEZgGgP-lkp@intel.com/

All warnings (new ones prefixed by >>):

   net/xfrm/xfrm_iptfs.c:468:17: warning: no previous prototype for 'iptfs_pskb_add_frags' [-Wmissing-prototypes]
     468 | struct sk_buff *iptfs_pskb_add_frags(struct sk_buff *tpl,
         |                 ^~~~~~~~~~~~~~~~~~~~
   net/xfrm/xfrm_iptfs.c: In function '__iptfs_reassem_done':
   net/xfrm/xfrm_iptfs.c:629:13: warning: variable 'ret' set but not used [-Wunused-but-set-variable]
     629 |         int ret;
         |             ^~~
   net/xfrm/xfrm_iptfs.c: In function '__reorder_drop':
   net/xfrm/xfrm_iptfs.c:1234:13: warning: variable 'wantseq' set but not used [-Wunused-but-set-variable]
    1234 |         u64 wantseq = xtfs->w_wantseq;
         |             ^~~~~~~
   net/xfrm/xfrm_iptfs.c: In function '__reorder_this':
   net/xfrm/xfrm_iptfs.c:1283:13: warning: variable 'wantseq' set but not used [-Wunused-but-set-variable]
    1283 |         u64 wantseq = xtfs->w_wantseq;
         |             ^~~~~~~
   net/xfrm/xfrm_iptfs.c: In function '__reorder_future_shifts':
   net/xfrm/xfrm_iptfs.c:1405:13: warning: variable 'last_drop_seq' set but not used [-Wunused-but-set-variable]
    1405 |         u64 last_drop_seq = xtfs->w_wantseq;
         |             ^~~~~~~~~~~~~
   net/xfrm/xfrm_iptfs.c: In function 'iptfs_drop_timer':
   net/xfrm/xfrm_iptfs.c:1625:20: warning: variable 'fcount' set but not used [-Wunused-but-set-variable]
    1625 |         u32 count, fcount;
         |                    ^~~~~~
   net/xfrm/xfrm_iptfs.c: In function 'iptfs_delay_timer':
   net/xfrm/xfrm_iptfs.c:2357:16: warning: variable 'osize' set but not used [-Wunused-but-set-variable]
    2357 |         size_t osize;
         |                ^~~~~
   net/xfrm/xfrm_iptfs.c: In function 'iptfs_encap_add_ipv4':
   net/xfrm/xfrm_iptfs.c:2405:13: warning: variable 'flags' set but not used [-Wunused-but-set-variable]
    2405 |         int flags;
         |             ^~~~~
   net/xfrm/xfrm_iptfs.c: At top level:
   net/xfrm/xfrm_iptfs.c:100:12: warning: '__trace_ip_proto_seq' defined but not used [-Wunused-function]
     100 | static u32 __trace_ip_proto_seq(struct iphdr *iph)
         |            ^~~~~~~~~~~~~~~~~~~~
>> net/xfrm/xfrm_iptfs.c:93:12: warning: '__trace_ip_proto' defined but not used [-Wunused-function]
      93 | static u32 __trace_ip_proto(struct iphdr *iph)
         |            ^~~~~~~~~~~~~~~~


vim +/__trace_ip_proto +93 net/xfrm/xfrm_iptfs.c

    92	
  > 93	static u32 __trace_ip_proto(struct iphdr *iph)
    94	{
    95		if (iph->version == 4)
    96			return iph->protocol;
    97		return ((struct ipv6hdr *)iph)->nexthdr;
    98	}
    99	

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [devel-ipsec] [RFC ipsec-next v2 6/8] iptfs: xfrm: Add mode_cbs module functionality

[Antony Antony]

[-- Attachment #1: Type: text/plain, Size: 11594 bytes --]

Hi Chris,

I got couple of more questions. I am trying flush them out. Here is one. And 
I am still formulating another question.

On Sun, Nov 12, 2023 at 10:52:17PM -0500, Christian Hopps via Devel wrote:
> From: Christian Hopps <chopps@labn.net>
> 
> Add a set of callbacks xfrm_mode_cbs to xfrm_state. These callbacks
> enable the addition of new xfrm modes, such as IP-TFS to be defined
> in modules.
> 
> Signed-off-by: Christian Hopps <chopps@labn.net>
> ---
>  include/net/xfrm.h     | 36 ++++++++++++++++++++++++++++++++++++
>  net/xfrm/xfrm_device.c |  3 ++-
>  net/xfrm/xfrm_input.c  | 14 ++++++++++++--
>  net/xfrm/xfrm_output.c |  9 +++++++--
>  net/xfrm/xfrm_policy.c | 18 +++++++++++-------
>  net/xfrm/xfrm_state.c  | 41 +++++++++++++++++++++++++++++++++++++++++
>  net/xfrm/xfrm_user.c   | 10 ++++++++++
>  7 files changed, 119 insertions(+), 12 deletions(-)
> 
> diff --git a/include/net/xfrm.h b/include/net/xfrm.h
> index d2e87344d175..aeeadadc9545 100644
> --- a/include/net/xfrm.h
> +++ b/include/net/xfrm.h
> @@ -204,6 +204,7 @@ struct xfrm_state {
>  		u16		family;
>  		xfrm_address_t	saddr;
>  		int		header_len;
> +		int		enc_hdr_len;
>  		int		trailer_len;
>  		u32		extra_flags;
>  		struct xfrm_mark	smark;
> @@ -289,6 +290,9 @@ struct xfrm_state {
>  	/* Private data of this transformer, format is opaque,
>  	 * interpreted by xfrm_type methods. */
>  	void			*data;
> +
> +	const struct xfrm_mode_cbs	*mode_cbs;
> +	void				*mode_data;
>  };
>  
>  static inline struct net *xs_net(struct xfrm_state *x)
> @@ -441,6 +445,38 @@ struct xfrm_type_offload {
>  int xfrm_register_type_offload(const struct xfrm_type_offload *type, unsigned short family);
>  void xfrm_unregister_type_offload(const struct xfrm_type_offload *type, unsigned short family);
>  
> +struct xfrm_mode_cbs {
> +	struct module	*owner;
> +	/* Add/delete state in the new xfrm_state in `x`. */
> +	int	(*create_state)(struct xfrm_state *x);
> +	void	(*delete_state)(struct xfrm_state *x);
> +
> +	/* Called while handling the user netlink options. */
> +	int	(*user_init)(struct net *net, struct xfrm_state *x,
> +			     struct nlattr **attrs);
> +	int	(*copy_to_user)(struct xfrm_state *x, struct sk_buff *skb);


I'm curious about how xfrm_state_clone() is supported, particularly when 
it's called in this sequence:


XFRM_MSG_MIGRATE
xfrm_do_migrate()
   xfrm_migrate()
     xfrm_state_migrate()
        xfrm_state_clone()

I've been pondering if perhaps IP-TFS is missing clone support?
It seems like something along the lines of the attached fragment might be 
needed.

And speaking of clone, it got me thinking: could there be packet drops when 
XFRM_MSG_UPDSA triggers iptfs_delete_state()? Given that an update creates a 
new state and deletes the old one, it's possible there might be some packet 
loss. Maybe it's unavoidable?

Also, I've been wondering state timers or limits expire. What happens when 
the byte or packet limit exceeds and the state gets deleted? Could that lead 
to packet loss? The same question applies to rekeying from userspace IKEd.  
Users are often very sensitive to packet loss during rekeying, and a lot of 
effort has gone into minimizing it. I'm curious if this has been carefully 
handled, or if there are considerations we should be aware of."

> +
> +	u32	(*get_inner_mtu)(struct xfrm_state *x, int outer_mtu);
> +
> +	/* Called to handle received xfrm (egress) packets. */
> +	int	(*input)(struct xfrm_state *x, struct sk_buff *skb);
> +
> +	/* Placed in dst_output of the dst when an xfrm_state is bound. */
> +	int	(*output)(struct net *net, struct sock *sk, struct sk_buff *skb);
> +
> +	/**
> +	 * Prepare the skb for output for the given mode. Returns:
> +	 *    Error value, if 0 then skb values should be as follows:
> +	 *    transport_header should point at ESP header
> +	 *    network_header should point at Outer IP header
> +	 *    mac_header should point at protocol/nexthdr of the outer IP
> +	 */
> +	int	(*prepare_output)(struct xfrm_state *x, struct sk_buff *skb);
> +};
> +
> +int xfrm_register_mode_cbs(u8 mode, const struct xfrm_mode_cbs *mode_cbs);
> +void xfrm_unregister_mode_cbs(u8 mode);
> +
>  static inline int xfrm_af2proto(unsigned int family)
>  {
>  	switch(family) {
> diff --git a/net/xfrm/xfrm_device.c b/net/xfrm/xfrm_device.c
> index 3784534c9185..8b848540ea47 100644
> --- a/net/xfrm/xfrm_device.c
> +++ b/net/xfrm/xfrm_device.c
> @@ -42,7 +42,8 @@ static void __xfrm_mode_tunnel_prep(struct xfrm_state *x, struct sk_buff *skb,
>  		skb->transport_header = skb->network_header + hsize;
>  
>  	skb_reset_mac_len(skb);
> -	pskb_pull(skb, skb->mac_len + x->props.header_len);
> +	pskb_pull(skb,
> +		  skb->mac_len + x->props.header_len - x->props.enc_hdr_len);
>  }
>  
>  static void __xfrm_mode_beet_prep(struct xfrm_state *x, struct sk_buff *skb,
> diff --git a/net/xfrm/xfrm_input.c b/net/xfrm/xfrm_input.c
> index bd4ce21d76d7..824f7b7f90e0 100644
> --- a/net/xfrm/xfrm_input.c
> +++ b/net/xfrm/xfrm_input.c
> @@ -437,6 +437,9 @@ static int xfrm_inner_mode_input(struct xfrm_state *x,
>  		WARN_ON_ONCE(1);
>  		break;
>  	default:
> +		if (x->mode_cbs && x->mode_cbs->input)
> +			return x->mode_cbs->input(x, skb);
> +
>  		WARN_ON_ONCE(1);
>  		break;
>  	}
> @@ -479,6 +482,10 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
>  
>  		family = x->props.family;
>  
> +		/* An encap_type of -3 indicates reconstructed inner packet */
> +		if (encap_type == -3)
> +			goto resume_decapped;
> +
>  		/* An encap_type of -1 indicates async resumption. */
>  		if (encap_type == -1) {
>  			async = 1;
> @@ -660,11 +667,14 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
>  
>  		XFRM_MODE_SKB_CB(skb)->protocol = nexthdr;
>  
> -		if (xfrm_inner_mode_input(x, skb)) {
> +		err = xfrm_inner_mode_input(x, skb);
> +		if (err == -EINPROGRESS)
> +			return 0;
> +		else if (err) {
>  			XFRM_INC_STATS(net, LINUX_MIB_XFRMINSTATEMODEERROR);
>  			goto drop;
>  		}
> -
> +resume_decapped:
>  		if (x->outer_mode.flags & XFRM_MODE_FLAG_TUNNEL) {
>  			decaps = 1;
>  			break;
> diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
> index 662c83beb345..4390c111410d 100644
> --- a/net/xfrm/xfrm_output.c
> +++ b/net/xfrm/xfrm_output.c
> @@ -280,7 +280,9 @@ static int xfrm4_tunnel_encap_add(struct xfrm_state *x, struct sk_buff *skb)
>  	skb_set_inner_network_header(skb, skb_network_offset(skb));
>  	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
>  
> -	skb_set_network_header(skb, -x->props.header_len);
> +	/* backup to add space for the outer encap */
> +	skb_set_network_header(skb,
> +			       -x->props.header_len + x->props.enc_hdr_len);
>  	skb->mac_header = skb->network_header +
>  			  offsetof(struct iphdr, protocol);
>  	skb->transport_header = skb->network_header + sizeof(*top_iph);
> @@ -325,7 +327,8 @@ static int xfrm6_tunnel_encap_add(struct xfrm_state *x, struct sk_buff *skb)
>  	skb_set_inner_network_header(skb, skb_network_offset(skb));
>  	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
>  
> -	skb_set_network_header(skb, -x->props.header_len);
> +	skb_set_network_header(skb,
> +			       -x->props.header_len + x->props.enc_hdr_len);
>  	skb->mac_header = skb->network_header +
>  			  offsetof(struct ipv6hdr, nexthdr);
>  	skb->transport_header = skb->network_header + sizeof(*top_iph);
> @@ -472,6 +475,8 @@ static int xfrm_outer_mode_output(struct xfrm_state *x, struct sk_buff *skb)
>  		WARN_ON_ONCE(1);
>  		break;
>  	default:
> +		if (x->mode_cbs->prepare_output)
> +			return x->mode_cbs->prepare_output(x, skb);
>  		WARN_ON_ONCE(1);
>  		break;
>  	}
> diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
> index d2dddc570f4f..3220b01121f3 100644
> --- a/net/xfrm/xfrm_policy.c
> +++ b/net/xfrm/xfrm_policy.c
> @@ -2707,13 +2707,17 @@ static struct dst_entry *xfrm_bundle_create(struct xfrm_policy *policy,
>  
>  		dst1->input = dst_discard;
>  
> -		rcu_read_lock();
> -		afinfo = xfrm_state_afinfo_get_rcu(inner_mode->family);
> -		if (likely(afinfo))
> -			dst1->output = afinfo->output;
> -		else
> -			dst1->output = dst_discard_out;
> -		rcu_read_unlock();
> +		if (xfrm[i]->mode_cbs && xfrm[i]->mode_cbs->output) {
> +			dst1->output = xfrm[i]->mode_cbs->output;
> +		} else {
> +			rcu_read_lock();
> +			afinfo = xfrm_state_afinfo_get_rcu(inner_mode->family);
> +			if (likely(afinfo))
> +				dst1->output = afinfo->output;
> +			else
> +				dst1->output = dst_discard_out;
> +			rcu_read_unlock();
> +		}
>  
>  		xdst_prev = xdst;
>  
> diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
> index bda5327bf34d..f5e1a17ebf74 100644
> --- a/net/xfrm/xfrm_state.c
> +++ b/net/xfrm/xfrm_state.c
> @@ -513,6 +513,36 @@ static const struct xfrm_mode *xfrm_get_mode(unsigned int encap, int family)
>  	return NULL;
>  }
>  
> +static struct xfrm_mode_cbs xfrm_mode_cbs_map[XFRM_MODE_MAX];
> +
> +int xfrm_register_mode_cbs(u8 mode, const struct xfrm_mode_cbs *mode_cbs)
> +{
> +	if (mode >= XFRM_MODE_MAX)
> +		return -EINVAL;
> +
> +	xfrm_mode_cbs_map[mode] = *mode_cbs;
> +	return 0;
> +}
> +EXPORT_SYMBOL(xfrm_register_mode_cbs);
> +
> +void xfrm_unregister_mode_cbs(u8 mode)
> +{
> +	if (mode >= XFRM_MODE_MAX)
> +		return;
> +
> +	memset(&xfrm_mode_cbs_map[mode], 0, sizeof(xfrm_mode_cbs_map[mode]));
> +}
> +EXPORT_SYMBOL(xfrm_unregister_mode_cbs);
> +
> +static const struct xfrm_mode_cbs *xfrm_get_mode_cbs(u8 mode)
> +{
> +	if (mode >= XFRM_MODE_MAX)
> +		return NULL;
> +	if (mode == XFRM_MODE_IPTFS && !xfrm_mode_cbs_map[mode].create_state)
> +		request_module("xfrm-iptfs");
> +	return &xfrm_mode_cbs_map[mode];
> +}
> +
>  void xfrm_state_free(struct xfrm_state *x)
>  {
>  	kmem_cache_free(xfrm_state_cache, x);
> @@ -521,6 +551,8 @@ EXPORT_SYMBOL(xfrm_state_free);
>  
>  static void ___xfrm_state_destroy(struct xfrm_state *x)
>  {
> +	if (x->mode_cbs && x->mode_cbs->delete_state)
> +		x->mode_cbs->delete_state(x);
>  	hrtimer_cancel(&x->mtimer);
>  	del_timer_sync(&x->rtimer);
>  	kfree(x->aead);
> @@ -2765,6 +2797,9 @@ u32 xfrm_state_mtu(struct xfrm_state *x, int mtu)
>  	case XFRM_MODE_TUNNEL:
>  		break;
>  	default:
> +		if (x->mode_cbs && x->mode_cbs->get_inner_mtu)
> +			return x->mode_cbs->get_inner_mtu(x, mtu);
> +
>  		WARN_ON_ONCE(1);
>  		break;
>  	}
> @@ -2850,6 +2885,12 @@ int __xfrm_init_state(struct xfrm_state *x, bool init_replay, bool offload,
>  			goto error;
>  	}
>  
> +	x->mode_cbs = xfrm_get_mode_cbs(x->props.mode);
> +	if (x->mode_cbs && x->mode_cbs->create_state) {
> +		err = x->mode_cbs->create_state(x);
> +		if (err)
> +			goto error;
> +	}
>  error:
>  	return err;
>  }
> diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c
> index fa2059de51f5..795da945fbc2 100644
> --- a/net/xfrm/xfrm_user.c
> +++ b/net/xfrm/xfrm_user.c
> @@ -779,6 +779,12 @@ static struct xfrm_state *xfrm_state_construct(struct net *net,
>  			goto error;
>  	}
>  
> +	if (x->mode_cbs && x->mode_cbs->user_init) {
> +		err = x->mode_cbs->user_init(net, x, attrs);
> +		if (err)
> +			goto error;
> +	}
> +
>  	return x;
>  
>  error:
> @@ -1192,6 +1198,10 @@ static int copy_to_user_state_extra(struct xfrm_state *x,
>  		if (ret)
>  			goto out;
>  	}
> +	if (x->mode_cbs && x->mode_cbs->copy_to_user)
> +		ret = x->mode_cbs->copy_to_user(x, skb);
> +	if (ret)
> +		goto out;
>  	if (x->mapping_maxage)
>  		ret = nla_put_u32(skb, XFRMA_MTIMER_THRESH, x->mapping_maxage);
>  out:
> -- 
> 2.42.0
> 
> -- 
> Devel mailing list
> Devel@linux-ipsec.org
> https://linux-ipsec.org/mailman/listinfo/devel

[-- Attachment #2: 0001-xfrm-iptfs-migrate-poc.patch --]
[-- Type: text/plain, Size: 2641 bytes --]

From 00fc5af96f90846f1e1882b141699fb62a9f0a73 Mon Sep 17 00:00:00 2001
From: Antony Antony <antony.antony@secunet.com>
Date: Mon, 13 Nov 2023 14:20:45 +0100
Subject: [PATCH] xfrm iptfs migrate poc

proof of concept for IP-TFS migrate support
---
 include/net/xfrm.h    |  1 +
 net/xfrm/xfrm_iptfs.c | 20 ++++++++++++++++++++
 net/xfrm/xfrm_state.c |  6 ++++++
 3 files changed, 27 insertions(+)

diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index a6e0e848918d..176ab5ac436e 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -456,6 +456,7 @@ struct xfrm_mode_cbs {
 	int	(*user_init)(struct net *net, struct xfrm_state *x,
 			     struct nlattr **attrs);
 	int	(*copy_to_user)(struct xfrm_state *x, struct sk_buff *skb);
+	int     (*clone)(struct xfrm_state *orig, struct xfrm_state *x);
 
 	u32	(*get_inner_mtu)(struct xfrm_state *x, int outer_mtu);
 
diff --git a/net/xfrm/xfrm_iptfs.c b/net/xfrm/xfrm_iptfs.c
index 65f7acdbe6a8..cef269a02b11 100644
--- a/net/xfrm/xfrm_iptfs.c
+++ b/net/xfrm/xfrm_iptfs.c
@@ -2613,6 +2613,25 @@ static int iptfs_copy_to_user(struct xfrm_state *x, struct sk_buff *skb)
 	return ret;
 }
 
+static int iptfs_clone(struct xfrm_state *orig,  struct xfrm_state *x)
+{
+
+	x->mode_data = kmemdup(orig->mode_data, sizeof(*x->mode_data),
+			       GFP_KERNEL);
+	/**  may be some values, such as the following, should not be copied ??
+	 * and need different handling ?
+	 * xtfs->iptfs_timer;
+	 * xtfs->drop_timer
+         * xtfs->drop_lock
+         * xtfs->w_saved
+         * xtfs;
+	 */
+
+	if (!x->mode_data)
+		return -ENOMEM;
+
+	return 0;
+}
 static int iptfs_create_state(struct xfrm_state *x)
 {
 	struct xfrm_iptfs_data *xtfs;
@@ -2667,6 +2686,7 @@ static const struct xfrm_mode_cbs iptfs_mode_cbs = {
 	.delete_state = iptfs_delete_state,
 	.user_init = iptfs_user_init,
 	.copy_to_user = iptfs_copy_to_user,
+	.clone = iptfs_clone,
 	.get_inner_mtu = iptfs_get_inner_mtu,
 	.input = iptfs_input,
 	.output = iptfs_output_collect,
diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
index 786f3fc0d428..c56d3be56229 100644
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -720,6 +720,7 @@ struct xfrm_state *xfrm_state_alloc(struct net *net)
 		x->replay_maxage = 0;
 		x->replay_maxdiff = 0;
 		spin_lock_init(&x->lock);
+		x->mode_data = NULL;
 	}
 	return x;
 }
@@ -1787,6 +1788,11 @@ static struct xfrm_state *xfrm_state_clone(struct xfrm_state *orig,
 	x->new_mapping = 0;
 	x->new_mapping_sport = 0;
 
+	if (x->mode_cbs && x->mode_cbs->clone && orig->mode_data) {
+		if (!x->mode_cbs->clone(x,orig))
+			goto error;
+	}
+
 	return x;
 
  error:
-- 
2.42.0

Re: [devel-ipsec] [RFC ipsec-next v2 6/8] iptfs: xfrm: Add mode_cbs module functionality

[Antony Antony]

[-- Attachment #1: Type: text/plain, Size: 15385 bytes --]

Hi Chris,

On Mon, Nov 13, 2023 at 03:07:59PM +0100, Antony Antony wrote:
> Hi Chris,
> 
> I got couple of more questions. I am trying flush them out. Here is one. And 
> I am still formulating another question.
> 
> On Sun, Nov 12, 2023 at 10:52:17PM -0500, Christian Hopps via Devel wrote:
> > From: Christian Hopps <chopps@labn.net>
> > 
> > Add a set of callbacks xfrm_mode_cbs to xfrm_state. These callbacks
> > enable the addition of new xfrm modes, such as IP-TFS to be defined
> > in modules.
> > 
> > Signed-off-by: Christian Hopps <chopps@labn.net>
> > ---
> >  include/net/xfrm.h     | 36 ++++++++++++++++++++++++++++++++++++
> >  net/xfrm/xfrm_device.c |  3 ++-
> >  net/xfrm/xfrm_input.c  | 14 ++++++++++++--
> >  net/xfrm/xfrm_output.c |  9 +++++++--
> >  net/xfrm/xfrm_policy.c | 18 +++++++++++-------
> >  net/xfrm/xfrm_state.c  | 41 +++++++++++++++++++++++++++++++++++++++++
> >  net/xfrm/xfrm_user.c   | 10 ++++++++++
> >  7 files changed, 119 insertions(+), 12 deletions(-)
> > 
> > diff --git a/include/net/xfrm.h b/include/net/xfrm.h
> > index d2e87344d175..aeeadadc9545 100644
> > --- a/include/net/xfrm.h
> > +++ b/include/net/xfrm.h
> > @@ -204,6 +204,7 @@ struct xfrm_state {
> >  		u16		family;
> >  		xfrm_address_t	saddr;
> >  		int		header_len;
> > +		int		enc_hdr_len;
> >  		int		trailer_len;
> >  		u32		extra_flags;
> >  		struct xfrm_mark	smark;
> > @@ -289,6 +290,9 @@ struct xfrm_state {
> >  	/* Private data of this transformer, format is opaque,
> >  	 * interpreted by xfrm_type methods. */
> >  	void			*data;
> > +
> > +	const struct xfrm_mode_cbs	*mode_cbs;
> > +	void				*mode_data;
> >  };
> >  
> >  static inline struct net *xs_net(struct xfrm_state *x)
> > @@ -441,6 +445,38 @@ struct xfrm_type_offload {
> >  int xfrm_register_type_offload(const struct xfrm_type_offload *type, unsigned short family);
> >  void xfrm_unregister_type_offload(const struct xfrm_type_offload *type, unsigned short family);
> >  
> > +struct xfrm_mode_cbs {
> > +	struct module	*owner;
> > +	/* Add/delete state in the new xfrm_state in `x`. */
> > +	int	(*create_state)(struct xfrm_state *x);
> > +	void	(*delete_state)(struct xfrm_state *x);
> > +
> > +	/* Called while handling the user netlink options. */
> > +	int	(*user_init)(struct net *net, struct xfrm_state *x,
> > +			     struct nlattr **attrs);
> > +	int	(*copy_to_user)(struct xfrm_state *x, struct sk_buff *skb);
> 
> 
> I'm curious about how xfrm_state_clone() is supported, particularly when 
> it's called in this sequence:
> 
> 
> XFRM_MSG_MIGRATE
> xfrm_do_migrate()
>    xfrm_migrate()
>      xfrm_state_migrate()
>         xfrm_state_clone()
> 
> I've been pondering if perhaps IP-TFS is missing clone support?
> It seems like something along the lines of the attached fragment might be 
> needed.

Here's a tested fix that supports XFRM_MSG_MIGRATE with iptfs. I tested it 
using IPv4, and it's working. I included some memset with zeros to be sure 
those timers are initialized properly. May be it is not necessary!

> And speaking of clone, it got me thinking: could there be packet drops 
> when XFRM_MSG_UPDSA triggers iptfs_delete_state()? Given that an update 
> creates a new state and deletes the old one, it's possible there might be 
> some packet loss. Maybe it's unavoidable?
> 
> Also, I've been wondering state timers or limits expire. What happens when 
> the byte or packet limit exceeds and the state gets deleted? Could that lead 
> to packet loss? The same question applies to rekeying from userspace IKEd.  
> Users are often very sensitive to packet loss during rekeying, and a lot of 
> effort has gone into minimizing it. I'm curious if this has been carefully 
> handled, or if there are considerations we should be aware of."
> 
> > +
> > +	u32	(*get_inner_mtu)(struct xfrm_state *x, int outer_mtu);
> > +
> > +	/* Called to handle received xfrm (egress) packets. */
> > +	int	(*input)(struct xfrm_state *x, struct sk_buff *skb);
> > +
> > +	/* Placed in dst_output of the dst when an xfrm_state is bound. */
> > +	int	(*output)(struct net *net, struct sock *sk, struct sk_buff *skb);
> > +
> > +	/**
> > +	 * Prepare the skb for output for the given mode. Returns:
> > +	 *    Error value, if 0 then skb values should be as follows:
> > +	 *    transport_header should point at ESP header
> > +	 *    network_header should point at Outer IP header
> > +	 *    mac_header should point at protocol/nexthdr of the outer IP
> > +	 */
> > +	int	(*prepare_output)(struct xfrm_state *x, struct sk_buff *skb);
> > +};
> > +
> > +int xfrm_register_mode_cbs(u8 mode, const struct xfrm_mode_cbs *mode_cbs);
> > +void xfrm_unregister_mode_cbs(u8 mode);
> > +
> >  static inline int xfrm_af2proto(unsigned int family)
> >  {
> >  	switch(family) {
> > diff --git a/net/xfrm/xfrm_device.c b/net/xfrm/xfrm_device.c
> > index 3784534c9185..8b848540ea47 100644
> > --- a/net/xfrm/xfrm_device.c
> > +++ b/net/xfrm/xfrm_device.c
> > @@ -42,7 +42,8 @@ static void __xfrm_mode_tunnel_prep(struct xfrm_state *x, struct sk_buff *skb,
> >  		skb->transport_header = skb->network_header + hsize;
> >  
> >  	skb_reset_mac_len(skb);
> > -	pskb_pull(skb, skb->mac_len + x->props.header_len);
> > +	pskb_pull(skb,
> > +		  skb->mac_len + x->props.header_len - x->props.enc_hdr_len);
> >  }
> >  
> >  static void __xfrm_mode_beet_prep(struct xfrm_state *x, struct sk_buff *skb,
> > diff --git a/net/xfrm/xfrm_input.c b/net/xfrm/xfrm_input.c
> > index bd4ce21d76d7..824f7b7f90e0 100644
> > --- a/net/xfrm/xfrm_input.c
> > +++ b/net/xfrm/xfrm_input.c
> > @@ -437,6 +437,9 @@ static int xfrm_inner_mode_input(struct xfrm_state *x,
> >  		WARN_ON_ONCE(1);
> >  		break;
> >  	default:
> > +		if (x->mode_cbs && x->mode_cbs->input)
> > +			return x->mode_cbs->input(x, skb);
> > +
> >  		WARN_ON_ONCE(1);
> >  		break;
> >  	}
> > @@ -479,6 +482,10 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
> >  
> >  		family = x->props.family;
> >  
> > +		/* An encap_type of -3 indicates reconstructed inner packet */
> > +		if (encap_type == -3)
> > +			goto resume_decapped;
> > +
> >  		/* An encap_type of -1 indicates async resumption. */
> >  		if (encap_type == -1) {
> >  			async = 1;
> > @@ -660,11 +667,14 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
> >  
> >  		XFRM_MODE_SKB_CB(skb)->protocol = nexthdr;
> >  
> > -		if (xfrm_inner_mode_input(x, skb)) {
> > +		err = xfrm_inner_mode_input(x, skb);
> > +		if (err == -EINPROGRESS)
> > +			return 0;
> > +		else if (err) {
> >  			XFRM_INC_STATS(net, LINUX_MIB_XFRMINSTATEMODEERROR);
> >  			goto drop;
> >  		}
> > -
> > +resume_decapped:
> >  		if (x->outer_mode.flags & XFRM_MODE_FLAG_TUNNEL) {
> >  			decaps = 1;
> >  			break;
> > diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
> > index 662c83beb345..4390c111410d 100644
> > --- a/net/xfrm/xfrm_output.c
> > +++ b/net/xfrm/xfrm_output.c
> > @@ -280,7 +280,9 @@ static int xfrm4_tunnel_encap_add(struct xfrm_state *x, struct sk_buff *skb)
> >  	skb_set_inner_network_header(skb, skb_network_offset(skb));
> >  	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
> >  
> > -	skb_set_network_header(skb, -x->props.header_len);
> > +	/* backup to add space for the outer encap */
> > +	skb_set_network_header(skb,
> > +			       -x->props.header_len + x->props.enc_hdr_len);
> >  	skb->mac_header = skb->network_header +
> >  			  offsetof(struct iphdr, protocol);
> >  	skb->transport_header = skb->network_header + sizeof(*top_iph);
> > @@ -325,7 +327,8 @@ static int xfrm6_tunnel_encap_add(struct xfrm_state *x, struct sk_buff *skb)
> >  	skb_set_inner_network_header(skb, skb_network_offset(skb));
> >  	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
> >  
> > -	skb_set_network_header(skb, -x->props.header_len);
> > +	skb_set_network_header(skb,
> > +			       -x->props.header_len + x->props.enc_hdr_len);
> >  	skb->mac_header = skb->network_header +
> >  			  offsetof(struct ipv6hdr, nexthdr);
> >  	skb->transport_header = skb->network_header + sizeof(*top_iph);
> > @@ -472,6 +475,8 @@ static int xfrm_outer_mode_output(struct xfrm_state *x, struct sk_buff *skb)
> >  		WARN_ON_ONCE(1);
> >  		break;
> >  	default:
> > +		if (x->mode_cbs->prepare_output)
> > +			return x->mode_cbs->prepare_output(x, skb);
> >  		WARN_ON_ONCE(1);
> >  		break;
> >  	}
> > diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
> > index d2dddc570f4f..3220b01121f3 100644
> > --- a/net/xfrm/xfrm_policy.c
> > +++ b/net/xfrm/xfrm_policy.c
> > @@ -2707,13 +2707,17 @@ static struct dst_entry *xfrm_bundle_create(struct xfrm_policy *policy,
> >  
> >  		dst1->input = dst_discard;
> >  
> > -		rcu_read_lock();
> > -		afinfo = xfrm_state_afinfo_get_rcu(inner_mode->family);
> > -		if (likely(afinfo))
> > -			dst1->output = afinfo->output;
> > -		else
> > -			dst1->output = dst_discard_out;
> > -		rcu_read_unlock();
> > +		if (xfrm[i]->mode_cbs && xfrm[i]->mode_cbs->output) {
> > +			dst1->output = xfrm[i]->mode_cbs->output;
> > +		} else {
> > +			rcu_read_lock();
> > +			afinfo = xfrm_state_afinfo_get_rcu(inner_mode->family);
> > +			if (likely(afinfo))
> > +				dst1->output = afinfo->output;
> > +			else
> > +				dst1->output = dst_discard_out;
> > +			rcu_read_unlock();
> > +		}
> >  
> >  		xdst_prev = xdst;
> >  
> > diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
> > index bda5327bf34d..f5e1a17ebf74 100644
> > --- a/net/xfrm/xfrm_state.c
> > +++ b/net/xfrm/xfrm_state.c
> > @@ -513,6 +513,36 @@ static const struct xfrm_mode *xfrm_get_mode(unsigned int encap, int family)
> >  	return NULL;
> >  }
> >  
> > +static struct xfrm_mode_cbs xfrm_mode_cbs_map[XFRM_MODE_MAX];
> > +
> > +int xfrm_register_mode_cbs(u8 mode, const struct xfrm_mode_cbs *mode_cbs)
> > +{
> > +	if (mode >= XFRM_MODE_MAX)
> > +		return -EINVAL;
> > +
> > +	xfrm_mode_cbs_map[mode] = *mode_cbs;
> > +	return 0;
> > +}
> > +EXPORT_SYMBOL(xfrm_register_mode_cbs);
> > +
> > +void xfrm_unregister_mode_cbs(u8 mode)
> > +{
> > +	if (mode >= XFRM_MODE_MAX)
> > +		return;
> > +
> > +	memset(&xfrm_mode_cbs_map[mode], 0, sizeof(xfrm_mode_cbs_map[mode]));
> > +}
> > +EXPORT_SYMBOL(xfrm_unregister_mode_cbs);
> > +
> > +static const struct xfrm_mode_cbs *xfrm_get_mode_cbs(u8 mode)
> > +{
> > +	if (mode >= XFRM_MODE_MAX)
> > +		return NULL;
> > +	if (mode == XFRM_MODE_IPTFS && !xfrm_mode_cbs_map[mode].create_state)
> > +		request_module("xfrm-iptfs");
> > +	return &xfrm_mode_cbs_map[mode];
> > +}
> > +
> >  void xfrm_state_free(struct xfrm_state *x)
> >  {
> >  	kmem_cache_free(xfrm_state_cache, x);
> > @@ -521,6 +551,8 @@ EXPORT_SYMBOL(xfrm_state_free);
> >  
> >  static void ___xfrm_state_destroy(struct xfrm_state *x)
> >  {
> > +	if (x->mode_cbs && x->mode_cbs->delete_state)
> > +		x->mode_cbs->delete_state(x);
> >  	hrtimer_cancel(&x->mtimer);
> >  	del_timer_sync(&x->rtimer);
> >  	kfree(x->aead);
> > @@ -2765,6 +2797,9 @@ u32 xfrm_state_mtu(struct xfrm_state *x, int mtu)
> >  	case XFRM_MODE_TUNNEL:
> >  		break;
> >  	default:
> > +		if (x->mode_cbs && x->mode_cbs->get_inner_mtu)
> > +			return x->mode_cbs->get_inner_mtu(x, mtu);
> > +
> >  		WARN_ON_ONCE(1);
> >  		break;
> >  	}
> > @@ -2850,6 +2885,12 @@ int __xfrm_init_state(struct xfrm_state *x, bool init_replay, bool offload,
> >  			goto error;
> >  	}
> >  
> > +	x->mode_cbs = xfrm_get_mode_cbs(x->props.mode);
> > +	if (x->mode_cbs && x->mode_cbs->create_state) {
> > +		err = x->mode_cbs->create_state(x);
> > +		if (err)
> > +			goto error;
> > +	}
> >  error:
> >  	return err;
> >  }
> > diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c
> > index fa2059de51f5..795da945fbc2 100644
> > --- a/net/xfrm/xfrm_user.c
> > +++ b/net/xfrm/xfrm_user.c
> > @@ -779,6 +779,12 @@ static struct xfrm_state *xfrm_state_construct(struct net *net,
> >  			goto error;
> >  	}
> >  
> > +	if (x->mode_cbs && x->mode_cbs->user_init) {
> > +		err = x->mode_cbs->user_init(net, x, attrs);
> > +		if (err)
> > +			goto error;
> > +	}
> > +
> >  	return x;
> >  
> >  error:
> > @@ -1192,6 +1198,10 @@ static int copy_to_user_state_extra(struct xfrm_state *x,
> >  		if (ret)
> >  			goto out;
> >  	}
> > +	if (x->mode_cbs && x->mode_cbs->copy_to_user)
> > +		ret = x->mode_cbs->copy_to_user(x, skb);
> > +	if (ret)
> > +		goto out;
> >  	if (x->mapping_maxage)
> >  		ret = nla_put_u32(skb, XFRMA_MTIMER_THRESH, x->mapping_maxage);
> >  out:
> > -- 
> > 2.42.0
> > 
> > -- 
> > Devel mailing list
> > Devel@linux-ipsec.org
> > https://linux-ipsec.org/mailman/listinfo/devel

> From 00fc5af96f90846f1e1882b141699fb62a9f0a73 Mon Sep 17 00:00:00 2001
> From: Antony Antony <antony.antony@secunet.com>
> Date: Mon, 13 Nov 2023 14:20:45 +0100
> Subject: [PATCH] xfrm iptfs migrate poc
> 
> proof of concept for IP-TFS migrate support
> ---
>  include/net/xfrm.h    |  1 +
>  net/xfrm/xfrm_iptfs.c | 20 ++++++++++++++++++++
>  net/xfrm/xfrm_state.c |  6 ++++++
>  3 files changed, 27 insertions(+)
> 
> diff --git a/include/net/xfrm.h b/include/net/xfrm.h
> index a6e0e848918d..176ab5ac436e 100644
> --- a/include/net/xfrm.h
> +++ b/include/net/xfrm.h
> @@ -456,6 +456,7 @@ struct xfrm_mode_cbs {
>  	int	(*user_init)(struct net *net, struct xfrm_state *x,
>  			     struct nlattr **attrs);
>  	int	(*copy_to_user)(struct xfrm_state *x, struct sk_buff *skb);
> +	int     (*clone)(struct xfrm_state *orig, struct xfrm_state *x);
>  
>  	u32	(*get_inner_mtu)(struct xfrm_state *x, int outer_mtu);
>  
> diff --git a/net/xfrm/xfrm_iptfs.c b/net/xfrm/xfrm_iptfs.c
> index 65f7acdbe6a8..cef269a02b11 100644
> --- a/net/xfrm/xfrm_iptfs.c
> +++ b/net/xfrm/xfrm_iptfs.c
> @@ -2613,6 +2613,25 @@ static int iptfs_copy_to_user(struct xfrm_state *x, struct sk_buff *skb)
>  	return ret;
>  }
>  
> +static int iptfs_clone(struct xfrm_state *orig,  struct xfrm_state *x)
> +{
> +
> +	x->mode_data = kmemdup(orig->mode_data, sizeof(*x->mode_data),
> +			       GFP_KERNEL);
> +	/**  may be some values, such as the following, should not be copied ??
> +	 * and need different handling ?
> +	 * xtfs->iptfs_timer;
> +	 * xtfs->drop_timer
> +         * xtfs->drop_lock
> +         * xtfs->w_saved
> +         * xtfs;
> +	 */
> +
> +	if (!x->mode_data)
> +		return -ENOMEM;
> +
> +	return 0;
> +}
>  static int iptfs_create_state(struct xfrm_state *x)
>  {
>  	struct xfrm_iptfs_data *xtfs;
> @@ -2667,6 +2686,7 @@ static const struct xfrm_mode_cbs iptfs_mode_cbs = {
>  	.delete_state = iptfs_delete_state,
>  	.user_init = iptfs_user_init,
>  	.copy_to_user = iptfs_copy_to_user,
> +	.clone = iptfs_clone,
>  	.get_inner_mtu = iptfs_get_inner_mtu,
>  	.input = iptfs_input,
>  	.output = iptfs_output_collect,
> diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
> index 786f3fc0d428..c56d3be56229 100644
> --- a/net/xfrm/xfrm_state.c
> +++ b/net/xfrm/xfrm_state.c
> @@ -720,6 +720,7 @@ struct xfrm_state *xfrm_state_alloc(struct net *net)
>  		x->replay_maxage = 0;
>  		x->replay_maxdiff = 0;
>  		spin_lock_init(&x->lock);
> +		x->mode_data = NULL;
>  	}
>  	return x;
>  }
> @@ -1787,6 +1788,11 @@ static struct xfrm_state *xfrm_state_clone(struct xfrm_state *orig,
>  	x->new_mapping = 0;
>  	x->new_mapping_sport = 0;
>  
> +	if (x->mode_cbs && x->mode_cbs->clone && orig->mode_data) {
> +		if (!x->mode_cbs->clone(x,orig))
> +			goto error;
> +	}
> +
>  	return x;
>  
>   error:
> -- 
> 2.42.0
> 


[-- Attachment #2: 0001-xfrm-iptfs-migrate-poc.patch --]
[-- Type: text/plain, Size: 3940 bytes --]

From 6d1ec5adddbe3a904591f465ff8487bc694de139 Mon Sep 17 00:00:00 2001
In-Reply-To: <20231113035219.920136-7-chopps@chopps.org>
References: <20231113035219.920136-7-chopps@chopps.org>
From: Antony Antony <antony@phenome.org>
Date: Mon, 13 Nov 2023 14:20:45 +0100
Subject: [PATCH] xfrm iptfs migrate poc
To: Christian Hopps <chopps@chopps.org>
Cc: devel@linux-ipsec.org,
    netdev@vger.kernel.org,
    Christian Hopps <chopps@labn.net>,
    Steffen Klassert via Devel <devel@linux-ipsec.org>

From: Antony Antony <antony.antony@secunet.com>

proof of concept for IP-TFS migrate support

Signed-off-by: Antony Antony <antony.antony@secunet.com>
---
 include/net/xfrm.h    |  1 +
 net/xfrm/xfrm_iptfs.c | 42 ++++++++++++++++++++++++++++++++++++------
 net/xfrm/xfrm_state.c |  6 ++++++
 3 files changed, 43 insertions(+), 6 deletions(-)

diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index a6e0e848918d..176ab5ac436e 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -456,6 +456,7 @@ struct xfrm_mode_cbs {
 	int	(*user_init)(struct net *net, struct xfrm_state *x,
 			     struct nlattr **attrs);
 	int	(*copy_to_user)(struct xfrm_state *x, struct sk_buff *skb);
+	int     (*clone)(struct xfrm_state *orig, struct xfrm_state *x);
 
 	u32	(*get_inner_mtu)(struct xfrm_state *x, int outer_mtu);
 
diff --git a/net/xfrm/xfrm_iptfs.c b/net/xfrm/xfrm_iptfs.c
index 65f7acdbe6a8..910c5e060931 100644
--- a/net/xfrm/xfrm_iptfs.c
+++ b/net/xfrm/xfrm_iptfs.c
@@ -2617,12 +2617,15 @@ static int iptfs_create_state(struct xfrm_state *x)
 {
 	struct xfrm_iptfs_data *xtfs;
 
-	xtfs = kzalloc(sizeof(*xtfs), GFP_KERNEL);
-	if (!xtfs)
-		return -ENOMEM;
-	x->mode_data = xtfs;
-
-	xtfs->x = x;
+	if (!x->mode_data) {
+		xtfs = kzalloc(sizeof(*xtfs), GFP_KERNEL);
+		if (!xtfs)
+			return -ENOMEM;
+		x->mode_data = xtfs;
+		xtfs->x = x;
+	} else { /* this is a cloned state */
+		xtfs = (struct xfrm_iptfs_data *) x->mode_data;
+	}
 
 	__skb_queue_head_init(&xtfs->queue);
 	xtfs->init_delay_ns = xtfs->cfg.init_delay_us * NSECS_IN_USEC;
@@ -2661,12 +2664,39 @@ static void iptfs_delete_state(struct xfrm_state *x)
 	kfree_sensitive(xtfs);
 }
 
+static int iptfs_clone(struct xfrm_state *orig,  struct xfrm_state *x)
+{
+	struct xfrm_iptfs_data *xtfs;
+	struct xfrm_iptfs_config *xc;
+
+	x->mode_data = kmemdup(orig->mode_data, sizeof(struct xfrm_iptfs_data),
+			       GFP_KERNEL);
+	if (IS_ERR_OR_NULL(x->mode_data))
+		return -ENOMEM;
+
+	xtfs = (struct xfrm_iptfs_data *)x->mode_data;
+	xtfs->x = x;
+	xc = &xtfs->cfg;
+	if (xc->reorder_win_size)
+		xtfs->w_saved = kcalloc(xc->reorder_win_size, sizeof(*xtfs->w_saved),
+					GFP_KERNEL);
+	xtfs->ra_newskb = NULL;
+	memset(&xtfs->iptfs_timer, 0, sizeof(xtfs->iptfs_timer));
+	memset(&xtfs->drop_timer, 0,sizeof(xtfs->drop_timer));
+	memset(&xtfs->drop_lock, 0, sizeof(xtfs->drop_lock));
+
+	/* x->mode_cbs->create_state(x) will initialize the rest of xtfs */
+
+	return 0;
+}
+
 static const struct xfrm_mode_cbs iptfs_mode_cbs = {
 	.owner = THIS_MODULE,
 	.create_state = iptfs_create_state,
 	.delete_state = iptfs_delete_state,
 	.user_init = iptfs_user_init,
 	.copy_to_user = iptfs_copy_to_user,
+	.clone = iptfs_clone,
 	.get_inner_mtu = iptfs_get_inner_mtu,
 	.input = iptfs_input,
 	.output = iptfs_output_collect,
diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
index 786f3fc0d428..fd592bf4d311 100644
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -720,6 +720,7 @@ struct xfrm_state *xfrm_state_alloc(struct net *net)
 		x->replay_maxage = 0;
 		x->replay_maxdiff = 0;
 		spin_lock_init(&x->lock);
+		x->mode_data = NULL;
 	}
 	return x;
 }
@@ -1787,6 +1788,11 @@ static struct xfrm_state *xfrm_state_clone(struct xfrm_state *orig,
 	x->new_mapping = 0;
 	x->new_mapping_sport = 0;
 
+	if (orig->mode_cbs && orig->mode_cbs->clone && orig->mode_data) {
+		if (orig->mode_cbs->clone(orig, x))
+			goto error;
+	}
+
 	return x;
 
  error:
-- 
2.42.0

Re: [devel-ipsec] [RFC ipsec-next v2 0/8] Add IP-TFS mode to xfrm

[Andrew Cagney]

> I did a multiple days peer review with Chris on this pachset. So my
> concerns are already addressed.
>
> Further reviews are welcome! This is a bigger change and it would
> be nice if more people could look at it.

I have a usability question.  What name should appear when a user
interacts with and sees log messages from this feature?
    ip-tfs, IP-TFS, IP_TFS
or:
   iptfs, IPTFS, ...

Re: [devel-ipsec] [RFC ipsec-next v2 0/8] Add IP-TFS mode to xfrm

[Antony Antony]

[-- Attachment #1: Type: text/plain, Size: 2383 bytes --]

On Sun, Nov 12, 2023 at 10:52:11PM -0500, Christian Hopps via Devel wrote:
> From: Christian Hopps <chopps@labn.net>
> 
> This patchset adds a new xfrm mode implementing on-demand IP-TFS. IP-TFS
> (AggFrag encapsulation) has been standardized in RFC9347.
> 
> Link: https://www.rfc-editor.org/rfc/rfc9347.txt
> 
> This feature supports demand driven (i.e., non-constant send rate) IP-TFS to
> take advantage of the AGGFRAG ESP payload encapsulation. This payload type
> supports aggregation and fragmentation of the inner IP packet stream which in
> turn yields higher small-packet bandwidth as well as reducing MTU/PMTU issues.
> Congestion control is unimplementated as the send rate is demand driven rather
> than constant.
> 
> In order to allow loading this fucntionality as a module a set of callbacks
> xfrm_mode_cbs has been added to xfrm as well.

Hi Chris,

I have further reviewed the code and have a few minor questions, mainly 
related to handling of XFRM_MODE_IPTFS. It appears to me be either some case  
missing support or/and in a few places it should be prohibited. I have three 
types of questions:

1. missing XFRM_MODE_IPTFS support?
2. Will XFRM_MODE_IPTFS be supported this combination?
3. Should be prohibited combination with XFRM_MODE_IPTFS 

1.  Missing:  

a.  wouldn't xfrm_sa_len() need extending? 

I could not find support for XFRM_MODE_IPTFS explicitly.

However, I'm not sure how the following code is working when xfrm_sa_len is 
missing IP-TFS xfrm_sa_len:

copy_to_user_state_extra() {
    if (x->mode_cbs && x->mode_cbs->copy_to_user)
        ret = x->mode_cbs->copy_to_user(x, skb);
}

I have attached what I imagine is a fix. Check with Steffen or others if 
this is necessary.

b. esp6_init_state() and esp_init_state():
These functions do not seem to handle XFRM_MODE_IPTFS. Would they default to work with it?

2. Would xfrm4_outer_mode_gso_segment() xfrm6_outer_mode_gso_segment(): 
support XFRM_MODE_IPTFS?
These functions appear to be missing. Is it possible that you don't support GSO and GRO?

3: Shouldn't these combinations return error?

a. ipcomp and  XFRM_MODE_IPTFS
I'm guessing that ipcomp would generate an error when userspace tries to add an SA with XFRM_MODE_IPTFS.
ipcomp6_init_state()
ipcomp4_init_state()

b: In xfrm_state_construct():

if (attrs[XFRMA_TFCPAD])
    x->tfcpad = nla_get_u32(attrs[XFRMA_TFCPAD]);

-antony

[-- Attachment #2: 0001-xfrm-iptfs-extend-xfrm_sa_len.patch --]
[-- Type: text/plain, Size: 2566 bytes --]

From 3d9ad9fab35f4efd6c2fb17853aaca986e7daaf3 Mon Sep 17 00:00:00 2001
From: Antony Antony <antony.antony@secunet.com>
Date: Thu, 16 Nov 2023 16:46:54 +0100
Subject: [PATCH] xfrm: iptfs extend xfrm_sa_len

think this extension is necessary. I wonder how it works now:)
May it is alreday handled:)

Signed-off-by: Antony Antony <antony.antony@secunet.com>
---
 include/net/xfrm.h    |  1 +
 net/xfrm/xfrm_iptfs.c | 18 ++++++++++++++++++
 net/xfrm/xfrm_user.c  |  3 +++
 3 files changed, 22 insertions(+)

diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index 176ab5ac436e..7ca508b0c46c 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -474,6 +474,7 @@ struct xfrm_mode_cbs {
 	 *    mac_header should point at protocol/nexthdr of the outer IP
 	 */
 	int	(*prepare_output)(struct xfrm_state *x, struct sk_buff *skb);
+	unsigned int (*xfrm_sa_len)(const struct xfrm_state *x);
 };
 
 int xfrm_register_mode_cbs(u8 mode, const struct xfrm_mode_cbs *mode_cbs);
diff --git a/net/xfrm/xfrm_iptfs.c b/net/xfrm/xfrm_iptfs.c
index 910c5e060931..d8b02cc3073b 100644
--- a/net/xfrm/xfrm_iptfs.c
+++ b/net/xfrm/xfrm_iptfs.c
@@ -2613,6 +2613,23 @@ static int iptfs_copy_to_user(struct xfrm_state *x, struct sk_buff *skb)
 	return ret;
 }
 
+static unsigned int iptfs_xfrm_sa_len(const struct xfrm_state *x)
+{
+	struct xfrm_iptfs_data *xtfs = x->mode_data;
+        struct xfrm_iptfs_config *xc = &xtfs->cfg;
+	unsigned int l = 0;
+
+	if (xc->dont_frag)
+		l += nla_total_size(sizeof(xc->reorder_win_size));
+	l += nla_total_size(sizeof(xc->reorder_win_size));
+	l += nla_total_size(sizeof(xc->pkt_size));
+	l += nla_total_size(sizeof(xc->max_queue_size));
+	l += nla_total_size(sizeof(xc->drop_time_us));
+	l += nla_total_size(sizeof(xc->init_delay_us));
+
+	return l;
+}
+
 static int iptfs_create_state(struct xfrm_state *x)
 {
 	struct xfrm_iptfs_data *xtfs;
@@ -2701,6 +2718,7 @@ static const struct xfrm_mode_cbs iptfs_mode_cbs = {
 	.input = iptfs_input,
 	.output = iptfs_output_collect,
 	.prepare_output = iptfs_prepare_output,
+	.xfrm_sa_len = iptfs_xfrm_sa_len,
 };
 
 static int __init xfrm_iptfs_init(void)
diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c
index 4b0e462d5e31..737636b2c267 100644
--- a/net/xfrm/xfrm_user.c
+++ b/net/xfrm/xfrm_user.c
@@ -3350,6 +3350,9 @@ static inline unsigned int xfrm_sa_len(struct xfrm_state *x)
 	if (x->mapping_maxage)
 		l += nla_total_size(sizeof(x->mapping_maxage));
 
+	if (x->mode_cbs && x->mode_cbs->copy_to_user)
+                l += x->mode_cbs->xfrm_sa_len(x);
+
 	return l;
 }
 
-- 
2.42.0

Re: [RFC ipsec-next v2 2/8] iptfs: uapi: ip: add ip_tfs_*_hdr packet formats

[Sabrina Dubroca]

2023-11-12, 22:52:13 -0500, Christian Hopps wrote:
> From: Christian Hopps <chopps@labn.net>
> 
> Add the on-wire basic and congestion-control IP-TFS packet headers.
> 
> Signed-off-by: Christian Hopps <chopps@labn.net>
> ---
>  include/uapi/linux/ip.h | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
>
> diff --git a/include/uapi/linux/ip.h b/include/uapi/linux/ip.h
> index 283dec7e3645..cc83878ecf08 100644
> --- a/include/uapi/linux/ip.h
> +++ b/include/uapi/linux/ip.h
> @@ -137,6 +137,23 @@ struct ip_beet_phdr {
>  	__u8 reserved;
>  };
>  
> +struct ip_iptfs_hdr {
> +	__u8 subtype;		/* 0*: basic, 1: CC */
> +	__u8 flags;
> +	__be16 block_offset;
> +};
> +
> +struct ip_iptfs_cc_hdr {
> +	__u8 subtype;		/* 0: basic, 1*: CC */
> +	__u8 flags;
> +	__be16 block_offset;
> +	__be32 loss_rate;
> +	__u8 rtt_and_adelay1[4];
> +	__u8 adelay2_and_xdelay[4];

Given how the fields are split, wouldn't it be more convenient to have
a single __be64, rather than reading some bits from multiple __u8?

> +	__be32 tval;
> +	__be32 techo;
> +};

I don't think these need to be part of uapi. Can we move them to
include/net/iptfs.h (or possibly net/xfrm/xfrm_iptfs.c)? It would also
make more sense to have them near the definitions for
IPTFS_SUBTYPE_*. And it would be easier to change how we split and
name fields for kernel consumption if we're not stuck with whatever we
put in uapi.

-- 
Sabrina

Re: [devel-ipsec] [RFC ipsec-next v2 0/8] Add IP-TFS mode to xfrm

[Christian Hopps]

[-- Attachment #1: Type: text/plain, Size: 597 bytes --]


Andrew Cagney <andrew.cagney@gmail.com> writes:

>> I did a multiple days peer review with Chris on this pachset. So my
>> concerns are already addressed.
>>
>> Further reviews are welcome! This is a bigger change and it would
>> be nice if more people could look at it.
>
> I have a usability question.  What name should appear when a user
> interacts with and sees log messages from this feature?
>     ip-tfs, IP-TFS, IP_TFS
> or:
>    iptfs, IPTFS, ...

I think no `-` or `_` in the code/api. For documentation it is probably better to hew closer to the RFC and use `IP-TFS`.

Thanks,
Chris.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 858 bytes --]

Re: [DKIM] Re: [devel-ipsec] [RFC ipsec-next v2 0/8] Add IP-TFS mode to xfrm

[Antony Antony]

On Mon, Nov 20, 2023 at 01:39:50PM -0500, Christian Hopps via Devel wrote:
> 
> Andrew Cagney <andrew.cagney@gmail.com> writes:
> 
> > > I did a multiple days peer review with Chris on this pachset. So my
> > > concerns are already addressed.
> > > 
> > > Further reviews are welcome! This is a bigger change and it would
> > > be nice if more people could look at it.
> > 
> > I have a usability question.  What name should appear when a user
> > interacts with and sees log messages from this feature?
> >     ip-tfs, IP-TFS, IP_TFS
> > or:
> >    iptfs, IPTFS, ...
> 
> I think no `-` or `_` in the code/api. For documentation it is probably better to hew closer to the RFC and use `IP-TFS`.

That sounds good. However,
iproute2 output, ip xfrm state, or "ip xfrm policy" is that documentation or code?

current unsubmitted patch shows: "iptfs"

src 192.1.2.23 dst 192.1.2.45
	proto esp spi 0x76ee6b87(1995336583) reqid 16389(0x00004005) mode iptfs

root@west:/testing/pluto/ikev2-74-iptfs-01 (iptfs-aa-20231120)# ip  x p
src 192.0.1.0/24 dst 192.0.2.0/24
	dir out priority 1757393 ptype main
	tmpl src 192.1.2.45 dst 192.1.2.23
		proto esp reqid 16389 mode iptfs

-antony

Re: [DKIM] Re: [devel-ipsec] [RFC ipsec-next v2 0/8] Add IP-TFS mode to xfrm

[Antony Antony]

On Mon, Nov 20, 2023 at 09:00:45PM +0100, Antony Antony wrote:
> On Mon, Nov 20, 2023 at 01:39:50PM -0500, Christian Hopps via Devel wrote:
> > 
> > Andrew Cagney <andrew.cagney@gmail.com> writes:
> > 
> > > > I did a multiple days peer review with Chris on this pachset. So my
> > > > concerns are already addressed.
> > > > 
> > > > Further reviews are welcome! This is a bigger change and it would
> > > > be nice if more people could look at it.
> > > 
> > > I have a usability question.  What name should appear when a user
> > > interacts with and sees log messages from this feature?
> > >     ip-tfs, IP-TFS, IP_TFS
> > > or:
> > >    iptfs, IPTFS, ...
> > 
> > I think no `-` or `_` in the code/api. For documentation it is probably better to hew closer to the RFC and use `IP-TFS`.
> 
> That sounds good. However,
> iproute2 output, ip xfrm state, or "ip xfrm policy" is that documentation or code?
> 
> current unsubmitted patch shows: "iptfs"
> 
> src 192.1.2.23 dst 192.1.2.45
> 	proto esp spi 0x76ee6b87(1995336583) reqid 16389(0x00004005) mode iptfs

there also the following line further down in ip x s

iptfs-opts pkt-size 0 max-queue-size 1048576 drop-time 1000000 reorder-window 3 init-delay 0

> 
> root@west:/testing/pluto/ikev2-74-iptfs-01 (iptfs-aa-20231120)# ip  x p
> src 192.0.1.0/24 dst 192.0.2.0/24
> 	dir out priority 1757393 ptype main
> 	tmpl src 192.1.2.45 dst 192.1.2.23
> 		proto esp reqid 16389 mode iptfs
> 
> -antony

Re: [DKIM] Re: [devel-ipsec] [RFC ipsec-next v2 0/8] Add IP-TFS mode to xfrm

[Christian Hopps]

[-- Attachment #1: Type: text/plain, Size: 1526 bytes --]


Antony Antony <antony@phenome.org> writes:

> On Mon, Nov 20, 2023 at 01:39:50PM -0500, Christian Hopps via Devel wrote:
>>
>> Andrew Cagney <andrew.cagney@gmail.com> writes:
>>
>> > > I did a multiple days peer review with Chris on this pachset. So my
>> > > concerns are already addressed.
>> > >
>> > > Further reviews are welcome! This is a bigger change and it would
>> > > be nice if more people could look at it.
>> >
>> > I have a usability question.  What name should appear when a user
>> > interacts with and sees log messages from this feature?
>> >     ip-tfs, IP-TFS, IP_TFS
>> > or:
>> >    iptfs, IPTFS, ...
>>
>> I think no `-` or `_` in the code/api. For documentation it is probably better to hew closer to the RFC and use `IP-TFS`.
>
> That sounds good. However,
> iproute2 output, ip xfrm state, or "ip xfrm policy" is that documentation or code?
>
> current unsubmitted patch shows: "iptfs"

That's code/api. I was thinking ui/api when I said api.

Documentation would be something ending in a .rst or within a comment as that might be referring to the RFC too.

I will go through and make sure things are consistent in the next version.

Thanks,
Chris.

>
> src 192.1.2.23 dst 192.1.2.45
> 	proto esp spi 0x76ee6b87(1995336583) reqid 16389(0x00004005) mode iptfs
>
> root@west:/testing/pluto/ikev2-74-iptfs-01 (iptfs-aa-20231120)# ip  x p
> src 192.0.1.0/24 dst 192.0.2.0/24
> 	dir out priority 1757393 ptype main
> 	tmpl src 192.1.2.45 dst 192.1.2.23
> 		proto esp reqid 16389 mode iptfs
>
> -antony


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 858 bytes --]

Re: [RFC ipsec-next v2 2/8] iptfs: uapi: ip: add ip_tfs_*_hdr packet formats

[Christian Hopps]

[-- Attachment #1: Type: text/plain, Size: 1901 bytes --]


Sabrina Dubroca <sd@queasysnail.net> writes:

> 2023-11-12, 22:52:13 -0500, Christian Hopps wrote:
>> From: Christian Hopps <chopps@labn.net>
>>
>> Add the on-wire basic and congestion-control IP-TFS packet headers.
>>
>> Signed-off-by: Christian Hopps <chopps@labn.net>
>> ---
>>  include/uapi/linux/ip.h | 17 +++++++++++++++++
>>  1 file changed, 17 insertions(+)
>>
>> diff --git a/include/uapi/linux/ip.h b/include/uapi/linux/ip.h
>> index 283dec7e3645..cc83878ecf08 100644
>> --- a/include/uapi/linux/ip.h
>> +++ b/include/uapi/linux/ip.h
>> @@ -137,6 +137,23 @@ struct ip_beet_phdr {
>>  	__u8 reserved;
>>  };
>>
>> +struct ip_iptfs_hdr {
>> +	__u8 subtype;		/* 0*: basic, 1: CC */
>> +	__u8 flags;
>> +	__be16 block_offset;
>> +};
>> +
>> +struct ip_iptfs_cc_hdr {
>> +	__u8 subtype;		/* 0: basic, 1*: CC */
>> +	__u8 flags;
>> +	__be16 block_offset;
>> +	__be32 loss_rate;
>> +	__u8 rtt_and_adelay1[4];
>> +	__u8 adelay2_and_xdelay[4];
>
> Given how the fields are split, wouldn't it be more convenient to have
> a single __be64, rather than reading some bits from multiple __u8?

This is a good point, I carried this over from an earlier implementation, let me give it some though but probably change it.

>> +	__be32 tval;
>> +	__be32 techo;
>> +};

> I don't think these need to be part of uapi. Can we move them to
> include/net/iptfs.h (or possibly net/xfrm/xfrm_iptfs.c)? It would also
> make more sense to have them near the definitions for
> IPTFS_SUBTYPE_*. And it would be easier to change how we split and
> name fields for kernel consumption if we're not stuck with whatever we
> put in uapi.

The other ipsec modes headers were added here, so I was simply following along. I don't mind moving things but would like to understand why iptfs would be different from the other modes, for example, `struct ip_comp_hdr` and `struct ip_beet_phdr` appears in this file.

Thanks!
Chris.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

Re: [RFC ipsec-next v2 2/8] iptfs: uapi: ip: add ip_tfs_*_hdr packet formats

[Sabrina Dubroca]

2023-11-20, 15:18:49 -0500, Christian Hopps wrote:
> 
> Sabrina Dubroca <sd@queasysnail.net> writes:
> 
> > 2023-11-12, 22:52:13 -0500, Christian Hopps wrote:
> > > From: Christian Hopps <chopps@labn.net>
> > > 
> > > Add the on-wire basic and congestion-control IP-TFS packet headers.
> > > 
> > > Signed-off-by: Christian Hopps <chopps@labn.net>
> > > ---
> > >  include/uapi/linux/ip.h | 17 +++++++++++++++++
> > >  1 file changed, 17 insertions(+)
> > > 
> > > diff --git a/include/uapi/linux/ip.h b/include/uapi/linux/ip.h
> > > index 283dec7e3645..cc83878ecf08 100644
> > > --- a/include/uapi/linux/ip.h
> > > +++ b/include/uapi/linux/ip.h
> > > @@ -137,6 +137,23 @@ struct ip_beet_phdr {
> > >  	__u8 reserved;
> > >  };
> > > 
> > > +struct ip_iptfs_hdr {
> > > +	__u8 subtype;		/* 0*: basic, 1: CC */
> > > +	__u8 flags;
> > > +	__be16 block_offset;
> > > +};
> > > +
> > > +struct ip_iptfs_cc_hdr {
> > > +	__u8 subtype;		/* 0: basic, 1*: CC */
> > > +	__u8 flags;
> > > +	__be16 block_offset;
> > > +	__be32 loss_rate;
> > > +	__u8 rtt_and_adelay1[4];
> > > +	__u8 adelay2_and_xdelay[4];
> > 
> > Given how the fields are split, wouldn't it be more convenient to have
> > a single __be64, rather than reading some bits from multiple __u8?
> 
> This is a good point, I carried this over from an earlier implementation, let me give it some though but probably change it.
> 
> > > +	__be32 tval;
> > > +	__be32 techo;
> > > +};
> 
> > I don't think these need to be part of uapi. Can we move them to
> > include/net/iptfs.h (or possibly net/xfrm/xfrm_iptfs.c)? It would also
> > make more sense to have them near the definitions for
> > IPTFS_SUBTYPE_*. And it would be easier to change how we split and
> > name fields for kernel consumption if we're not stuck with whatever we
> > put in uapi.
> 
> The other ipsec modes headers were added here, so I was simply
> following along. I don't mind moving things but would like to
> understand why iptfs would be different from the other modes, for
> example, `struct ip_comp_hdr` and `struct ip_beet_phdr` appears in
> this file.

IMHO it was a mistake that was made when include/uapi was created,
they should not have been part of uapi since there's never an exchange
between kernel and userspace using those. I guess it's less
problematic because the header formats are simple (everything fits
nicely into a u8/u16/u32) and they were already used in the kernel for
a long time so pretty much stable (ie no doubt about whether the
split and naming that was chosen was right).

But if others feel strongly about putting those structs in uapi, I can
live with that.

I'll send comments on the rest of the series as I continue making my
way through it. The main patch is going to take me a while :(

-- 
Sabrina

Re: [RFC ipsec-next v2 4/8] iptfs: sysctl: allow configuration of global default values

[Sabrina Dubroca]

2023-11-12, 22:52:15 -0500, Christian Hopps wrote:
> From: Christian Hopps <chopps@labn.net>
> 
> Add sysctls for the changing the IPTFS default SA values.
> 
> Signed-off-by: Christian Hopps <chopps@labn.net>
> ---
>  Documentation/networking/xfrm_sysctl.rst | 29 ++++++++++++++++++
>  include/net/netns/xfrm.h                 |  6 ++++
>  include/net/xfrm.h                       |  7 +++++
>  net/xfrm/xfrm_sysctl.c                   | 38 ++++++++++++++++++++++++
>  4 files changed, 80 insertions(+)
> 
> diff --git a/Documentation/networking/xfrm_sysctl.rst b/Documentation/networking/xfrm_sysctl.rst
> index 47b9bbdd0179..9e628806c110 100644
> --- a/Documentation/networking/xfrm_sysctl.rst
> +++ b/Documentation/networking/xfrm_sysctl.rst
> @@ -9,3 +9,32 @@ XFRM Syscall
>  
>  xfrm_acq_expires - INTEGER
>  	default 30 - hard timeout in seconds for acquire requests
> +
> +xfrm_iptfs_maxqsize - UNSIGNED INTEGER
> +        The default IPTFS max output queue size in octets. The output queue is
> +        where received packets destined for output over an IPTFS tunnel are
> +        stored prior to being output in aggregated/fragmented form over the
> +        IPTFS tunnel.
> +
> +        Default 1M.
> +
> +xfrm_iptfs_drptime - UNSIGNED INTEGER

nit: Can we make those names a bit more human-readable?
xfrm_iptfs_droptime, or possibly xfrm_iptfs_drop_time for better
consistency with the netlink API.

> +        The default IPTFS drop time in microseconds. The drop time is the amount
> +        of time before a missing out-of-order IPTFS tunnel packet is considered
> +        lost. See also the reorder window.
> +
> +        Default 1s (1000000).
> +
> +xfrm_iptfs_idelay - UNSIGNED INTEGER

I would suggest xfrm_iptfs_initial_delay (or at least init_delay like
the netlink attribute).

> +        The default IPTFS initial output delay in microseconds. The initial
> +        output delay is the amount of time prior to servicing the output queue
> +        after queueing the first packet on said queue.

Does that also apply if the queue was fully drained (no traffic for a
while) and starts being used again? That might be worth documenting
either way (sorry, I haven't processed the main patch far enough to
answer this question myself yet).

And it might be worth mentioning that all these sysctls can be
overridden per SA via the netlink API.

> +        Default 0.
> +
> +xfrm_iptfs_rewin - UNSIGNED INTEGER

xfrm_iptfs_reorderwin (or reorder_win, or reorder_window)?
I'd also make the equivalent netlink attribute's name a bit longer (at
least spell out REORDER, I can live with WIN for WINDOW).


[...]
> diff --git a/include/net/xfrm.h b/include/net/xfrm.h
> index c9bb0f892f55..d2e87344d175 100644
> --- a/include/net/xfrm.h
> +++ b/include/net/xfrm.h
> @@ -2190,4 +2190,11 @@ static inline int register_xfrm_interface_bpf(void)
>  
>  #endif
>  
> +#if IS_ENABLED(CONFIG_XFRM_IPTFS)
> +#define XFRM_IPTFS_DEFAULT_MAX_QUEUE_SIZE (1024 * 1024)
> +#define XFRM_IPTFS_DEFAULT_INIT_DELAY_USECS (0)
> +#define XFRM_IPTFS_DEFAULT_DROP_TIME_USECS (1000000)
> +#define XFRM_IPTFS_DEFAULT_REORDER_WINDOW (3)
> +#endif

nit: move those to net/xfrm/xfrm_sysctl.c ? they're only used in that file.


> diff --git a/net/xfrm/xfrm_sysctl.c b/net/xfrm/xfrm_sysctl.c
> index 7fdeafc838a7..bf8e73a6c38e 100644
> --- a/net/xfrm/xfrm_sysctl.c
> +++ b/net/xfrm/xfrm_sysctl.c
[...]
> @@ -55,6 +87,12 @@ int __net_init xfrm_sysctl_init(struct net *net)
>  	table[1].data = &net->xfrm.sysctl_aevent_rseqth;
>  	table[2].data = &net->xfrm.sysctl_larval_drop;
>  	table[3].data = &net->xfrm.sysctl_acq_expires;
> +#if IS_ENABLED(CONFIG_XFRM_IPTFS)
> +	table[4].data = &net->xfrm.sysctl_iptfs_drptime;
> +	table[5].data = &net->xfrm.sysctl_iptfs_idelay;
> +	table[6].data = &net->xfrm.sysctl_iptfs_maxqsize;
> +	table[7].data = &net->xfrm.sysctl_iptfs_rewin;
> +#endif

Is it time to switch to a loop like in ipv6_sysctl_net_init? See
commit d2f7e56d1e40 ("ipv6: Use math to point per net sysctls into the
appropriate struct net"). OTOH we don't add sysctls for xfrm very
often so it's not critical.

-- 
Sabrina

Re: [devel-ipsec] [RFC ipsec-next v2 0/8] Add IP-TFS mode to xfrm

[Antony Antony]

On Mon, Nov 13, 2023 at 08:15:47AM +0100, Steffen Klassert via Devel wrote:
> On Sun, Nov 12, 2023 at 10:52:11PM -0500, Christian Hopps wrote:
> > From: Christian Hopps <chopps@labn.net>
> > 
> > This patchset adds a new xfrm mode implementing on-demand IP-TFS. IP-TFS
> > (AggFrag encapsulation) has been standardized in RFC9347.
> > 
> > Link: https://www.rfc-editor.org/rfc/rfc9347.txt
> > 
> > This feature supports demand driven (i.e., non-constant send rate) IP-TFS to
> > take advantage of the AGGFRAG ESP payload encapsulation. This payload type
> > supports aggregation and fragmentation of the inner IP packet stream which in
> > turn yields higher small-packet bandwidth as well as reducing MTU/PMTU issues.
> > Congestion control is unimplementated as the send rate is demand driven rather
> > than constant.
> > 
> > In order to allow loading this fucntionality as a module a set of callbacks
> > xfrm_mode_cbs has been added to xfrm as well.
> 
> I did a multiple days peer review with Chris on this pachset. So my
> concerns are already addressed.
> 
> Further reviews are welcome! This is a bigger change and it would
> be nice if more people could look at it.

I'd like to pose a basic question to understand the new IP-TFS config 
options for an SA.

When configuring IP-TFS parameters on an SA, are all parameters actively 
used by that SA? or does their usage vary based on the direction of the 
traffic?
Currently, each IP-TFS SA includes both init-delay and drop-time parameters. 
I would like to understand whether both parameters are necessary for every SA.

ip xfrm state

src 2001:db8:1:2::45 dst 2001:db8:1:2::23
proto esp spi 0x32af1f6d reqid 16389 mode iptfs
replay-window 0 flag af-unspec esn
aead rfc4106(gcm(aes)) 0x0... 128
anti-replay esn context:
seq-hi 0x0, seq 0x0, oseq-hi 0x0, oseq 0x0
replay_window 128, bitmap-length 4
00000000 00000000 00000000 00000000
iptfs-opts pkt-size 0 max-queue-size 1048576 drop-time 1000000 reorder-window 3 init-delay 0

Chris, would like to you share which of iptfs-opts are for input and which 
are for output?

My current understanding suggests that, depending on the traffic direction, 
onlysome of these parameters might be in active use for each SA. If this is 
indeed the case, I propose we discuss the possibility of refining our 
configuration approach. Could we consider making these parameters mutually 
exclusive and dependent on the traffic direction? Furthermore, given that 
the xfrm community has previously discussed the idea of adding direction to 
the SA — a concept also used in hardware offload scenarios — why not explore 
adding direction to the SA?

We currently have DIR with offload.

ip xfrm state { add | update } ... offload [ crypto|packet ] dev DEV dir DIR

Implementing direction could imply that an IP-TFS SA would require only fewer parameters
– init-delay or drop-time, depending on the specified direction. This would 
make ip x s output simple and comprehensible, thereby reducing potential 
confusion. Also avoid confusing when creating state say input, why add 
output parameters to an input SA?"

Re: [devel-ipsec] [RFC ipsec-next v2 0/8] Add IP-TFS mode to xfrm

[Antony Antony]

Hi Chris,

I got a crash when running inside a  namespace using veth. I got twice. It 
is not 100% reproducable. 

Note: I have KASAN enabled.

[  226.323824] BUG: KASAN: null-ptr-deref in 
iptfs_output_collect+0x1f5/0x61a
[  226.325155] Read of size 8 at addr 0000000000000478 by task ping/5276
[  226.325980]
[  226.326220] CPU: 2 PID: 5276 Comm: ping Not tainted 6.6.0-rc1-00528-gadd9146753b7 #95
[  226.327219] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
[  226.328396] Call Trace:
[  226.328791]  <TASK>
[  226.329099]  dump_stack_lvl+0x33/0x42
[  226.329597]  kasan_report+0xa0/0xc3
[  226.330074]  ? iptfs_output_collect+0x1f5/0x61a
[  226.330678]  iptfs_output_collect+0x1f5/0x61a
[  226.331259]  ip_send_skb+0x25/0x58
[  226.331724]  raw_sendmsg+0xec4/0xfee
[  226.332223]  ? raw_hash_sk+0x224/0x224
[  226.332741]  ? kasan_unpoison+0x44/0x52
[  226.333258]  ? kernel_init_pages+0x42/0x51
[  226.333810]  ? post_alloc_hook+0xba/0xe0
[  226.334339]  ? prep_new_page+0x44/0x51
[  226.334849]  ? ma_data_end+0x6e/0x88
[  226.335338]  ? preempt_count_sub+0x14/0xb5
[  226.335894]  ? zone_watermark_fast.isra.0+0x12b/0x12b
[  226.336577]  ? __might_resched+0x8d/0x248
[  226.337116]  ? __might_sleep+0x27/0xa2
[  226.337626]  ? first_zones_zonelist+0x2c/0x43
[  226.338211]  ? do_raw_spin_lock+0x72/0xbb
[  226.338752]  ? __might_resched+0x8d/0x248
[  226.339291]  ? __might_sleep+0x27/0xa2
[  226.339801]  ? inet_send_prepare+0x59/0x59
[  226.340356]  ? sock_sendmsg_nosec+0x42/0x6c
[  226.340924]  sock_sendmsg_nosec+0x42/0x6c
[  226.341465]  __sys_sendto+0x172/0x1e1
[  226.341964]  ? __x64_sys_getpeername+0x44/0x44
[  226.342558]  ? folio_batch_add_and_move+0x6a/0x7d
[  226.343189]  ? find_vma+0x6b/0x8b
[  226.343646]  ? find_vma_intersection+0x8a/0x8a
[  226.344246]  ? handle_mm_fault+0xfa/0x163
[  226.344788]  ? preempt_latency_start+0x41/0x4c
[  226.345385]  ? preempt_count_sub+0x14/0xb5
[  226.345937]  __x64_sys_sendto+0x76/0x82
[  226.346456]  do_syscall_64+0x58/0x78
[  226.346946]  entry_SYSCALL_64_after_hwframe+0x46/0xb0
[  226.347611] RIP: 0033:0x7f378b00ba73
[  226.348095] Code: 8b 15 a9 83 0c 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b8 0f 1f 00 80 3d 71 0b 0d 00 00 41 89 ca 74 14 b8 2c 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 75 c3 0f 1f 40 00 55 48 83 ec 30 44 89 4c 24
[  226.350433] RSP: 002b:00007fff58f9c148 EFLAGS: 00000202 ORIG_RAX: 000000000000002c
[  226.351400] RAX: ffffffffffffffda RBX: 00005591b05fa340 RCX: 00007f378b00ba73
[  226.352318] RDX: 0000000000000040 RSI: 00005591b06003c0 RDI: 0000000000000003
[  226.353233] RBP: 00005591b06003c0 R08: 00005591b05fc5c0 R09: 0000000000000010
[  226.354147] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000040
[  226.355061] R13: 00007fff58f9d830 R14: 0000001d00000001 R15: 00005591b05fd680
[  226.355978]  </TASK>
[  226.356301] ==================================================================
[  226.357274] Disabling lock debugging due to kernel taint
[  226.357973] BUG: kernel NULL pointer dereference, address: 0000000000000478
[  226.358865] #PF: supervisor read access in kernel mode
[  226.359540] #PF: error_code(0x0000) - not-present page
[  226.360220] PGD 0 P4D 0
[  226.360594] Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC KASAN
[  226.361346] CPU: 2 PID: 5276 Comm: ping Tainted: G    B              6.6.0-rc1-00528-gadd9146753b7 #95
[  226.362520] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
[  226.363693] RIP: 0010:iptfs_output_collect+0x1f5/0x61a
[  226.364376] Code: c1 ff ff 65 ff 0d 12 29 35 7e 75 05 e8 c3 f0 31 ff 48 8d 7b 10 e8 4b 16 6b ff 4c 8b 73 10 49 8d be 78 04 00 00 e8 3b 16 6b ff <4d> 8b b6 78 04 00 00 49 8d be c0 01 00 00 e8 28 16 6b ff 49 8b 86
[  226.366704] RSP: 0018:ffffc9000a357998 EFLAGS: 00010296
[  226.367389] RAX: 0000000000000001 RBX: ffff8881123a7a00 RCX: fffffbfff075e9f5
[  226.368308] RDX: fffffbfff075e9f5 RSI: fffffbfff075e9f5 RDI: ffffffff83af4fa0
[  226.369237] RBP: ffff88810f45ac00 R08: 0000000000000008 R09: 0000000000000001
[  226.370154] R10: ffffffff83af4fa7 R11: fffffbfff075e9f4 R12: 0000000000000000
[  226.371070] R13: 0000000000000000 R14: 0000000000000000 R15: ffff888114cbeb01
[  226.371984] FS:  00007f378ad4fc40(0000) GS:ffff88815ad00000(0000) knlGS:0000000000000000
[  226.373029] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  226.373777] CR2: 0000000000000478 CR3: 00000001083e9000 CR4: 0000000000350ee0
[  226.374693] Call Trace:
[  226.375042]  <TASK>
[  226.375352]  ? __die_body+0x1a/0x58
[  226.375828]  ? page_fault_oops+0x459/0x4c7
[  226.376387]  ? irq_work_queue+0x2c/0x41
[  226.376915]  ? dump_pagetable+0x1db/0x1db
[  226.377453]  ? vprintk_emit+0x187/0x195
[  226.377972]  ? iptfs_output_collect+0x1f5/0x61a
[  226.378576]  ? _printk+0xb2/0xe1
[  226.379021]  ? syslog_print+0x340/0x340
[  226.379541]  ? do_user_addr_fault+0x156/0x59f
[  226.380713]  ? exc_page_fault+0xaa/0xc3
[  226.381234]  ? asm_exc_page_fault+0x22/0x30
[  226.381795]  ? iptfs_output_collect+0x1f5/0x61a
[  226.382399]  ? iptfs_output_collect+0x1f5/0x61a
[  226.383002]  ip_send_skb+0x25/0x58
[  226.383469]  raw_sendmsg+0xec4/0xfee
[  226.383958]  ? raw_hash_sk+0x224/0x224
[  226.385083]  ? kasan_unpoison+0x44/0x52
[  226.385866]  ? kernel_init_pages+0x42/0x51
[  226.386687]  ? post_alloc_hook+0xba/0xe0
[  226.387559]  ? prep_new_page+0x44/0x51
[  226.388522]  ? ma_data_end+0x6e/0x88
[  226.389365]  ? preempt_count_sub+0x14/0xb5
[  226.390575]  ? zone_watermark_fast.isra.0+0x12b/0x12b
[  226.391555]  ? __might_resched+0x8d/0x248
[  226.392127]  ? __might_sleep+0x27/0xa2
[  226.392794]  ? first_zones_zonelist+0x2c/0x43
[  226.393684]  ? do_raw_spin_lock+0x72/0xbb
[  226.394501]  ? __might_resched+0x8d/0x248
[  226.395324]  ? __might_sleep+0x27/0xa2
[  226.396204]  ? inet_send_prepare+0x59/0x59
[  226.397042]  ? sock_sendmsg_nosec+0x42/0x6c
[  226.397879]  sock_sendmsg_nosec+0x42/0x6c
[  226.398704]  __sys_sendto+0x172/0x1e1
[  226.399465]  ? __x64_sys_getpeername+0x44/0x44
[  226.400367]  ? folio_batch_add_and_move+0x6a/0x7d
[  226.401327]  ? find_vma+0x6b/0x8b
[  226.401812]  ? find_vma_intersection+0x8a/0x8a
[  226.402442]  ? handle_mm_fault+0xfa/0x163
[  226.403014]  ? preempt_latency_start+0x41/0x4c
[  226.403641]  ? preempt_count_sub+0x14/0xb5
[  226.404224]  __x64_sys_sendto+0x76/0x82
[  226.404790]  do_syscall_64+0x58/0x78
[  226.405306]  entry_SYSCALL_64_after_hwframe+0x46/0xb0
[  226.406010] RIP: 0033:0x7f378b00ba73
[  226.406524] Code: 8b 15 a9 83 0c 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b8 0f 1f 00 80 3d 71 0b 0d 00 00 41 89 ca 74 14 b8 2c 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 75 c3 0f 1f 40 00 55 48 83 ec 30 44 89 4c 24
[  226.408990] RSP: 002b:00007fff58f9c148 EFLAGS: 00000202 ORIG_RAX: 000000000000002c
[  226.410020] RAX: ffffffffffffffda RBX: 00005591b05fa340 RCX: 00007f378b00ba73
[  226.410992] RDX: 0000000000000040 RSI: 00005591b06003c0 RDI: 0000000000000003
[  226.411968] RBP: 00005591b06003c0 R08: 00005591b05fc5c0 R09: 0000000000000010
[  226.412948] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000040
[  226.413923] R13: 00007fff58f9d830 R14: 0000001d00000001 R15: 00005591b05fd680
[  226.414904]  </TASK>
[  226.415243] Modules linked in:
[  226.415697] CR2: 0000000000000478
[  226.416181] ---[ end trace 0000000000000000 ]---
[  226.416848] RIP: 0010:iptfs_output_collect+0x1f5/0x61a
[  226.417569] Code: c1 ff ff 65 ff 0d 12 29 35 7e 75 05 e8 c3 f0 31 ff 48 8d 7b 10 e8 4b 16 6b ff 4c 8b 73 10 49 8d be 78 04 00 00 e8 3b 16 6b ff <4d> 8b b6 78 04 00 00 49 8d be c0 01 00 00 e8 28 16 6b ff 49 8b 86
[  226.420031] RSP: 0018:ffffc9000a357998 EFLAGS: 00010296
[  226.420764] RAX: 0000000000000001 RBX: ffff8881123a7a00 RCX: fffffbfff075e9f5
[  226.421733] RDX: fffffbfff075e9f5 RSI: fffffbfff075e9f5 RDI: ffffffff83af4fa0
[  226.422704] RBP: ffff88810f45ac00 R08: 0000000000000008 R09: 0000000000000001
[  226.423671] R10: ffffffff83af4fa7 R11: fffffbfff075e9f4 R12: 0000000000000000
[  226.424646] R13: 0000000000000000 R14: 0000000000000000 R15: ffff888114cbeb01
[  226.425616] FS:  00007f378ad4fc40(0000) GS:ffff88815ad00000(0000) knlGS:0000000000000000
[  226.426709] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  226.427501] CR2: 0000000000000478 CR3: 00000001083e9000 CR4: 0000000000350ee0
[  226.428492] Kernel panic - not syncing: Fatal exception in interrupt
[  226.429587] Kernel Offset: disabled
[  226.430096] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---

(gdb) list *iptfs_output_collect+0x1f5
0xffffffff81ce40d6 is in iptfs_output_collect (./include/net/net_namespace.h:385).
380	}
381
382	static inline struct net *read_pnet(const possible_net_t *pnet)
383	{
384	#ifdef CONFIG_NET_NS
385		return pnet->net;
386	#else
387		return &init_net;
388	#endif
389	}


On Sun, Nov 12, 2023 at 10:52:11PM -0500, Christian Hopps via Devel wrote:
> From: Christian Hopps <chopps@labn.net>
> 
> This patchset adds a new xfrm mode implementing on-demand IP-TFS. IP-TFS
> (AggFrag encapsulation) has been standardized in RFC9347.
> 
> Link: https://www.rfc-editor.org/rfc/rfc9347.txt
> 
> This feature supports demand driven (i.e., non-constant send rate) IP-TFS to
> take advantage of the AGGFRAG ESP payload encapsulation. This payload type
> supports aggregation and fragmentation of the inner IP packet stream which in
> turn yields higher small-packet bandwidth as well as reducing MTU/PMTU issues.
> Congestion control is unimplementated as the send rate is demand driven rather
> than constant.
> 
> In order to allow loading this fucntionality as a module a set of callbacks
> xfrm_mode_cbs has been added to xfrm as well.
> -- 
> Devel mailing list
> Devel@linux-ipsec.org
> https://linux-ipsec.org/mailman/listinfo/devel

Re: [RFC ipsec-next v2 8/8] iptfs: impl: add new iptfs xfrm mode impl

[Sabrina Dubroca]

2023-11-12, 22:52:19 -0500, Christian Hopps wrote:
> From: Christian Hopps <chopps@labn.net>
> 
> Add a new xfrm mode implementing AggFrag/IP-TFS from RFC9347.
> 
> This utilizes the new xfrm_mode_cbs to implement demand-driven IP-TFS
> functionality. This functionality can be used to increase bandwidth
> utilization through small packet aggregation, as well as help solve PMTU
> issues through it's efficient use of fragmentation.
> 
> Link: https://www.rfc-editor.org/rfc/rfc9347.txt
> 
> Signed-off-by: Christian Hopps <chopps@labn.net>
> ---
>  include/net/iptfs.h    |   18 +
>  net/xfrm/Makefile      |    1 +
>  net/xfrm/trace_iptfs.h |  224 ++++
>  net/xfrm/xfrm_iptfs.c  | 2696 ++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 2939 insertions(+)
>  create mode 100644 include/net/iptfs.h
>  create mode 100644 net/xfrm/trace_iptfs.h
>  create mode 100644 net/xfrm/xfrm_iptfs.c
> 
> diff --git a/include/net/iptfs.h b/include/net/iptfs.h
> new file mode 100644
> index 000000000000..d8f2e494f251
> --- /dev/null
> +++ b/include/net/iptfs.h

Is this header needed? It's only included by net/xfrm/xfrm_iptfs.c,
why not put those #defines directly in the file?

> @@ -0,0 +1,18 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _NET_IPTFS_H
> +#define _NET_IPTFS_H
> +
> +#include <linux/types.h>
> +#include <linux/ip.h>
> +
> +#define IPTFS_SUBTYPE_BASIC 0
> +#define IPTFS_SUBTYPE_CC 1
> +#define IPTFS_SUBTYPE_LAST IPTFS_SUBTYPE_CC

_LAST is never used.

> +#define IPTFS_CC_FLAGS_ECN_CE 0x1
> +#define IPTFS_CC_FLAGS_PLMTUD 0x2

Not used either.

> +extern void xfrm_iptfs_get_rtt_and_delays(struct ip_iptfs_cc_hdr *cch, u32 *rtt,
> +					  u32 *actual_delay, u32 *xmit_delay);

Not implemented in this series, drop this.

[...]
> diff --git a/net/xfrm/xfrm_iptfs.c b/net/xfrm/xfrm_iptfs.c
> new file mode 100644
> index 000000000000..65f7acdbe6a8
> --- /dev/null
> +++ b/net/xfrm/xfrm_iptfs.c
[...]
> +struct sk_buff *iptfs_pskb_add_frags(struct sk_buff *tpl,

nit: static? afaict it's not used outside this file.

> +				     struct skb_frag_walk *walk, u32 off,
> +				     u32 len, struct skb_seq_state *st,
> +				     u32 copy_len)
> +{


[...]
> +
> +/**
> + * iptfs_input_ordered() - handle next in order IPTFS payload.
> + *
> + * Process the IPTFS payload in `skb` and consume it afterwards.
> + */
> +static int iptfs_input_ordered(struct xfrm_state *x, struct sk_buff *skb)
> +{

Can we try to not introduce a worse problem than xfrm_input already
is? 326 lines and 20+ local variables is way too much. And then it
calls another 200+ lines function...

I did try to understand what the main loop does but I got completely
lost the 3 times I tried :/


> +static u32 __reorder_this(struct xfrm_iptfs_data *xtfs, struct sk_buff *inskb,
> +			  struct list_head *list)
> +

nit: extra blank line

> +{


[...]
> +/**
> + * iptfs_input() - handle receipt of iptfs payload
> + * @x: xfrm state.
> + * @skb: the packet.
> + *
> + * We have an IPTFS payload order it if needed, then process newly in order
> + * packetsA.

typo? "packetsA"


[...]
> +/* IPv4/IPv6 packet ingress to IPTFS tunnel, arrange to send in IPTFS payload
> + * (i.e., aggregating or fragmenting as appropriate).
> + * This is set in dst->output for an SA.
> + */
> +static int iptfs_output_collect(struct net *net, struct sock *sk,
> +				struct sk_buff *skb)
> +{
> +	struct dst_entry *dst = skb_dst(skb);
> +	struct xfrm_state *x = dst->xfrm;
> +	struct xfrm_iptfs_data *xtfs = x->mode_data;
> +	struct sk_buff *segs, *nskb;
> +	u32 count, qcount;
> +	u32 pmtu = 0;
> +	bool ok = true;
> +	bool was_gso;
> +
> +	/* We have hooked into dst_entry->output which means we have skipped the
> +	 * protocol specific netfilter (see xfrm4_output, xfrm6_output).
> +	 * when our timer runs we will end up calling xfrm_output directly on
> +	 * the encapsulated traffic.
> +	 *
> +	 * For both cases this is the NF_INET_POST_ROUTING hook which allows
> +	 * changing the skb->dst entry which then may not be xfrm based anymore
> +	 * in which case a REROUTED flag is set. and dst_output is called.
> +	 *
> +	 * For IPv6 we are also skipping fragmentation handling for local
> +	 * sockets, which may or may not be good depending on our tunnel DF
> +	 * setting. Normally with fragmentation supported we want to skip this
> +	 * fragmentation.
> +	 */
> +
> +	BUG_ON(!xtfs);

Or drop the packet and add a DEBUG_NET_WARN_ON_ONCE? This should never
happen, but why crash the system when we have a way to deal with this
error?

> +
> +	if (xtfs->cfg.dont_frag)
> +		pmtu = iptfs_get_cur_pmtu(x, xtfs, skb);
> +
> +	/* Break apart GSO skbs. If the queue is nearing full then we want the
> +	 * accounting and queuing to be based on the individual packets not on the
> +	 * aggregate GSO buffer.
> +	 */
> +	was_gso = skb_is_gso(skb);
> +	if (!was_gso) {
> +		segs = skb;
> +	} else {
> +		segs = skb_gso_segment(skb, 0);
> +		if (IS_ERR_OR_NULL(segs)) {
> +			XFRM_INC_STATS(net, LINUX_MIB_XFRMOUTERROR);
> +			kfree_skb(skb);
> +			return PTR_ERR(segs);
> +		}
> +		consume_skb(skb);
> +		skb = NULL;
> +	}
> +
> +	count = 0;
> +	qcount = 0;

nit: both of those get incremented through the main loop but never read

> +
> +	/* We can be running on multiple cores and from the network softirq or
> +	 * from user context depending on where the packet is coming from.
> +	 */
> +	spin_lock_bh(&x->lock);
> +
> +	skb_list_walk_safe(segs, skb, nskb) {
> +		skb_mark_not_on_list(skb);
> +		count++;
> +
> +		/* Once we drop due to no queue space we continue to drop the
> +		 * rest of the packets from that GRO.
> +		 */
> +		if (!ok) {
> +nospace:
> +			trace_iptfs_no_queue_space(skb, xtfs, pmtu, was_gso);
> +			XFRM_INC_STATS(dev_net(skb->dev), LINUX_MIB_XFRMOUTNOQSPACE);
> +			kfree_skb_reason(skb, SKB_DROP_REASON_FULL_RING);
> +			continue;
> +		}
> +
> +		/* If the user indicated no iptfs fragmenting check before
> +		 * enqueue.
> +		 */
> +		if (xtfs->cfg.dont_frag && iptfs_is_too_big(sk, skb, pmtu)) {
> +			trace_iptfs_too_big(skb, xtfs, pmtu, was_gso);
> +			kfree_skb_reason(skb, SKB_DROP_REASON_PKT_TOO_BIG);
> +			continue;
> +		}
> +
> +		/* Enqueue to send in tunnel */
> +

nit: unneeded blank line

> +		ok = iptfs_enqueue(xtfs, skb);
> +		if (!ok)
> +			goto nospace;
> +
> +		trace_iptfs_enqueue(skb, xtfs, pmtu, was_gso);
> +		qcount++;
> +	}
> +
> +	/* Start a delay timer if we don't have one yet */
> +	if (!hrtimer_is_queued(&xtfs->iptfs_timer)) {
> +		/* softirq blocked lest the timer fire and interrupt us */
> +		BUG_ON(!in_interrupt());

Why is that a fatal condition?

> +		hrtimer_start(&xtfs->iptfs_timer, xtfs->init_delay_ns,
> +			      IPTFS_HRTIMER_MODE);
> +
> +		xtfs->iptfs_settime = ktime_get_raw_fast_ns();
> +		trace_iptfs_timer_start(xtfs, xtfs->init_delay_ns);
> +	}
> +
> +	spin_unlock_bh(&x->lock);
> +	return 0;
> +}
> +

[...]
> +static int iptfs_copy_create_frags(struct sk_buff **skbp,
> +				   struct xfrm_iptfs_data *xtfs, u32 mtu)
> +{
[...]
> +	/* prepare the initial fragment with an iptfs header */
> +	iptfs_output_prepare_skb(skb, 0);
> +
> +	/* Send all but last fragment. */
> +	list_for_each_entry_safe(skb, nskb, &sublist, list) {
> +		skb_list_del_init(skb);
> +		xfrm_output(NULL, skb);

Should we stop if xfrm_output fails? Or is it still useful to send the
rest of the iptfs frags if we lose one in the middle?

> +	}
> +
> +	return 0;
> +}
> +

[...]
> +static int iptfs_first_skb(struct sk_buff **skbp, struct xfrm_iptfs_data *xtfs,
> +			   u32 mtu)
> +{
> +	struct sk_buff *skb = *skbp;
> +	int err;
> +
> +	/* Classic ESP skips the don't fragment ICMP error if DF is clear on
> +	 * the inner packet or ignore_df is set. Otherwise it will send an ICMP
> +	 * or local error if the inner packet won't fit it's MTU.
> +	 *
> +	 * With IPTFS we do not care about the inner packet DF bit. If the
> +	 * tunnel is configured to "don't fragment" we error back if things
> +	 * don't fit in our max packet size. Otherwise we iptfs-fragment as
> +	 * normal.
> +	 */
> +
> +	/* The opportunity for HW offload has ended */
> +	if (skb->ip_summed == CHECKSUM_PARTIAL) {
> +		err = skb_checksum_help(skb);
> +		if (err)
> +			return err;
> +	}
> +
> +	/* We've split these up before queuing */
> +	BUG_ON(skb_is_gso(skb));

Drop and DEBUG_NET_WARN_ON_ONCE?

> +
> +	trace_iptfs_first_dequeue(skb, mtu, 0, ip_hdr(skb));
> +
> +	/* Simple case -- it fits. `mtu` accounted for all the overhead
> +	 * including the basic IPTFS header.
> +	 */
> +	if (skb->len <= mtu) {
> +		iptfs_output_prepare_skb(skb, 0);
> +		return 0;
> +	}
> +
> +	BUG_ON(xtfs->cfg.dont_frag);

and here?

> +	if (iptfs_first_should_copy(skb, mtu))
> +		return iptfs_copy_create_frags(skbp, xtfs, mtu);

Since we end up copying anyway, drop this (and
iptfs_first_should_copy). You can introduce the optimization later on.


> +	/* For now we always copy */
> +	return iptfs_copy_create_frags(skbp, xtfs, mtu);
> +}
> +
> +static struct sk_buff **iptfs_rehome_fraglist(struct sk_buff **nextp,
> +					      struct sk_buff *child)
> +{
> +	u32 fllen = 0;
> +
> +	BUG_ON(!skb_has_frag_list(child));

Not needed, this was tested just before calling this function.

> +
> +	/* It might be possible to account for a frag list in addition to page
> +	 * fragment if it's a valid state to be in. The page fragments size
> +	 * should be kept as data_len so only the frag_list size is removed,
> +	 * this must be done above as well took
> +	 */
> +	BUG_ON(skb_shinfo(child)->nr_frags);

Again not worth crashing the system?

> +	*nextp = skb_shinfo(child)->frag_list;
> +	while (*nextp) {
> +		fllen += (*nextp)->len;
> +		nextp = &(*nextp)->next;
> +	}
> +	skb_frag_list_init(child);
> +	BUG_ON(fllen > child->data_len);
> +	child->len -= fllen;
> +	child->data_len -= fllen;
> +
> +	return nextp;
> +}

[...]
> +static void iptfs_output_queued(struct xfrm_state *x, struct sk_buff_head *list)
> +{
> +	struct xfrm_iptfs_data *xtfs = x->mode_data;
> +	u32 payload_mtu = xtfs->payload_mtu;
> +	struct sk_buff *skb, *skb2, **nextp;
> +	struct skb_shared_info *shi, *shi2;
> +
> +	/* For now we are just outputting packets as fast as we can, so if we
> +	 * are fragmenting we will do so until the last inner packet has been
> +	 * consumed.
> +	 *
> +	 * When we are fragmenting we need to output all outer packets that
> +	 * contain the fragments of a single inner packet, consecutively (ESP
> +	 * seq-wise). So we need a lock to keep another CPU from sending the
> +	 * next batch of packets (it's `list`) and trying to output those, while
> +	 * we output our `list` resuling with interleaved non-spec-client inner
> +	 * packet streams. Thus we need to lock the IPTFS output on a per SA
> +	 * basis while we process this list.
> +	 */

This talks about a lock but I don't see one. What am I missing?

> +
> +	/* NOTE: for the future, for timed packet sends, if our queue is not
> +	 * growing longer (i.e., we are keeping up) and a packet we are about to
> +	 * fragment will not fragment in then next outer packet, we might consider
> +	 * holding on to it to send whole in the next slot. The question then is
> +	 * does this introduce a continuous delay in the inner packet stream
> +	 * with certain packet rates and sizes?
> +	 */
> +
> +	/* and send them on their way */
> +
> +	while ((skb = __skb_dequeue(list))) {
> +		struct xfrm_dst *xdst = (struct xfrm_dst *)skb_dst(skb);
> +		u32 mtu = __iptfs_get_inner_mtu(x, xdst->child_mtu_cached);
> +		bool share_ok = true;
> +		int remaining;
> +
> +		/* protocol comes to us cleared sometimes */
> +		skb->protocol = x->outer_mode.family == AF_INET ?
> +					htons(ETH_P_IP) :
> +					htons(ETH_P_IPV6);
> +
> +		if (payload_mtu && payload_mtu < mtu)
> +			mtu = payload_mtu;

Isn't that iptfs_get_cur_pmtu?


[...]
> +static enum hrtimer_restart iptfs_delay_timer(struct hrtimer *me)
> +{
> +	struct sk_buff_head list;
> +	struct xfrm_iptfs_data *xtfs;
> +	struct xfrm_state *x;
> +	time64_t settime;
> +	size_t osize;
> +
> +	xtfs = container_of(me, typeof(*xtfs), iptfs_timer);
> +	x = xtfs->x;
> +
> +	/* Process all the queued packets
> +	 *
> +	 * softirq execution order: timer > tasklet > hrtimer
> +	 *
> +	 * Network rx will have run before us giving one last chance to queue
> +	 * ingress packets for us to process and transmit.
> +	 */
> +
> +	spin_lock(&x->lock);
> +	__skb_queue_head_init(&list);
> +	skb_queue_splice_init(&xtfs->queue, &list);
> +	osize = xtfs->queue_size;

Unused variable?

[...]
> +static int iptfs_user_init(struct net *net, struct xfrm_state *x,
> +			   struct nlattr **attrs)
> +{
> +	struct xfrm_iptfs_data *xtfs = x->mode_data;
> +	struct xfrm_iptfs_config *xc;
> +
> +	if (x->props.mode != XFRM_MODE_IPTFS)
> +		return -EINVAL;

Is that necessary? This only gets called via ->user_init for this
mode.

> +	xc = &xtfs->cfg;
> +	xc->reorder_win_size = net->xfrm.sysctl_iptfs_rewin;
> +	xc->max_queue_size = net->xfrm.sysctl_iptfs_maxqsize;
> +	xc->init_delay_us = net->xfrm.sysctl_iptfs_idelay;
> +	xc->drop_time_us = net->xfrm.sysctl_iptfs_drptime;
> +
> +	if (attrs[XFRMA_IPTFS_DONT_FRAG])
> +		xc->dont_frag = true;
> +	if (attrs[XFRMA_IPTFS_REORD_WIN])
> +		xc->reorder_win_size =
> +			nla_get_u16(attrs[XFRMA_IPTFS_REORD_WIN]);
> +	/* saved array is for saving 1..N seq nums from wantseq */
> +	if (xc->reorder_win_size)
> +		xtfs->w_saved = kcalloc(xc->reorder_win_size,
> +					sizeof(*xtfs->w_saved), GFP_KERNEL);

We probably need a reasonable bound on reorder_win_size so that we
don't try to allocate crazy amounts of memory here.

> +	if (attrs[XFRMA_IPTFS_PKT_SIZE]) {
> +		xc->pkt_size = nla_get_u32(attrs[XFRMA_IPTFS_PKT_SIZE]);
> +		if (!xc->pkt_size)
> +			xtfs->payload_mtu = 0;

That's already set to 0 via kzalloc, right? So passing 0 as
XFRMA_IPTFS_PKT_SIZE is equivalent to not providing it?

> +		else if (xc->pkt_size > x->props.header_len)
> +			xtfs->payload_mtu = xc->pkt_size - x->props.header_len;
> +		else
> +			return -EINVAL;

This could probably use an extack to explain why the value was rejected.

> +	}
> +	if (attrs[XFRMA_IPTFS_MAX_QSIZE])
> +		xc->max_queue_size = nla_get_u32(attrs[XFRMA_IPTFS_MAX_QSIZE]);
> +	if (attrs[XFRMA_IPTFS_DROP_TIME])
> +		xc->drop_time_us = nla_get_u32(attrs[XFRMA_IPTFS_DROP_TIME]);
> +	if (attrs[XFRMA_IPTFS_INIT_DELAY])
> +		xc->init_delay_us = nla_get_u32(attrs[XFRMA_IPTFS_INIT_DELAY]);
> +
> +	xtfs->ecn_queue_size = (u64)xc->max_queue_size * 95 / 100;
> +	xtfs->drop_time_ns = xc->drop_time_us * NSECS_IN_USEC;
> +	xtfs->init_delay_ns = xc->init_delay_us * NSECS_IN_USEC;

Can't we get rid of the _us version? Why store both in kernel memory?


[...]
> +static void iptfs_delete_state(struct xfrm_state *x)
> +{
> +	struct xfrm_iptfs_data *xtfs = x->mode_data;
> +
> +	if (IS_ERR_OR_NULL(xtfs))

Can mode_data ever be an error pointer?

> +		return;
> +
> +	spin_lock(&xtfs->drop_lock);
> +	hrtimer_cancel(&xtfs->iptfs_timer);
> +	hrtimer_cancel(&xtfs->drop_timer);
> +	spin_unlock(&xtfs->drop_lock);
> +
> +	kfree_sensitive(xtfs->w_saved);
> +	kfree_sensitive(xtfs);
> +}
> +
> +static const struct xfrm_mode_cbs iptfs_mode_cbs = {
> +	.owner = THIS_MODULE,
> +	.create_state = iptfs_create_state,
> +	.delete_state = iptfs_delete_state,
> +	.user_init = iptfs_user_init,
> +	.copy_to_user = iptfs_copy_to_user,
> +	.get_inner_mtu = iptfs_get_inner_mtu,
> +	.input = iptfs_input,
> +	.output = iptfs_output_collect,
> +	.prepare_output = iptfs_prepare_output,
> +};
> +
> +static int __init xfrm_iptfs_init(void)
> +{
> +	int err;
> +
> +	pr_info("xfrm_iptfs: IPsec IP-TFS tunnel mode module\n");
> +
> +	err = xfrm_register_mode_cbs(XFRM_MODE_IPTFS, &iptfs_mode_cbs);
> +	if (err < 0)
> +		pr_info("%s: can't register IP-TFS\n", __func__);
> +
> +	return err;
> +}
> +
> +static void __exit xfrm_iptfs_fini(void)
> +{
> +	xfrm_unregister_mode_cbs(XFRM_MODE_IPTFS);
> +}

If the module is unloaded, existing xfrm states will be left but
silently broken?

-- 
Sabrina

Re: [RFC ipsec-next v2 6/8] iptfs: xfrm: Add mode_cbs module functionality

[Sabrina Dubroca]

2023-11-12, 22:52:17 -0500, Christian Hopps wrote:
> From: Christian Hopps <chopps@labn.net>
> 
> Add a set of callbacks xfrm_mode_cbs to xfrm_state. These callbacks
> enable the addition of new xfrm modes, such as IP-TFS to be defined
> in modules.

Not a big fan of bringing back modes in modules :(
Florian's work made the code a lot more readable.

> diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
> index 662c83beb345..4390c111410d 100644
> --- a/net/xfrm/xfrm_output.c
> +++ b/net/xfrm/xfrm_output.c
> @@ -280,7 +280,9 @@ static int xfrm4_tunnel_encap_add(struct xfrm_state *x, struct sk_buff *skb)
>  	skb_set_inner_network_header(skb, skb_network_offset(skb));
>  	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
>  
> -	skb_set_network_header(skb, -x->props.header_len);
> +	/* backup to add space for the outer encap */
> +	skb_set_network_header(skb,
> +			       -x->props.header_len + x->props.enc_hdr_len);

Since this only gets called for XFRM_MODE_TUNNEL, and only iptfs sets
enc_hdr_len, do we need this change? (and same for xfrm6_tunnel_encap_add)

>  	skb->mac_header = skb->network_header +
>  			  offsetof(struct iphdr, protocol);
>  	skb->transport_header = skb->network_header + sizeof(*top_iph);
> @@ -325,7 +327,8 @@ static int xfrm6_tunnel_encap_add(struct xfrm_state *x, struct sk_buff *skb)
>  	skb_set_inner_network_header(skb, skb_network_offset(skb));
>  	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
>  
> -	skb_set_network_header(skb, -x->props.header_len);
> +	skb_set_network_header(skb,
> +			       -x->props.header_len + x->props.enc_hdr_len);
>  	skb->mac_header = skb->network_header +
>  			  offsetof(struct ipv6hdr, nexthdr);
>  	skb->transport_header = skb->network_header + sizeof(*top_iph);
> @@ -472,6 +475,8 @@ static int xfrm_outer_mode_output(struct xfrm_state *x, struct sk_buff *skb)
>  		WARN_ON_ONCE(1);
>  		break;
>  	default:
> +		if (x->mode_cbs->prepare_output)

Can x->mode_cbs be NULL here? Every other use of mode_cbs does
    if (x->mode_cbs && x->mode_cbs->FOO)

(I think not at the moment since only IPTFS (and IN_TRIGGER) can reach
this, but this inconsistency with the rest of the series struck me)

-- 
Sabrina

Re: [RFC ipsec-next v2 5/8] iptfs: netlink: add config (netlink) options

[Sabrina Dubroca]

2023-11-12, 22:52:16 -0500, Christian Hopps wrote:
> @@ -3046,6 +3056,12 @@ const struct nla_policy xfrma_policy[XFRMA_MAX+1] = {
>  	[XFRMA_SET_MARK_MASK]	= { .type = NLA_U32 },
>  	[XFRMA_IF_ID]		= { .type = NLA_U32 },
>  	[XFRMA_MTIMER_THRESH]   = { .type = NLA_U32 },
> +	[XFRMA_IPTFS_PKT_SIZE]	= { .type = NLA_U32 },
> +	[XFRMA_IPTFS_MAX_QSIZE]	= { .type = NLA_U32 },
> +	[XFRMA_IPTFS_DONT_FRAG]	= { .type = NLA_FLAG },

Do we want to be able to turn this off with an update? Flags don't
really allow this.

-- 
Sabrina

Re: [RFC ipsec-next v2 4/8] iptfs: sysctl: allow configuration of global default values

[Christian Hopps]

[-- Attachment #1: Type: text/plain, Size: 4750 bytes --]


Sabrina Dubroca <sd@queasysnail.net> writes:

> 2023-11-12, 22:52:15 -0500, Christian Hopps wrote:
>> From: Christian Hopps <chopps@labn.net>
>>
>> Add sysctls for the changing the IPTFS default SA values.
>>
>> Signed-off-by: Christian Hopps <chopps@labn.net>
>> ---
>>  Documentation/networking/xfrm_sysctl.rst | 29 ++++++++++++++++++
>>  include/net/netns/xfrm.h                 |  6 ++++
>>  include/net/xfrm.h                       |  7 +++++
>>  net/xfrm/xfrm_sysctl.c                   | 38 ++++++++++++++++++++++++
>>  4 files changed, 80 insertions(+)
>>
>> diff --git a/Documentation/networking/xfrm_sysctl.rst b/Documentation/networking/xfrm_sysctl.rst
>> index 47b9bbdd0179..9e628806c110 100644
>> --- a/Documentation/networking/xfrm_sysctl.rst
>> +++ b/Documentation/networking/xfrm_sysctl.rst
>> @@ -9,3 +9,32 @@ XFRM Syscall
>>
>>  xfrm_acq_expires - INTEGER
>>  	default 30 - hard timeout in seconds for acquire requests
>> +
>> +xfrm_iptfs_maxqsize - UNSIGNED INTEGER
>> +        The default IPTFS max output queue size in octets. The output queue is
>> +        where received packets destined for output over an IPTFS tunnel are
>> +        stored prior to being output in aggregated/fragmented form over the
>> +        IPTFS tunnel.
>> +
>> +        Default 1M.
>> +
>> +xfrm_iptfs_drptime - UNSIGNED INTEGER
>
> nit: Can we make those names a bit more human-readable?
> xfrm_iptfs_droptime, or possibly xfrm_iptfs_drop_time for better
> consistency with the netlink API.

Changed to xfrm_iptfs_drop_time.

>> +        The default IPTFS drop time in microseconds. The drop time is the amount
>> +        of time before a missing out-of-order IPTFS tunnel packet is considered
>> +        lost. See also the reorder window.
>> +
>> +        Default 1s (1000000).
>> +
>> +xfrm_iptfs_idelay - UNSIGNED INTEGER
>
> I would suggest xfrm_iptfs_initial_delay (or at least init_delay like
> the netlink attribute).

Changed to xfrm_iptfs_init_delay.

>
>> +        The default IPTFS initial output delay in microseconds. The initial
>> +        output delay is the amount of time prior to servicing the output queue
>> +        after queueing the first packet on said queue.
>
> Does that also apply if the queue was fully drained (no traffic for a
> while) and starts being used again? That might be worth documenting
> either way (sorry, I haven't processed the main patch far enough to
> answer this question myself yet).

Added: "This applies anytime the output queue was previously empty."

> And it might be worth mentioning that all these sysctls can be
> overridden per SA via the netlink API.

The description of these values as the "default"s implies the fact that they can be changed, I think.

>> +        Default 0.
>> +
>> +xfrm_iptfs_rewin - UNSIGNED INTEGER
>
> xfrm_iptfs_reorderwin (or reorder_win, or reorder_window)?
> I'd also make the equivalent netlink attribute's name a bit longer (at
> least spell out REORDER, I can live with WIN for WINDOW).
>

Changed to xfrm_iptfs_reorder_window.

Also renamed the netlink attribute REORD_WIN to match (i.e., not XFRMA_IPTFS_REORDER_WINDOW).

> [...]
>> diff --git a/include/net/xfrm.h b/include/net/xfrm.h
>> index c9bb0f892f55..d2e87344d175 100644
>> --- a/include/net/xfrm.h
>> +++ b/include/net/xfrm.h
>> @@ -2190,4 +2190,11 @@ static inline int register_xfrm_interface_bpf(void)
>>
>>  #endif
>>
>> +#if IS_ENABLED(CONFIG_XFRM_IPTFS)
>> +#define XFRM_IPTFS_DEFAULT_MAX_QUEUE_SIZE (1024 * 1024)
>> +#define XFRM_IPTFS_DEFAULT_INIT_DELAY_USECS (0)
>> +#define XFRM_IPTFS_DEFAULT_DROP_TIME_USECS (1000000)
>> +#define XFRM_IPTFS_DEFAULT_REORDER_WINDOW (3)
>> +#endif
>
> nit: move those to net/xfrm/xfrm_sysctl.c ? they're only used in that file.

Rather than move them directly above where they are only used I just removed them entirely.

Thanks,
Chris.

>> diff --git a/net/xfrm/xfrm_sysctl.c b/net/xfrm/xfrm_sysctl.c
>> index 7fdeafc838a7..bf8e73a6c38e 100644
>> --- a/net/xfrm/xfrm_sysctl.c
>> +++ b/net/xfrm/xfrm_sysctl.c
> [...]
>> @@ -55,6 +87,12 @@ int __net_init xfrm_sysctl_init(struct net *net)
>>  	table[1].data = &net->xfrm.sysctl_aevent_rseqth;
>>  	table[2].data = &net->xfrm.sysctl_larval_drop;
>>  	table[3].data = &net->xfrm.sysctl_acq_expires;
>> +#if IS_ENABLED(CONFIG_XFRM_IPTFS)
>> +	table[4].data = &net->xfrm.sysctl_iptfs_drptime;
>> +	table[5].data = &net->xfrm.sysctl_iptfs_idelay;
>> +	table[6].data = &net->xfrm.sysctl_iptfs_maxqsize;
>> +	table[7].data = &net->xfrm.sysctl_iptfs_rewin;
>> +#endif
>
> Is it time to switch to a loop like in ipv6_sysctl_net_init? See
> commit d2f7e56d1e40 ("ipv6: Use math to point per net sysctls into the
> appropriate struct net"). OTOH we don't add sysctls for xfrm very
> often so it's not critical.


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 858 bytes --]

Re: [RFC ipsec-next v2 2/8] iptfs: uapi: ip: add ip_tfs_*_hdr packet formats

[Christian Hopps]

[-- Attachment #1: Type: text/plain, Size: 1709 bytes --]


Sabrina Dubroca <sd@queasysnail.net> writes:

> 2023-11-12, 22:52:13 -0500, Christian Hopps wrote:
>> From: Christian Hopps <chopps@labn.net>
>>
>> Add the on-wire basic and congestion-control IP-TFS packet headers.
>>
>> Signed-off-by: Christian Hopps <chopps@labn.net>
>> ---
>>  include/uapi/linux/ip.h | 17 +++++++++++++++++
>>  1 file changed, 17 insertions(+)
>>
>> diff --git a/include/uapi/linux/ip.h b/include/uapi/linux/ip.h
>> index 283dec7e3645..cc83878ecf08 100644
>> --- a/include/uapi/linux/ip.h
>> +++ b/include/uapi/linux/ip.h
>> @@ -137,6 +137,23 @@ struct ip_beet_phdr {
>>  	__u8 reserved;
>>  };
>>
>> +struct ip_iptfs_hdr {
>> +	__u8 subtype;		/* 0*: basic, 1: CC */
>> +	__u8 flags;
>> +	__be16 block_offset;
>> +};
>> +
>> +struct ip_iptfs_cc_hdr {
>> +	__u8 subtype;		/* 0: basic, 1*: CC */
>> +	__u8 flags;
>> +	__be16 block_offset;
>> +	__be32 loss_rate;
>> +	__u8 rtt_and_adelay1[4];
>> +	__u8 adelay2_and_xdelay[4];
>
> Given how the fields are split, wouldn't it be more convenient to have
> a single __be64, rather than reading some bits from multiple __u8?

Changed this to __be64.


>> +	__be32 tval;
>> +	__be32 techo;
>> +};
>
> I don't think these need to be part of uapi. Can we move them to
> include/net/iptfs.h (or possibly net/xfrm/xfrm_iptfs.c)? It would also
> make more sense to have them near the definitions for
> IPTFS_SUBTYPE_*. And it would be easier to change how we split and
> name fields for kernel consumption if we're not stuck with whatever we
> put in uapi.

I saw this also as the place documenting the packet format. Userland can have raw packets delivered to them... Mostly though I was following the pattern that existed already.

Thanks,
Chris.


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

Re: [RFC ipsec-next v2 6/8] iptfs: xfrm: Add mode_cbs module functionality

[Christian Hopps]

[-- Attachment #1: Type: text/plain, Size: 2453 bytes --]


Sabrina Dubroca <sd@queasysnail.net> writes:

> 2023-11-12, 22:52:17 -0500, Christian Hopps wrote:
>> From: Christian Hopps <chopps@labn.net>
>>
>> Add a set of callbacks xfrm_mode_cbs to xfrm_state. These callbacks
>> enable the addition of new xfrm modes, such as IP-TFS to be defined
>> in modules.
>
> Not a big fan of bringing back modes in modules :(
> Florian's work made the code a lot more readable.
>
>> diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
>> index 662c83beb345..4390c111410d 100644
>> --- a/net/xfrm/xfrm_output.c
>> +++ b/net/xfrm/xfrm_output.c
>> @@ -280,7 +280,9 @@ static int xfrm4_tunnel_encap_add(struct xfrm_state *x, struct sk_buff *skb)
>>  	skb_set_inner_network_header(skb, skb_network_offset(skb));
>>  	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
>>
>> -	skb_set_network_header(skb, -x->props.header_len);
>> +	/* backup to add space for the outer encap */
>> +	skb_set_network_header(skb,
>> +			       -x->props.header_len + x->props.enc_hdr_len);
>
> Since this only gets called for XFRM_MODE_TUNNEL, and only iptfs sets
> enc_hdr_len, do we need this change? (and same for xfrm6_tunnel_encap_add)

You're right, removed. This particular code actually predated the callbacks.

>>  	skb->mac_header = skb->network_header +
>>  			  offsetof(struct iphdr, protocol);
>>  	skb->transport_header = skb->network_header + sizeof(*top_iph);
>> @@ -325,7 +327,8 @@ static int xfrm6_tunnel_encap_add(struct xfrm_state *x, struct sk_buff *skb)
>>  	skb_set_inner_network_header(skb, skb_network_offset(skb));
>>  	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
>>
>> -	skb_set_network_header(skb, -x->props.header_len);
>> +	skb_set_network_header(skb,
>> +			       -x->props.header_len + x->props.enc_hdr_len);
>>  	skb->mac_header = skb->network_header +
>>  			  offsetof(struct ipv6hdr, nexthdr);
>>  	skb->transport_header = skb->network_header + sizeof(*top_iph);
>> @@ -472,6 +475,8 @@ static int xfrm_outer_mode_output(struct xfrm_state *x, struct sk_buff *skb)
>>  		WARN_ON_ONCE(1);
>>  		break;
>>  	default:
>> +		if (x->mode_cbs->prepare_output)
>
> Can x->mode_cbs be NULL here? Every other use of mode_cbs does
>     if (x->mode_cbs && x->mode_cbs->FOO)
>
> (I think not at the moment since only IPTFS (and IN_TRIGGER) can reach
> this, but this inconsistency with the rest of the series struck me)

Still worth putting the guard in, fixed.

Thanks,
Chris.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

Re: [RFC ipsec-next v2 8/8] iptfs: impl: add new iptfs xfrm mode impl

[Christian Hopps]

[-- Attachment #1: Type: text/plain, Size: 19361 bytes --]


Sabrina Dubroca <sd@queasysnail.net> writes:

> 2023-11-12, 22:52:19 -0500, Christian Hopps wrote:
>> From: Christian Hopps <chopps@labn.net>
>>
>> Add a new xfrm mode implementing AggFrag/IP-TFS from RFC9347.
>>
>> This utilizes the new xfrm_mode_cbs to implement demand-driven IP-TFS
>> functionality. This functionality can be used to increase bandwidth
>> utilization through small packet aggregation, as well as help solve PMTU
>> issues through it's efficient use of fragmentation.
>>
>> Link: https://www.rfc-editor.org/rfc/rfc9347.txt
>>
>> Signed-off-by: Christian Hopps <chopps@labn.net>
>> ---
>>  include/net/iptfs.h    |   18 +
>>  net/xfrm/Makefile      |    1 +
>>  net/xfrm/trace_iptfs.h |  224 ++++
>>  net/xfrm/xfrm_iptfs.c  | 2696 ++++++++++++++++++++++++++++++++++++++++
>>  4 files changed, 2939 insertions(+)
>>  create mode 100644 include/net/iptfs.h
>>  create mode 100644 net/xfrm/trace_iptfs.h
>>  create mode 100644 net/xfrm/xfrm_iptfs.c
>>
>> diff --git a/include/net/iptfs.h b/include/net/iptfs.h
>> new file mode 100644
>> index 000000000000..d8f2e494f251
>> --- /dev/null
>> +++ b/include/net/iptfs.h
>
> Is this header needed? It's only included by net/xfrm/xfrm_iptfs.c,
> why not put those #defines directly in the file?

Moved into xfrm_iptfs.c and removed this file.

>
>> @@ -0,0 +1,18 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +#ifndef _NET_IPTFS_H
>> +#define _NET_IPTFS_H
>> +
>> +#include <linux/types.h>
>> +#include <linux/ip.h>
>> +
>> +#define IPTFS_SUBTYPE_BASIC 0
>> +#define IPTFS_SUBTYPE_CC 1
>> +#define IPTFS_SUBTYPE_LAST IPTFS_SUBTYPE_CC
>
> _LAST is never used.

Removed.

>> +#define IPTFS_CC_FLAGS_ECN_CE 0x1
>> +#define IPTFS_CC_FLAGS_PLMTUD 0x2
>
> Not used either.

Removed (until we need them).

>> +extern void xfrm_iptfs_get_rtt_and_delays(struct ip_iptfs_cc_hdr *cch, u32 *rtt,
>> +					  u32 *actual_delay, u32 *xmit_delay);
>
> Not implemented in this series, drop this.

Gone.

> [...]
>> diff --git a/net/xfrm/xfrm_iptfs.c b/net/xfrm/xfrm_iptfs.c
>> new file mode 100644
>> index 000000000000..65f7acdbe6a8
>> --- /dev/null
>> +++ b/net/xfrm/xfrm_iptfs.c
> [...]
>> +struct sk_buff *iptfs_pskb_add_frags(struct sk_buff *tpl,
>
> nit: static? afaict it's not used outside this file.

static node.

>> +				     struct skb_frag_walk *walk, u32 off,
>> +				     u32 len, struct skb_seq_state *st,
>> +				     u32 copy_len)
>> +{
>
>
> [...]
>> +
>> +/**
>> + * iptfs_input_ordered() - handle next in order IPTFS payload.
>> + *
>> + * Process the IPTFS payload in `skb` and consume it afterwards.
>> + */
>> +static int iptfs_input_ordered(struct xfrm_state *x, struct sk_buff *skb)
>> +{
>
> Can we try to not introduce a worse problem than xfrm_input already
> is? 326 lines and 20+ local variables is way too much. And then it
> calls another 200+ lines function...
>
> I did try to understand what the main loop does but I got completely
> lost the 3 times I tried :/

I'm not sure what to do with this comment. :)

Its a fairly complex process of reassembling the fragmented inner packets, and it's already divided into 2 functions as you noted. I don't think breaking this up more is going to do much for simplification b/c now we'll need to be passing pointers to what are currently local variables to maintain the state, and the singular concept of what is going on would be fragmented (no pun intended) across multiple functions when it's really just a single process.

>> +static u32 __reorder_this(struct xfrm_iptfs_data *xtfs, struct sk_buff *inskb,
>> +			  struct list_head *list)
>> +
>
> nit: extra blank line

Fixed.

>> +{
>
>
> [...]
>> +/**
>> + * iptfs_input() - handle receipt of iptfs payload
>> + * @x: xfrm state.
>> + * @skb: the packet.
>> + *
>> + * We have an IPTFS payload order it if needed, then process newly in order
>> + * packetsA.
>
> typo? "packetsA"

Fixed.

> [...]
>> +/* IPv4/IPv6 packet ingress to IPTFS tunnel, arrange to send in IPTFS payload
>> + * (i.e., aggregating or fragmenting as appropriate).
>> + * This is set in dst->output for an SA.
>> + */
>> +static int iptfs_output_collect(struct net *net, struct sock *sk,
>> +				struct sk_buff *skb)
>> +{
>> +	struct dst_entry *dst = skb_dst(skb);
>> +	struct xfrm_state *x = dst->xfrm;
>> +	struct xfrm_iptfs_data *xtfs = x->mode_data;
>> +	struct sk_buff *segs, *nskb;
>> +	u32 count, qcount;
>> +	u32 pmtu = 0;
>> +	bool ok = true;
>> +	bool was_gso;
>> +
>> +	/* We have hooked into dst_entry->output which means we have skipped the
>> +	 * protocol specific netfilter (see xfrm4_output, xfrm6_output).
>> +	 * when our timer runs we will end up calling xfrm_output directly on
>> +	 * the encapsulated traffic.
>> +	 *
>> +	 * For both cases this is the NF_INET_POST_ROUTING hook which allows
>> +	 * changing the skb->dst entry which then may not be xfrm based anymore
>> +	 * in which case a REROUTED flag is set. and dst_output is called.
>> +	 *
>> +	 * For IPv6 we are also skipping fragmentation handling for local
>> +	 * sockets, which may or may not be good depending on our tunnel DF
>> +	 * setting. Normally with fragmentation supported we want to skip this
>> +	 * fragmentation.
>> +	 */
>> +
>> +	BUG_ON(!xtfs);
>
> Or drop the packet and add a DEBUG_NET_WARN_ON_ONCE? This should never
> happen, but why crash the system when we have a way to deal with this
> error?

So this is running into basic assert philosophy I suppose. For true invariant's I like to simplify things by asserting them being true (which also documents them) rather than create more logic branches in the code. The added complexity of those extra branches becomes more apparent when you are doing code-coverage analysis. I'd rather not add code that will never execute that I then have to create impossible situation tests to test.

>> +	if (xtfs->cfg.dont_frag)
>> +		pmtu = iptfs_get_cur_pmtu(x, xtfs, skb);
>> +
>> +	/* Break apart GSO skbs. If the queue is nearing full then we want the
>> +	 * accounting and queuing to be based on the individual packets not on the
>> +	 * aggregate GSO buffer.
>> +	 */
>> +	was_gso = skb_is_gso(skb);
>> +	if (!was_gso) {
>> +		segs = skb;
>> +	} else {
>> +		segs = skb_gso_segment(skb, 0);
>> +		if (IS_ERR_OR_NULL(segs)) {
>> +			XFRM_INC_STATS(net, LINUX_MIB_XFRMOUTERROR);
>> +			kfree_skb(skb);
>> +			return PTR_ERR(segs);
>> +		}
>> +		consume_skb(skb);
>> +		skb = NULL;
>> +	}
>> +
>> +	count = 0;
>> +	qcount = 0;
>
> nit: both of those get incremented through the main loop but never read

Removed.

>> +
>> +	/* We can be running on multiple cores and from the network softirq or
>> +	 * from user context depending on where the packet is coming from.
>> +	 */
>> +	spin_lock_bh(&x->lock);
>> +
>> +	skb_list_walk_safe(segs, skb, nskb) {
>> +		skb_mark_not_on_list(skb);
>> +		count++;
>> +
>> +		/* Once we drop due to no queue space we continue to drop the
>> +		 * rest of the packets from that GRO.
>> +		 */
>> +		if (!ok) {
>> +nospace:
>> +			trace_iptfs_no_queue_space(skb, xtfs, pmtu, was_gso);
>> +			XFRM_INC_STATS(dev_net(skb->dev), LINUX_MIB_XFRMOUTNOQSPACE);
>> +			kfree_skb_reason(skb, SKB_DROP_REASON_FULL_RING);
>> +			continue;
>> +		}
>> +
>> +		/* If the user indicated no iptfs fragmenting check before
>> +		 * enqueue.
>> +		 */
>> +		if (xtfs->cfg.dont_frag && iptfs_is_too_big(sk, skb, pmtu)) {
>> +			trace_iptfs_too_big(skb, xtfs, pmtu, was_gso);
>> +			kfree_skb_reason(skb, SKB_DROP_REASON_PKT_TOO_BIG);
>> +			continue;
>> +		}
>> +
>> +		/* Enqueue to send in tunnel */
>> +
>
> nit: unneeded blank line

A style choice, it's an important point in the code, and I was trying to call that out. I'll remove the newline though.

>
>> +		ok = iptfs_enqueue(xtfs, skb);
>> +		if (!ok)
>> +			goto nospace;
>> +
>> +		trace_iptfs_enqueue(skb, xtfs, pmtu, was_gso);
>> +		qcount++;
>> +	}
>> +
>> +	/* Start a delay timer if we don't have one yet */
>> +	if (!hrtimer_is_queued(&xtfs->iptfs_timer)) {
>> +		/* softirq blocked lest the timer fire and interrupt us */
>> +		BUG_ON(!in_interrupt());
>
> Why is that a fatal condition?

Removed this.

>
>> +		hrtimer_start(&xtfs->iptfs_timer, xtfs->init_delay_ns,
>> +			      IPTFS_HRTIMER_MODE);
>> +
>> +		xtfs->iptfs_settime = ktime_get_raw_fast_ns();
>> +		trace_iptfs_timer_start(xtfs, xtfs->init_delay_ns);
>> +	}
>> +
>> +	spin_unlock_bh(&x->lock);
>> +	return 0;
>> +}
>> +
>
> [...]
>> +static int iptfs_copy_create_frags(struct sk_buff **skbp,
>> +				   struct xfrm_iptfs_data *xtfs, u32 mtu)
>> +{
> [...]
>> +	/* prepare the initial fragment with an iptfs header */
>> +	iptfs_output_prepare_skb(skb, 0);
>> +
>> +	/* Send all but last fragment. */
>> +	list_for_each_entry_safe(skb, nskb, &sublist, list) {
>> +		skb_list_del_init(skb);
>> +		xfrm_output(NULL, skb);
>
> Should we stop if xfrm_output fails? Or is it still useful to send the
> rest of the iptfs frags if we lose one in the middle?

At the time it didn't seem worth handling this special case, but I've added freeing the rest now if an xfrm_output returns an error.

>> +	}
>> +
>> +	return 0;
>> +}
>> +
>
> [...]
>> +static int iptfs_first_skb(struct sk_buff **skbp, struct xfrm_iptfs_data *xtfs,
>> +			   u32 mtu)
>> +{
>> +	struct sk_buff *skb = *skbp;
>> +	int err;
>> +
>> +	/* Classic ESP skips the don't fragment ICMP error if DF is clear on
>> +	 * the inner packet or ignore_df is set. Otherwise it will send an ICMP
>> +	 * or local error if the inner packet won't fit it's MTU.
>> +	 *
>> +	 * With IPTFS we do not care about the inner packet DF bit. If the
>> +	 * tunnel is configured to "don't fragment" we error back if things
>> +	 * don't fit in our max packet size. Otherwise we iptfs-fragment as
>> +	 * normal.
>> +	 */
>> +
>> +	/* The opportunity for HW offload has ended */
>> +	if (skb->ip_summed == CHECKSUM_PARTIAL) {
>> +		err = skb_checksum_help(skb);
>> +		if (err)
>> +			return err;
>> +	}
>> +
>> +	/* We've split these up before queuing */
>> +	BUG_ON(skb_is_gso(skb));
>
> Drop and DEBUG_NET_WARN_ON_ONCE?

Earlier comment on asserts applies here.

>> +
>> +	trace_iptfs_first_dequeue(skb, mtu, 0, ip_hdr(skb));
>> +
>> +	/* Simple case -- it fits. `mtu` accounted for all the overhead
>> +	 * including the basic IPTFS header.
>> +	 */
>> +	if (skb->len <= mtu) {
>> +		iptfs_output_prepare_skb(skb, 0);
>> +		return 0;
>> +	}
>> +
>> +	BUG_ON(xtfs->cfg.dont_frag);
>
> and here?

I'll just drop this one.

>> +	if (iptfs_first_should_copy(skb, mtu))
>> +		return iptfs_copy_create_frags(skbp, xtfs, mtu);
>
> Since we end up copying anyway, drop this (and
> iptfs_first_should_copy). You can introduce the optimization later on.

I was going to remove this, but I'd really like to leave it. It provides the start of the optimized code, as well as making it obvious to someone coming after me that there is something that can be (somewhat easily?) done here to finish it. So I see a real loss in removing this code, but I don't see any real gain in doing so.

>> +	/* For now we always copy */
>> +	return iptfs_copy_create_frags(skbp, xtfs, mtu);
>> +}
>> +
>> +static struct sk_buff **iptfs_rehome_fraglist(struct sk_buff **nextp,
>> +					      struct sk_buff *child)
>> +{
>> +	u32 fllen = 0;
>> +
>> +	BUG_ON(!skb_has_frag_list(child));
>
> Not needed, this was tested just before calling this function.

Removed.

>> +
>> +	/* It might be possible to account for a frag list in addition to page
>> +	 * fragment if it's a valid state to be in. The page fragments size
>> +	 * should be kept as data_len so only the frag_list size is removed,
>> +	 * this must be done above as well took
>> +	 */
>> +	BUG_ON(skb_shinfo(child)->nr_frags);
>
> Again not worth crashing the system?

The function does not handle this unexpected case, so this seems like a valid use of BUG_ON() even if one doesn't subscribe to the general assert philosophy I mentioned above. :)

>> +	*nextp = skb_shinfo(child)->frag_list;
>> +	while (*nextp) {
>> +		fllen += (*nextp)->len;
>> +		nextp = &(*nextp)->next;
>> +	}
>> +	skb_frag_list_init(child);
>> +	BUG_ON(fllen > child->data_len);
>> +	child->len -= fllen;
>> +	child->data_len -= fllen;
>> +
>> +	return nextp;
>> +}
>
> [...]
>> +static void iptfs_output_queued(struct xfrm_state *x, struct sk_buff_head *list)
>> +{
>> +	struct xfrm_iptfs_data *xtfs = x->mode_data;
>> +	u32 payload_mtu = xtfs->payload_mtu;
>> +	struct sk_buff *skb, *skb2, **nextp;
>> +	struct skb_shared_info *shi, *shi2;
>> +
>> +	/* For now we are just outputting packets as fast as we can, so if we
>> +	 * are fragmenting we will do so until the last inner packet has been
>> +	 * consumed.
>> +	 *
>> +	 * When we are fragmenting we need to output all outer packets that
>> +	 * contain the fragments of a single inner packet, consecutively (ESP
>> +	 * seq-wise). So we need a lock to keep another CPU from sending the
>> +	 * next batch of packets (it's `list`) and trying to output those, while
>> +	 * we output our `list` resuling with interleaved non-spec-client inner
>> +	 * packet streams. Thus we need to lock the IPTFS output on a per SA
>> +	 * basis while we process this list.
>> +	 */
>
> This talks about a lock but I don't see one. What am I missing?

I will fix the comment as no (further) locking is required here. The reason is that iptfs_output_queued() is always being run from a timer (iptfs_delay_timer()) which will only be running on a single CPU at any given time.

>> +	/* NOTE: for the future, for timed packet sends, if our queue is not
>> +	 * growing longer (i.e., we are keeping up) and a packet we are about to
>> +	 * fragment will not fragment in then next outer packet, we might consider
>> +	 * holding on to it to send whole in the next slot. The question then is
>> +	 * does this introduce a continuous delay in the inner packet stream
>> +	 * with certain packet rates and sizes?
>> +	 */
>> +
>> +	/* and send them on their way */
>> +
>> +	while ((skb = __skb_dequeue(list))) {
>> +		struct xfrm_dst *xdst = (struct xfrm_dst *)skb_dst(skb);
>> +		u32 mtu = __iptfs_get_inner_mtu(x, xdst->child_mtu_cached);
>> +		bool share_ok = true;
>> +		int remaining;
>> +
>> +		/* protocol comes to us cleared sometimes */
>> +		skb->protocol = x->outer_mode.family == AF_INET ?
>> +					htons(ETH_P_IP) :
>> +					htons(ETH_P_IPV6);
>> +
>> +		if (payload_mtu && payload_mtu < mtu)
>> +			mtu = payload_mtu;
>
> Isn't that iptfs_get_cur_pmtu?

You're right; replaced the locals and conditional with iptfs_get_cur_pmtu instead.

> [...]
>> +static enum hrtimer_restart iptfs_delay_timer(struct hrtimer *me)
>> +{
>> +	struct sk_buff_head list;
>> +	struct xfrm_iptfs_data *xtfs;
>> +	struct xfrm_state *x;
>> +	time64_t settime;
>> +	size_t osize;
>> +
>> +	xtfs = container_of(me, typeof(*xtfs), iptfs_timer);
>> +	x = xtfs->x;
>> +
>> +	/* Process all the queued packets
>> +	 *
>> +	 * softirq execution order: timer > tasklet > hrtimer
>> +	 *
>> +	 * Network rx will have run before us giving one last chance to queue
>> +	 * ingress packets for us to process and transmit.
>> +	 */
>> +
>> +	spin_lock(&x->lock);
>> +	__skb_queue_head_init(&list);
>> +	skb_queue_splice_init(&xtfs->queue, &list);
>> +	osize = xtfs->queue_size;
>
> Unused variable?

Removed.

> [...]
>> +static int iptfs_user_init(struct net *net, struct xfrm_state *x,
>> +			   struct nlattr **attrs)
>> +{
>> +	struct xfrm_iptfs_data *xtfs = x->mode_data;
>> +	struct xfrm_iptfs_config *xc;
>> +
>> +	if (x->props.mode != XFRM_MODE_IPTFS)
>> +		return -EINVAL;
>
> Is that necessary? This only gets called via ->user_init for this
> mode.

It predated the callbacks. I've removed it.

>> +	xc = &xtfs->cfg;
>> +	xc->reorder_win_size = net->xfrm.sysctl_iptfs_rewin;
>> +	xc->max_queue_size = net->xfrm.sysctl_iptfs_maxqsize;
>> +	xc->init_delay_us = net->xfrm.sysctl_iptfs_idelay;
>> +	xc->drop_time_us = net->xfrm.sysctl_iptfs_drptime;
>> +
>> +	if (attrs[XFRMA_IPTFS_DONT_FRAG])
>> +		xc->dont_frag = true;
>> +	if (attrs[XFRMA_IPTFS_REORD_WIN])
>> +		xc->reorder_win_size =
>> +			nla_get_u16(attrs[XFRMA_IPTFS_REORD_WIN]);
>> +	/* saved array is for saving 1..N seq nums from wantseq */
>> +	if (xc->reorder_win_size)
>> +		xtfs->w_saved = kcalloc(xc->reorder_win_size,
>> +					sizeof(*xtfs->w_saved), GFP_KERNEL);
>
> We probably need a reasonable bound on reorder_win_size so that we
> don't try to allocate crazy amounts of memory here.

It's a u16 so there's a built in limit. :)

>> +	if (attrs[XFRMA_IPTFS_PKT_SIZE]) {
>> +		xc->pkt_size = nla_get_u32(attrs[XFRMA_IPTFS_PKT_SIZE]);
>> +		if (!xc->pkt_size)
>> +			xtfs->payload_mtu = 0;
>
> That's already set to 0 via kzalloc, right? So passing 0 as
> XFRMA_IPTFS_PKT_SIZE is equivalent to not providing it?

Not providing is supposed to mean don't change. You're right they are equivalent currently.

>> +		else if (xc->pkt_size > x->props.header_len)
>> +			xtfs->payload_mtu = xc->pkt_size - x->props.header_len;
>> +		else
>> +			return -EINVAL;
>
> This could probably use an extack to explain why the value was rejected.

OK, also added extack to the callback so to be able to do this.

>> +	}
>> +	if (attrs[XFRMA_IPTFS_MAX_QSIZE])
>> +		xc->max_queue_size = nla_get_u32(attrs[XFRMA_IPTFS_MAX_QSIZE]);
>> +	if (attrs[XFRMA_IPTFS_DROP_TIME])
>> +		xc->drop_time_us = nla_get_u32(attrs[XFRMA_IPTFS_DROP_TIME]);
>> +	if (attrs[XFRMA_IPTFS_INIT_DELAY])
>> +		xc->init_delay_us = nla_get_u32(attrs[XFRMA_IPTFS_INIT_DELAY]);
>> +
>> +	xtfs->ecn_queue_size = (u64)xc->max_queue_size * 95 / 100;
>> +	xtfs->drop_time_ns = xc->drop_time_us * NSECS_IN_USEC;
>> +	xtfs->init_delay_ns = xc->init_delay_us * NSECS_IN_USEC;
>
> Can't we get rid of the _us version? Why store both in kernel memory?

Removed the *_us copies from the config struct.

> [...]
>> +static void iptfs_delete_state(struct xfrm_state *x)
>> +{
>> +	struct xfrm_iptfs_data *xtfs = x->mode_data;
>> +
>> +	if (IS_ERR_OR_NULL(xtfs))
>
> Can mode_data ever be an error pointer?

I don't believe so, changed to (!xtfs).

>> +		return;
>> +
>> +	spin_lock(&xtfs->drop_lock);
>> +	hrtimer_cancel(&xtfs->iptfs_timer);
>> +	hrtimer_cancel(&xtfs->drop_timer);
>> +	spin_unlock(&xtfs->drop_lock);
>> +
>> +	kfree_sensitive(xtfs->w_saved);
>> +	kfree_sensitive(xtfs);
>> +}
>> +
>> +static const struct xfrm_mode_cbs iptfs_mode_cbs = {
>> +	.owner = THIS_MODULE,
>> +	.create_state = iptfs_create_state,
>> +	.delete_state = iptfs_delete_state,
>> +	.user_init = iptfs_user_init,
>> +	.copy_to_user = iptfs_copy_to_user,
>> +	.get_inner_mtu = iptfs_get_inner_mtu,
>> +	.input = iptfs_input,
>> +	.output = iptfs_output_collect,
>> +	.prepare_output = iptfs_prepare_output,
>> +};
>> +
>> +static int __init xfrm_iptfs_init(void)
>> +{
>> +	int err;
>> +
>> +	pr_info("xfrm_iptfs: IPsec IP-TFS tunnel mode module\n");
>> +
>> +	err = xfrm_register_mode_cbs(XFRM_MODE_IPTFS, &iptfs_mode_cbs);
>> +	if (err < 0)
>> +		pr_info("%s: can't register IP-TFS\n", __func__);
>> +
>> +	return err;
>> +}
>> +
>> +static void __exit xfrm_iptfs_fini(void)
>> +{
>> +	xfrm_unregister_mode_cbs(XFRM_MODE_IPTFS);
>> +}
>
> If the module is unloaded, existing xfrm states will be left but
> silently broken?

You're right. I've added try_module_get()/module_put() calls in the create and delete state callbacks now.

All the changes noted above will be in the next published patch set.

Thanks for the thorough review!
Chris.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

Re: [devel-ipsec] [RFC ipsec-next v2 0/8] Add IP-TFS mode to xfrm

[Christian Hopps]

[-- Attachment #1: Type: text/plain, Size: 2932 bytes --]


Antony Antony <antony@phenome.org> writes:

> On Sun, Nov 12, 2023 at 10:52:11PM -0500, Christian Hopps via Devel wrote:
>> From: Christian Hopps <chopps@labn.net>
>>
>> This patchset adds a new xfrm mode implementing on-demand IP-TFS. IP-TFS
>> (AggFrag encapsulation) has been standardized in RFC9347.
>>
>> Link: https://www.rfc-editor.org/rfc/rfc9347.txt
>>
>> This feature supports demand driven (i.e., non-constant send rate) IP-TFS to
>> take advantage of the AGGFRAG ESP payload encapsulation. This payload type
>> supports aggregation and fragmentation of the inner IP packet stream which in
>> turn yields higher small-packet bandwidth as well as reducing MTU/PMTU issues.
>> Congestion control is unimplementated as the send rate is demand driven rather
>> than constant.
>>
>> In order to allow loading this fucntionality as a module a set of callbacks
>> xfrm_mode_cbs has been added to xfrm as well.
>
> Hi Chris,
>
> I have further reviewed the code and have a few minor questions, mainly
> related to handling of XFRM_MODE_IPTFS. It appears to me be either some case
> missing support or/and in a few places it should be prohibited. I have three
> types of questions:
>
> 1. missing XFRM_MODE_IPTFS support?
> 2. Will XFRM_MODE_IPTFS be supported this combination?
> 3. Should be prohibited combination with XFRM_MODE_IPTFS
>
> 1.  Missing:
>
> a.  wouldn't xfrm_sa_len() need extending?
>
> I could not find support for XFRM_MODE_IPTFS explicitly.
>
> However, I'm not sure how the following code is working when xfrm_sa_len is
> missing IP-TFS xfrm_sa_len:
>
> copy_to_user_state_extra() {
>     if (x->mode_cbs && x->mode_cbs->copy_to_user)
>         ret = x->mode_cbs->copy_to_user(x, skb);
> }
>
> I have attached what I imagine is a fix. Check with Steffen or others if
> this is necessary.

I have adopted this change with a couple small changes, thanks!

> b. esp6_init_state() and esp_init_state():
> These functions do not seem to handle XFRM_MODE_IPTFS. Would they default to work with it?

That's b/c IPTFS uses ESP w/o modification. IP-TFS makes its own mode based changes (similar to what `esp_init_state()` does) in it's `create_state` callback which like `esp_init_state()` is called from `__xfrm_init_state()`.

> 2. Would xfrm4_outer_mode_gso_segment() xfrm6_outer_mode_gso_segment():
> support XFRM_MODE_IPTFS?
> These functions appear to be missing. Is it possible that you don't support GSO and GRO?

Correct.

> 3: Shouldn't these combinations return error?
>
> a. ipcomp and  XFRM_MODE_IPTFS
> I'm guessing that ipcomp would generate an error when userspace tries to add an SA with XFRM_MODE_IPTFS.
> ipcomp6_init_state()
> ipcomp4_init_state()

Correct.

> b: In xfrm_state_construct():
>
> if (attrs[XFRMA_TFCPAD])
>     x->tfcpad = nla_get_u32(attrs[XFRMA_TFCPAD]);

Can you explain this more?

Thanks,
Chris.

>
> -antony
>
> [2. text/plain; 0001-xfrm-iptfs-extend-xfrm_sa_len.patch]...


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

Re: [devel-ipsec] [RFC ipsec-next v2 0/8] Add IP-TFS mode to xfrm

[Antony Antony]

On Thu, Mar 07, 2024 at 06:05:37AM -0500, Christian Hopps wrote:
> 
> Antony Antony <antony@phenome.org> writes:
> 
> > On Sun, Nov 12, 2023 at 10:52:11PM -0500, Christian Hopps via Devel wrote:
> > > From: Christian Hopps <chopps@labn.net>
> > > 

> > b: In xfrm_state_construct():
> > 
> > if (attrs[XFRMA_TFCPAD])
> >     x->tfcpad = nla_get_u32(attrs[XFRMA_TFCPAD]);
> 
> Can you explain this more?

I haven't used it. I just know it from *swan code. And I wondered how it 
would work with IP-TFS. Looking in the kernel code, I noticed, it is not 
allowed in xfrm_user.c

 270                 if (attrs[XFRMA_TFCPAD] &&
 271                     p->mode != XFRM_MODE_TUNNEL) {
 272                         NL_SET_ERR_MSG(extack, "TFC padding can only be used in tunnel mode");
 273                         goto out;
 274                 }